In reply toby Djoerd Hiemstra
I’m looking forward to following this project, Djoerd! It sounds sort-of IndieWeb like. Where e.g. MicroSub decouples feed fetching from feed reading and Micropub writing from posting, this project decouples index building from the search. Within IndieWeb that allows the creation of a variety of personal tools, to read and write the web. I’ve long been musing about personal search engines and personal agents and crawlers without putting anything into action. I’m curious to see if this project will actually deliver some of the things I dreamt of over time, by enabling personal tools for search.
A new EU project OpenWebSearch.eu … [in which] … the key idea is to separate index construction from the search engines themselves, where the most expensive step to create index shards can be carried out on large clusters while the search engine itself can be operated locally. …[including] an Open-Web-Search Engine Hub, [where anyone can] share their specifications of search engines and pre-computed, regularly updated search indices. … that would enable a new future of human-centric search without privacy concerns.