Keyword search for web application

[Ekliptor]

How could I implement a keyword search for a web application?
I have read this topic about dynamic content: #28
But keyword items are public, not private.

Let's assume the following use case:

• a web app where EVERY user can publish items under keywords
• all items and their keywords are considered public (read-only)

I understand that there is no server side script or central database with an index. So the browser with javascript could do something like this:
let keywordHash = someHashFunction("MyKeyword"); // SHA2 or something similar
let results = ipfsFind(keywordHash);
for (let result of results) // loop through results and download the desired items

1. What about publishing and searching for multiple keywords in your DHT? The best approach should be to publish them under all keyword hashes. (Only then can users find them under every single keyword.)
   Normalizing spaces/word-separators and publishing every possible keyword combination for a phrase of length n would require 2^n publications.
2. Should I add a prefix to all my keywords to avoid conflicts with potential other apps when searching for common keywords?
   let keywordHash = someHashFunction(appPrefix + "MyKeyword");

Alternatively my app could built a database file where every user can write new entries to and load it for every user on when the user opens the app. Search results would appear instantly, but it's not scalable for larger indices?

Or is there already an official IPFS search API in development? I assume more apps would require something like that?
Or is it your intention that every app should deal with this on its own? In that case a reference implementation might still be helpful.

[mcast]

If you want to search a static search index, client-side in Javascript, I think there are choices with & without downloading the full index. I found

• http://jssindex.sourceforge.net/ (intention: index data is already local, e.g. on CD-ROM)
• BWT-based search
  • allows downloading just the chunks of index providing the result
  • not sure how you would update it incrementally. Perhaps main + incremental patches?
  • can't find the one I remember, but http://dl.acm.org/citation.cfm?doid=2361354.2361404 (abstract only / :moneybag:)

[flyingzumwalt]

This issue was moved to https://discuss.ipfs.io/t/keyword-search-for-web-application/340