List of categories seems to be static

[benoit74]

The list of categories seems to be static, e.g. it misses at least phet, video, iFixit, ... and blank category (more a library problem, but we need to be able to search these ZIMs as well).

Do we want to make this list of categories dynamic or should we simply update the list before each release?

Updating the list before each release allows to continue to transform the OPDS category (e.g. stack_exchange) into a nice looking label (e.g. StackExchange).

Making the list dynamic allows to not care about updating it and being very reactive to library changes. It more or less forces us to stop transforming OPDS category into nice looking label.

But this transformation of OPDS category is anyway a bad idea from my PoV, we should represent the category as is stored in the OPDS catalog, and AFAIK nothing forces us to have ugly technical categories in the catalog / ZIM (this needs to be answered / solved in kiwix/overview#104).

From my PoV, making the list dynamic is in fact mostly mandatory because we have old devices in the wild, stuck on old iOS versions and hence stuck on old Kiwix app versions. If the list is not dynamic, it means the app will never update its categories ... which is really bad.

[kelson42]

Do we want to make this list of categories dynamic or should we simply update the list before each release?

Should be (mostly) dynamic, via https://kiwix-tools.readthedocs.io/en/latest/kiwix-serve.html#catalog-v2-categories. But we can rely on libkiwix or/and cache if offline.

[BPerlakiH]

Thank you @benoit74. I am looking into this.

One argument is not really clear for me:

From my PoV, making the list dynamic is in fact mostly mandatory because we have old devices in the wild, stuck on old iOS versions and hence stuck on old Kiwix app versions. If the list is not dynamic, it means the app will never update its categories ... which is really bad.

From a practical point of view, old Kiwix app versions (and old iOS versions), will be stuck forever on static categories, there's no remedy for them, since the latest Kiwix app releases will support higher and higher iOS versions gradually.
I do see we have a V2 API, but the "old" clients - I mean everyone before this ticket - is on the url:
https://library.kiwix.org/catalog/v2/entries?count=-1
roughly since 16 Feb 2024:

On the other-hand it would be nice to have insights into how many users are still stuck on old devices, to see if it's worth to support them, and if so, for how long.

[BPerlakiH]

ZimFiles are not missing

I looked into those categories as well, and currently we do have all the ZIM files, so it seems to me that nothing is missing. Each file that is not matching the static list of categories are falling into the "other" category on the reader side. Including the aforementioned iFixit:

With dynamic categories we might have 1 item per category

Now there's another caveat here, we do filter down the entries by the user defined languages, which means that the user might have selected only "English", this will filter down the items in each (dynamic) category as well. Currently we display only items from a selected category at one time.

This means that for example for iFixit we will have a single item category - assuming that the user has "English" only selected. I am not sure if this is going to be an improvement for the user.

Additionally there's another rule we currently apply, that we do not display empty categories, meaning even if a category would have otherwise items, if those items are not matching the language(s) preferred by the user, the filtered down result would give an empty category, and we do not display the category at all.

Search is not global, but per category

The list of categories seems to be static, e.g. it misses at least phet, video, iFixit, ... and blank category (more a library problem, but we need to be able to search these ZIMs as well).

The problem we have with the search is that it is not global, but per category. So it is possible to search for a ZIM file, but the user needs to know upfront in which category it is in: