[BUG]: In query mode, return answer citations with mach rates ranging from 0% to 3%.

[kaanvur]

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

I added two files.
I selected query mode in chat settings.
Document similarity threshold
medium is set in vector database settings.
But the model still returns answers to irrelevant questions.
As citations, it shows a few examples with mach rates ranging from 0% to 3%.

Are there known steps to reproduce?

[timothycarambat]

Turn off reranking when in query mode - that is the root cause of this (and also why it is not the default mode for a workspace)

[kaanvur]

Turn off reranking when in query mode - that is the root cause of this (and also why it is not the default mode for a workspace)

Thanks for your answer @timothycarambat but I couldn't find any instructions on where and how to do this setting. Can you help me?

[timothycarambat]

Of course, in the UI we call reranking "Search Preference" since re-ranking is very ambiguous to non-technical people. If you click on a workspace's settings (gear icon) and navigate to "Vector Database" you will find a field "search preference" which you can change from "Accuracy Optimized" back to "default"

https://docs.anythingllm.com/llm-not-using-my-docs#vector-database-settings--search-preference

[kaanvur]

Thank you for your answer @timothycarambat . When I read your explanations, it gives the opposite impression :) I made the changes. This time it brought up more relevant information. However, none of the quotes that it said were 65% relevant to my question "egg bread recipe" have anything to do with this topic. How can I improve this?

[kaanvur]

Would changing the Embedding model here provide better results?

[timothycarambat]

@kaanvur The default embedder is pretty bad at multi-lingual embedding (see #658). However we support using Ollama/LMStudio with any embedder you can find on HF, which goes a long way.

You would indeed get better responses with the non-default. The default we have is super small so that is why we use it. Some other models like Jina can be more comprehensive but also are 300MB

[kaanvur]

Thank you for all your answers and patience. I guess I need to try different combinations and see what the results are for the better results :) Thanks for your efforts!