fix: use configured models instead of hardcoded defaults#564
Open
fix: use configured models instead of hardcoded defaults#564
Conversation
…mbed/pull The `status`, `embed`, and `pull` commands used the hardcoded DEFAULT_EMBED_MODEL_URI / DEFAULT_GENERATE_MODEL_URI / DEFAULT_RERANK_MODEL_URI constants instead of reading the user-configured models from index.yml. This means `models.embed` (and generate/rerank) in the YAML config as well as QMD_EMBED_MODEL env var were silently ignored by these three commands, even though getStore() correctly calls setDefaultLlamaCpp() with the config values. Fix: read model URIs from the LlamaCpp instance (which already resolves config → env → default) instead of using the compile-time constants. Also adds generateModelName and rerankModelName getters to LlamaCpp (embedModelName already existed). Fixes tobi#562
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
status,embed, andpullcommands ignoremodels.embed/models.generate/models.rerankfromindex.ymland the correspondingQMD_*_MODELenv vars. They always use the hardcoded defaults (e.g. embeddinggemma-300M).Root cause: These commands reference the compile-time constants (
DEFAULT_EMBED_MODEL_URIetc.) directly, bypassing theLlamaCppinstance thatgetStore()already configures viasetDefaultLlamaCpp().Fix: Read model URIs from
getDefaultLlamaCpp().embedModelName(and newgenerateModelName/rerankModelNamegetters) instead of the constants.Changes
src/cli/qmd.ts: 3 call sites updated (status display, embed command, pull command)src/llm.ts: AddedgenerateModelNameandrerankModelNamegetters toLlamaCpp(mirrors existingembedModelName)Test plan
qmd statusshows configured model (not default) whenmodels.embedis set in index.ymlqmd embed -fuses configured modelqmd pulldownloads configured modelsFixes #562