feat(plugins): add llm-switch example plugin for local server management#3672
Open
crxssrazr93 wants to merge 1 commit intoNousResearch:mainfrom
Open
feat(plugins): add llm-switch example plugin for local server management#3672crxssrazr93 wants to merge 1 commit intoNousResearch:mainfrom
crxssrazr93 wants to merge 1 commit intoNousResearch:mainfrom
Conversation
Example plugin demonstrating the lifecycle hooks activated in NousResearch#3542. Auto-manages a local llama-server (or any OpenAI-compatible server) when the active model matches a locally configured model name. Features: - pre_llm_call hook: auto-starts the correct server on first message when hermes is configured with a local model name - on_session_end hook: kills the server on exit - switch_local_llm tool: mid-session model switching — the agent swaps the server when asked ("switch to the code model") - Declarative YAML config for model definitions (GGUF paths, context sizes, KV cache quantization, sampling params) replacing shell scripts The plugin is self-contained in docs/llm-switch-plugin-example/ with a README, example config, and full implementation. Users copy it to ~/.hermes/plugins/llm-switch/ to install. Complements NousResearch#3360 and NousResearch#3548 which restore /model as a slash command — once merged, /model custom:write would trigger the pre_llm_call hook to auto-start the right server seamlessly. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Example plugin demonstrating the lifecycle hooks activated in #3542. Auto-manages a local llama-server (or any OpenAI-compatible server) when the active model matches a locally configured model name.
This is a plugin-only PR — no core changes. All hook infrastructure was already merged in #3542.
Supersedes #2930 (which included core hook patches before #3542 was merged).
What it does
pre_llm_callhook: Detects when the active model name matches a key inmodels.yaml. If the correct server isn't running, starts it automatically before the LLM call proceeds.on_session_endhook: Kills the server when the session ends.switch_local_llmtool: Mid-session model switching — the agent calls this when asked "switch to the code model". Swaps the server behind the scenes while the endpoint stays the same.Example
models.yamlUser flow
hermes model→ select custom provider → pick model name matchingmodels.yamlkeypre_llm_callhook auto-starts the server on first messageswitch_local_llmtool → server swapson_session_endkills serverRelationship to other PRs
/modelis restored as a slash command,/model custom:writewould trigger thepre_llm_callhook to auto-start the right server seamlesslyChanges
6 new files in
docs/llm-switch-plugin-example/:plugin.yaml__init__.pyschemas.pyswitch_local_llmserver.pymodels.yaml.exampleREADME.mdTesting
~/.hermes/plugins/llm-switch/, addmodels.yaml, verify with/pluginsPlatforms tested