What problem does this solve?
The config is currently opinionated with temp and doesn't set top_k and top_p.
Proposed solution
Add env vars for temp, top_k and top_p for llm and swarm models.
Alternatives considered
No response
Which area does this affect?
Configuration
What problem does this solve?
The config is currently opinionated with temp and doesn't set top_k and top_p.
Proposed solution
Add env vars for temp, top_k and top_p for llm and swarm models.
Alternatives considered
No response
Which area does this affect?
Configuration