Skip to content

Roadmap: temperature is sent unconditionally while models are env-overridable — a model bump silently degrades every AI reply to the apology fallback #149

Description

@NathanTarbert

Roadmap item, not urgent today — but it turns a routine config change into a silent outage.

What breaks

Every Anthropic call in packages/outpost/ai passes temperature unconditionally, while the model on each is env-overridable:

Call site Temperature Model env var Default
generator.ts:117 (generate) config.responseTemperature 0.3 AI_RESPONSE_MODEL claude-sonnet-4-6
generator.ts:185 (generateStream) config.responseTemperature 0.3 AI_RESPONSE_MODEL claude-sonnet-4-6
confidence.ts:67 config.confidenceTemperature 0.1 AI_CONFIDENCE_MODEL claude-haiku-4-5-20251001
classifier.ts:60 config.classifierTemperature 0.1 AI_CLASSIFIER_MODEL claude-haiku-4-5-20251001
sentiment.ts:69 config.sentimentTemperature 0.1 AI_SENTIMENT_MODEL claude-haiku-4-5-20251001

Per the Anthropic API reference, temperature / top_p / top_k are removed on Claude Opus 4.7, Opus 4.8, Opus 5, and Fable 5 — sending any of them returns 400 invalid_request_error. On Claude Sonnet 5, a non-default value is rejected the same way. The defaults above are all currently-safe models, so nothing is broken right now.

Why it's worth a roadmap slot rather than a backlog line

The failure is silent and total. Set AI_RESPONSE_MODEL=claude-opus-5 — a completely reasonable upgrade — and:

  1. Every messages.create in generator.ts 400s.
  2. generator.ts catches the error and returns its fallback: "I apologize, but I was unable to generate a response at this time. A human support agent will follow up shortly."
  3. That text is posted publicly to every GitHub issue and Discord thread.
  4. classifier, sentiment, and confidence each catch their own 400 and degrade to heuristics.

So the bot keeps replying, keeps scoring, keeps classifying — and every answer is the apology. Nothing in the response path distinguishes "the model rejected our request shape" from "the API had a bad minute", and the only signal is a console.error line in the worker.

Worth noting this interacts with the groundedness gate (#146, #147): a 400'd generation produces empty text, which scores as perfectly grounded — the gate has nothing to object to.

Suggested shape

  • Stop sending temperature when the target model doesn't accept it. Either drop it entirely (the modern equivalent is output_config.effort, which the same models do accept) or gate it behind a model-capability check.
  • Fail loudly on a 400. A malformed-request error is a deploy-time bug, not a transient fault — it should surface distinctly from a rate limit or a timeout rather than being absorbed into the same fallback path.
  • Consider adopting output_config: { effort: ... } on the generator while touching this, since it replaces what temperature was reaching for on these models.

Found by the CR loop on #143 (all three rounds, multiple reviewers). Out of that PR's subject, so filed rather than fixed there.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: aiAI response pipeline, groundedness, confidenceroadmapTracked on the Outpost roadmaproadmap: nowRoadmap horizon: V1 launch critical path / safety

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions