refactor(model-routing): centralize explicit foreground fallback policy

[alteixeira20]

Summary

Centralize explicit foreground fallback policy for Chat and Agent routing.

Fallback remains disabled by default, advances only on eligible availability
failures, preserves owner/model/credential boundaries, pins the answering Agent
route after substantive output or a completed tool call, and records truthful
per-round model/endpoint provenance and usage attribution.

This is the clean rebased replacement for #5811 on top of merged #5801. The
implementation was reconciled against current dev and the rebase regressions
found during final audit were fixed.

Target branch

• This PR targets dev, not main.

Linked Issue

Fixes #5626

Supersedes #5811.

Part of #5625.

Type of Change

• Bug fix
• New feature
• Refactor / cleanup

Checklist

• I searched open issues and open PRs — this is the rebased replacement for refactor(model-routing): centralize explicit foreground fallback policy #5811, not a competing proposal.
• This PR targets dev
• My changes are limited to the explicit foreground fallback policy and its routing/provenance/test surface.
• I actually ran the app (docker compose up or uvicorn app:app) and verified the change works end-to-end.

Behavior contract

• Missing, disabled, malformed, or empty policy state is strict.
• Candidates are concrete endpoint/model pairs in explicit order.
• Named users cannot inherit single-user fallback consent.
• Endpoint ownership and model allowlists are enforced before credential resolution.
• Only documented availability failures may advance before substantive output.
• Request, authentication, authorization, model, schema, and configuration failures do not advance.
• Empty successful responses do not trigger fallback.
• Accepted output or a completed tool call pins the route for later Agent rounds.
• Streaming Chat, non-stream Chat, Agent events, history, metrics, and usage identify the route that actually answered.
• Legacy default_model_fallbacks remains stored but is not used as foreground fallback policy.

Validation

Final rebase audit was performed against current dev.

• Clean one-commit history on top of merged fix(model-routing): keep user-selected models strict #5801.
• Focused foreground routing and Agent provenance tests passed.
• Reconciliation-sensitive Agent/Chat regression tests passed.
• Full authoritative Python suite passed in a clean virtual environment.
• Python compilation passed.
• JavaScript syntax checks passed.
• No conflict markers remain.
• Stale _ody_general_no_tool_mode reference removed.

How to Test

1. Install requirements.txt in a clean virtual environment.
2. Run the foreground routing, Agent provenance, stream-error, context-compaction,
   prefs, and LLM fallback tests.
3. Run the full Python test suite with python -m pytest -q.
4. Run python -m compileall -q core routes src.
5. Run node --check against the modified/new Chat JavaScript modules.
6. Confirm disabled policy stays strict and eligible availability failures alone
   may advance to an explicitly configured route.
7. Confirm Agent fallback routes remain pinned after accepted output/tool effects
   and saved metrics identify the actual answering route.

Visual / UI changes — REQUIRED if you touched anything that renders

This changes existing Chat provenance and terminal-stream behavior rather than
introducing a new visual component or layout. Existing visual styling is
preserved. Browser-side JavaScript regression tests cover provenance, terminal
errors, background terminal state, and interrupted streams.

No new visual surface is introduced.

[StressTestor]

ran two independent review passes on this, one diffing every finding against base b522964 in a throwaway worktree, one probing the server paths at runtime. the fallback-policy design itself looks solid: owner-scoped consent, prefs migration, and credential identity all verify clean. but the rebase dropped real code on the floor, and two of those drops are ship blockers. requesting changes.

rebase casualties (blockers)

1. chat send is broken in the browser. the rebase deleted the definitions of _touchStreamActivity, _syncForegroundStreamGlobals, and _getForegroundStreamState (they exist at base b522964, chat.js:585-609) but left the call sites, 16+ of them. every send hits _touchStreamActivity(streamSessionId) at chat.js:1375, outside any try/catch, and throws ReferenceError before the request dispatches. also breaks _releaseSendFlag (composer stays disabled), the stop button (chat.js:4455), and the slash-command streaming check (chat.js:716). quick repro: grep -n "function _touchStreamActivity" static/js/chat.js finds nothing at head, finds it at base. node --check passes because the failure only happens at runtime.

2. notes mode disables every tool, including manage_notes. the general no-tool branch got fused into the notes branch during the rebase: agent_loop.py:3960 now runs disabled_tools.update(known_tool_names()) inside elif _ody_notes_finetune_mode, so "add a note to buy milk" on odysseus-qwen3 gets the minimal notes prompt with zero callable tools. the fusion is visible in the logs, "notes finetune tool clamp" and "general no-tool clamp active" fire back to back. the inverse also regressed: general qwen turns (zero tools at base) now keep full schemas.

3. calendar routing is dead code. notes_mode at agent_loop.py:3926 collapsed to (A or B) and B, which is just B, so pure calendar phrasing ("what's on my calendar today") never routes. the calendar-followup branch from base (3527-3540, _looks_like_notes_calendar_followup + recent-tool-context) is gone too.

4. the notes clamp lost manage_calendar and manage_tasks. agent_loop.py:3944 clamps to {manage_notes, ask_user, update_plan}; base had all three domain tools plus the difference_update that re-enabled them when tool-RAG had disabled them.

5. workspace rules never reach the system prompt. the sole _build_system_prompt call at agent_loop.py:4095 dropped the workspace=workspace kwarg from base, so _workspace_coding_rules (confinement, get_workspace orientation) is never injected. tool selection still reads workspace, so the agent gets file tools without the guardrails.

6. the qwen3 temperature clamp is gone. base force-capped odysseus-qwen3 to 0.2 (base 3148-3152); that was deleted and never re-homed. the only remaining 0.2 clamp (llm_core.py:1079) is scoped to local MiniMax MLX.

bugs in the new code

7. IPv6 endpoints are never cost-tracked. endpoint_cost_tracked (endpoint_resolver.py:58) short-circuits on "." not in host before parsing, so http://[2606:4700::1]/v1 classifies as not-cost-tracked. the private-range list also omits link-local 169.254/16, which classifies as cost-tracked. chat_helpers._is_local_host already handles both families correctly; reusing it fixes both cases and removes a third copy of this classification.

8. a None endpoint_url turns the clean 400 into a 500. chat_routes.py:711 and 1063 use getattr(sess, "endpoint_url", "").strip(). when the attribute exists and is None, getattr returns None and .strip() raises AttributeError. lines 403 and 461 in the same file already use (getattr(...) or "").strip().

9. PoolTimeout blocks availability fallback. llm_core.py:2493 groups httpx.PoolTimeout with WriteTimeout as fallback-ineligible, but PoolTimeout fires while waiting for a connection, before anything is sent. splitting it out keeps the fail-closed intent for ambiguous writes while letting the unambiguous case fall back.

10. terminal-then-EOF clobbers the saved result. when the canonical terminal event arrives and the connection then dies, chat.js:4297 checks isRecoverableStreamError(err) before _canonicalTerminalSaved (only consulted at 4299, inside the terminal branch), so auto-recover kicks in, resume 404s for the completed run, and "connection lost" replaces the persisted result. checking the flag first fixes it.

11. rapid resend can strand the old run's SSE forever. agent_runs.py:170 cancels the previous task; when the cancel lands before _drain first runs, the finally that flips status and arms eviction never executes, and a subscriber bound to the old run heartbeats forever. flipping the old run's status synchronously in start() closes it.

12. stop has two dead zones. (a) the run id is only captured when response headers arrive (chat.js:1926), but the UI is stoppable before that, so a fast stop is a silent no-op (_stopExactRun at chat.js:614 returns false without making a request). (b) only the detached-run and resume responses set X-Odysseus-Run-Id (chat_routes.py:2380/2396), so header-less paths and non-browser clients of POST /api/chat/stop lost the ability to cancel. probably one design pass: queue the stop until the id arrives, and decide what header-less callers should get.

13. the fallback allowlist filters after slicing. foreground_model_routing.py:80 applies entries[:MAX_FOREGROUND_FALLBACKS] before the allowed_models filter, so allowed entries past position 10 get dropped and the policy can silently resolve to strict. filter first, then slice.

lower priority

• chatRenderer.js:997: the cost ledger does an unlocked whole-map localStorage read-modify-write. the comment claims two-tab safety via per-runId idempotence, but two tabs writing different runIds are last-writer-wins. worth at least fixing the comment.
• agent_loop.py:5726: persisted tool_events dropped desc and _resolved_tool_event_name(), so saved history permanently shows generic names for MCP tools.
• agent_loop.py:4366 / endpoint_resolver.py:568: route descriptors get recovered and re-aligned by equality checks and index scans with silent fallbacks (stale headers degrade to a generic descriptor, identical endpoints attribute by row order). threading the known endpoint id through with the candidate removes the whole desync class. bigger refactor, fine as a follow-up.
• foreground_model_routing.py:106: the resolver-seam compat branch pairs descriptors positionally against a deduped list, so indices desync. only tests reassign the seam today, but it's a trap on a documented seam.
• llm_core.py:3441: a symbolic-only status like {"status": "RATE_LIMITED"} returns the 400 default before the marker heuristics would map it to 429, so fallback stalls on that shape. the docstring reads like fail-closed is intentional here, so flagging rather than requesting. your call.

the meta issue

everything in the blockers section is invisible to node --check and the focused pytest additions, which is all this PR ran (the e2e checklist item is unchecked, which tracks). worth launching the app and sending one message before merge (that alone catches finding 1), or a browser smoke test in CI that does the same, or the next rebase does this again.

the two CodeQL prototype-pollution alerts look refutable btw: the dynamic keys trace to server-generated uuids (agent_runs.py:37) and never reach shared or server state.

happy to push the restoration commits for 1-6 if useful, they're mostly copy-back-from-base.

[StressTestor]

pushed the fixes to this branch, 10 commits on top of eb3a094, head cdeb4fe.

everything from the review is addressed:

• the three deleted chat.js helpers (_getForegroundStreamState, _syncForegroundStreamGlobals, _touchStreamActivity) are restored, chat send works again
• notes/calendar routing unfused: notes mode keeps manage_notes, manage_calendar, manage_tasks, ask_user, and update_plan, the general no-tool clamp is back in its own branch, calendar phrasing and follow-ups route again
• workspace= kwarg restored, qwen temperature cap re-homed (applied per candidate now, so mixed qwen/non-qwen fallback chains can't inherit each other's values)
• cost classification handles public IPv6 and link-local via ip.is_global, a None endpoint_url 400s instead of 500ing, PoolTimeout is fallback-eligible again (split from WriteTimeout), and the fallback allowlist filters before slicing
• run lifecycle: rapid resend can't strand the old SSE (agent_runs terminalizes the replaced run synchronously), terminal-then-EOF reloads the canonical record instead of auto-recovering into a 404, and the cost ledger serializes cross-tab writes behind a web lock
• stop is correct across the whole pre-header window: a per-send generation is reserved synchronously at send commit, queued stops are generation-keyed and flush from their own send's header arrival, and a superseded send can't clear the replacement's state, reopen the send gate mid-preflight, or consume the newer run's identity

test story: ~30 new regression tests, including browser tests that execute the real chat.js code under node (extracted and run, which catches what node --check can't), each proven failing on the commit before its fix. full suite: 5,266 passed; the 6 failures are pre-existing on eb3a094 (docker-socket/SSRF/glob env tests, reproduced unchanged on the unpatched head).

deliberate scope calls, flagging them so you can veto:

• headerless POST /api/chat/stop stays fail-closed by design, so non-browser clients can't cancel. say the word if you want an opt-in there.
• the symbolic-status fix maps only rate-limit symbols (RATE_LIMITED, RATE_LIMIT_EXCEEDED, RESOURCE_EXHAUSTED) to 429; everything else still fails closed per the docstring's intent.
• resolve_route_descriptor's equality-scan fallback still exists; the selected_endpoint_id path bypasses it when the client sends the id. threading identity through everywhere is a bigger refactor, left for a follow-up.
• known residual: a POST that never produces headers can only be hard-aborted client-side at the 2/6-minute request timeout plus a 2s grace; its run is only reachable by its replacement. same behavior as base.

the request-changes review above predates these commits. ping me if any of this needs more detail.