Skip to content

refactor(model-routing): centralize explicit foreground fallback policy - #5811

Open
RaresKeY wants to merge 2 commits into
odysseus-dev:devfrom
RaresKeY:refactor/explicit-foreground-fallback-policy
Open

refactor(model-routing): centralize explicit foreground fallback policy#5811
RaresKeY wants to merge 2 commits into
odysseus-dev:devfrom
RaresKeY:refactor/explicit-foreground-fallback-policy

Conversation

@RaresKeY

Copy link
Copy Markdown
Member

Summary

This replaces #5683, which GitHub closed during the repository transfer and fork-network separation. The branch has been rebuilt on the current dev history; the implementation scope is unchanged.

Foreground Chat, non-stream Chat, and Agent runs now share one explicit per-user fallback policy. Selected models remain strict unless the current user enables an ordered list of exact endpoint/model candidates. Only availability failures may advance; request, authentication, authorization, unsupported-model, and configuration failures remain visible. Agent runs pin a fallback route after substantive output or a completed tool call, and model provenance follows the route that actually answered.

Target branch

  • This PR targets dev, not main. All PRs land in dev; main is curated by the maintainer at each release. If your PR is on main by accident, click "Edit" on this PR and change the base.

Linked Issue

Fixes #5626

Part of #5625

Depends on #5632

Type of Change

  • Bug fix (non-breaking — fixes a confirmed issue)
  • New feature (non-breaking — adds new behaviour)
  • Breaking change (changes or removes existing behaviour)
  • Refactor / cleanup (behaviour unchanged)
  • Documentation only
  • CI / tooling / configuration

Checklist

  • I searched open issues and open PRs — this is not a duplicate.
  • This PR targets dev
  • My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in.
  • I actually ran the app (docker compose up or uvicorn app:app) and verified the change works end-to-end. Type-checks and unit tests are not enough.

Stack

This is the second PR in #5625: #5632 → this PR → #5684.

Until #5632 merges, GitHub shows both stack commits. The issue-specific commit is 1d0e7cdf32d162aaa1993d771348c36b61fd4387.

Behavior contract

  • Missing, disabled, malformed, or empty policy state is strict.
  • Candidates are concrete endpoint/model pairs in explicit order.
  • Named users cannot inherit single-user fallback consent.
  • Endpoint ownership and model allowlists are enforced before credential resolution.
  • Only documented availability failures may advance before substantive output.
  • Request, authentication, authorization, model, schema, and configuration failures do not advance.
  • Empty successful responses do not trigger fallback.
  • Accepted output or a completed tool call pins the route for later Agent rounds.
  • Streaming Chat, non-stream Chat, Agent events, history, metrics, and usage identify the route that actually answered.
  • Legacy default_model_fallbacks data remains stored but is not read, exposed, migrated, or used by foreground routing.

Scope

In scope: foreground Chat and Agent routing, explicit per-user policy, owner-scoped resolution, availability classification, Agent pinning, provenance, accounting, and removal of dead silent-switching paths.

Out of scope: Settings controls, Utility, Vision, Deep Research, scheduled tasks, teacher, image, embedding, search-provider routing, inferred model tiers, and legacy-data migration.

Validation

  • Canonical GitHub Actions Python tests: passed.
  • Python compilation: passed.
  • JavaScript syntax checks: passed.
  • CodeQL, dependency review, secret scanning, workflow security, and container checks: passed.
  • Focused routing and provenance regression suite: 506 passed.

Live-provider and interactive browser validation remain outstanding, so this PR remains draft.

How to Test

  1. Leave the policy disabled and fail the selected route in streaming Chat, non-stream Chat, and Agent mode; confirm no second endpoint is called.
  2. Enable an ordered list and return an eligible availability failure; confirm the next exact candidate answers.
  3. Repeat with request, authentication, authorization, unsupported-model, schema, and configuration failures; confirm none advances.
  4. Let an Agent fallback complete a tool call; confirm later rounds stay pinned to that route.
  5. Confirm events, saved history, response metadata, usage, and final metrics identify the winning endpoint/model.
  6. Confirm separate users cannot read or activate one another's fallback policy.
  7. Confirm legacy default_model_fallbacks values remain stored but have no runtime or Settings effect.

Visual / UI changes — REQUIRED if you touched anything that renders

  • Screenshot or short clip of the change in the running app, attached below. Mobile screenshot too if the change affects mobile.
  • Style match: the change uses Odysseus's existing visual language. Specifically:
    • Reuse existing CSS variables (--red, --fg, --bg, --card, --border, etc.) — do not introduce new color values, font sizes, or spacing units.
    • Reuse existing button/input/card/border classes. Don't invent parallel styling.
    • No Unicode emoji in UI or code. Use inline SVG (matching the monochrome icon style already in static/index.html) or plain text.
    • Monospaced font (Fira Code) for primary UI text. Don't override.
    • Dark theme is the default; any light-mode work must be wired through the existing theme system, not hard-coded.
  • No new component patterns. If a similar widget already exists in the app, extend it instead of writing a parallel one.
  • I am not an LLM agent submitting a bulk PR. If you are, please open an issue describing the problem first — bulk auto-generated PRs that don't match the project's visual style are closed on sight, even when the underlying fix is correct.

Screenshots / clips

Not included. This PR changes existing fallback and provenance labels; manual browser validation remains outstanding.

RaresKeY added 2 commits July 27, 2026 20:34
Make foreground fallback an explicit per-user, availability-only policy shared by streaming Chat, non-stream Chat, and Agent runs.

Preserve strict defaults, owner/model and credential boundaries, pinned Agent routes, and truthful per-round provenance/accounting. Carry provider-reported model identifiers through native streaming adapters, non-stream responses, and caches, and keep legacy default_model_fallbacks as tombstoned raw storage that generic settings APIs and agent tools cannot expose or mutate.
@github-actions github-actions Bot added the ready for review Description complete — ready for maintainer review label Jul 28, 2026
@RaresKeY
RaresKeY marked this pull request as ready for review July 28, 2026 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for review Description complete — ready for maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(models): centralize explicit foreground fallback policy

1 participant