Skip to content

fix(model-routing): keep user-selected models strict - #5801

Merged
alteixeira20 merged 2 commits into
odysseus-dev:devfrom
RaresKeY:fix/strict-selected-model
Aug 12, 2026
Merged

fix(model-routing): keep user-selected models strict#5801
alteixeira20 merged 2 commits into
odysseus-dev:devfrom
RaresKeY:fix/strict-selected-model

Conversation

@RaresKeY

Copy link
Copy Markdown
Member

Summary

This replaces #5632, which GitHub closed during the repository transfer and fork-network separation. The branch has been rebuilt on the current dev history; the implementation scope is unchanged.

Keep foreground Chat and every Agent tool-call round on the endpoint and model selected for the session. A shared foreground policy boundary now supplies the Chat candidate list and the Agent fallback slice, so both paths are strict by default instead of silently consulting the historical default-model list.

Existing nonempty default_model_fallbacks values remain stored as legacy data for rollback and manual reference. They are not migrated, cleared during endpoint cleanup, inherited by Utility routing, used to choose a new default chat, or returned as candidates to background callers. The retired Settings editor is hidden, and ordinary default-model saves no longer rewrite those preserved values.

The generic fallback transport remains available for workload-specific policies. This PR only changes which candidates are authorized; the explicit per-user replacement policy is tracked separately in #5626 and #5627.

Target branch

  • This PR targets dev, not main.

Linked Issue

Fixes #4561

Part of #5625

Type of Change

  • Bug fix (non-breaking — fixes a confirmed issue)
  • New feature (non-breaking — adds new behaviour)
  • Breaking change (changes or removes existing behaviour)
  • Refactor / cleanup (behaviour unchanged)
  • Documentation only
  • CI / tooling / configuration

Checklist

  • I searched open issues and open PRs — closed PR fix(agent): do not switch models mid agent loop (#4561) #5376 is related but unmerged, Agent-only, and lacks the Chat parity and legacy-disconnection coverage in this branch.
  • This PR targets dev.
  • My changes are limited to strict selected-model routing, preservation/disconnection of the legacy default list, and focused regression coverage.
  • I ran the app end-to-end. No disposable two-provider environment was available; route-level, Agent-loop, persistence, and adjacent automated checks were run in the secretless runner instead.

How to Test

  1. Run the focused and adjacent suite:

    python -m pytest -q \
      tests/test_foreground_model_routing.py \
      tests/test_legacy_default_fallback_ui.py \
      tests/test_agent_loop.py \
      tests/test_agent_rounds_exhausted.py \
      tests/test_chat_helpers.py \
      tests/test_chat_metrics.py \
      tests/test_chat_preprocess_tool_policy.py \
      tests/test_chat_route_tool_policy.py \
      tests/test_chat_stream_scope.py \
      tests/test_aux_llm_owner_scope.py \
      tests/test_endpoint_resolver_headers.py \
      tests/test_resolve_endpoint_fallbacks.py \
      tests/test_model_defaults.py \
      tests/test_model_routes.py \
      tests/test_session_endpoint_owner_scope.py \
      tests/test_llm_core_fallback.py \
      tests/test_llm_core_streaming.py \
      tests/test_task_endpoint_normalization.py

    Expected result: 330 tests pass.

  2. Run syntax checks:

    python -m py_compile \
      src/foreground_model_routing.py \
      src/endpoint_resolver.py \
      routes/chat_routes.py \
      routes/model_routes.py \
      routes/email_routes.py \
      src/task_endpoint.py
    node --check static/js/settings.js
  3. Configure two model endpoints and retain a nonempty legacy default fallback list. Select the primary endpoint/model in a Chat session, make it fail before output, and verify the stream surfaces that failure without calling the legacy endpoint.

  4. Repeat in Agent mode with a tool call followed by another model round. Verify every recorded request uses the selected endpoint/model and no fallback event is emitted.

  5. Change the default endpoint/model in Settings and verify the preserved legacy list is not rewritten. Delete an endpoint referenced only by that legacy list and verify the stored data remains intact but has no routing effect.

The focused/adjacent suite passes 330 tests. Python compilation, JavaScript syntax, and diff checks pass. A full isolated run reached 4,671 passing tests and 5 skips; its 11 unrelated failures are contained in a 12-failure baseline set reproduced on clean current origin/dev.

Visual / UI changes — REQUIRED if you touched anything that renders

The obsolete Default Chat Model fallback editor is hidden so Settings no longer promises behavior that is intentionally disconnected. No new style, component, spacing, or mobile behavior is introduced.

  • Screenshot or short clip of the change in the running app. The secretless review runner has no browser engine, so this remains an explicit draft-PR validation item.
  • Style match: the existing Settings structure is retained; no new styling is introduced.
  • No new component patterns. This only hides the retired control until the replacement per-user UI lands.
  • I am not an LLM agent submitting a bulk PR. Contributor attestation remains for the human author.

Screenshots / clips

Pending browser verification. Expected result: the Default Chat Model card shows only Endpoint and Model controls; the legacy Fallbacks row is absent.

@github-actions github-actions Bot added the ready for review Description complete — ready for maintainer review label Jul 28, 2026
@RaresKeY
RaresKeY marked this pull request as ready for review July 28, 2026 16:56

@alteixeira20 alteixeira20 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Audited against current dev. The strict foreground routing boundary is appropriately scoped, preserves generic fallback transport, keeps legacy default_model_fallbacks inert for foreground Chat/Agent routing, and CI is green. Approved.

@alteixeira20
alteixeira20 merged commit b522964 into odysseus-dev:dev Aug 12, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for review Description complete — ready for maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent-mode tool-call rounds can resolve to a different model than the UI-selected one, causing unexpected API calls and rate-limit errors

2 participants