Skip to content

fix(model-routing): keep user-selected models strict#5632

Closed
RaresKeY wants to merge 1932 commits into
odysseus-dev:devfrom
RaresKeY:fix/strict-selected-model
Closed

fix(model-routing): keep user-selected models strict#5632
RaresKeY wants to merge 1932 commits into
odysseus-dev:devfrom
RaresKeY:fix/strict-selected-model

Conversation

@RaresKeY

Copy link
Copy Markdown
Member

Summary

Keep foreground Chat and every Agent tool-call round on the endpoint and model selected for the session. A shared foreground policy boundary now supplies the Chat candidate list and the Agent fallback slice, so both paths are strict by default instead of silently consulting the historical default-model list.

Existing nonempty default_model_fallbacks values remain stored as legacy data for rollback and manual reference. They are not migrated, cleared during endpoint cleanup, inherited by Utility routing, used to choose a new default chat, or returned as candidates to background callers. The retired Settings editor is hidden, and ordinary default-model saves no longer rewrite those preserved values.

The generic fallback transport remains available for workload-specific policies. This PR only changes which candidates are authorized; the explicit per-user replacement policy is tracked separately in #5626 and #5627.

Target branch

  • This PR targets dev, not main.

Linked Issue

Fixes #4561

Part of #5625

Type of Change

  • Bug fix (non-breaking — fixes a confirmed issue)
  • New feature (non-breaking — adds new behaviour)
  • Breaking change (changes or removes existing behaviour)
  • Refactor / cleanup (behaviour unchanged)
  • Documentation only
  • CI / tooling / configuration

Checklist

  • I searched open issues and open PRs — closed PR fix(agent): do not switch models mid agent loop (#4561) #5376 is related but unmerged, Agent-only, and lacks the Chat parity and legacy-disconnection coverage in this branch.
  • This PR targets dev.
  • My changes are limited to strict selected-model routing, preservation/disconnection of the legacy default list, and focused regression coverage.
  • I ran the app end-to-end. No disposable two-provider environment was available; route-level, Agent-loop, persistence, and adjacent automated checks were run in the secretless runner instead.

How to Test

  1. Run the focused and adjacent suite:

    python -m pytest -q \
      tests/test_foreground_model_routing.py \
      tests/test_legacy_default_fallback_ui.py \
      tests/test_agent_loop.py \
      tests/test_agent_rounds_exhausted.py \
      tests/test_chat_helpers.py \
      tests/test_chat_metrics.py \
      tests/test_chat_preprocess_tool_policy.py \
      tests/test_chat_route_tool_policy.py \
      tests/test_chat_stream_scope.py \
      tests/test_aux_llm_owner_scope.py \
      tests/test_endpoint_resolver_headers.py \
      tests/test_resolve_endpoint_fallbacks.py \
      tests/test_model_defaults.py \
      tests/test_model_routes.py \
      tests/test_session_endpoint_owner_scope.py \
      tests/test_llm_core_fallback.py \
      tests/test_llm_core_streaming.py \
      tests/test_task_endpoint_normalization.py

    Expected result: 330 tests pass.

  2. Run syntax checks:

    python -m py_compile \
      src/foreground_model_routing.py \
      src/endpoint_resolver.py \
      routes/chat_routes.py \
      routes/model_routes.py \
      routes/email_routes.py \
      src/task_endpoint.py
    node --check static/js/settings.js
  3. Configure two model endpoints and retain a nonempty legacy default fallback list. Select the primary endpoint/model in a Chat session, make it fail before output, and verify the stream surfaces that failure without calling the legacy endpoint.

  4. Repeat in Agent mode with a tool call followed by another model round. Verify every recorded request uses the selected endpoint/model and no fallback event is emitted.

  5. Change the default endpoint/model in Settings and verify the preserved legacy list is not rewritten. Delete an endpoint referenced only by that legacy list and verify the stored data remains intact but has no routing effect.

The focused/adjacent suite passes 330 tests. Python compilation, JavaScript syntax, and diff checks pass. A full isolated run reached 4,671 passing tests and 5 skips; its 11 unrelated failures are contained in a 12-failure baseline set reproduced on clean current origin/dev.

Visual / UI changes — REQUIRED if you touched anything that renders

The obsolete Default Chat Model fallback editor is hidden so Settings no longer promises behavior that is intentionally disconnected. No new style, component, spacing, or mobile behavior is introduced.

  • Screenshot or short clip of the change in the running app. The secretless review runner has no browser engine, so this remains an explicit draft-PR validation item.
  • Style match: the existing Settings structure is retained; no new styling is introduced.
  • No new component patterns. This only hides the retired control until the replacement per-user UI lands.
  • I am not an LLM agent submitting a bulk PR. Contributor attestation remains for the human author.

Screenshots / clips

Pending browser verification. Expected result: the Default Chat Model card shows only Endpoint and Model controls; the legacy Fallbacks row is absent.

pewdiepie-archdaemon and others added 30 commits June 27, 2026 21:18
Keep an unhealthy MemoryVectorStore instance available for health reporting instead of discarding it as disabled. This lets health checks report a degraded/down vector-store state while preserving focused regression coverage for initializer behavior.
ap0ught and others added 13 commits July 17, 2026 16:22
fix(docker): bump Docker CLI to a patched release
fix(docker): bump Docker CLI to a patched release
* feat(models): define capability schema and readers

* fix(models): harden Google catalog probing

Restrict native catalog probing to the Gemini host, keep provider keys out of request URLs, filter non-chat model resources, and preserve the manual refresh default in the built-in Google add flow.
…(#5474)

* security(url-safety): reject RFC 6598 shared address space in strict mode

Strict mode (block_private=True) is a full SSRF lockdown, but it only
rejected is_private and is_loopback targets. CPython does not classify RFC
6598 shared/CGNAT space (100.64.0.0/10) as is_private (it is "shared", not
"private"), so a public redirect into 100.64.0.1 passed the per-hop guard
and still issued the request to a potentially internal CGNAT service.

not is_global would also exclude it, but only on CPython 3.11.10+/3.12.4+/
3.13+; the CI matrix runs 3.11/3.12, so reject the range explicitly to stay
correct across patch levels and the 3.14 runtime image. Default local-first
mode is unchanged. Adds strict-mode coverage for shared, non-global, and
public targets.

* docs(url-safety): correct CGNAT is_global rationale in strict-mode comment

The prior comment claimed `not is_global` catches 100.64.0.0/10 only on
CPython 3.11.10+/3.12.4+/3.13+. That is inaccurate for CGNAT: is_global
is False for 100.64.0.1 on every supported version (verified 3.10-3.14).
The version-fragility applies to other ranges gh-113171 touched, not CGNAT.
The explicit range reject is still the right choice; restate the reason as
is_private not covering shared space, and not coupling strict mode to
is_global's broader, cross-version definition. No behavior change.
…… (#5491)

* fix(llm): enhance fallback logic to handle empty completions and improve metadata handling

* fix(llm): stream tool call deltas immediately
Slice 2f of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves note_routes.py into
routes/note/, leaving a backward-compat sys.modules shim at the old path.
Pure file reorganization, no behavior change.

The shim uses sys.modules replacement (same pattern as the merged gallery
#4903, research #4975, memory #5007, history #5090, and contacts #5227
slices) so that `import routes.note_routes`, `from routes.note_routes import
X`, `importlib.import_module(...)`, and the `import ... as note_routes` +
`monkeypatch.setattr(note_routes, "SessionLocal", ...)` pattern used by
test_note_reminder_fire_scope.py / test_notes_fail_closed_auth.py all
operate on the same module object the application uses.

The canonical module does NOT depend on the shim — routes/note/note_routes.py
imports only from core/, src/, and stdlib. The outbound email cross-domain
imports (routes.email_routes._get_email_config, routes.email_helpers.
_send_smtp_message) are function-local lazy imports that keep resolving
through the email module's own path (email is not yet migrated).

One source-introspection test site repointed to the new canonical path:
- test_model_helper_owner_scope.py (shared with history; history entry
  already repointed in #5090, note entry repointed here)

Adds tests/test_note_routes_shim.py to pin the sys.modules shim contract
(legacy and canonical paths resolve to the same module object; monkeypatch
via legacy alias reaches the canonical module).

Verified: compileall clean; full suite 4487 passed, 3 skipped.
@github-actions github-actions Bot added the ready for review Description complete — ready for maintainer review label Jul 20, 2026
@RaresKeY
RaresKeY marked this pull request as ready for review July 21, 2026 10:35
ydonghao and others added 5 commits July 21, 2026 12:38
… (#5658)

Slice 2g of the route-domain reorganization (#4082/#4071). Moves
cleanup_routes.py into routes/cleanup/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.

The shim uses sys.modules replacement so string-targeted
monkeypatch.setattr("routes.cleanup_routes.*", ...) in
test_cleanup_owner_scope.py reaches the canonical module.

Canonical module imports only from src/ and stdlib (zero internal
routes/ coupling). Zero source-introspection landmines.

Adds tests/test_cleanup_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
…ackage (#5659)

Slice 2h of the route-domain reorganization (#4082/#4071). Moves
admin_wipe_routes.py into routes/admin_wipe/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.

The shim uses sys.modules replacement so the `import ... as
admin_wipe_routes` + `monkeypatch.setattr(admin_wipe_routes, "SessionLocal",
...)` / `"require_admin"` pattern in test_admin_wipe_gallery.py reaches
the canonical module.

Canonical module imports only from core/, src/, and stdlib (zero internal
routes/ coupling). Zero source-introspection landmines.

Adds tests/test_admin_wipe_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
… (#5660)

Slice 2i of the route-domain reorganization (#4082/#4071). Moves
compare_routes.py into routes/compare/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.

The shim uses sys.modules replacement so the `import ... as cr` +
`monkeypatch.setattr(cr, "SessionLocal", ...)` / `"_owned_endpoint_by_url"`
/ `"_owned_endpoint_by_id"` pattern in test_endpoint_owner_scope_followup.py
reaches the canonical module.

Canonical module imports only from core/, src/, and routes.session_routes
(zero dependency on the legacy shim). One source-introspection test site
repointed: test_endpoint_owner_scope_followup.py (shared with other domains;
only the compare entry repointed here).

Adds tests/test_compare_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
fix(docker): detect snap+WSL2 GPU passthrough incompatibility
* ci: add CodeQL advanced setup to scan pull requests before merge

* ci(codeql): preserve scheduled scans

Add a weekly advanced-setup scan and update the security CI guide so default setup remains disabled.

---------

Co-authored-by: RaresKeY <158580472+RaresKeY@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for review Description complete — ready for maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent-mode tool-call rounds can resolve to a different model than the UI-selected one, causing unexpected API calls and rate-limit errors