Skip to content

fix(context): use Ollama loaded runtime allocation#5631

Closed
RaresKeY wants to merge 1933 commits into
odysseus-dev:devfrom
RaresKeY:fix/ollama-runtime-context-allocation
Closed

fix(context): use Ollama loaded runtime allocation#5631
RaresKeY wants to merge 1933 commits into
odysseus-dev:devfrom
RaresKeY:fix/ollama-runtime-context-allocation

Conversation

@RaresKeY

Copy link
Copy Markdown
Member

Summary

Use Ollama's /api/ps loaded-model allocation for context budgeting instead
of promoting a static model maximum to effective runtime truth. The runtime
response is normalized through the provider adapter layer, and
src/model_context.py remains the consumer that performs the local probe,
handles caching, and returns the effective value.

Manually added local Ollama endpoints stored as endpoint_kind="api" now probe
before the existing API/proxy short-circuit. A valid Ollama response with no
matching loaded model remains runtime-unknown, so the static model table is not
sent back as options.num_ctx. Remote endpoints and configured proxies do not
receive the probe, and non-Ollama local endpoints keep their existing request
sequence.

Stack

This draft depends on #5576 and the focused runtime-context evidence adapter in
#5628 / #5630. After those layers land, rebase this branch so the PR contains
only src/model_context.py and tests/test_model_context.py.

Related implementation

PR #5355 targets the same bug but reads /api/show model metadata as the
serving window. This implementation instead uses /api/ps, which reports the
allocation for currently running models, and keeps model maximum, configured
num_ctx, and loaded allocation distinct.

Target branch

  • This PR targets dev, not main.

Linked Issue

Fixes #5193

Part of #2737

Type of Change

  • Bug fix (non-breaking — fixes a confirmed issue)
  • New feature (non-breaking — adds new behaviour)
  • Breaking change (changes or removes existing behaviour)
  • Refactor / cleanup (behaviour unchanged)
  • Documentation only
  • CI / tooling / configuration

Checklist

  • I searched open issues and open PRs — fix(context): probe Ollama /api/show serving window (#5193) #5355 is related but uses a different and incomplete evidence source.
  • This PR targets dev.
  • My changes are limited to the scope described above — no unrelated refactors or whitespace changes are mixed in.
  • I ran the app end-to-end. Live Ollama validation remains to be performed; focused, adjacent, and full automated checks were run in isolation.

How to Test

  1. Run:

    python -m pytest -q \
      tests/test_model_context.py \
      tests/test_model_capability_readers.py \
      tests/test_llama_server_models_url.py \
      tests/test_budget_auto_sentinel.py \
      tests/test_llm_core_ollama.py

    Expected result: 94 tests pass.

  2. Configure a local Ollama endpoint as endpoint_kind="api", return a
    matching /api/ps row with context_length: 65536, and verify
    get_context_length() returns 65536 without querying /v1/models.

  3. Return a valid /api/ps response before the model is loaded and verify the
    result remains unknown; load the model at 32768 and verify the next lookup
    returns 32768 rather than a cached static-table value.

  4. Verify remote endpoints, configured proxies, and an unconfigured
    localhost:8080 OpenAI-compatible endpoint do not receive /api/ps probes.

Broader context, compaction, capability, Ollama, and endpoint tests pass 285
tests. Python compilation passes for all changed modules. The full suite
reaches 4,711 passing tests and 5 skips; its 12 unrelated failures reproduce
unchanged on the #5576 base.

Optional live check:

  1. Start Ollama with a model loaded below its theoretical maximum.
  2. Compare GET /api/ps with Odysseus's logged context length.
  3. Confirm agent prompt budgeting follows the loaded allocation after the
    model is running.

Visual / UI changes

N/A — backend context discovery and tests only; no UI rendering changed.

pewdiepie-archdaemon and others added 30 commits June 27, 2026 21:20
Keep an unhealthy MemoryVectorStore instance available for health reporting instead of discarding it as disabled. This lets health checks report a degraded/down vector-store state while preserving focused regression coverage for initializer behavior.
EmirCobanOfficial and others added 25 commits July 14, 2026 08:33
…mporter (#5261)

* Harden skill importer against SSRF: block private targets + revalidate redirects per hop

The skill importer validated only the initial URL with the lenient SSRF guard
(block_private=False) and then fetched with follow_redirects=True, so a 3xx to
an internal/metadata address (169.254.169.254, 127.0.0.1, RFC-1918) was still
connected to — inconsistent with the hardened services/search/content.py
:_get_public_url path.

Add a _get_checked() helper that follows redirects manually and re-runs the
SSRF guard with block_private=True on every hop, and route all three fetch
sites (skills.sh unwrap, _fetch_bytes, _list_github_dir) through it. GitHub's
own redirects and the final-host _assert_github_url checks are preserved.

Adds hermetic regression tests (IP-literal hosts, faked HTTP layer) and updates
the existing mock signature for the new block_private kwarg.

Defense-in-depth: the endpoint is admin-gated (require_admin) and admins are
trusted per THREAT_MODEL.md, so this is not a cross-boundary vulnerability.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: enforce follow_redirects=False invariant in mock client

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Add Arch-specific package installation and NVIDIA runtime configuration
to the Docker setup guide. Cover passthrough verification, the NVIDIA
Compose overlay, and the distinction between GPU passthrough and
CUDA-backed model serving

Refs #831
…7-odysseus

chore: sync upstream changes from odysseus-dev/odysseus
fix(docker): bump Docker CLI to a patched release
fix(docker): bump Docker CLI to a patched release
* feat(models): define capability schema and readers

* fix(models): harden Google catalog probing

Restrict native catalog probing to the Gemini host, keep provider keys out of request URLs, filter non-chat model resources, and preserve the manual refresh default in the built-in Google add flow.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for review Description complete — ready for maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Static context-window table overrides the endpoint's actual serving context — silent truncation in long agent sessions