Add task-aware quality routing

[0xzr]

Summary

• add a conservative local grounded-reading classifier with explicit task-hint precedence
• rank exact model identities using versioned, repeated synthetic-fixture evidence without overriding quota, health, latency, or failover priorities
• propagate task hints through sync/async APIs, CLI, MCP, proxy protocols, panels, roles, and cache keys
• package provenance-bearing task evidence and document the public behavior

Validation

• 985 tests passed
• coverage: 86.55% lines, 74.62% branches
• Ruff and focused strict mypy passed
• catalog, count, release-readiness, build, twine, and fresh-wheel smoke checks passed
• Bandit, pip-audit, zizmor, and security-policy checks passed
• proxy stress: 720 requests at concurrency 64, zero failures
• Codex Sol 5.6 xhigh approved staged tree bd642baa57b6245bd263e9a6859e2d777e4c6cae

Fixes #57

Summary by Sourcery

Add task-aware quality routing with grounded-reading classification and task evidence, and propagate explicit task hints through routing APIs, CLI, MCP tools, proxy, roles, cache, and panels.

New Features:

• Introduce local grounded-reading vs general task classification with an explicit task hint API and automatic fallback.
• Add task-specific, provenance-bearing quality evidence for exact model identities that influences quality routing without overriding quota, health, or latency.
• Expose a grounded-reader role and task hints across CLI, MCP tools, proxy headers/body extensions, async APIs, and panel selection.

Enhancements:

• Extend quality routing to blend task fit with capability and latency while preserving ordering when no task evidence is available.
• Include task hints in cache keys and routing explanations to keep intent-aligned caching and debugging.
• Update documentation to describe task-aware quality routing, grounded-reading usage, and new MCP tool parameters.

Build:

• Bundle a versioned grounded-reading task evidence JSON and fixture into the wheel for offline use.

Tests:

• Add extensive tests for task classification, evidence validation, routing behavior under quota/failure, CLI/MCP/async parity, cache-keying, roles, and proxy validation of task hints.

[sourcery-ai]

Sorry @0xzr, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[sourcery-ai]

Reviewer's Guide

Introduce task-aware quality routing with grounded Markdown reading detection, validated per-model task evidence, and explicit task hints propagated through sync/async APIs, CLI, MCP tools, proxy, roles, panel, and cache keying, while ensuring quota/health constraints and existing ordering behavior remain intact when no evidence applies.

Sequence diagram for task-aware quality routing and cache keying

sequenceDiagram
    actor Client
    participant CLI
    participant Proxy
    participant Pool
    participant task_quality
    participant Cache

    Client->>CLI: freellmpool ask/chat --routing quality --task hint
    CLI->>Pool: ask/chat(messages, routing, task)
    Client->>Proxy: OpenAI/Anthropic request + X-Freellmpool-Task
    Proxy->>task_quality: task_resolution(messages, task)
    Proxy->>Pool: chat/stream_chat(messages, routing, task)

    Pool->>task_quality: resolve_task(messages, task)
    task_quality-->>Pool: resolved_task

    Pool->>Cache: make_key(messages, model, providers, routing, task)
    Cache-->>Pool: cache_key (hit or miss)

    Pool->>Pool: rank_targets(messages, routing, task)
    Pool->>task_quality: task_evidence_table(resolved_task)
    task_quality-->>Pool: evidence_table
    loop each candidate model
        Pool->>task_quality: model_task_score(model, evidence_table)
        task_quality-->>Pool: task_score or None
    end
    Pool->>Pool: _order(targets, difficulty, routing, task)
    Pool-->>CLI: Reply (provider_id, model)
    Pool-->>Proxy: Reply/stream (provider_id, model)

Loading

File-Level Changes

Change	Details	Files
Add local task classifier and validated task evidence for grounded Markdown reading and wire it into quality routing scoring.	Introduce task_quality module with grounded-reading classifier, task resolution/validation, evidence loading, and model task scoring utilities. Update Pool.rank_targets and _order to resolve task intent, fetch per-task evidence tables, and blend a bounded task-fit penalty into quality-mode ordering. Add bundled task_evidence.json to the package and ensure user overrides are merged and validated via a cached loader.	src/freellmpool/task_quality.py src/freellmpool/router.py src/freellmpool/task_evidence.json pyproject.toml tests/test_task_quality.py tests/test_routing.py
Propagate explicit task hints through public sync/async APIs, CLI, MCP tools, panels, proxy, and cache keys, preserving precedence rules.	Extend chat/ask/stream_chat/rank_targets APIs and AsyncPool.achat to accept a task hint, resolving or validating it based on routing mode and caching the resolved task for reuse. Thread task hints through CLI ask command (including new --task flag and role defaults), panel selection and execution, and the SQLite cache key so differing task intents do not share entries. Update MCP tool schemas and handlers and proxy routing to accept/validate task hints from arguments, JSON body, or X-Freellmpool-Task header, resolving conflicts and surfacing invalid tasks as 400 errors.	src/freellmpool/router.py src/freellmpool/aio.py src/freellmpool/cli.py src/freellmpool/panel.py src/freellmpool/mcp_server.py src/freellmpool/proxy.py src/freellmpool/cache.py tests/test_cli.py tests/test_mcp.py tests/test_proxy.py tests/test_cache.py
Expose a grounded-reading role and documentation for task-aware quality routing and grounded evidence behavior.	Add grounded-reader RoleSpec with quality routing, grounded-reading task hint, and faithful-reader system prompt, and include task in role formatting output. Document task-aware quality routing, grounded-reading usage, and client integration in README, AGENTS, MCP docs, and changelog entries. Ensure roles tests cover grounded-reader presence and task declaration.	src/freellmpool/roles.py README.md docs/AGENTS.md docs/MCP.md CHANGELOG.md tests/test_roles.py

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#57	Make the quality routing mode task-aware (in addition to difficulty-aware), at least for grounded/faithful document reading, by introducing task classification and using task-specific evidence in model ranking.	✅
#57	Expose a way for callers (CLI, MCP tools, async/sync APIs, proxy) to provide explicit task hints that override or disable automatic task classification, and propagate these through routing and caching.	✅
#57	Document the new task-aware quality routing behavior, including the grounded-reading task, task hints, and how evidence is used and stored.	✅

Possibly linked issues

• Question: Should quality routing be task-aware in addition to difficulty-aware? #57: The PR implements task-aware quality routing for grounded Markdown reading, classifiers, and task hints, answering the issue.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.