[Tracker] TinyAgents harness improvements — 2026-07 audit

[M3gA-Mind]

Audit of the vendored TinyAgents harness (vendor/tinyagents, pinned b1f0d82) for concrete improvements to agent reliability, cost, and long-horizon behaviour. Verified against the code at that commit (the crate's own docs/audit.md correctness/security findings are all already resolved; the crate's docs/sdk-gaps.md backlog is partly stale — several items are now implemented). Recurring theme: the harness aborts the whole run on recoverable conditions, and re-does expensive work it should cache/persist.

Improvements/PRs land in the upstream tinyhumansai/tinyagents repo (via the M3gA-Mind fork); this tracker lives here because that's where the TinyAgents migration work is coordinated.

Tier 1 — quick wins (small, surgical) — PRs in progress

• [T1] Unary model calls ignore mid-flight cancellation → billed to completion (streaming path already races the cancel token, unary doesn't). agent_loop/model_call.rs.
• [T1] Token estimator counts only text blocks → images / tool-result JSON / reasoning ≈ 0 tokens, so compaction & trim silently never trigger on multimodal/tool-heavy runs. message/mod.rs, summarization/mod.rs.
• [T1] Default OpenAI(-compatible) profiles omit context windows → context-window-aware compaction no-ops (a hint table exists but is unused). providers/openai/transport.rs.
• [T1] Streamed provider failures collapse to an unstructured Model error → permanent errors (401/quota/bad-request) get retried & fallback-churned. model/mod.rs, retry/mod.rs.
• [T1] Empty provider response → empty assistant row that terminates the run (SDK root of adapter Turn finalization: empty assistant row on EmptyProviderResponse, stale error slot, duplicate TurnCompleted #4457). agent_loop/run_loop.rs.

Tier 2 — stop aborting recoverable turns; stop re-summarizing

• [T2] Schema-validation failure hard-aborts the turn (the still-open core of Harness: fatal tool-arg schema validation aborts the whole turn (legacy loop self-corrected) #4451) — no InvalidArgsPolicy to mirror the recoverable UnknownToolPolicy; forces the adapter's SchemaGuardMiddleware stub-hack.
• [T2] Compaction never persists to the durable transcript → an LLM summarizer re-runs on the entire growing head every turn once over threshold. middleware/library/context.rs, agent_loop/run_loop.rs.
• [T2] Summarizer failure aborts the run (no fallback-trim/degrade policy) — precisely on the longest threads. middleware/library/context.rs.
• [T2] Any wrap_tool middleware disables the concurrent tool path → with ≥4 wrap middlewares in prod, multi-tool turns run fully serial. Needs a before_tool short-circuit outcome.

Tier 3 — strategic (durability, orchestration, policy migration)

• [T3] Execution runtime binding the durable TaskStore to live runs + boot resume — the SDK persists task metadata but never runs/resumes a task, so running_subagents.rs can't be retired.
• [T3] Durable status store + events→status listener (in-memory only today) so a UI can attach late and reconstruct a run.
• [T3] Async, context-aware tool policy + async approval — SDK policy is name-only/sync and approval is a sync bool that raises a fatal interrupt; classification runs before unknown-tool recovery (mutually exclusive).

Tier 4 — observability & completeness

Field-typed redaction (not just literal secrets) · silent front-trim emits no event · fallback chain skips capability gating · sub-agent stream deltas don't reach the parent · push→pull stream helper (absorb the adapter's #4460 shim) · concurrent-fold drops succeeded siblings on failure · parallel-agent builder over map_reduce · couple compaction threshold to reserved output tokens · no-progress ladder has no SDK loop driver / halt control outcome.

Already implemented upstream (don't re-open)

Recoverable UnknownToolPolicy (sentinel deleted) · fail-closed allowlist composition · BudgetMiddleware (preflight/reserve/reconcile) · top-level reasoning+tool-arg streaming (ThinkingForwarder retired) · durable JsonlTaskStore + event journals · map_reduce combinator · WorkspaceIsolation primitive · correct middleware onion ordering.

Method: 4 parallel subsystem analyses (tool-path, model/streaming/budget, orchestration/durability, long-horizon), each verifying current state against the code rather than the docs.

[M3gA-Mind]

Tier-1 PRs raised on tinyhumansai/tinyagents (via the M3gA-Mind fork), each with tests + fmt/clippy/test green:

• [TinyAgents][T1] Token estimator ignores non-text blocks — compaction/trim silently never trigger #4633 → harness: size token estimation over all content blocks, not just text tinyagents#35 (token estimator over all content blocks)
• [TinyAgents][T1] Default OpenAI(-compatible) profiles omit context windows — window-aware compaction no-ops #4634 → harness(openai): populate profile context window from the hint table tinyagents#36 (OpenAI profile context window from hint table)
• [TinyAgents][T1] Unary model calls ignore mid-flight cancellation — billed to completion #4635 → harness: race in-flight unary model calls against cancellation tinyagents#37 (cancel in-flight unary model calls)

Remaining Tier-1 items (streamed-error retry classification, empty-response guard) + Tiers 2–4 still to be scheduled.

[M3gA-Mind]

Dispatched the next 5 fixes across the worker fleet (each → a PR on tinyhumansai/tinyagents via the M3gA-Mind fork, closing its issue):

• W1 → [TinyAgents][T1] Streamed provider failures lose retry classification (retried as transient) #4637 (streamed-error retry classification)
• W2 → [TinyAgents][T1] Empty provider response terminates the run with a blank answer #4638 (empty-response guard)
• W3 → [TinyAgents][T2] Schema-validation failure hard-aborts the turn — add recoverable InvalidArgsPolicy (#4451 core) #4639 (recoverable InvalidArgsPolicy — Harness: fatal tool-arg schema validation aborts the whole turn (legacy loop self-corrected) #4451 core)
• W4 → [TinyAgents][T2] Summarizer failure aborts the run — add CompressionFailurePolicy fallback #4640 (summarizer-failure CompressionFailurePolicy)
• W5 → [TinyAgents][T4] Runtime fallback chain skips capability/lifecycle gating #4641 (fallback capability gating)

Each worker builds + tests + clippy-checks the crate before opening its PR. Tier 3 architectural items (execution runtime, status store, async policy) remain unscheduled — too large for a single worker PR.

[M3gA-Mind]

Update: W5 reassigned from #4641 to standing CI-guardian duty — watches all open M3gA-Mind PRs on tinyhumansai/tinyagents and fixes any red CI (fmt/clippy/test/build), never merges. #4641 deferred. Active feature workers now: W1 #4637, W2 #4638, W3 #4639, W4 #4640.