Skip to content

Unify resource bounds across runtimes and add cap-hit observability + regression case #51

Description

@nabinpkl

Why

Per the audit in docs/securing-agents/06-resource-bounds.md, the agent's resource caps are scattered, not unified, and not regression-pinned. A prompt-injected agent today can fail in expensive ways (tool-call loops, token-burn injections), and the eval suite would not catch it because the suite measures correctness, not budget.

Current state

  • agent-service/src/agent_service/core/run.py:88: _USAGE_LIMITS = UsageLimits(request_limit=10, tool_calls_limit=8) for the pydantic-ai loop.
  • Codex runtime: an internal cap inside the codex helper. Not surfaced in our code as a constant, not documented, not verified to match the pydantic-ai cap.
  • agent-service/src/agent_service/thread_state.py:92: MAX_THREAD_TOOL_CALL_TURNS = 20, a state-pruning bound (not a per-turn cap).
  • No per-session token budget, no wall-clock cap, no alerting on cap hits, no eval case proving a cap fires.

What to land

  1. Audit the codex-side cap and document it alongside the pydantic-ai cap. Either factor both runtimes' caps to a shared constant or document the per-runtime values + verify they match by structural test.
  2. Stamp an OTel attribute on cap hit (e.g. mcae.turn.cap_hit=tool_calls, plus the cap value). Both runtimes.
  3. Emit a refusal narrative when a cap fires, distinguishing the wording from the topical-rail rejection (cap-hit means "we tried, the work got too expensive", not "we did not run your input").
  4. Hermetic eval case runaway_tool_call_loop.yaml: payload designed to trick the agent into a tool-call loop (the mock substrate can return a result the agent thinks needs follow-up calls). Asserts mcae.turn.cap_hit=tool_calls and mcae.turn.tool_calls clamped to the cap value. Runs under both runtimes; identical structural outcomes.
  5. (Optional, separate scope) Per-session cost ceiling for long conversations; wall-clock cap per turn.

Done when

  • One documented per-turn cap policy that both runtimes obey.
  • OTel attribute fires on cap hit; eval probe can assert on it.
  • One hermetic case under both runtimes that proves the cap fires.
  • Doc updated to remove the "what is missing" section and reference the new contract.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions