Skip to content

feat: warn the agent at 80% of a run budget before hard stop - #4179

Draft
salignatmoandal wants to merge 1 commit into
docker:mainfrom
salignatmoandal:feat/soft-budget-warn
Draft

salignatmoandal wants to merge 1 commit into
docker:mainfrom
salignatmoandal:feat/soft-budget-warn

Conversation

@salignatmoandal

@salignatmoandal salignatmoandal commented Sep 5, 2026

Copy link
Copy Markdown

Summary

The budget already existed. The agent never saw it until the run was killed.

  • Warn once at 80% of max_cost / max_tokens / max_time, then keep going.
  • On the next turn the model gets a system message so it can cheapen tools, summarize, or finish instead of slamming into the hard stop.
  • The budget_exceeded JSON contract and kill-switch at 100% are unchanged.

Why this is interesting

enforceBudget was a fuse, not a policy. The runtime already counted cost, tokens, and time (record / snapshot / per-agent spend), but the model only learned the ceiling after iterationStop. From the user's point of view that looks like a crash: Execution stopped after reaching budget.max_cost.

That is the EcoAgent-Bench gap: completing under a budget is not the same as choosing economical actions. A kill-switch with no feedback trains the agent to spend until the wall.

This PR turns the counters into a signal inside the loop — a change in agency policy, not an extra log line.

Before After
Budget = kill-switch Budget = signal + kill-switch
Agent is blind until death Agent can still adapt
Counters unused by the model Counters feed the next prompt

What that buys docker-agent:

  1. Fewer runs cut mid-task (code, MCP, delegation).
  2. Less waste: the agent can stop repeating tools or finish sooner.
  3. A reusable primitive: the runtime can now talk to the model from a metric. Soft budget today; later, model routing or denying expensive tools.

What it does not claim:

  • It does not pick cheaper tools by itself — it gives the LLM the information.
  • It does not add YAML (80% is an internal constant, no schema bump).
  • It does not relax the ceiling. Ignore the warning and the 100% stop still fires.

Behavior

At the existing turn-boundary check in RunStream:

  1. If any ceiling is hit → same as today (budget_exceeded + assistant stop message + iterationStop).
  2. Else if a limit is in [80%, 100%) and that tracker has not warned for that limit yet:
    • emit the existing Warning event (TUI / JSON stream)
    • append a system message the model will see on the next GetMessages call
  3. Each tracker warns once per limit (cost, then tokens, then time — same priority as exceeded()).
  4. Unpriced spend still does not invent a cost warning (the existing unpriced Warning remains).

Test plan

  • go test ./pkg/runtime -count=1 -run 'TestBudget|TestNilBudget|TestEnforceBudget|TestRecordBudget'
  • go test ./pkg/runtime -count=1 -run 'TestBudgetExceededEventJSONContract|TestEnforceBudgetWarnsOnceThenStillHardStops'
  • task test
  • task lint

@salignatmoandal
salignatmoandal requested a review from a team as a code owner September 5, 2026 10:28
@aheritier aheritier added area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/feat PR adds a new feature (maps to feat:). Use on PRs only. status/needs-signed-commits Some commits in the PR are signed with a valid SSH/GPG key labels Sep 5, 2026
@aheritier

Copy link
Copy Markdown
Collaborator

👋 Some commits in this PR are not signed and verified by GitHub. Please sign your commits with a GPG or SSH key registered in your GitHub account, then force-push.

Commits that are not verified: 8858c9a

See GitHub's guide on signing commits for setup instructions. I've added status/needs-signed-commits; it will be removed automatically once every commit in this PR carries a valid GitHub-verified signature.

Give the model one chance to cheapen or finish before the existing
kill-switch fires, without changing the budget_exceeded contract.
@aheritier aheritier removed the status/needs-signed-commits Some commits in the PR are signed with a valid SSH/GPG key label Sep 5, 2026

@aheritier aheritier left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prompt-cache concern: this persists the 80%-budget warning as a system message in session history (pkg/runtime/budget.go:506). That rewrites the cached system/instructions prefix for Anthropic, Bedrock, and ChatGPT/Codex; with num_history_items, trimming can also relocate it ahead of the conversation. The first call after warning may therefore require a full-context cache write at exactly the point this feature is meant to limit cost.

Please route this through the existing transient extraSystemMessages mechanism instead of persisting it, using stable wording to preserve cache reuse. This also avoids stale/accumulating warnings after session resume and loss of the warning after compaction.

Separate blocker: CI lint is currently red (budget_wiring_test.go:245, use assert.Len).

@dgageot @rumpl — requesting your eyes on the prompt-caching implications, as this is a nuanced provider-level behaviour.

@aheritier aheritier added the status/needs-rebase PR has merge conflicts or is out of date with main label Sep 14, 2026
@aheritier
aheritier marked this pull request as draft September 14, 2026 17:09
@aheritier aheritier removed the status/needs-rebase PR has merge conflicts or is out of date with main label Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants