Skip to content

feat: per-provider monthly budget limits - #109

Merged
steipete merged 5 commits into
mainfrom
feat/provider-budgets
Aug 3, 2026
Merged

feat: per-provider monthly budget limits#109
steipete merged 5 commits into
mainfrom
feat/provider-budgets

Conversation

@steipete

@steipete steipete commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Enforces per-provider monthly budgets via a second budget-ledger dimension, including reservation and settlement.
  • Returns 402 provider_budget_exhausted when a provider budget is exhausted.
  • Adds an admin editor with month-to-date spend in the Catalog inspector.
  • Includes admin UX fixes for tablist-only tab reveal, scroll reset on view change, and $0.00 spend formatting.

Verification

  • pnpm check all suites green: 123+26+74 tests.
  • The Playwright darwin snapshot diff on fusion-readiness is pre-existing on this machine and reproduces without these changes; CI Linux snapshots are authoritative.
  • Codex and Claude autoreview clean.

Deployment

Will be deployed to clawrouter.openclaw.ai via the Deploy Cloudflare workflow after merge.

Remaining risk

Provider budgets default to null/unmetered; only metered providers pay one extra Durable Object round-trip.

…ew change

The Access screen's active-tab scrollIntoView also scrolled ancestors during
mount, loading the page yanked down past the header. Reveal the active tab by
adjusting only the tablist's horizontal scroll, reset window scroll on SPA view
changes, and show $0.00 instead of 'none' for zero provider-budget spend.
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P1 Urgent regression or broken agent/channel workflow affecting real users now. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. labels Aug 3, 2026
@clawsweeper

clawsweeper Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed August 3, 2026, 1:07 PM ET / 17:07 UTC.

ClawSweeper review

What this changes

This PR adds monthly per-provider budget settings, reserves and settles provider budgets alongside policy budgets, and displays provider spend controls in the admin catalog.

Merge readiness

Blocked until real behavior proof from a real setup is added - 7 items remain

Keep this PR open, but do not merge it yet. Its new provider-budget Durable Object key is hard-coded to the default tenant and the connection configuration is provider-only, while current ClawRouter policies support distinct tenant IDs; traffic from one tenant can therefore exhaust another tenant’s provider allowance. The PR also needs after-fix real behavior proof beyond reported test results.

Priority: P1
Reviewed head: d3800d72925fd95c57d19f30994e2ac2721c0521
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) The branch has focused accounting changes and relevant unit tests, but the unresolved tenant boundary and missing real behavior proof prevent merge readiness.
Proof confidence 🦪 silver shellfish (2/6) Needs real behavior proof before merge: The PR body reports checks and unit suites but provides no after-fix real Worker or admin behavior proof. Add redacted runtime output or a short recording that demonstrates the decided budget scope, then update the PR body; if no automatic review occurs, ask a maintainer to comment @clawsweeper re-review.
Patch quality 🦐 gold shrimp (3/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Needs proof Needs real behavior proof before merge: The PR body reports checks and unit suites but provides no after-fix real Worker or admin behavior proof. Add redacted runtime output or a short recording that demonstrates the decided budget scope, then update the PR body; if no automatic review occurs, ask a maintainer to comment @clawsweeper re-review.
Evidence reviewed 7 items Repository policy: The reviewed repository policy requires provider-neutral Worker logic, a small hot path, authoritative hard-budget enforcement, focused touched-surface validation, and autoreview.
Cross-tenant provider ledger: Policy-ledger addresses derive tenant from policy.tenantId, but the added provider-ledger address fixes it to default and names the ledger only by provider. All tenants therefore reserve and settle against the same provider:<id> ledger.
Provider budget configuration has no tenant boundary: The added budget field lives on ProviderConnection, which contains provider identity and no tenant identity; the admin mutation persists that provider-wide record.
Findings 1 actionable finding [P1] Scope the provider ledger to the policy tenant
Security None None.

How this fits together

ClawRouter’s Worker data plane authorizes and routes provider requests, reserves budget before sending traffic upstream, then settles actual cost through Durable Object ledgers. The admin console configures provider connections and displays their budget state.

flowchart LR
  A[Admin catalog] --> B[Provider connection settings]
  B --> C[Worker request routing]
  C --> D[Policy budget reservation]
  D --> E[Provider budget reservation]
  E --> F[Upstream provider request]
  F --> G[Ledger settlement]
  G --> H[Usage and admin spend status]
Loading

Decision needed

Question Recommendation
Should a configured provider monthly budget be isolated per policy tenant, or intentionally shared gateway-wide across every tenant using that provider? Tenant-scoped provider budgets: Store, read, and enforce each provider limit by tenant and provider so existing tenant isolation continues through the new budget dimension.

Why: The branch calls the feature tenant-wide while its configuration and ledger implementation are provider-global; only maintainers can set that permanent billing and admission-control contract.

Before merge

  • Add real behavior proof - Needs real behavior proof before merge: The PR body reports checks and unit suites but provides no after-fix real Worker or admin behavior proof. Add redacted runtime output or a short recording that demonstrates the decided budget scope, then update the PR body; if no automatic review occurs, ask a maintainer to comment @clawsweeper re-review.
  • Scope the provider ledger to the policy tenant (P1) - providerBudgetLedgerAddress always uses default and provider:<id>, while current policies carry different tenant IDs. Two tenants using the same provider therefore reserve and settle one shared balance, so one can receive provider_budget_exhausted because another exhausted the allowance. Preserve tenant identity in the chosen provider-budget contract and add an isolation regression.
  • Resolve merge risk (P1) - Existing multi-tenant deployments could have one tenant’s provider spend deny another tenant’s traffic with provider_budget_exhausted, while the catalog shows one shared balance.
  • Resolve merge risk (P1) - The new provider-budget contract is configuration and routing admission surface; choosing a gateway-wide budget instead of a tenant-wide one would be an intentional compatibility and operations decision that must be documented and approved.
  • Resolve merge risk (P1) - No after-fix real Worker/admin behavior proof accompanies the branch, and the supplied check digest reports worker-package failures.
  • Complete next step (P2) - A maintainer must decide the permanent tenant-versus-gateway provider-budget contract before a repair can safely change configuration and persistent connection semantics.

Findings

  • [P1] Scope the provider ledger to the policy tenant — worker/budget-scope.ts:25-31
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Change surface 20 files affected; 322 added, 49 removed The feature crosses Worker enforcement, Durable Object settlement, admin configuration, documentation, and tests, making its tenant contract merge-critical.
Regression coverage 1 new provider-budget test file; 3 scenarios The added cases cover denial, settlement, and unmetered fast path but not multi-tenant isolation.

Merge-risk options

Maintainer options:

  1. Resolve tenant scope before merge (recommended)
    Decide the intended provider-budget boundary, then make ledger keys, configuration, admin status, and tests conform to that one contract.
  2. Accept a shared gateway pool
    Merge only after maintainers explicitly approve gateway-wide depletion semantics and the UI/docs clearly explain that all tenants share each provider limit.
  3. Pause the budget feature
    Pause this PR if tenant-scoped provider configuration requires broader design work that should not be decided during repair.

Technical review

Best possible solution:

Choose and document the supported ownership boundary for provider budgets; if the intended behavior is tenant-wide, persist and address budgets by tenant plus provider, then prove two tenants cannot affect each other’s balance before merge.

Do we have a high-confidence way to reproduce the issue?

Yes, from source: configure two distinct tenant policies that route to the same metered provider, exhaust the first tenant’s provider spend, then send the second tenant’s request. Both use the provider:<id> ledger under default, so the second request can be denied by the first tenant’s spend.

Is this the best way to solve the issue?

No. Dual reservation and settlement are suitable, but the provider-budget configuration and Durable Object address omit tenant identity despite the stated tenant-wide behavior. The maintainer must first confirm whether the feature is tenant-scoped or gateway-wide.

Full review comments:

  • [P1] Scope the provider ledger to the policy tenant — worker/budget-scope.ts:25-31
    providerBudgetLedgerAddress always uses default and provider:<id>, while current policies carry different tenant IDs. Two tenants using the same provider therefore reserve and settle one shared balance, so one can receive provider_budget_exhausted because another exhausted the allowance. Preserve tenant identity in the chosen provider-budget contract and add an isolation regression.
    Confidence: 0.99

Overall correctness: patch is incorrect
Overall confidence: 0.99

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f82765ad3821.

Labels

Label changes:

  • add P1: A new budget limit can incorrectly block real tenant traffic because another tenant used the same provider.
  • add merge-risk: 🚨 compatibility: The new global provider ledger conflicts with existing tenant-scoped policy-budget behavior and may change who can consume a configured allowance.
  • add merge-risk: 🚨 auth-provider: Provider routing admission now depends on a ledger whose configured boundary does not preserve tenant identity.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body reports checks and unit suites but provides no after-fix real Worker or admin behavior proof. Add redacted runtime output or a short recording that demonstrates the decided budget scope, then update the PR body; if no automatic review occurs, ask a maintainer to comment @clawsweeper re-review.

Label justifications:

  • P1: A new budget limit can incorrectly block real tenant traffic because another tenant used the same provider.
  • merge-risk: 🚨 compatibility: The new global provider ledger conflicts with existing tenant-scoped policy-budget behavior and may change who can consume a configured allowance.
  • merge-risk: 🚨 auth-provider: Provider routing admission now depends on a ledger whose configured boundary does not preserve tenant identity.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body reports checks and unit suites but provides no after-fix real Worker or admin behavior proof. Add redacted runtime output or a short recording that demonstrates the decided budget scope, then update the PR body; if no automatic review occurs, ask a maintainer to comment @clawsweeper re-review.

Evidence

What I checked:

  • Repository policy: The reviewed repository policy requires provider-neutral Worker logic, a small hot path, authoritative hard-budget enforcement, focused touched-surface validation, and autoreview. (AGENTS.md:1, f82765ad3821)
  • Cross-tenant provider ledger: Policy-ledger addresses derive tenant from policy.tenantId, but the added provider-ledger address fixes it to default and names the ledger only by provider. All tenants therefore reserve and settle against the same provider:<id> ledger. (worker/budget-scope.ts:25, d3800d72925f)
  • Provider budget configuration has no tenant boundary: The added budget field lives on ProviderConnection, which contains provider identity and no tenant identity; the admin mutation persists that provider-wide record. (worker/types.ts:120, d3800d72925f)
  • Multi-tenant behavior exists on main: The existing demo configuration defines policies for the distinct openclaw, research, and sandbox tenants, so tenant isolation is an active product model. (admin/src/demo-data.ts:112, f82765ad3821)
  • Coverage omits isolation: The new provider-budget tests exercise one provider ledger and do not verify that two tenant policies using the same provider have independent exhaustion and spend status. (worker/test/provider-budget.test.mjs:16, d3800d72925f)
  • Feature-history provenance: Current main’s budget-scope and accounting files trace to the v0.1.0 release commit by Peter Steinberger; the PR author also owns the existing budget-addressing implementation history. (worker/budget-scope.ts:12, 13845ee2bdc0)

Likely related people:

  • steipete: Peter Steinberger authored the current main budget-addressing implementation in the v0.1.0 release commit and authored this PR’s accounting extension. (role: current budget-scope author and feature contributor; confidence: high; commits: 13845ee2bdc0, c3740c5d1c2c; files: worker/budget-scope.ts, worker/accounting.ts, worker/admin.ts)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Get a maintainer decision on tenant-scoped versus gateway-wide provider budgets.
  • Implement the chosen boundary in configuration, ledger keys, status, and cross-tenant regression coverage.
  • Post redacted real Worker/admin proof showing exhaustion, balance display, and the selected isolation behavior.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete
steipete merged commit 98e78e8 into main Aug 3, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P1 Urgent regression or broken agent/channel workflow affecting real users now. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant