Skip to content

fix(kiro): distinguish confirmed credit exhaustion from ambiguous 402s#2664

Open
SemonCat wants to merge 3 commits into
decolua:masterfrom
SemonCat:fix/kiro-credit-exhaustion-cooldown
Open

fix(kiro): distinguish confirmed credit exhaustion from ambiguous 402s#2664
SemonCat wants to merge 3 commits into
decolua:masterfrom
SemonCat:fix/kiro-credit-exhaustion-cooldown

Conversation

@SemonCat

@SemonCat SemonCat commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Today every Kiro 402 gets the same flat 2-minute cooldown (open-sse/config/errorConfig.js ERROR_RULES, status: 402). That's a reasonable default for an ambiguous 402 (bad payment method, suspended account), but wrong for a confirmed monthly credit-limit exhaustion: the account keeps getting re-tried and re-402ing every 2 minutes until the real reset, burning requests and cycling needlessly through account/combo fallback — sometimes for weeks.

This PR adds a narrow, backward-compatible path: only a recognized exhaustion signal gets special handling; everything else keeps today's behavior exactly as-is.

Is a Kiro 402 distinguishable from "really out of credits"? Yes — for one well-known shape

A CodeWhisperer GenerateAssistantResponse 402 for monthly-quota exhaustion has a documented, reproducible shape (AWS ServiceQuotaExceededException), confirmed via public reports (e.g. aws/aws-toolkit-vscode#8577):

{
  "code": "QModelResponse",
  "name": "Error",
  "message": "You have reached the limit.",
  "cause": {
    "$metadata": { "httpStatusCode": 402 },
    "name": "ServiceQuotaExceededException",
    "reason": "MONTHLY_REQUEST_COUNT"
  }
}

KiroExecutor.parseError() (open-sse/executors/kiro.js) now matches this shape — and a flattened variant some surfaces use (name/reason at the top level instead of nested under cause) — to classify a 402 as confirmed vs. ambiguous. Anything that doesn't match (unrecognized body, non-JSON, a different reason) is left ambiguous and keeps the exact current 2-minute cooldown.

I also checked the related credits/quota reports in this repo (#1328, #1854, #1901, #2374) and the recent Kiro PRs #2580/#2618/#2639 — none of them touch 402 classification or the reactive cooldown path; #1854 (open) adds a complementary proactive pre-flight quota check that skips known-depleted Kiro accounts before dispatch. This PR is the reactive fallback for when that pre-flight either isn't present or is stale — the two aren't in conflict.

Where does the reset time come from?

The 402 body itself never carries a reset date (confirmed against the real-world shape above). The only trustworthy source is Kiro's own quota endpoint, GetUsageLimits, which 9router already parses (open-sse/services/usage/kiro.js, nextDateReset/resetDate). So on a confirmed match, parseError makes a best-effort follow-up call to that endpoint (8s timeout) and takes the earliest resetAt among fully-depleted quota buckets.

If that lookup is unreachable, times out, or nothing comes back depleted, it does not guess a date — it falls back to a fixed default (see below) rather than fabricating provider semantics.

Why the existing resetsAtMs cap needed a per-provider override

resetsAtMs already flows through the pipeline (parseUpstreamErrorcreateErrorResultmarkAccountUnavailable), and was previously capped at a single shared MAX_RATE_LIMIT_COOLDOWN_MS (30 min) — sized for short-window resets like Codex's resets_at (hours). Applying that cap to a month-scale Kiro reset would mean re-probing a known-exhausted account every 30 minutes.

markAccountUnavailable (src/sse/services/auth.js) now looks up a per-provider cap via RESET_COOLDOWN_CAP_MS (open-sse/config/errorConfig.js), with Kiro set to 24h:

const cooldownCapMs = RESET_COOLDOWN_CAP_MS[provider] ?? MAX_RATE_LIMIT_COOLDOWN_MS;
cooldownMs = Math.min(resetsAtMs - Date.now(), cooldownCapMs);

Other providers (Codex, etc.) are untouched — they still resolve to MAX_RATE_LIMIT_COOLDOWN_MS.

Behavior

Scenario Before After
Ambiguous 402 (unrecognized body) 2-min cooldown unchanged — 2-min cooldown
Confirmed exhaustion, quota API says reset in 3h 2-min cooldown, retried every 2 min for 3h locked ~3h (real reset honored)
Confirmed exhaustion, quota API says reset in 20 days 2-min cooldown, retried every 2 min for 20 days locked in ≤24h increments; re-probed roughly once/day until it recovers or the real reset passes
Confirmed exhaustion, quota lookup fails/times out/nothing depleted 2-min cooldown locked 24h (safe default, not fabricated), then re-probed
Multi-account account cycles fallback every 2 min locked account is skipped; sibling accounts keep serving normally

Tests

New: tests/unit/kiro-credit-exhaustion.test.js, tests/unit/kiro-credit-cooldown.test.js

  • confirmed exhaustion → resetsAtMs derived from the quota API
  • api-key credential precedence for the quota lookup
  • quota lookup returns nothing depleted → daily-probe fallback
  • quota lookup fails/rejects → daily-probe fallback
  • ambiguous 402 → unclassified, no resetsAtMs (old cooldown preserved)
  • non-402 statuses untouched
  • flattened-shape (top-level name/reason) also recognized
  • far-future reset capped at the daily-probe interval, not the full window
  • near-term reset respected (not stretched to 24h)
  • ambiguous Kiro 402 still gets the old 2-minute cooldown
  • other providers' resetsAtMs cap (Codex) unchanged
  • lock expiry — account becomes selectable again once the cooldown elapses
  • multi-account fallback — a locked account is skipped while a sibling stays available
npx vitest run unit/kiro-credit-exhaustion.test.js unit/kiro-credit-cooldown.test.js
# 2 files, 13 passed

npx vitest run unit/openai-to-kiro.test.js unit/cached-token-usage.test.js unit/kiro-external-idp.test.js \
  unit/kiro-model-slots.test.js unit/kiro-profile-arn.test.js unit/kiro-thinking-strip.test.js unit/rtkKiro.test.js \
  unit/codex-reset-credits.test.js unit/codex-fast-capacity.test.js translator/claude-kiro-direct.test.js translator/bugs-kiro.test.js
# 11 files, 72 passed | 2 expected fail (pre-existing, unrelated)

Full suite: ran on a clean master checkout before and after this change — identical set of unrelated failures both times (environment-dependent: live-network tests, CWD-relative fs.readFileSync in security-audit.test.js, SQLite concurrency timing, golden snapshot drift). No new failures introduced.

npx eslint clean on all changed files.

Limitations

  • The lock is still per (account, model), like every other error type in this file — an account rotating through several Kiro model aliases can independently probe each alias within the cap window. Making confirmed credit exhaustion an account-wide lock is a natural follow-up, but changing that shared locking mechanism felt out of scope for a focused fix — happy to do it here if maintainers would rather have it bundled.
  • The quota-lookup follow-up call adds a bit of latency to the (already-failed) 402 response — bounded to 8s worst case, and only on a confirmed match, not on the hot path.

🤖 Generated with Claude Code

Account-lock aggregation hardening

This branch now also keeps retry timing, HTTP status, and error text paired to the same locked account when every account is unavailable. A transient 502/503 account with the earliest 30-second retry can no longer inherit MONTHLY_REQUEST_COUNT from a different 402 account. Genuine all-account quota exhaustion remains 402.

The follow-up also clears stale errorCode after a successful cooldown recovery and makes model-specific plus account-wide locks compose by waiting for the later active expiry on each account before selecting the earliest account across the pool.

Regression coverage adds the mixed 3x402/120s plus 2x502/503/30s case, genuine all-402 preservation, cooldown recovery, stale error cleanup, and overlapping model/account lock expiry. Focused verification: 4 files, 24 tests; ESLint and production build pass. Independent Claude Code review verdict: SHIP.

SemonCat and others added 3 commits July 17, 2026 14:36
A Kiro 402 is currently always treated the same way: a flat 2-minute cooldown
(open-sse/config/errorConfig.js ERROR_RULES). That's fine for an ambiguous 402
(bad payment method, suspended account) but wrong for a confirmed monthly
credit-limit exhaustion, where the account will keep 402ing every 2 minutes
until the real reset — wasting requests and cycling through combo/account
fallback for no reason.

A real CodeWhisperer GenerateAssistantResponse 402 for this case has a
recognizable, verified shape (AWS ServiceQuotaExceededException):

  { "message": "You have reached the limit.",
    "cause": { "$metadata": { "httpStatusCode": 402 },
               "name": "ServiceQuotaExceededException",
               "reason": "MONTHLY_REQUEST_COUNT" } }

KiroExecutor.parseError() now matches this shape (and the flattened variant
some surfaces use) to classify the 402 as confirmed vs. ambiguous. Ambiguous
402s are untouched and keep the existing 2-minute cooldown.

The 402 body itself never carries a reset time, so on a confirmed match this
makes a best-effort follow-up call to Kiro's own quota API (GetUsageLimits),
which does surface a trustworthy resetAt (services/usage/kiro.js already
parses `nextDateReset`/`resetDate`). If that lookup is unreachable, times out,
or shows nothing depleted, it falls back to a bounded default instead of
guessing a date.

That resetsAtMs already flows through the existing pipeline (parseUpstreamError
-> createErrorResult -> markAccountUnavailable), which previously capped any
resetsAtMs-derived cooldown at a shared 30-minute ceiling meant for short-window
rate limits (e.g. Codex). Applying that ceiling to a month-scale Kiro reset would
re-probe a known-exhausted account every 30 minutes. markAccountUnavailable now
looks up a per-provider cap (RESET_COOLDOWN_CAP_MS), with Kiro set to 24h — so a
near-term reset is honored as-is, and a far-future reset is capped to a daily
probe, recovering within a day of an early provider-side change (manual top-up,
plan change) without hammering the account meanwhile. Other providers'
resetsAtMs handling (e.g. Codex) is unchanged.

Tests: confirmed exhaustion + resetsAtMs derivation, quota-lookup-failure and
nothing-depleted fallback, ambiguous 402 preserving the old cooldown, the daily
probe cap vs. a near-term reset, the existing 30-min cap for other providers,
lock expiry, and multi-account fallback around a locked Kiro account.

Limitations: the lock is still per (account, model) like every other error type
here, so an account rotating through several Kiro model aliases can still probe
each alias independently within the cap window; making credit exhaustion an
account-wide lock is a natural follow-up but is out of scope for this change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Keep retry timing, status, and error tied to the earliest model lock owner, and clear stale error codes after successful recovery.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant