Skip to content

feat(wrap): auto-detect CLAUDE_CODE_USE_BEDROCK and re-sign with SigV4#1220

Open
didhd wants to merge 1 commit into
headroomlabs-ai:mainfrom
didhd:feat/wrap-claude-bedrock
Open

feat(wrap): auto-detect CLAUDE_CODE_USE_BEDROCK and re-sign with SigV4#1220
didhd wants to merge 1 commit into
headroomlabs-ai:mainfrom
didhd:feat/wrap-claude-bedrock

Conversation

@didhd

@didhd didhd commented Jun 21, 2026

Copy link
Copy Markdown

Description

headroom wrap claude special-cases Vertex (CLAUDE_CODE_USE_VERTEX) and Foundry (CLAUDE_CODE_USE_FOUNDRY), but has no Bedrock branch. With CLAUDE_CODE_USE_BEDROCK=1, wrap claude falls through to setting ANTHROPIC_BASE_URL — which Claude Code ignores in Bedrock mode (it reads ANTHROPIC_BEDROCK_BASE_URL). The result: Claude Code talks straight to AWS Bedrock and Headroom compresses nothing, silently. This is the same failure mode the Vertex guide warns about, but with no code path to prevent it.

This PR makes CLAUDE_CODE_USE_BEDROCK=1 a turnkey, compressing flow — the Bedrock analogue of the existing native Vertex path.

The wrinkle vs. Vertex: Claude Code signs each Bedrock InvokeModel request with AWS SigV4, and the signature covers a hash of the request body. Compressing the body invalidates that signature, so simply repointing the endpoint would make AWS reject every request with InvalidSignatureException. So this PR also adds post-compression SigV4 re-signing to the Python proxy (boto3 is already a dependency), forwarding direct to the regional Bedrock endpoint with the user's own AWS credentials — no re-signing gateway required.

Closes #

Type of Change

  • New feature (non-breaking change that adds functionality)
  • Documentation update

Changes Made

  • headroom/proxy/bedrock_signer.py (new) — BedrockSigner: re-signs the post-compression body with SigV4 via the standard boto3 credential chain (env, profile, SSO, IMDS, ECS/EKS). Drops stale Authorization/X-Amz-*/host headers, preserves content-type and anthropic-* / x-amzn-bedrock-* passthrough headers, fails loud when no credentials resolve.
  • headroom/proxy/handlers/bedrock.py — forward to the regional AWS endpoint and re-sign when in signing mode; sign over the exact outbound bytes (after compression). An explicit --bedrock-api-url gateway still wins and is forwarded to verbatim (unchanged behavior).
  • headroom/proxy/models.py — new bedrock_sign: bool config field.
  • headroom/providers/proxy_routes.py — register the Bedrock routes when either bedrock_api_url or bedrock_sign is set.
  • headroom/cli/proxy.py, headroom/proxy/server.py--bedrock-sign flag + HEADROOM_BEDROCK_SIGN env, on both the click CLI and the argparse/native entry points.
  • headroom/cli/wrap.py — detect CLAUDE_CODE_USE_BEDROCK, set ANTHROPIC_BEDROCK_BASE_URL (not ANTHROPIC_BASE_URL), start the proxy with --bedrock-sign, resolve region from --regionAWS_REGIONAWS_DEFAULT_REGION, and persist/restore the correct settings.json key for daemon-spawned workers ([BUG] headroom wrap claude: daemon child sessions don't inherit ANTHROPIC_BASE_URL, bypassing proxy after first conversation #951 parity). unwrap claude also clears the Bedrock key.
  • docs — new claude-code-bedrock.mdx guide (mirrors the Vertex guide); registered in meta.json; README agent-matrix note.
  • tests — signer unit tests, handler signing-mode tests, and wrap claude branch CLI tests.

Testing

  • Unit tests pass (pytest)
  • Linting passes (ruff check .)
  • Type checking passes (mypy headroom) — not run (see Not tested)
  • New tests added for new functionality
  • Manual testing performed (unit-level; live AWS smoke test pending — see below)

Test Output

$ python -m pytest tests/test_proxy/test_bedrock_passthrough.py \
    tests/test_proxy/test_bedrock_signer.py \
    tests/test_cli/test_wrap_claude_bedrock.py \
    tests/test_cli/test_unwrap_claude.py tests/test_bedrock_region.py -q
66 passed, 2 warnings in 6.99s

$ python -m pytest tests/test_cli/ tests/test_proxy/ -q
477 passed, 3 skipped, 2 warnings in 16.68s

$ ruff check <changed files>
All checks passed!

Real Behavior Proof

  • Environment: macOS (darwin 25.5.0), Python 3.11.11, headroom _core built via maturin develop --release, a real AWS account with bedrock:InvokeModel in us-west-2, botocore SigV4, on branch feat/wrap-claude-bedrock.

  • Exact command / steps: Start the proxy in signing mode, then route the AWS CLI through it via AWS_ENDPOINT_URL_BEDROCK_RUNTIME so the CLI signs to the proxy, the proxy re-signs the post-compression body, and forwards direct to real AWS — which validates the signature. The exact commands:

    # proxy
    python -m headroom.cli proxy --port 8799 --bedrock-sign --region us-west-2
    # → /health: {"status":"healthy","ready":true}
    # → log: botocore.credentials - Found credentials in shared credentials file
    
    # invoke through the proxy (us. inference profile)
    AWS_ENDPOINT_URL_BEDROCK_RUNTIME=http://127.0.0.1:8799 \
      aws bedrock-runtime invoke-model --region us-west-2 \
        --model-id us.anthropic.claude-haiku-4-5-20251001-v1:0 \
        --body fileb://body.json out.json
  • Observed result: 3/3 live invokes accepted by AWS (no InvalidSignatureException), returning BEDROCK_SIGN_OK / LARGE_OK / COMPRESS_OK with stop_reason: end_turn, including a 116KB body and a multi-block tool_use/tool_result conversation; GET /metrics reported headroom_requests_by_provider{provider="bedrock"} 3. This proves the re-signed signature is valid against AWS's real validator. Unit and integration coverage is green too: signer tests (the signature tracks the body, stale signing headers are dropped, passthrough headers are signed, missing credentials raise BedrockSigningError); handler tests on a mocked-compressing pipeline (the compressed body is forwarded to https://bedrock-runtime.<region>.amazonaws.com/... carrying an AWS4-HMAC-SHA256 Authorization, and --bedrock-api-url takes precedence with no re-sign); CLI tests (CLAUDE_CODE_USE_BEDROCK=1 sets ANTHROPIC_BEDROCK_BASE_URL, not ANTHROPIC_BASE_URL); and the full CLI + proxy suites at 477 passed, 3 skipped — no regressions.

  • Not tested: Live compression and re-sign in the same request — the synthetic payloads used in the live run landed in the protected live-zone, so the live invokes re-signed the body unchanged; the compress-then-sign path is instead covered by a handler unit test with a mocked compressing pipeline, and this PR does not change compression logic. mypy was not run in this environment. Live streaming (invoke-with-response-stream) re-sign is covered structurally via the same _sign_if_needed call but not against a live stream.

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows the project's style guidelines (ruff clean)
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove the feature works
  • New and existing unit tests pass locally with my changes
  • I have updated the CHANGELOG.md if applicable (release-please managed — left to maintainer)

@github-actions

github-actions Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

PR governance

This PR follows the template and is marked ready for human review.

@github-actions github-actions Bot added status: needs author action Pull request body or readiness checklist still needs author updates status: ready for review Pull request body is complete and the author marked it ready for human review and removed status: needs author action Pull request body or readiness checklist still needs author updates labels Jun 21, 2026
@chopratejas

Copy link
Copy Markdown
Collaborator

Hi - thanks for this change - can we resolve some conflicts.

@codecov

codecov Bot commented Jun 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 24.69136% with 61 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
headroom/proxy/bedrock_signer.py 0.00% 39 Missing ⚠️
headroom/proxy/handlers/bedrock.py 12.50% 21 Missing ⚠️
headroom/providers/proxy_routes.py 0.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@didhd didhd force-pushed the feat/wrap-claude-bedrock branch from 45e4db8 to 3b0bce9 Compare June 21, 2026 18:04
@didhd

didhd commented Jun 21, 2026

Copy link
Copy Markdown
Author

Thanks for the review! I've rebased onto the latest main (6129808) and resolved everything:

Conflicts resolved (4 files)

  • headroom/cli/proxy.py had the only real conflicts (2 hunks): your new --telemetry opt-in flag landed next to my --bedrock-sign flag. Kept both — the option decorators and the function signature now carry --telemetry/telemetry: bool and --bedrock-sign/bedrock_sign: bool. Verified headroom proxy --help renders and all three flags (--bedrock-sign, --telemetry, --no-telemetry) are present, so the click option↔param mapping is intact.
  • cli/wrap.py, providers/proxy_routes.py, proxy/server.py merged cleanly; confirmed my Bedrock changes (route registration, --bedrock-sign arg, CLAUDE_CODE_USE_BEDROCK detection) all survived alongside your changes.

CI failures fixed

  • lint (ruff format --check): reformatted tests/test_cli/test_wrap_claude_bedrock.py (one nested with line). ruff format --check and ruff check are now clean across all changed files.
  • test shard botocore failure: boto3 lives in the bedrock extra, not dev, so the test shards (pip install .[dev]) run without botocore. test_bedrock_signer.py already guarded with pytest.importorskip("botocore"), but test_bedrock_passthrough.py had two tests importing botocore.credentials directly inside the function body — those raised ModuleNotFoundError instead of skipping. Switched both to pytest.importorskip("botocore.credentials"). Verified by simulating a botocore-absent environment: both now skip cleanly (the other 17 tests in the file, which don't need botocore, still run).

Local verification

  • Bedrock + wrap tests: 30 passed (with botocore present).
  • Full tests/test_cli/ tests/test_proxy/: 474 passed, 3 skipped.
  • The 3 remaining failures in tests/test_proxy/test_transformations_feed.py reproduce identically on pristine main (I swapped in the unmodified main server.py to confirm) — they're caused by the _require_loopback guard added to /transformations/feed on main without the test being updated, unrelated to this PR. Happy to fix that in a separate PR if useful.

Note: the CI checks show action_required — that's the fork-PR approval gate, so they'll need a maintainer "Approve and run" to execute.

@github-actions github-actions Bot added status: has conflicts Pull request has merge conflicts with the base branch and removed status: ready for review Pull request body is complete and the author marked it ready for human review labels Jun 22, 2026

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough implementation and the live AWS validation notes. The signing flow itself is headed in the right direction, but I think this needs one packaging/install fix before it can land.

headroom wrap claude now automatically enables --bedrock-sign whenever CLAUDE_CODE_USE_BEDROCK=1 is present, and headroom/proxy/bedrock_signer.py imports boto3 on the first signed request. However, boto3 is only declared under the optional bedrock extra in pyproject.toml, and even [all] currently omits that extra. The new guide also tells users to run pip install headroom before headroom wrap claude.

That means a normal wrapper/proxy install can enter the new turnkey Bedrock path and then fail at runtime with BedrockSigningError: boto3 is required for Bedrock SigV4 signing. Because this feature is auto-detected rather than an explicit advanced mode, the dependency story needs to be made consistent with the behavior before merge.

Concrete fixes that would satisfy this:

  • Add boto3/botocore to the install surface that supports wrap claude + proxy, or include bedrock from the relevant extras such as proxy and all; and update the lockfile if this repo tracks it for dependency changes.
  • Update the Bedrock guide install command to the real PyPI package/extra, e.g. pip install "headroom-ai[proxy,bedrock]" if Bedrock remains optional.
  • Add a small test or packaging assertion so bedrock is not silently excluded from [all] again if that is the intended comprehensive install.

Once that is addressed, I’d be comfortable taking another pass on the signing/routing details.

`headroom wrap claude` special-cases Vertex (CLAUDE_CODE_USE_VERTEX) and
Foundry (CLAUDE_CODE_USE_FOUNDRY) but had no Bedrock branch: with
CLAUDE_CODE_USE_BEDROCK=1 it fell through to setting ANTHROPIC_BASE_URL,
which Claude Code ignores in Bedrock mode — so traffic went straight to AWS
and Headroom compressed nothing.

Claude Code's Bedrock path reads ANTHROPIC_BEDROCK_BASE_URL and signs each
InvokeModel request with SigV4 (the signature hashes the body). Compressing
the body invalidates that signature, so a Vertex-style "just repoint the
endpoint" branch alone would 403. This adds direct-to-AWS re-signing so the
flow is turnkey, with no re-signing gateway required.

- proxy: new BedrockSigner (boto3 default credential chain) re-signs the
  post-compression body; `--bedrock-sign` / HEADROOM_BEDROCK_SIGN enables it.
- handler: forward to the regional endpoint and re-sign when signing; an
  explicit --bedrock-api-url gateway still wins (forwarded verbatim).
- routes register when either --bedrock-api-url or --bedrock-sign is set.
- wrap claude: detect CLAUDE_CODE_USE_BEDROCK, set ANTHROPIC_BEDROCK_BASE_URL
  (not ANTHROPIC_BASE_URL), start the proxy with --bedrock-sign, resolve the
  region from AWS_REGION/AWS_DEFAULT_REGION, persist+restore the right
  settings.json key.
- packaging: boto3 is required by the auto-detected signing path, so add the
  bedrock extra to [all], document the real install
  (pip install "headroom-ai[proxy,bedrock]"), point the BedrockSigningError at
  the [bedrock] extra, and add tests/test_bedrock_packaging.py to keep bedrock
  from being silently dropped from [all].
- docs: Claude Code on Amazon Bedrock guide; README matrix note.
- tests: signer unit tests, handler signing-mode tests, wrap-branch CLI tests.
@didhd didhd force-pushed the feat/wrap-claude-bedrock branch from 3b0bce9 to 2b65e30 Compare June 23, 2026 00:49
@didhd

didhd commented Jun 23, 2026

Copy link
Copy Markdown
Author

Thanks @JerrettDavis — you're right, the auto-detected path made boto3 a real runtime dependency while the packaging still treated it as fully optional. Fixed all three, and rebased onto the latest main.

1. Install surface now carries boto3 for the wrap/proxy + Bedrock flow

  • Added bedrock to the all aggregate in pyproject.toml, so headroom-ai[all] now ships boto3. (It was the omission you flagged.)
  • Updated uv.lock to match (the all extra's requires-dist now references bedrock).
  • I kept bedrock an opt-in extra rather than folding boto3 into [proxy] itself: botocore is ~14 MB, and the large majority of [proxy] users never touch Bedrock. wrap claude only enters the signing path when CLAUDE_CODE_USE_BEDROCK=1, so the explicit [proxy,bedrock] (or [all]) install keeps non-AWS installs lean. Happy to move it into [proxy] instead if you'd prefer zero-config over install size — your call.

2. Guide install command fixed
docs/content/docs/claude-code-bedrock.mdx now says:

pip install "headroom-ai[proxy,bedrock]"

(plus a note that bedrock provides boto3 for SigV4 re-signing, proxy provides the local proxy, and [all] includes both). The old pip install headroom was wrong on the package name and missing the deps.

3. Guard so bedrock can't be silently dropped from [all] again
Added tests/test_bedrock_packaging.py:

  • test_bedrock_extra_declares_boto3 — the [bedrock] extra exists and declares boto3.
  • test_all_extra_includes_bedrock — the [all] aggregate references bedrock (parses pyproject.toml, asserts the self-referential extra set contains it). This is the exact regression you described.

Bonus: made the runtime failure actionable — BedrockSigningError now says pip install "headroom-ai[bedrock]" instead of the generic pip install boto3, so anyone who does hit it lands on the right extra.

Verification (local, with boto3 present): ruff format --check + ruff check clean on all changed files; bedrock signer + new packaging tests pass; confirmed boto3 resolves under both [all] and [proxy,bedrock]. CI will re-run on the rebased head once a maintainer approves the fork-PR workflow gate.

@github-actions github-actions Bot added status: ready for review Pull request body is complete and the author marked it ready for human review status: ci failing Required or reported CI checks are failing and removed status: has conflicts Pull request has merge conflicts with the base branch status: ready for review Pull request body is complete and the author marked it ready for human review labels Jun 23, 2026

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latest commit resolves the packaging/install blocker I raised. The Bedrock signing path no longer depends on an undocumented runtime dependency: [bedrock] declares boto3, [all] now includes bedrock, the Bedrock guide uses pip install "headroom-ai[proxy,bedrock]", and the new packaging tests guard against dropping the extra again.

Code review passes on the signing flow as well: Bedrock mode routes Claude Code through ANTHROPIC_BEDROCK_BASE_URL, starts the proxy with signing enabled, signs the post-compression outbound bytes, keeps explicit gateway URLs taking precedence, and has focused signer/handler/wrap coverage. CI still needs to run/finish on the final head, but the previous code blocker is resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: ci failing Required or reported CI checks are failing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants