fix(runner): make config getters nil-safe before initialization by G4614 · Pull Request #680 · boxlite-ai/boxlite

G4614 · 2026-06-08T14:26:04Z

runner cannot panic while getting a nil config

Tests

go test ./cmd/runner/config

Summary by CodeRabbit

Bug Fixes

Improved stability by ensuring configuration accessor functions safely handle cases where the configuration is not yet initialized, with graceful fallbacks to environment variables for runtime, network, and build engine settings.

Tests

Added comprehensive test coverage for configuration fallback behavior when the configuration is uninitialized, including tests for default value handling.

coderabbitai · 2026-06-08T14:26:12Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 91a9e2db-9ee4-47bc-b1bd-b4f70d9efd5e

📥 Commits

Reviewing files that changed from the base of the PR and between 77e932c and 88feee6.

📒 Files selected for processing (2)

apps/runner/cmd/runner/config/config.go
apps/runner/cmd/runner/config/config_test.go

📝 Walkthrough

Walkthrough

Four config getter functions (GetContainerRuntime, GetContainerNetwork, GetEnvironment, GetBuildEngine) are updated to return corresponding environment variable values when the global config is nil. GetBuildEngine additionally defaults to "buildkit" when the env var is empty. Tests covering these fallbacks are added.

Changes

Nil-safe config getter fallbacks

Layer / File(s)	Summary
Nil-safe getter implementation and tests `apps/runner/cmd/runner/config/config.go`, `apps/runner/cmd/runner/config/config_test.go`	Each getter adds a `config == nil` guard that reads the matching environment variable instead of dereferencing the nil pointer. `GetBuildEngine` also falls back to `"buildkit"` when `BUILD_ENGINE` is unset. Tests nil the global config, set env vars, and assert all four fallback values plus the `"buildkit"` default.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~5 minutes

Poem

🐇 When config goes missing, no panic, no fright,
I sniff the environment and set things just right.
BUILD_ENGINE? Buildkit, by default I'll choose,
A nil-safe hop forward — no data to lose.
flop 🌿

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: making config getters nil-safe before initialization, which matches the core focus of the changeset.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

cla-assistant · 2026-06-10T07:43:16Z

All committers have signed the CLA.

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR boxlite-ai#680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR boxlite-ai#680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR boxlite-ai#680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR boxlite-ai#680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR boxlite-ai#680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR boxlite-ai#680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… gap PR boxlite-ai#680 is sourced from a fork (G4614/boxlite). GitHub Actions does NOT pass repository variables (the `vars` context) to workflow runs triggered by `pull_request` from external forks — security rule to prevent fork PRs from reading upstream secrets/vars. Result: even though `gh variable set AWS_E2E_CLOUD_REGION` and `AWS_E2E_CLOUD_ROLE_ARN` succeeded on boxlite-ai/boxlite, the workflow run on this PR's head saw both as empty strings, and `aws-actions/configure-aws-credentials@v4` failed at input validation with "Input required and not supplied: aws-region" before any OIDC handshake could occur. Fix: hardcode the values in the workflow's env: block. Both are acceptable to commit: - `ap-northeast-1`: region name, public infrastructure information. - IAM role ARN: contains the AWS account ID (12 digits). AWS docs classify account IDs as "sensitive, not secret"; the role's safety is enforced by its trust policy (which restricts AssumeRole to GitHub OIDC tokens with sub matching `repo:boxlite-ai/boxlite:*` patterns), not by hiding the ARN. Industry-standard pattern in OSS CI infrastructure. After this PR merges, future PRs sourced from same-repo branches WILL see the repo vars again, but the hardcoded values remain the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

G4614 · 2026-06-10T14:09:52Z

Superseded by #724 — same content but as a same-repo PR (not from fork) so the e2e-cloud workflow's OIDC handshake can actually run. GitHub strips id-token: write permission from workflow runs triggered by PRs from forks, which made #680's CI structurally unable to pass.

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR #680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR #680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… gap PR #680 is sourced from a fork (G4614/boxlite). GitHub Actions does NOT pass repository variables (the `vars` context) to workflow runs triggered by `pull_request` from external forks — security rule to prevent fork PRs from reading upstream secrets/vars. Result: even though `gh variable set AWS_E2E_CLOUD_REGION` and `AWS_E2E_CLOUD_ROLE_ARN` succeeded on boxlite-ai/boxlite, the workflow run on this PR's head saw both as empty strings, and `aws-actions/configure-aws-credentials@v4` failed at input validation with "Input required and not supplied: aws-region" before any OIDC handshake could occur. Fix: hardcode the values in the workflow's env: block. Both are acceptable to commit: - `ap-northeast-1`: region name, public infrastructure information. - IAM role ARN: contains the AWS account ID (12 digits). AWS docs classify account IDs as "sensitive, not secret"; the role's safety is enforced by its trust policy (which restricts AssumeRole to GitHub OIDC tokens with sub matching `repo:boxlite-ai/boxlite:*` patterns), not by hiding the ARN. Industry-standard pattern in OSS CI infrastructure. After this PR merges, future PRs sourced from same-repo branches WILL see the repo vars again, but the hardcoded values remain the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Hide the AWS account ID + region + role ARN behind repo Variables — same pattern as .github/workflows/e2e-test.yml uses for AWS_ACCOUNT_ID. Previous version had them hardcoded because the original PR (#680) was a fork-PR and GitHub strips the `vars` context on fork-PR runs. PR #724 is now a same-repo PR (branch lives on boxlite-ai/boxlite), so the fork-PR workaround is no longer needed. Reads from: vars.AWS_E2E_CLOUD_REGION = ap-northeast-1 vars.AWS_E2E_CLOUD_ROLE_ARN = arn:aws:iam::<account>:role/boxlite-e2e-cloud-github-actions vars.AWS_ACCOUNT_ID (consumed by setup-e2e-cloud-oidc.sh) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR #680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR #680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… gap PR #680 is sourced from a fork (G4614/boxlite). GitHub Actions does NOT pass repository variables (the `vars` context) to workflow runs triggered by `pull_request` from external forks — security rule to prevent fork PRs from reading upstream secrets/vars. Result: even though `gh variable set AWS_E2E_CLOUD_REGION` and `AWS_E2E_CLOUD_ROLE_ARN` succeeded on boxlite-ai/boxlite, the workflow run on this PR's head saw both as empty strings, and `aws-actions/configure-aws-credentials@v4` failed at input validation with "Input required and not supplied: aws-region" before any OIDC handshake could occur. Fix: hardcode the values in the workflow's env: block. Both are acceptable to commit: - `ap-northeast-1`: region name, public infrastructure information. - IAM role ARN: contains the AWS account ID (12 digits). AWS docs classify account IDs as "sensitive, not secret"; the role's safety is enforced by its trust policy (which restricts AssumeRole to GitHub OIDC tokens with sub matching `repo:boxlite-ai/boxlite:*` patterns), not by hiding the ARN. Industry-standard pattern in OSS CI infrastructure. After this PR merges, future PRs sourced from same-repo branches WILL see the repo vars again, but the hardcoded values remain the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR #680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR #680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… gap PR #680 is sourced from a fork (G4614/boxlite). GitHub Actions does NOT pass repository variables (the `vars` context) to workflow runs triggered by `pull_request` from external forks — security rule to prevent fork PRs from reading upstream secrets/vars. Result: even though `gh variable set AWS_E2E_CLOUD_REGION` and `AWS_E2E_CLOUD_ROLE_ARN` succeeded on boxlite-ai/boxlite, the workflow run on this PR's head saw both as empty strings, and `aws-actions/configure-aws-credentials@v4` failed at input validation with "Input required and not supplied: aws-region" before any OIDC handshake could occur. Fix: hardcode the values in the workflow's env: block. Both are acceptable to commit: - `ap-northeast-1`: region name, public infrastructure information. - IAM role ARN: contains the AWS account ID (12 digits). AWS docs classify account IDs as "sensitive, not secret"; the role's safety is enforced by its trust policy (which restricts AssumeRole to GitHub OIDC tokens with sub matching `repo:boxlite-ai/boxlite:*` patterns), not by hiding the ARN. Industry-standard pattern in OSS CI infrastructure. After this PR merges, future PRs sourced from same-repo branches WILL see the repo vars again, but the hardcoded values remain the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Hide the AWS account ID + region + role ARN behind repo Variables — same pattern as .github/workflows/e2e-test.yml uses for AWS_ACCOUNT_ID. Previous version had them hardcoded because the original PR (#680) was a fork-PR and GitHub strips the `vars` context on fork-PR runs. PR #724 is now a same-repo PR (branch lives on boxlite-ai/boxlite), so the fork-PR workaround is no longer needed. Reads from: vars.AWS_E2E_CLOUD_REGION = ap-northeast-1 vars.AWS_E2E_CLOUD_ROLE_ARN = arn:aws:iam::<account>:role/boxlite-e2e-cloud-github-actions vars.AWS_ACCOUNT_ID (consumed by setup-e2e-cloud-oidc.sh) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR #680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR #680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… gap PR #680 is sourced from a fork (G4614/boxlite). GitHub Actions does NOT pass repository variables (the `vars` context) to workflow runs triggered by `pull_request` from external forks — security rule to prevent fork PRs from reading upstream secrets/vars. Result: even though `gh variable set AWS_E2E_CLOUD_REGION` and `AWS_E2E_CLOUD_ROLE_ARN` succeeded on boxlite-ai/boxlite, the workflow run on this PR's head saw both as empty strings, and `aws-actions/configure-aws-credentials@v4` failed at input validation with "Input required and not supplied: aws-region" before any OIDC handshake could occur. Fix: hardcode the values in the workflow's env: block. Both are acceptable to commit: - `ap-northeast-1`: region name, public infrastructure information. - IAM role ARN: contains the AWS account ID (12 digits). AWS docs classify account IDs as "sensitive, not secret"; the role's safety is enforced by its trust policy (which restricts AssumeRole to GitHub OIDC tokens with sub matching `repo:boxlite-ai/boxlite:*` patterns), not by hiding the ARN. Industry-standard pattern in OSS CI infrastructure. After this PR merges, future PRs sourced from same-repo branches WILL see the repo vars again, but the hardcoded values remain the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR #680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR #680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… gap PR #680 is sourced from a fork (G4614/boxlite). GitHub Actions does NOT pass repository variables (the `vars` context) to workflow runs triggered by `pull_request` from external forks — security rule to prevent fork PRs from reading upstream secrets/vars. Result: even though `gh variable set AWS_E2E_CLOUD_REGION` and `AWS_E2E_CLOUD_ROLE_ARN` succeeded on boxlite-ai/boxlite, the workflow run on this PR's head saw both as empty strings, and `aws-actions/configure-aws-credentials@v4` failed at input validation with "Input required and not supplied: aws-region" before any OIDC handshake could occur. Fix: hardcode the values in the workflow's env: block. Both are acceptable to commit: - `ap-northeast-1`: region name, public infrastructure information. - IAM role ARN: contains the AWS account ID (12 digits). AWS docs classify account IDs as "sensitive, not secret"; the role's safety is enforced by its trust policy (which restricts AssumeRole to GitHub OIDC tokens with sub matching `repo:boxlite-ai/boxlite:*` patterns), not by hiding the ARN. Industry-standard pattern in OSS CI infrastructure. After this PR merges, future PRs sourced from same-repo branches WILL see the repo vars again, but the hardcoded values remain the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Hide the AWS account ID + region + role ARN behind repo Variables — same pattern as .github/workflows/e2e-test.yml uses for AWS_ACCOUNT_ID. Previous version had them hardcoded because the original PR (#680) was a fork-PR and GitHub strips the `vars` context on fork-PR runs. PR #724 is now a same-repo PR (branch lives on boxlite-ai/boxlite), so the fork-PR workaround is no longer needed. Reads from: vars.AWS_E2E_CLOUD_REGION = ap-northeast-1 vars.AWS_E2E_CLOUD_ROLE_ARN = arn:aws:iam::<account>:role/boxlite-e2e-cloud-github-actions vars.AWS_ACCOUNT_ID (consumed by setup-e2e-cloud-oidc.sh) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR #680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR #680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… gap PR #680 is sourced from a fork (G4614/boxlite). GitHub Actions does NOT pass repository variables (the `vars` context) to workflow runs triggered by `pull_request` from external forks — security rule to prevent fork PRs from reading upstream secrets/vars. Result: even though `gh variable set AWS_E2E_CLOUD_REGION` and `AWS_E2E_CLOUD_ROLE_ARN` succeeded on boxlite-ai/boxlite, the workflow run on this PR's head saw both as empty strings, and `aws-actions/configure-aws-credentials@v4` failed at input validation with "Input required and not supplied: aws-region" before any OIDC handshake could occur. Fix: hardcode the values in the workflow's env: block. Both are acceptable to commit: - `ap-northeast-1`: region name, public infrastructure information. - IAM role ARN: contains the AWS account ID (12 digits). AWS docs classify account IDs as "sensitive, not secret"; the role's safety is enforced by its trust policy (which restricts AssumeRole to GitHub OIDC tokens with sub matching `repo:boxlite-ai/boxlite:*` patterns), not by hiding the ARN. Industry-standard pattern in OSS CI infrastructure. After this PR merges, future PRs sourced from same-repo branches WILL see the repo vars again, but the hardcoded values remain the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Hide the AWS account ID + region + role ARN behind repo Variables — same pattern as .github/workflows/e2e-test.yml uses for AWS_ACCOUNT_ID. Previous version had them hardcoded because the original PR (#680) was a fork-PR and GitHub strips the `vars` context on fork-PR runs. PR #724 is now a same-repo PR (branch lives on boxlite-ai/boxlite), so the fork-PR workaround is no longer needed. Reads from: vars.AWS_E2E_CLOUD_REGION = ap-northeast-1 vars.AWS_E2E_CLOUD_ROLE_ARN = arn:aws:iam::<account>:role/boxlite-e2e-cloud-github-actions vars.AWS_ACCOUNT_ID (consumed by setup-e2e-cloud-oidc.sh) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR #680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR #680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… gap PR #680 is sourced from a fork (G4614/boxlite). GitHub Actions does NOT pass repository variables (the `vars` context) to workflow runs triggered by `pull_request` from external forks — security rule to prevent fork PRs from reading upstream secrets/vars. Result: even though `gh variable set AWS_E2E_CLOUD_REGION` and `AWS_E2E_CLOUD_ROLE_ARN` succeeded on boxlite-ai/boxlite, the workflow run on this PR's head saw both as empty strings, and `aws-actions/configure-aws-credentials@v4` failed at input validation with "Input required and not supplied: aws-region" before any OIDC handshake could occur. Fix: hardcode the values in the workflow's env: block. Both are acceptable to commit: - `ap-northeast-1`: region name, public infrastructure information. - IAM role ARN: contains the AWS account ID (12 digits). AWS docs classify account IDs as "sensitive, not secret"; the role's safety is enforced by its trust policy (which restricts AssumeRole to GitHub OIDC tokens with sub matching `repo:boxlite-ai/boxlite:*` patterns), not by hiding the ARN. Industry-standard pattern in OSS CI infrastructure. After this PR merges, future PRs sourced from same-repo branches WILL see the repo vars again, but the hardcoded values remain the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Hide the AWS account ID + region + role ARN behind repo Variables — same pattern as .github/workflows/e2e-test.yml uses for AWS_ACCOUNT_ID. Previous version had them hardcoded because the original PR (#680) was a fork-PR and GitHub strips the `vars` context on fork-PR runs. PR #724 is now a same-repo PR (branch lives on boxlite-ai/boxlite), so the fork-PR workaround is no longer needed. Reads from: vars.AWS_E2E_CLOUD_REGION = ap-northeast-1 vars.AWS_E2E_CLOUD_ROLE_ARN = arn:aws:iam::<account>:role/boxlite-e2e-cloud-github-actions vars.AWS_ACCOUNT_ID (consumed by setup-e2e-cloud-oidc.sh) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR #680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR #680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… gap PR #680 is sourced from a fork (G4614/boxlite). GitHub Actions does NOT pass repository variables (the `vars` context) to workflow runs triggered by `pull_request` from external forks — security rule to prevent fork PRs from reading upstream secrets/vars. Result: even though `gh variable set AWS_E2E_CLOUD_REGION` and `AWS_E2E_CLOUD_ROLE_ARN` succeeded on boxlite-ai/boxlite, the workflow run on this PR's head saw both as empty strings, and `aws-actions/configure-aws-credentials@v4` failed at input validation with "Input required and not supplied: aws-region" before any OIDC handshake could occur. Fix: hardcode the values in the workflow's env: block. Both are acceptable to commit: - `ap-northeast-1`: region name, public infrastructure information. - IAM role ARN: contains the AWS account ID (12 digits). AWS docs classify account IDs as "sensitive, not secret"; the role's safety is enforced by its trust policy (which restricts AssumeRole to GitHub OIDC tokens with sub matching `repo:boxlite-ai/boxlite:*` patterns), not by hiding the ARN. Industry-standard pattern in OSS CI infrastructure. After this PR merges, future PRs sourced from same-repo branches WILL see the repo vars again, but the hardcoded values remain the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Hide the AWS account ID + region + role ARN behind repo Variables — same pattern as .github/workflows/e2e-test.yml uses for AWS_ACCOUNT_ID. Previous version had them hardcoded because the original PR (#680) was a fork-PR and GitHub strips the `vars` context on fork-PR runs. PR #724 is now a same-repo PR (branch lives on boxlite-ai/boxlite), so the fork-PR workaround is no longer needed. Reads from: vars.AWS_E2E_CLOUD_REGION = ap-northeast-1 vars.AWS_E2E_CLOUD_ROLE_ARN = arn:aws:iam::<account>:role/boxlite-e2e-cloud-github-actions vars.AWS_ACCOUNT_ID (consumed by setup-e2e-cloud-oidc.sh) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

G4614 · 2026-06-22T08:14:12Z

Added the runner config nil-safe fix here per request to reuse PR #680.

What changed:

GetEnvironment, GetContainerRuntime, GetContainerNetwork, and GetBuildEngine no longer panic when the package-level runner config has not been initialized.
Added config tests covering env/default fallback before GetConfig().

Validated locally:

go test ./cmd/runner/config

@v5

The previous e2e-stack.yml bootstrapped a full local stack (postgres, redis, docker registry, boxlite-runner systemd, boxlite-api) inside a self-hosted KVM runner per PR — slow to maintain (bootstrap drift, KVM runner contention) and only covers self-bootstrap state. This workflow keeps the same 3-job structure introduced by PR boxlite-ai#680 (changes / e2e / e2e-gate) so the `E2E gate (required)` branch protection check is unchanged, but replaces the `e2e` job: - runs-on: ubuntu-latest (no /dev/kvm needed) - authenticates to AWS via OIDC using the new boxlite-e2e-cloud-github-actions role - builds the Api container image + boxlite-runner musl binary from this checkout - deploys to the always-on Tokyo stack: rolling-updates the Api ECS service to the new image and SSM-replaces the runner EC2's binary - initialises the admin-org sandbox quota row in RDS via ECS Exec into the Api task (DB_PASSWORD stays in container env — never in the SSM command body) - builds & installs the Python SDK from this checkout (path_prefix and other source-tree additions land ahead of the PyPI release) - runs `pytest scripts/test/e2e/cases/` against http://<api-lb>/api with BOXLITE_E2E_SKIP_PATH_VERIFY=1 (journalctl-based path verify doesn't reach the remote runner) - on failure, dumps Api CloudWatch logs + boxlite-runner journalctl Concurrency uses a constant group (`e2e-cloud-shared`) because every run — push + PR — competes for the same singleton Tokyo stack. Per-ref grouping would let two PRs interleave ECS rolling updates. Resource discovery (cluster id, LB DNS, runner instance id, S3 bucket, ECR repo) is by-pattern at workflow start, with a strict count=1 assertion so an orphaned `ApiLoadBalancer-*` from a partial SST teardown fails the job loudly instead of silently binding to the wrong target. SSM agent ping-status is verified before the runner update step to fail fast on a dead agent. Adversarial review (multiple lenses) caught and this commit addresses: - credentials.toml written via printf rather than unquoted heredoc so a `$` inside the admin key can't be shell-expanded; ADMIN_KEY immediately registered via `::add-mask::` - ECS execute-command output is captured and grepped for the expected SELECT row (Session Manager exit-code propagation has historically been unreliable through --interactive without a tty) - `wait services-stable` is followed by ALB target-health polling AND a /api/health curl with retry — services-stable alone races the LB health check - pytest wrapped in `timeout 35m` plus `--timeout=180 --timeout-method=thread` so a hung VM doesn't burn the 45-min job timeout before `Collect logs on failure` runs - Rust build cached via Swatinem/rust-cache@v2 so repeat PR builds are incremental - both actions/checkout occurrences bumped to @v5 to match the newer workflows in the repo Known limits (TODO/follow-up, not blocking): - boxlite-runner has no documented in-flight drain; restart drops any request landing during the swap window. Acceptable for e2e (no concurrent users) but a production-grade rollout would need a runner-side graceful drain. - No post-run restore of `main` HEAD on the Tokyo stack: the stack is left running the last e2e-cloud run's image. Console inspection of `api-<sha>` identifies the running revision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…visioned PR boxlite-ai#680's original goal was to make the e2e suite a required merge gate. With the workflow now retargeted at the always-on Tokyo stack (previous commit), keeping `e2e-gate` as `(required)` would block every PR until the Tokyo IAM OIDC role + SSM parameter + admin-org quota are provisioned — a chicken-and-egg. Removing the gate job for now. The two-job structure (changes → e2e) still works for `workflow_dispatch` trial runs and for PR CI feedback (non-blocking). Re-add an `e2e-gate` job and the corresponding branch protection rule once a first green run is observed. Header comment updated to flag the deferred-required-check status so future contributors know why the workflow exists without being gated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… gap PR boxlite-ai#680 is sourced from a fork (G4614/boxlite). GitHub Actions does NOT pass repository variables (the `vars` context) to workflow runs triggered by `pull_request` from external forks — security rule to prevent fork PRs from reading upstream secrets/vars. Result: even though `gh variable set AWS_E2E_CLOUD_REGION` and `AWS_E2E_CLOUD_ROLE_ARN` succeeded on boxlite-ai/boxlite, the workflow run on this PR's head saw both as empty strings, and `aws-actions/configure-aws-credentials@v4` failed at input validation with "Input required and not supplied: aws-region" before any OIDC handshake could occur. Fix: hardcode the values in the workflow's env: block. Both are acceptable to commit: - `ap-northeast-1`: region name, public infrastructure information. - IAM role ARN: contains the AWS account ID (12 digits). AWS docs classify account IDs as "sensitive, not secret"; the role's safety is enforced by its trust policy (which restricts AssumeRole to GitHub OIDC tokens with sub matching `repo:boxlite-ai/boxlite:*` patterns), not by hiding the ARN. Industry-standard pattern in OSS CI infrastructure. After this PR merges, future PRs sourced from same-repo branches WILL see the repo vars again, but the hardcoded values remain the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

G4614 force-pushed the chore/e2e-required-merge-gate branch from 8df6ade to 41bbb6d Compare June 9, 2026 03:27

G4614 force-pushed the chore/e2e-required-merge-gate branch from 213359b to 97af472 Compare June 10, 2026 07:44

G4614 force-pushed the chore/e2e-required-merge-gate branch from be6793a to 347ee5b Compare June 10, 2026 08:42

G4614 force-pushed the chore/e2e-required-merge-gate branch from 5bfa094 to 0cdf723 Compare June 10, 2026 09:14

G4614 mentioned this pull request Jun 10, 2026

ci(deploy): isolate non-release runner runtime caches #724

Draft

12 tasks

G4614 closed this Jun 10, 2026

G4614 reopened this Jun 22, 2026

G4614 changed the title ~~ci(e2e): make e2e suite a required merge gate (stacks on #678)~~ fix(runner): make config getters nil-safe before initialization Jun 22, 2026

G4614 force-pushed the chore/e2e-required-merge-gate branch from bd29f49 to aa05aec Compare June 22, 2026 08:47

fix(runner): guard config getters before initialization

88feee6

G4614 force-pushed the chore/e2e-required-merge-gate branch from aa05aec to 88feee6 Compare June 22, 2026 08:52

G4614 marked this pull request as ready for review June 22, 2026 09:01

G4614 requested a review from a team as a code owner June 22, 2026 09:01

G4614 enabled auto-merge June 22, 2026 09:01

G4614 marked this pull request as draft June 22, 2026 10:35

auto-merge was automatically disabled June 22, 2026 10:35
Pull request was converted to draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(runner): make config getters nil-safe before initialization#680

fix(runner): make config getters nil-safe before initialization#680
G4614 wants to merge 1 commit into
boxlite-ai:mainfrom
G4614:chore/e2e-required-merge-gate

G4614 commented Jun 8, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jun 8, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

Poem

❌ Failed checks (1 warning)

Uh oh!

cla-assistant Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

G4614 commented Jun 10, 2026

Uh oh!

G4614 commented Jun 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

G4614 commented Jun 8, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tests

Summary by CodeRabbit

Bug Fixes

Tests

Uh oh!

coderabbitai Bot commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

❌ Failed checks (1 warning)

Uh oh!

cla-assistant Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

G4614 commented Jun 10, 2026

Uh oh!

G4614 commented Jun 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

G4614 commented Jun 8, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 8, 2026 •

edited

Loading

cla-assistant Bot commented Jun 10, 2026 •

edited

Loading