Remediate AWS integration CI (closes #30)#31
Merged
Conversation
starburst_setup() always ended by building the multi-platform worker image (~4 min). Add build_image = TRUE (trailing, backward compatible); set FALSE to provision S3/ECR/ECS/VPC + write config + check quotas without the image build. The image is then built lazily on first worker launch via ensure_environment(). Enables a fast, cheap CI connectivity check. Refs #30
#30) The inspect/create calls discarded stdout/stderr, so a failed builder setup was swallowed and surfaced later as an opaque 'no builder found' at buildx build. Capture stdout/stderr (so safe_system's stop() carries the real stderr) and add 'docker buildx use <name>' after create so the explicit --builder reference resolves even if the create didn't register it as current. Refs #30
- starburst_setup(build_image=FALSE) does not call build_initial_environment; build_image=TRUE (default) does. - ensure_buildx_builder selects the builder via 'buildx use' after create. Refs #30
#30) - Add job env SUITE = inputs.test_suite || 'quick' and gate the 5 suite steps on env.SUITE, so the monthly schedule (empty inputs) runs the quick smoke suite instead of being a no-op. - Configure staRburst step now uses build_image=FALSE so provisioning is fast for every run; suites that launch workers build the image lazily. Closes #30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes all four findings in #30 so the AWS integration workflow runs green end-to-end and the monthly schedule is a real (cheap) drift check rather than a no-op.
A —
quickno longer triggers a 4-min image buildstarburst_setup()gains a trailingbuild_image = TRUEparam (backward compatible). Withbuild_image = FALSEit provisions S3/ECR/ECS/VPC + writes config + checks quotas but skipsbuild_initial_environment(); the worker image is built lazily on first launch. The CIConfigure staRburststep now passesbuild_image = FALSE.B — buildx failures surface instead of "no builder found"
ensure_buildx_builder()now captures stdout/stderr (sosafe_system'sstop()carries the real error) and runsdocker buildx use <name>after create so the explicit--builderreference resolves. Multi-platform is unchanged (ARM64 Graviton workers need it).C — monthly schedule runs a default suite
Added job
env: SUITE: ${{ github.event.inputs.test_suite || 'quick' }}and switched the 5 suite-stepif:gates fromgithub.event.inputs.test_suite(empty onschedule) toenv.SUITE. The monthly cron now runs the read-onlyquicksmoke suite (~$0) — which is exactly what catches creds/IAM/config drift.D — 2 missing IAM perms (applied out-of-band)
Added
s3:PutEncryptionConfigurationandservicequotas:ListRequestedServiceQuotaChangeHistoryByQuotato thestarburst-github-actionsrole'sstarburst-integrationpolicy via AWS CLI (not a repo change). Verified present.Verification
R CMD check: back to the pre-existing baseline (1 donttest ERROR + 2 file NOTEs, all environmental — the new Rd/usage warnings I briefly introduced were fixed by regenerating docs).quickdispatch onmainis the final green check (the workflow only runs on dispatch/schedule, and the IAM trust is scoped tomain).Closes #30