Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use the k8s-staging-images project to build cloud-provider-kind images #33794

Merged
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ postsubmits:
command:
- /run.sh
args:
- --project=k8s-staging-kind
- --scratch-bucket=gs://k8s-staging-kind-gcb
- --project=k8s-staging-images
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we running all cloudbuilds here? we're going to have quota issues fetching the logs etc and we're reducing the isolation of these subprojects??

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(now someone's job that's only supposed to push some other subproject can also push to this staging location, right?)

lots of subprojects are not checking their logs closely when promoting images, if I understand what we're doing correctly: this is risky and we should go back to separate projects per subproject.

cc @kubernetes/sig-k8s-infra-leads

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the IAM permissions/ACL defined here not enough to handle this ?

Copy link
Member Author

@upodroid upodroid Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plan is for all subprojects to share the single cloudbuild project. The quotas are very high(something like 30 concurrent jobs). k/k will run cloudbuild jobs on its own project given its complexity.

We enforce the expected behaviour by only allowing promoting images from specified sources that we review. In the extremely unlikely situation that a maintainer misconfigures their cloudbuild definition and publishes it against another subprojects registry, the image can't be promoted and is deleted after 90d and they will spot the errors.

Also, humans only have privileges to push to their own respective registries and only prow and infra admins can submit cloudbuild jobs. The only way a cloudbuild job can be submitted by maintainers is via a postsubmit defined in the image pushing folder in the trusted cluster and commits to main branches approved by maintainers.

This approach is much simpler and with a reasonable set of guardrails.

- --scratch-bucket=gs://k8s-staging-images-gcb
- --env-passthrough=PULL_BASE_SHA
- --build-dir=.
- --with-git-dir
Expand Down