Skip to content

Conversation

@sylvainsf
Copy link
Contributor

@sylvainsf sylvainsf commented Aug 29, 2025

Description

This PR consolidates build/test workflows and artifacts, reduces registry pushes, enables local functional test parity, adds first‑party image purge, and introduces build metrics.

What changed:

  • Build system
    • New consolidated artifacts target (build/artifacts.mk) to produce artifacts for tests/releases; accepts DOCKER_TAG_VERSION and honors GIT_COMMIT.
    • Docker image save/load helpers (docker-save-images, docker-load-images) produce dist/images/*.tar.
    • GitHub Actions Docker caching enabled by default via DOCKER_CACHE_GHA=1 in CI environments.
    • GIT_COMMIT override wiring in build/version.mk.
  • GHCR registry optimization with approval-triggered architecture
    • Build workflow saves container images as artifacts instead of pushing to GHCR.
    • Functional test workflows download artifacts and load them directly into the KinD cluster.
    • Eliminates fork permission issues - all PRs (fork or main repo) follow same artifact-based flow.
  • Functional test workflows
    • Cloud and noncloud workflows optimized to pull from GHCR instead of rebuilding images.
    • rad install uses Helm global.imageRegistry and global.imageTag with explicit de.image/de.tag.
    • Noncloud: secure local registry + KinD integration, publish Bicep types to local registry, download CLI via artifact.
  • Purge old images
    • Replace external purge dependency with a first-party GitHub script; default retention 48 hours.
    • Add workflow_dispatch inputs: image_names, tag_filter, cut_off_hours, dry_run.
  • Build metrics
    • New make build-metrics target: measures duration, image count, and total bytes; writes dist/metrics/metrics.json and metrics.txt.
    • CI uploads metrics per workflow with unique names (cloud vs noncloud) and appends a job summary.

Workflow architecture:

graph TD
    subgraph "Fork PR Context (Untrusted)"
        A[PR Created/Updated] --> B(build.yaml);
        B --> C{Build Images};
        C --> D[Save Images as .tar];
        D --> E[Upload Artifact 'container-images'];
    end

    subgraph "Main Repo Context (Trusted)"
        F[Maintainer Approves] --> G(functional-tests-approval.yaml);
        G --> H(functional-test-*.yaml);
        H --> I{Download Artifact 'container-images'};
        I --> J[Load .tar Images into KinD];
        J --> K[Run Functional Tests];
    end

    E --> F;
Loading

Security analysis:

  • GHCR push uses elevated token (GH_RAD_CI_BOT_PAT) but is protected by approval-based workflow.
  • Functional tests require explicit approval via "functional-tests" environment before elevated permissions are used.
  • Fork-based PRs cannot trigger elevated permissions without maintainer approval, preventing unauthorized package pushes.
  • Bot token scope limited to package writes within radius-project organization.
  • Approval workflow acts as security gate between artifact creation and registry push.

Notes:

  • Fork PR workflow improved - no permission conflicts, consistent artifact-based flow for all PRs.
  • Functional test performance improved through GHCR image reuse and GitHub Actions caching.
  • Build metrics collection enabled by default for development workflow visibility.
  • Developers can run make build-metrics locally; CI collects metrics without altering build outputs.

Type of change

  • This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).

Fixes: N/A

Contributor checklist

Please verify that the PR meets the following requirements, where applicable:

  • An overview of proposed schema changes is included in a linked GitHub issue.
    - [ ] Yes
    - [x] Not applicable
  • A design document PR is created in the design-notes repository, if new APIs are being introduced.
    - [ ] Yes
    - [x] Not applicable
  • The design document has been reviewed and approved by Radius maintainers/approvers.
    - [ ] Yes
    - [x] Not applicable
  • A PR for the samples repository is created, if existing samples are affected by the changes in this PR.
    - [ ] Yes
    - [x] Not applicable
  • A PR for the documentation repository is created, if the changes in this PR affect the documentation or any user facing updates are made.
    - [ ] Yes
    - [x] Not applicable
  • A PR for the recipes repository is created, if existing recipes are affected by the changes in this PR.
    - [ ] Yes
    - [x] Not applicable

Signed-off-by: Sylvain Niles <[email protected]>
@radius-functional-tests
Copy link

radius-functional-tests bot commented Aug 29, 2025

Radius functional test overview

🔍 Go to test action run

Name Value
Repository sylvainsf/radius
Commit ref dabb2b6
Unique ID funcd04ddd5180
Image tag pr-funcd04ddd5180
Click here to see the list of tools in the current test run
  • gotestsum 1.12.0
  • KinD: v0.29.0
  • Dapr:
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcd04ddd5180
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcd04ddd5180
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-funcd04ddd5180
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funcd04ddd5180
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcd04ddd5180
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

Signed-off-by: Sylvain Niles <[email protected]>
@codecov
Copy link

codecov bot commented Aug 29, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 50.40%. Comparing base (0218b4c) to head (0cc6dcb).

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10272      +/-   ##
==========================================
- Coverage   50.42%   50.40%   -0.02%     
==========================================
  Files         661      661              
  Lines       41072    41072              
==========================================
- Hits        20709    20703       -6     
- Misses      18399    18402       +3     
- Partials     1964     1967       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Sylvain Niles <[email protected]>
@radius-functional-tests
Copy link

radius-functional-tests bot commented Sep 2, 2025

Radius functional test overview

🔍 Go to test action run

Name Value
Repository sylvainsf/radius
Commit ref 3a3c068
Unique ID func3859bdbc5f
Image tag pr-func3859bdbc5f
Click here to see the list of tools in the current test run
  • gotestsum 1.12.0
  • KinD: v0.29.0
  • Dapr:
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func3859bdbc5f
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func3859bdbc5f
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func3859bdbc5f
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func3859bdbc5f
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func3859bdbc5f
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

Signed-off-by: Sylvain Niles <[email protected]>
Signed-off-by: Sylvain Niles <[email protected]>
@radius-functional-tests
Copy link

radius-functional-tests bot commented Sep 3, 2025

Radius functional test overview

🔍 Go to test action run

Name Value
Repository sylvainsf/radius
Commit ref c06dca3
Unique ID func696122e543
Image tag pr-func696122e543
Click here to see the list of tools in the current test run
  • gotestsum 1.12.0
  • KinD: v0.29.0
  • Dapr:
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func696122e543
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func696122e543
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func696122e543
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func696122e543
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func696122e543
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

Signed-off-by: Sylvain Niles <[email protected]>
@radius-functional-tests
Copy link

radius-functional-tests bot commented Sep 3, 2025

Radius functional test overview

🔍 Go to test action run

Name Value
Repository sylvainsf/radius
Commit ref 6fc44f7
Unique ID func509d3374a2
Image tag pr-func509d3374a2
Click here to see the list of tools in the current test run
  • gotestsum 1.12.0
  • KinD: v0.29.0
  • Dapr:
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func509d3374a2
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func509d3374a2
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func509d3374a2
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func509d3374a2
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func509d3374a2
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

@radius-functional-tests
Copy link

radius-functional-tests bot commented Sep 3, 2025

Radius functional test overview

🔍 Go to test action run

Name Value
Repository sylvainsf/radius
Commit ref 5664a11
Unique ID func2c544f8aaa
Image tag pr-func2c544f8aaa
Click here to see the list of tools in the current test run
  • gotestsum 1.12.0
  • KinD: v0.29.0
  • Dapr:
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func2c544f8aaa
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func2c544f8aaa
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func2c544f8aaa
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func2c544f8aaa
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func2c544f8aaa
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

@radius-functional-tests
Copy link

radius-functional-tests bot commented Sep 12, 2025

Radius functional test overview

🔍 Go to test action run

Name Value
Repository sylvainsf/radius
Commit ref feb777d
Unique ID func52bd39da04
Image tag pr-func52bd39da04
Click here to see the list of tools in the current test run
  • gotestsum 1.12.0
  • KinD: v0.29.0
  • Dapr:
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func52bd39da04
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func52bd39da04
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func52bd39da04
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func52bd39da04
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func52bd39da04
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

@sylvainsf sylvainsf marked this pull request as ready for review September 12, 2025 23:34
@sylvainsf sylvainsf requested review from a team as code owners September 12, 2025 23:34
@lakshmimsft
Copy link
Contributor

maybe we are missing the approval workflow which will finally push these artifacts to GHCR. should this be a required step for a PR post running tests, so that edge always has the latest images??

@sylvainsf
Copy link
Contributor Author

maybe we are missing the approval workflow which will finally push these artifacts to GHCR. should this be a required step for a PR post running tests, so that edge always has the latest images??

That's out of scope for this change, isn't that a workflow triggered by merge to main? Running tests is completely separate from managing artifacts in GHCR.

@radius-functional-tests
Copy link

radius-functional-tests bot commented Nov 6, 2025

Radius functional test overview

🔍 Go to test action run

Click here to see the test run details
Name Value
Repository sylvainsf/radius
Commit ref 0cc6dcb
Unique ID func3e26b0bed5
Image tag pr-func3e26b0bed5
  • gotestsum 1.13.0
  • KinD: v0.29.0
  • Dapr:
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.3.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func3e26b0bed5
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func3e26b0bed5
  • dynamic-rp test image location: ghcr.io/radius-project/dev/dynamic-rp:pr-func3e26b0bed5
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func3e26b0bed5
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func3e26b0bed5
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

Copy link
Member

@brooke-hamilton brooke-hamilton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

Comment on lines +111 to +112
- name: Check out code
uses: actions/checkout@v5
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- name: Check out code
uses: actions/checkout@v5
- name: Check out code
uses: actions/checkout@v5
with:
submodules: recursive

| head -n1 \
| cut -d" " -f1
)
POD_NAME=$(kubectl get pods -n radius-system -o jsonpath='{range .items[*]}{.metadata.name}{" "}{.spec.containers[*].name}{"\n"}{end}' | grep "ucp" | head -n1 | cut -d" " -f1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for the formatting changes in this section and the removal of the code comments?

cache-dependency-path: go.sum
cache: true

- name: Pull images from GHCR or build locally (with fallback + controller check)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to be duplicated code from functional-test-cloud.yaml. Could it be consolidated into a script or action?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need the purge-old-images.yaml workflow if we are no longer putting PR images into GHCR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants