Skip to content

ci: reuse cached signed/notarized native binaries on release#775

Merged
theoephraim merged 3 commits into
mainfrom
ci/reuse-cached-native-binaries
Jun 12, 2026
Merged

ci: reuse cached signed/notarized native binaries on release#775
theoephraim merged 3 commits into
mainfrom
ci/reuse-cached-native-binaries

Conversation

@theoephraim

Copy link
Copy Markdown
Member

Problem

The release workflow rebuilt and re-notarized the macOS / Rust native encryption binaries on every publish, even when nothing under the native source changed. The three native jobs were gated only on mode == 'publish' && includes-varlock == 'true' — never on whether the binary-relevant source actually changed.

The Apple notarization round-trip (xcrun notarytool submit --wait) is the worst offender: it's slow, externally dependent, and occasionally flaky, and re-running it for a byte-identical binary is pure waste.

What this does

Gates the build-native-macos, notarize-native-macos, and build-native-rust jobs on a content-addressed cache probe in the plan job. On a cache hit, the build + notarize jobs are skipped and the previously notarized bundle / Rust binaries are reused.

  • Cache key = content hash of the native source + build/bundle scripts + the workflows that drive them, plus a manual v1 salt to force a fleet-wide rebuild on toolchain bumps the file hash can't see (runner image / Xcode / Rust / UPX).
  • No git diff change-detection — that's empty on main. The cache key is the change detector.
  • The macOS .app is already version-independent in CI (the release path doesn't pass --version, so the bundle embeds a constant placeholder, not the npm version), which is what makes the notarized bundle byte-stable across releases. Documented inline.

"Never publish the wrong thing" safeguards

This was the explicit design constraint. Layers:

  1. Content-addressed keys — a restored binary always corresponds to the source that produced it.
  2. Cache scope — a release on main can only restore caches produced by prior main runs; feature-branch caches aren't visible. Combined with branch protection, that's the anti-poisoning boundary.
  3. Re-verification before publish — a reused macOS bundle is re-validated on a macOS runner (codesign --verify --deep --strict + xcrun stapler validate + a functional status smoke) in a new verify-native-macos job. If it can't be verified, the release is blocked (!failure()).
  4. Fail-loud presence check — the publish job aborts if any native binary is missing before building the npm package.
  5. fail-on-cache-miss: true on the publish-side restores, so a mid-run eviction blocks rather than silently shipping an incomplete package.

Rollout

The first release after this lands sees no cache under the new v1 keys, so it rebuilds + notarizes once to seed the cache. Subsequent releases with unchanged native source reuse it (skipping the macOS build + the Apple notarization round-trip, and the Rust matrix).

Compatibility

  • notarize-native-macos.yaml gains an optional cache-key input (default empty); all caching steps are gated on it being set. The other caller (binary-release.yaml) doesn't pass it and is unaffected.
  • build-native-rust.yaml is unchanged — the release path just passes a salted, comprehensive source-hash into the existing source-hash input.

Notes / caveats for review

  • The reused-bundle reverification is the key thing to scrutinize: it relies on the .app round-tripping through actions/cache with its signature + stapled ticket intact. Those are regular files in the bundle (not xattrs), and test.yaml already round-trips the signed .app through cache today, so the path is exercised — but the verify-native-macos job exists precisely to catch it if a future cache/runner change breaks it.
  • No changeset added — this is a CI-workflow-only change, which per AGENTS.md does not require a bump file.
  • Not runnable locally; correctness needs to be observed on the first real release (watch that it seeds the cache, and the one after reuses + verifies).

The release workflow rebuilt and re-notarized the macOS/Rust native
encryption binaries on every publish, even when nothing under the native
source changed. The Apple notarization round-trip in particular is slow
and externally dependent, and re-running it for a byte-identical binary
is wasted work.

Gate the native build/notarize/Rust jobs on a content-addressed cache
probe. The cache key covers the native source, the build/bundle scripts,
and the workflows that drive them, plus a manual 'v1' salt for
toolchain bumps the file hash can't see. On a cache hit the build and
notarize jobs are skipped and the previously notarized bundle is reused.

To never publish the wrong thing:
- keys are content-addressed, so a restored binary always matches the
  source that produced it
- caches restored on a release are main-scoped only (feature-branch
  caches are not visible), so a release can only reuse a binary a prior
  main run produced
- a reused macOS bundle is re-verified on a macOS runner (codesign +
  notarization staple + functional smoke) before the publish step; if it
  can't be verified the release is blocked
- the publish job fails loudly if any native binary is missing

First release after this lands rebuilds once to seed the cache, then
subsequent releases with unchanged native source reuse it.
The macOS .app reuse relied on the bundle being byte-identical across
releases, but that was only true by accident: release.yaml happened not
to pass a --version, so the bundle fell back to a constant default.
Anyone wiring a real version into that call would have silently defeated
the notarized-bundle cache with no error.

Make it stable by construction. CFBundleVersion is vestigial — nothing
reads it (Swift binary, Rust binary, varlock CLI, notarization) — so
hardcode it to a fixed constant in build-swift.ts and remove the
--version plumbing from the macOS build workflow and its callers. The
bundle is now byte-stable in every path, not just the release one.

First release after this rebuilds + re-notarizes once (the build script
hash changed), then reuse kicks in.
The bundle version is now a fixed constant (build-swift.ts), so the
binary-release path no longer threads a version into the macOS build.
That left the debug job's version output unused and a vestigial
'needs: debug' on build-native-macos. Remove both; the context-printing
debug step stays. The re-release flow (workflow_dispatch version input ->
RELEASE_VERSION -> release upload / homebrew / docker) is unchanged.
@theoephraim theoephraim merged commit 8a4a91d into main Jun 12, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant