Feature request: snapshot-aware tool setup for standard GitHub Actions

[garysassano]

Summary

I have been testing runs-on/snapshot@v1 as a way to preserve Rust/Cargo build state between ephemeral CI runners. The snapshot approach works very well for Cargo build artifacts, but there is another source of repeated overhead: normal setup actions often install tools outside the snapshot path.

Examples include actions that install or configure tools such as:

Rust toolchain / rustup components
Zig
cargo-lambda
Node / pnpm / other language toolchains

For the snapshot approach to be most useful, it would be helpful to have a more elegant way to keep using normal GitHub Actions while having the files those actions create land inside the restored snapshot automatically.

The goal is not to persist these tool installs through GitHub Actions Cache or RunsOn Magic Cache. The goal is to keep using the convenient setup actions that already exist, but make their side effects snapshot-backed.

Right now, the reliable workaround is to stop using some setup actions and replace them with manual install logic that writes into the snapshot root. That works, but it makes the workflow less idiomatic and more fragile.

Problem

The build-state snapshot root might look like this:

/mnt/build-snapshot/workspace
/mnt/build-snapshot/cargo-home
/mnt/build-snapshot/tools
/mnt/build-snapshot/tool-cache
/mnt/build-snapshot/zig-cache

But many standard setup actions install elsewhere, for example:

/opt/hostedtoolcache
/home/runner/.cargo
/home/runner/.install-action/bin
/home/runner/_work/<repo>/<repo>/.zig-cache
/home/runner/.cache

Those locations are outside the snapshot path, so the work is repeated on every fresh runner. Some actions may also use GitHub Actions Cache backed by Azure Storage or RunsOn Magic Cache backed by S3/R2 when configured that way, but that is still an archive/cache layer separate from the restored filesystem snapshot. For this use case, I want the tool installation itself to become part of the same block-level build-state snapshot.

In testing, the actual Cargo build became almost a no-op after restoring the snapshot, but setup actions still consumed meaningful time because they were doing work outside the snapshot.

flowchart TD
    start[Fresh ephemeral runner]
    snap[Restore build-state snapshot]
    cargo[Cargo build state restored]
    setup[Setup actions install tools]
    outside[Tools installed outside snapshot]
    repeat[Repeated work every run]
    build[Build is fast, setup remains slow]

    start --> snap --> cargo --> build
    start --> setup --> outside --> repeat --> build

Loading

Why this matters

For Cargo/Rust workloads, block-level snapshots can preserve the expensive build state:

target/
fingerprints
dep-info files
incremental state
build script outputs
Cargo registry source dirs

Once that is solved, the next visible bottleneck becomes setup work:

installing Zig
installing cargo-lambda
installing Rust components / targets
restoring or extracting tool caches

It is possible to solve this manually by installing tools into the snapshot root, but that means losing the convenience and maintenance benefits of existing setup actions.

Current workaround

Instead of using normal setup actions for some tools, the workflow can manually install tools into the snapshot root:

env:
  SNAPSHOT_ROOT: /mnt/build-snapshot
  SNAPSHOT_WORKSPACE: /mnt/build-snapshot/workspace
  CARGO_HOME: /mnt/build-snapshot/cargo-home
  CARGO_TARGET_DIR: /mnt/build-snapshot/workspace/app/target
  ZIG_GLOBAL_CACHE_DIR: /mnt/build-snapshot/zig-cache/global
  ZIG_LOCAL_CACHE_DIR: /mnt/build-snapshot/zig-cache/local

steps:
  - uses: runs-on/snapshot@v1
    with:
      path: ${{ env.SNAPSHOT_ROOT }}
      version: build-state-v1
      volume_size: 10

  - name: Configure snapshot paths
    run: |
      mkdir -p "$SNAPSHOT_WORKSPACE" "$CARGO_HOME" "$SNAPSHOT_ROOT/tools"
      echo "$CARGO_HOME/bin" >> "$GITHUB_PATH"
      echo "$SNAPSHOT_ROOT/tools/zig" >> "$GITHUB_PATH"
      echo "$SNAPSHOT_ROOT/tools/cargo-lambda" >> "$GITHUB_PATH"

  - name: Install Zig into snapshot
    run: |
      if [ ! -x "$SNAPSHOT_ROOT/tools/zig/zig" ]; then
        # download and extract Zig into the snapshot path
        true
      fi

  - name: Install cargo-lambda into snapshot
    run: |
      if [ ! -x "$SNAPSHOT_ROOT/tools/cargo-lambda/cargo-lambda" ]; then
        # download and extract cargo-lambda into the snapshot path
        true
      fi

This works, but it replaces maintained setup actions with custom download/extract logic.

Desired workflow shape

Ideally, a workflow could still use normal setup actions:

steps:
  - uses: runs-on/snapshot@v1
    with:
      path: /mnt/build-snapshot
      key: build-state-${{ matrix.target }}

  - uses: runs-on/snapshot-toolcache@v1
    with:
      path: /mnt/build-snapshot

  - uses: dtolnay/rust-toolchain@stable
    with:
      targets: aarch64-unknown-linux-gnu

  - uses: mlugg/setup-zig@v2

  - uses: taiki-e/install-action@v2
    with:
      tool: cargo-lambda

And the RunsOn integration would make those actions write their installed tools and caches into snapshot-backed locations, or at least make the standard paths point into the snapshot. From the workflow author's perspective, the setup actions would stay normal and convenient; from the filesystem perspective, their outputs would be persisted by the snapshot rather than by GitHub Actions Cache on Azure Storage or RunsOn Magic Cache on S3/R2.

Possible approaches

Option 1: Snapshot-aware toolcache helper

A helper action could configure standard tool/cache environment variables before setup actions run:

- uses: runs-on/snapshot-toolcache@v1
  with:
    snapshot-path: /mnt/build-snapshot
    tool-cache-path: /mnt/build-snapshot/tool-cache
    home-cache-path: /mnt/build-snapshot/home-cache

It could set or prepare common paths such as:

RUNNER_TOOL_CACHE
CARGO_HOME
RUSTUP_HOME
ZIG_GLOBAL_CACHE_DIR
ZIG_LOCAL_CACHE_DIR
HOME-based cache directories where safe

This would let many setup actions continue using their normal logic while the resulting installs land in the snapshot instead of relying on a separate cache backend.

flowchart LR
    snapshot[Restore snapshot]
    helper[Configure snapshot-aware tool paths]
    setup[Normal setup actions]
    tools[Tools installed inside snapshot]
    later[Later run restores tools]

    snapshot --> helper --> setup --> tools --> later

Loading

Option 2: Mount or symlink common tool paths

RunsOn could provide a supported way to map standard runner paths into the snapshot volume:

/opt/hostedtoolcache -> /mnt/build-snapshot/tool-cache
/home/runner/.cargo -> /mnt/build-snapshot/cargo-home
/home/runner/.install-action -> /mnt/build-snapshot/install-action
/home/runner/.cache -> /mnt/build-snapshot/home-cache

This could be opt-in, because redirecting broad home directories may be risky. A narrowly scoped mapping for known tool directories would be safer.

Example:

- uses: runs-on/snapshot@v1
  with:
    path: /mnt/build-snapshot
    key: build-state-${{ matrix.target }}
    map-tool-cache: true
    map-cargo-home: true

Option 3: A documented recipe for standard setup actions

Even if this is not implemented as a new action, documentation could provide a blessed recipe:

env:
  SNAPSHOT_ROOT: /mnt/build-snapshot
  CARGO_HOME: /mnt/build-snapshot/cargo-home
  RUSTUP_HOME: /mnt/build-snapshot/rustup-home
  ZIG_GLOBAL_CACHE_DIR: /mnt/build-snapshot/zig-cache/global
  ZIG_LOCAL_CACHE_DIR: /mnt/build-snapshot/zig-cache/local

Then explain which popular actions respect which environment variables, and which actions do not.

For example:

dtolnay/rust-toolchain: respects CARGO_HOME/RUSTUP_HOME behavior through rustup/cargo
setup-zig: supports Zig cache env vars but may still use tool-cache behavior
install-action: may install into ~/.install-action unless configured otherwise

Useful outputs or diagnostics

It would also be helpful if the helper could print a summary like:

Snapshot-aware tool paths:
  RUNNER_TOOL_CACHE=/mnt/build-snapshot/tool-cache
  CARGO_HOME=/mnt/build-snapshot/cargo-home
  RUSTUP_HOME=/mnt/build-snapshot/rustup-home
  ZIG_GLOBAL_CACHE_DIR=/mnt/build-snapshot/zig-cache/global
  ZIG_LOCAL_CACHE_DIR=/mnt/build-snapshot/zig-cache/local

Detected tool installs:
  zig: restored from snapshot
  cargo-lambda: restored from snapshot
  rust target aarch64-unknown-linux-gnu: restored from snapshot

This would make it obvious whether setup work is being reused or repeated.

Security considerations

This feature would need to avoid accidentally snapshotting secrets.

For Cargo, credentials commonly live in:

$CARGO_HOME/credentials
$CARGO_HOME/credentials.toml
$CARGO_HOME/config.toml

If CARGO_HOME is inside the snapshot, workflows need a clean way to remove credentials before save, or the snapshot action/helper could support excludes/scrubbing hooks.

Potential API:

with:
  exclude-before-save: |
    /mnt/build-snapshot/cargo-home/credentials
    /mnt/build-snapshot/cargo-home/credentials.toml
    /mnt/build-snapshot/cargo-home/config.toml

Or:

with:
  scrub-cargo-credentials: true

Why this is separate from normal cache actions

This is not trying to replace actions/cache or Magic Cache.

The use case is specifically:

I already restored a block-level filesystem snapshot.
I want normal setup actions to populate that same filesystem snapshot.
I want later runs to skip repeated tool downloads/extractions.

Archive cache actions are still useful for many dependency caches, but for build-state reuse the goal is to preserve one coherent filesystem state.

Desired end state

The ideal result is that a workflow can keep using maintained setup actions while RunsOn handles the “make this setup state snapshot-backed” part:

- uses: runs-on/snapshot@v1
  with:
    path: /mnt/build-snapshot
    key: build-state-${{ matrix.target }}

- uses: runs-on/snapshot-toolcache@v1
  with:
    path: /mnt/build-snapshot

- uses: normal/tool-setup-action@v1

- run: build-command

That would preserve the main benefit of the snapshot approach without requiring custom install scripts for every tool.