Skip to content

fix(ci): cache node_modules across runs to skip npm ci on an exact hit - #2446

Merged
JSONbored merged 1 commit into
mainfrom
fix/ci-node-modules-cache
Jul 2, 2026
Merged

fix(ci): cache node_modules across runs to skip npm ci on an exact hit#2446
JSONbored merged 1 commit into
mainfrom
fix/ci-node-modules-cache

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • actions/checkout's default clean: true runs git clean -ffdx before every checkout, wiping node_modules on every run even though the self-hosted "gittensory" runner containers are persistent VPS processes, not fresh machines. npm ci also unconditionally deletes and reinstalls node_modules by design regardless of what's already on disk. Together these meant node_modules got zero cross-run reuse no matter how long the runner container stays alive.
  • Added an actions/cache/restore + actions/cache/save pair (GitHub's own cache service, not local disk) around both the root npm ci and review-enrichment's separate npm ci (it isn't an npm workspace member, so it has its own package-lock.json and needs its own cache entry). The install step is skipped entirely when the restore is an exact hit; the save step is placed immediately after a successful install (not as an automatic post-job hook), so a job that fails installing never reaches it — a broken/partial node_modules can never get written to the cache for a future run to inherit.
  • Cache keys are scoped separately per fork-vs-trusted (mirroring the existing runs-on split): the self-hosted runner's Docker image and GitHub's ubuntu-latest are not guaranteed binary-compatible for native modules (sharp, workerd, fsevents, @sentry/cli all compile platform-specific binaries), so crossing them could load an incompatible binary. Fork PRs get read-only cache tokens (documented actions/cache behavior) and can never write a "fork"-keyed entry — they keep doing a full npm ci exactly as before, no regression on that no-retry population.
  • Adversarial review caught one real bug before this shipped: the first draft's cache key only hashed the lockfile, so a Node version bump (.nvmrc) with no lockfile change would still hit and silently reuse node_modules whose native addons were compiled against the OLD Node's ABI. Both keys now also hash .nvmrc.
  • Added test/unit/ci-dependency-cache.test.ts to pin the restore/skip/save wiring for both install paths, including the .nvmrc fix and that REES's build/test step still runs unconditionally (independent of whether install ran or was skipped this run).

Scope

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage — full unsharded run green on Node 22.23.1 (matching CI's pinned .nvmrc): 314 passed / 2 skipped, 5927 tests passed, 0 failures.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • The 5 existing guardrail tests that assert on ci.yml's structure (workflow-runner-labels, codecov-policy, change-guardrail, agent-guardrail-paths, path-matchers) all pass unchanged.
  • Verified node_modules layout empirically: a fresh root npm ci also creates apps/gittensory-ui/node_modules (a small, gitignored, npm-workspaces hoisting-conflict artifact) — both paths are included in the root cache entry.
  • Verified actions/cache's documented restore/save split semantics against its own README before implementing (not from memory): cache-primary-key output feeding save's key input is the officially documented pattern; fork-PR read-only cache tokens are documented behavior, not an assumption.
  • Ran an independent adversarial review pass against this exact diff — see Summary for the one real bug it caught (missing .nvmrc in the cache key), which is fixed and now has a dedicated assertion in the new test.
  • One transient failure surfaced in an early local test:coverage run (selfhost-sqlite-queue.test.ts, unrelated to this diff) while the machine was heavily loaded from parallel background work; reran that file in isolation (52/52 passed) and reran the full suite cleanly (0 failures) to confirm it was local system-load flakiness, not a regression.

If any required check was skipped, explain why:

  • N/A — everything ran.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • N/A — no auth, cookie, CORS, GitHub App, Cloudflare, or session changes.
  • N/A — no API/OpenAPI/MCP behavior change.
  • N/A — no UI code changes.
  • N/A — no visible UI change; this is CI-workflow-only.
  • N/A — no docs/changelog changes needed.

Notes

  • Follow-up from ci(gate): cut PR validation cost — cached test fixtures, correct worker count, narrower UI scope #2426 / fix(ci): stop regenerating the OpenAPI spec twice in the same UI build #2444 (found while auditing validate-code's remaining cost after both merged).
  • The first run against a given .nvmrc + lockfile combination will still be a full npm ci (cache miss, then save) — the skip is only observable on a second run against the same combination.
  • Out of scope here (considered and rejected/deferred, per the same audit): TypeScript incremental-build caching showed real but highly inconsistent local speedup (4.7x on one change, no benefit or worse on another, depending on which file changed) — not confident enough to ship; ESLint --cache was technically verified (~4x local speedup with cache-strategy: content) but the net win is marginal-to-negative once fork-PR risk and actions/cache's own overhead are weighed in.

actions/checkout's default clean:true runs `git clean -ffdx` before every
checkout, wiping node_modules on every run even though the self-hosted
"gittensory" runner containers are persistent VPS processes, not fresh
machines. npm ci also unconditionally deletes and reinstalls node_modules by
design regardless of what's on disk. Together these meant node_modules got
zero cross-run reuse no matter how long the runner container stays alive.

Add an actions/cache/restore + actions/cache/save pair (GitHub's own cache
service, not local disk) around both the root npm ci and review-enrichment's
separate npm ci (it isn't an npm workspace member, so it has its own
package-lock.json and needs its own cache entry). The install step is skipped
entirely when the restore is an exact hit; the save step is placed
immediately after a successful install (not as an automatic post-job hook),
so a job that fails installing never reaches it -- a broken/partial
node_modules can never get written to the cache for a future run to inherit.

Cache keys are scoped separately per fork-vs-trusted (mirroring the existing
runs-on split): the self-hosted runner's Docker image and GitHub's
ubuntu-latest are not guaranteed binary-compatible for native modules (sharp,
workerd, fsevents, @sentry/cli all compile platform-specific binaries), so
crossing them could load an incompatible binary. Fork PRs get read-only cache
tokens (documented actions/cache behavior) and can never write a "fork"-keyed
entry -- they keep doing a full npm ci exactly as before, no regression on
that no-retry population.

Adversarial review caught one real bug before this shipped: the first draft's
cache key only hashed the lockfile, so a Node version bump (.nvmrc) with no
lockfile change would still hit and silently reuse node_modules whose native
addons were compiled against the OLD Node's ABI. Both keys now also hash
.nvmrc.

Added test/unit/ci-dependency-cache.test.ts to pin the restore/skip/save
wiring for both install paths, including the .nvmrc fix and that REES's
build/test step still runs unconditionally (independent of whether install
ran or was skipped this run).
@loopover-orb

loopover-orb Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-02 01:37:33 UTC

2 files · 1 AI reviewer · no blockers · readiness 93/100 · CI pending · blocked

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review

Review summary
The workflow adds exact-hit node_modules caching for the root install and review-enrichment install, with keys that include the relevant lockfile and .nvmrc, then skips npm ci only when the cache action reports an exact hit. The install/save ordering is sensible and the added test locks down the key ingredients and REES gating. I do not see a reachable break in the visible diff, but the PR text overstates fork cache behavior and the test is mostly structural rather than proving cache service semantics.

Nits — 5 non-blocking
  • nit: .github/workflows/ci.yml:144 relies on the assumption that fork PRs cannot write cache entries, but GitHub Actions cache scoping is subtle enough that this should be verified in docs or softened because the workflow still executes the save step for fork PRs.
  • nit: test/unit/ci-dependency-cache.test.ts:41 only checks that the path string contains node_modules, so it would not catch accidental extra cache paths or removal of the root path formatting; asserting the parsed multiline path entries exactly would make the regression test tighter.
  • In .github/workflows/ci.yml:144, either add an explicit fork guard to the save steps or change the comment to describe the actual cache scope you have verified for pull_request runs.
  • In test/unit/ci-dependency-cache.test.ts:41, split the multiline cache path and assert the exact expected entries for root and UI node_modules.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ No-issue rationale PR body explains why no issue is linked.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (size label size:XS; no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 65 registered-repo PR(s), 55 merged, 563 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 65 PR(s), 563 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: not available
  • Official Gittensor activity: 65 PR(s), 563 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Triage stale or unlinked PRs.
  • No action.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added gittensor gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jul 2, 2026
@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.91%. Comparing base (fa6accb) to head (e1cd56a).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2446   +/-   ##
=======================================
  Coverage   95.91%   95.91%           
=======================================
  Files         224      224           
  Lines       25240    25240           
  Branches     9177     9177           
=======================================
  Hits        24210    24210           
  Misses        417      417           
  Partials      613      613           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored
JSONbored merged commit 3f3dc3b into main Jul 2, 2026
12 checks passed
@JSONbored
JSONbored deleted the fix/ci-node-modules-cache branch July 2, 2026 01:44
@github-project-automation github-project-automation Bot moved this from Todo to Done in gittensory - v1 roadmap Jul 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant