Skip to content

fix(docker): document a proven CPU-priority pattern for --profile runners - #2449

Merged
JSONbored merged 2 commits into
mainfrom
fix/runner-cpu-priority-override
Jul 2, 2026
Merged

fix(docker): document a proven CPU-priority pattern for --profile runners#2449
JSONbored merged 2 commits into
mainfrom
fix/runner-cpu-priority-override

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • A box running --profile runners (self-hosted GitHub Actions runners) alongside the main gittensory app has no CPU limits set by default — every container competes for the host's CPUs on equal footing. Under a burst of CI jobs, this can starve the app itself of the CPU it needs to do the review work it exists for.
  • This exact pattern already bit us in production on an 8-vCPU box: uncapped runner containers left the app starved under load. It was fixed live via docker update --cpu-shares/--cpus, but that tuning only ever existed as an uncommitted host file (docker-compose.override.yml, never committed) and shell history — invisible to anyone else running this pattern, and lost if the box were ever rebuilt from the repo alone.
  • Added docker-compose.override.yml.example (a real docker-compose.override.yml stays gitignored and operator-owned, mirroring the existing .env/.env.example split) documenting the fix: cpu_shares is a relative weight that only matters once the host is genuinely contended — setting the app's shares high and the runner's low means review work wins a race for CPU when both want it at the same instant, without ever blocking CI from running merely because the app exists. cpus is a separate, absolute per-container ceiling; the file's own comments include a sizing guide for a different host/replica count.
  • Added a one-line pointer to this file from docker-compose.yml's own runner: service comment block, where anyone enabling that profile would already be looking.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused: docs/config for the self-host Docker stack + a matching guardrail test, no product code changes.
  • This follows CONTRIBUTING.md.
  • No issue is linked — this is infra/ops documentation captured from a production incident already resolved live, not a pre-filed bug.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage — full unsharded run green on Node 22.23.1 (matching CI's pinned .nvmrc): 315 passed / 2 skipped, 5930 tests passed, 0 failures.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • docker compose -f docker-compose.yml -f docker-compose.override.yml.example config --quiet — validates the merged config parses with no errors (confirms cpus/cpu_shares are valid top-level compose-spec keys honored outside Swarm mode, not the Swarm-only deploy.resources.limits block).
  • The 3 existing tests that read the real docker-compose.yml (selfhost-image-deploy.test.ts, selfhost-observability-config.test.ts, path-matchers.test.ts) all pass unchanged.

If any required check was skipped, explain why:

  • N/A — everything ran.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed. The example file contains only generic resource-limit numbers, no host-specific values beyond what's already documented in this PR's own summary.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • N/A — no auth, cookie, CORS, GitHub App, Cloudflare, or session changes.
  • N/A — no API/OpenAPI/MCP behavior change.
  • N/A — no UI code changes.
  • N/A — no visible UI change.
  • N/A — no docs/changelog changes needed beyond the in-file comments this PR adds.

Notes

…ners

A box running --profile runners (self-hosted GitHub Actions runners)
alongside the main gittensory app has no CPU limits set by default -- every
container competes for the host's CPUs on equal footing. Under a burst of CI
jobs, this can starve the app itself of the CPU it needs to do the review
work it exists for. Discovered running exactly this pattern in production on
an 8-vCPU box: uncapped runner containers left the app starved under load,
fixed live via `docker update --cpu-shares`/`--cpus`, but that tuning only
ever existed as an uncommitted host file and shell history -- lost if the box
were ever rebuilt, and invisible to anyone else running this same pattern.

Add docker-compose.override.yml.example (gitignored `docker-compose.override.yml`
stays the operator's actual file, mirroring the existing .env/.env.example
split) documenting the fix: cpu_shares is a relative weight that only matters
once the host is genuinely contended, so setting the app's shares high and
the runner's low means review work wins a race for CPU when both want it at
the same instant, without ever blocking CI from running merely because the
app exists. cpus is a separate, absolute per-container ceiling, sized to the
sizing guide in the file's own comments for a different host/replica count.

Verified `docker compose config` (merging both files) accepts the syntax with
no errors. Added test/unit/docker-compose-override-example.test.ts (pure YAML
parsing, no `docker` CLI invocation -- the self-hosted runner container this
would actually run on doesn't have Docker-in-Docker access, so shelling out
to `docker compose config` inside a test would be unreliable) to pin: the
example's service names match the real docker-compose.yml, the app's
cpu_shares is set higher than the runner's, and .gitignore keeps a real
override file out of version control while tracking the example.
@dosubot dosubot Bot added the size:M label Jul 2, 2026
@loopover-orb

loopover-orb Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-02 03:37:55 UTC

4 files · 1 AI reviewer · no blockers · readiness 86/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This PR adds operator-facing Docker Compose guidance for shared-host runner CPU prioritization and backs the example file with structural tests. The visible change is coherent with the PR description, keeps the real override operator-owned via .gitignore, and does not change runtime behavior unless someone explicitly copies the example into place. I do not see a reachable breaking defect in the diff.

Nits — 5 non-blocking
  • nit: docker-compose.override.yml.example:22 documents an aggressive 8:1 shares ratio, but the example does not mention that `cpu_shares` behavior can vary under Docker Desktop / non-Linux backends, so operators on those environments may over-trust the tuning.
  • nit: test/unit/docker-compose-override-example.test.ts:11 duplicates the YAML-reading helper pattern already present in selfhost-observability-config.test.ts; consider extracting a tiny shared test helper if more Compose structural tests are expected.
  • nit: .gitignore:15 adds `!docker-compose.override.yml.example`, but the preceding ignore pattern is the exact filename `docker-compose.override.yml`, so the negation is mostly documentary rather than required.
  • docker-compose.override.yml.example:22: add one short comment that this tuning is intended for Docker Engine/Linux hosts, since that is the production scenario described by the file.
  • test/unit/docker-compose-override-example.test.ts:20: add an assertion that the base `runner` service still has no committed `cpu_shares`/`cpus`, which would protect the documented default from drifting.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (size label size:M; no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 65 registered-repo PR(s), 55 merged, 554 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 65 PR(s), 554 issue(s).
Gate result ✅ Passing No configured blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: not available
  • Official Gittensor activity: 65 PR(s), 554 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Triage stale or unlinked PRs.
  • No action.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added gittensor gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jul 2, 2026
@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.91%. Comparing base (38647fc) to head (ea283b5).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2449   +/-   ##
=======================================
  Coverage   95.91%   95.91%           
=======================================
  Files         224      224           
  Lines       25235    25235           
  Branches     9177     9177           
=======================================
  Hits        24205    24205           
  Misses        417      417           
  Partials      613      613           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…u_shares

The prior commit's test only protected the relative cpu_shares contract
(app > runner) -- the documented cpus: "4.0" ceiling had no assertion at all,
so it could silently drift from the example's own sizing guide with nothing
to catch it. Verified this addition actually catches a regression: dropping
it to "2.0" fails the new assertion before restoring the real value.
@JSONbored
JSONbored merged commit 2a4e7bd into main Jul 2, 2026
13 checks passed
@JSONbored
JSONbored deleted the fix/runner-cpu-priority-override branch July 2, 2026 04:32
@github-project-automation github-project-automation Bot moved this from Todo to Done in gittensory - v1 roadmap Jul 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant