Skip to content

fix(github): surface a warning when CI completeness can't be verified - #2403

Merged
JSONbored merged 1 commit into
mainfrom
claude/ci-completeness-advisory
Jul 1, 2026
Merged

fix(github): surface a warning when CI completeness can't be verified#2403
JSONbored merged 1 commit into
mainfrom
claude/ci-completeness-advisory

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

What

When no branch-protection required contexts are resolvable (enforceRequiredOnly === false — common on repos without branch protection configured, or when the required-contexts fetch 403s), the only "absence" safety net in reduceLiveCiAggregate was missingConventionalValidateAggregate — hardcoded to this repo's own CI convention (a validate context gated behind changes/security/validate-code). For any other self-hosted repo with different job names, a workflow that never triggers for a given PR (path-filtered out, or a broken workflow YAML trigger) produces neither a check-run nor a check-suite entry — indistinguishable from "no such check exists" — and folds into "passed" as long as at least one other check ran and passed. A self-hosted maintainer expecting a non-required, path-filtered e2e-tests job to run on every PR would get no signal that it silently never ran.

Fix

Ships the issue's own "lower-effort interim mitigation" rather than the full generalized fix (a learned-or-.gittensory.yml-declared expected-workflow-set comparison), which the issue itself frames as a separate, larger follow-up.

  • Added a ciCompletenessWarning: string | null field to LiveCiAggregate. reduceLiveCiAggregate populates it when the aggregate resolves to "passed" with no branch-protection required contexts configured. ciState/disposition is never touched — forcing "pending" in this scenario would leave a self-hosted repo with no expected-checks config stuck waiting forever on a workflow that can structurally never complete (worse than the silent-pass it replaces).
  • Since gittensory's own convention (missingConventionalValidateAggregate) already forces "pending" — not "passed" — when its aggregate check is legitimately missing, the new warning can never fire for a properly-configured gittensory-convention repo; it's mathematically orthogonal to the existing mechanism, not a replacement for it.
  • The main planner (maybeRunAgentMaintenance in src/queue/processors.ts) records the warning as an informational github_app.ci_completeness_unverified audit event (outcome: "completed", never "denied") — a nudge for the operator, not a gate blocker.

Tests

  • Fixed one pre-existing test whose full-object toEqual assertion broke at runtime from the new field (not caught by tsc since expect().toEqual() isn't type-checked against the exact shape).
  • 3 new direct tests on fetchLiveCiAggregate: the warning is set when CI resolves to passed with no required contexts (workflow A passes, workflow B never appears at all); the warning is absent when required contexts ARE configured; the warning is absent when ciState is anything other than "passed".
  • 2 new integration tests via processJob/agent-regate-pr: the audit event fires with the expected detail when the mocked aggregate carries a warning; it does NOT fire when the aggregate carries none.
  • npx tsc --noEmit clean — widening LiveCiAggregate's required fields surfaced 5 test-side mock objects needing the new field via the compiler.
  • Scoped: backfill.test.ts — 143 passed; full queue.test.ts — 204 passed.
  • Regression sweep: graphql-status-rollup.test.ts (the other reduceLiveCiAggregate call path) — 30 passed.
  • Diff-range coverage-gap check on both changed source files: fully covered (two genuinely-defensive fallbacks marked /* v8 ignore next */ with a verified, non-hand-wavy invariant justification — ciCompletenessWarning is only ever set when ciState === "passed", which itself requires a truthy headSha).
  • Full unsharded npm run test:coverage: 5605 passed, 4 skipped (pre-existing/unrelated), 0 failed.
  • npm audit --audit-level=moderate: 0 vulnerabilities.

Advances #1936. Closes #2137.

reduceLiveCiAggregate's only "absence" safety net for a workflow that
never triggers at all (path-filtered out, or a broken YAML trigger —
producing neither a check-run nor a check-suite entry) was
missingConventionalValidateAggregate, hardcoded to this repo's own
validate/changes/security/validate-code convention. For any other
self-hosted repo without branch protection configured, a workflow
that silently never starts is indistinguishable from one that doesn't
exist, and folds into "passed" as long as some OTHER check ran and
passed.

Add a ciCompletenessWarning field to LiveCiAggregate, populated when
the aggregate resolves to "passed" with no branch-protection required
contexts configured (enforceRequiredOnly false). This never changes
ciState or disposition — forcing "pending" would leave a self-hosted
repo with no expected-checks config stuck waiting forever on a
workflow that can structurally never complete. The main planner
records it as an informational audit event
(github_app.ci_completeness_unverified) so operators can notice and
configure branch protection or an expected-checks list.

The full fix (a generic, learned-or-configured expected-workflow-set
comparison) is a separate, larger follow-up; this ships the "lower
effort interim mitigation" the issue calls out as sufficient ahead of
that.
@dosubot dosubot Bot added the size:M label Jul 1, 2026
@loopover-orb

loopover-orb Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-01 20:41:48 UTC

4 files · 1 AI reviewer · no blockers · readiness 75/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review

Review summary
The change threads an informational CI-completeness warning through the live aggregate and records it from the maintenance processor without changing the aggregate disposition. The core condition is scoped to the intended case: only a passed aggregate with required-context enforcement disabled gets the warning, while unverified, failed, pending, and required-context-backed results stay unchanged. The visible tests cover the new reducer behavior and the processor audit side effect, so the diff is safe enough to proceed.

Nits — 6 non-blocking
  • nit: src/queue/processors.ts:1627 records the audit event every time the same PR/head is reprocessed with the warning, so consider deduping by event type, targetKey, and headSha if audit volume matters.
  • nit: src/github/backfill.ts:2343 tells operators to configure an expected-checks list, but this PR description says that config-driven comparison is a follow-up; make sure the message only recommends mechanisms that exist today.
  • src/queue/processors.ts:1627: consider making the audit write idempotent per PR head so repeated maintenance passes do not create identical informational rows.
  • src/github/backfill.ts:2343: either shorten the warning or move the text to a shared constant so tests and future UI/log surfaces do not drift.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #2137
Related work ⚠️ 2 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (size label size:M; 1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 67 registered-repo PR(s), 57 merged, 589 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 67 PR(s), 589 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Review context
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Review top overlaps.
  • Add a concise scope and risk note.
  • Triage stale or unlinked PRs.
  • No action.
  • Check active issues and PRs before submitting.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added gittensor gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jul 1, 2026
@codecov

codecov Bot commented Jul 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.73%. Comparing base (385990b) to head (10891d8).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2403   +/-   ##
=======================================
  Coverage   95.73%   95.73%           
=======================================
  Files         223      223           
  Lines       24755    24758    +3     
  Branches     8970     8972    +2     
=======================================
+ Hits        23699    23702    +3     
  Misses        433      433           
  Partials      623      623           
Files with missing lines Coverage Δ
src/github/backfill.ts 96.15% <100.00%> (+<0.01%) ⬆️
src/queue/processors.ts 90.23% <100.00%> (+0.01%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored JSONbored self-assigned this Jul 1, 2026
@JSONbored
JSONbored merged commit 3a94b10 into main Jul 1, 2026
12 checks passed
@JSONbored
JSONbored deleted the claude/ci-completeness-advisory branch July 1, 2026 22:11
@github-project-automation github-project-automation Bot moved this from Todo to Done in gittensory - v1 roadmap Jul 1, 2026
JSONbored added a commit that referenced this pull request Jul 1, 2026
…2427)

#2403 referenced args.deliveryId on a plan-and-execute context that never
declared or received the field, breaking typecheck on main. Thread it
through from the caller and backfill the ciCompletenessWarning field on
existing LiveCiAggregate test mocks that predated that type change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

fix(github): a non-required check that never appears is silently treated as CI-passed

1 participant