feat(miner): add structured reviewer-consensus calibration signal

[galuis116]

Summary

Adds a new pure engine module, packages/gittensory-engine/src/reviewer-consensus-calibration.ts, in the same opt-in calibration family as objective-anchor, pairwise-judge, gate-verdict, and finding-severity calibration. It gives the miner replay harness a structured, default-off signal for how well a review's verdicts agree across independent reviewers.

When a review runs more than one independent reviewer (multiple models, or the same model sampled multiple times), each reviewer casts a per-dimension verdict (pass / warn / fail). This module ingests, per replayed PR, the set of votes per gate dimension and scores the consensus:

• Per dimension, unrecognized/abstention votes are dropped, the rest are tallied, the plurality outcome is chosen (ties broken toward the more severe outcome so a genuine split never rounds a real fail/warn down to pass), and the agreement fraction is the plurality's share of the definite votes.
• The per-PR score is the vote-count-weighted mean of the per-dimension agreements, so a dimension reviewed by more reviewers carries more weight than one seen by a single reviewer.
• It composes with the objective-anchor and pairwise signals via computeReviewerConsensusCompositeCalibrationScore, renormalizing weights when a signal is absent.

It complements the pairwise judge (which measures the order-stability of a single judge) by measuring agreement across independent reviewers, and it is distinct from gate-verdict calibration (which scores the outcome) and finding-severity calibration (which scores per-tier confirmation). Exactly like the sibling signals, the type surface is deliberately narrow — repo/run ids and per-dimension votes only. No raw review text, secrets, trust scores, reward values, private rankings, or maintainer evidence is represented, and renderReviewerConsensusCalibrationAuditMarkdown Markdown-escapes and newline-collapses every caller-supplied id.

Public API (exported from the package barrel)

• resolveReviewerConsensusCalibrationConfig(manifest) — default-off opt-in from miner.calibration.shareStructuredReviewerConsensus (with a top-level calibration.* alias) plus an optional non-negative weight; malformed values fail closed with warnings.
• ingestReviewerConsensusCalibrationSignals(signals) — validates + normalizes opted-in signals, rejecting with a specific reason (not_opted_in / empty_dimensions / invalid_repo / invalid_run_id); aggregates repeated dimensions and normalizes dimension + vote aliases.
• computeReviewerConsensusCompositeCalibrationScore(input) — blends objective-anchor + pairwise + reviewer-consensus (numbers or the sibling score objects), renormalizing weights and falling back to objective-only when all weights are zero.
• renderReviewerConsensusCalibrationAuditMarkdown(result) — deterministic, public-safe Markdown.

This is deliberately default-off and safe to call at ingestion time. No app routes, deploy config, or existing modules change — only the new module, its barrel export, its README.md section, and its unit test.

No linked issue: it sits in the established calibration-signal family and is a self-contained, additive, default-off engine module whose rationale is fully described here (issue creation on this repo is collaborator-only).

Scope

• The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
• This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
• This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
• I linked an issue, or this is small enough that the summary explains why an issue is not needed.

Validation

• git diff --check
• npm run actionlint
• npm run typecheck
• npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
• npm run test:workers
• npm run build:mcp
• npm run test:mcp-pack
• npm run ui:openapi:check
• npm run ui:lint
• npm run ui:typecheck
• npm run ui:build
• npm audit --audit-level=moderate
• New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

• The full npm run test:ci aggregate ran green (including build:miner, which compiles the engine src/ — the new module and barrel export type-check clean — and typecheck) plus npm audit --audit-level=moderate (0 vulnerabilities). A dedicated node:test suite (packages/gittensory-engine/test/reviewer-consensus-calibration.test.ts, mirroring the sibling calibration tests) covers config resolution, ingestion/rejection reasons, dimension + vote normalization, plurality tie-breaking, agreement weighting, composite weight renormalization, and the Markdown renderer. The module lives under packages/ (outside the root src/** Codecov scope), so the 99% patch rule does not apply to it.

Safety

• No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
• Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
• Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
• API/OpenAPI/MCP behavior is updated and tested where needed.
• UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
• Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
• Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

The module's type surface is deliberately private-data-free (repo/run ids + per-dimension votes only), it is default-off (ingestion requires an explicit shareStructuredReviewerConsensus: true opt-in), and the renderer escapes and newline-collapses all caller-supplied ids. Pure and deterministic: no auth/CORS/session surface, no API/OpenAPI shape change, no UI.

Notes

• New file reviewer-consensus-calibration.ts + its barrel export + a README.md section + a node:test suite. No schema, migration, OpenAPI, wrangler, or generated-artifact impact.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[loopover-orb]

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review result - reject/close recommended

_{Review updated: 2026-07-05 06:07:00 UTC}

4 files · 1 AI reviewer · 1 blocker · readiness 80/100 · CI green · clean

🛑 Suggested Action - Reject/Close

• AI reviewers agree on a likely critical defect: packages/gittensory-engine/src/reviewer-consensus-calibration.ts:314 makes the later objective-only fallback at line 482 unreachable for `weights: { objectiveAnchor: 0, pairwiseJudge: 0, structuredReviewerConsensus: 0 }`, so a caller explicitly disabling every component gets the default 0.45/0.35/0.2 blend instead of the documented/test-named fallback
• change the zero-total branch to preserve zero weights so line 482 can handle it, e.g. `if (total <= 0) return { objectiveAnchor: 0, pairwiseJudge: 0, structuredReviewerConsensus: 0 }
• `, or change the API/test/docs to explain why all-zero means defaults. — Resolve the flagged defect, or override if the AI reviewers are mistaken, then re-run the gate.

Review summary
This adds a self-contained reviewer-consensus calibration module with opt-in config parsing, signal ingestion, composite scoring, audit rendering, barrel exports, tests, and README docs. The main scoring path matches the stated per-dimension plurality/tie-break and vote-count-weighted per-PR agreement behavior, and the Markdown rendering consistently escapes caller-supplied identifiers. I found one reachable custom-weight handling bug around the all-zero case; otherwise the new surface is well covered for normal ingestion, aliasing, rejection, and rendering paths.

Blockers

• packages/gittensory-engine/src/reviewer-consensus-calibration.ts:314 makes the later objective-only fallback at line 482 unreachable for `weights: { objectiveAnchor: 0, pairwiseJudge: 0, structuredReviewerConsensus: 0 }`, so a caller explicitly disabling every component gets the default 0.45/0.35/0.2 blend instead of the documented/test-named fallback; change the zero-total branch to preserve zero weights so line 482 can handle it, e.g. `if (total <= 0) return { objectiveAnchor: 0, pairwiseJudge: 0, structuredReviewerConsensus: 0 };`, or change the API/test/docs to explain why all-zero means defaults.

Nits — 5 non-blocking

• nit: packages/gittensory-engine/src/reviewer-consensus-calibration.ts:202 maps `comment` to `warn`, but the module says abstention/unrecognized votes are dropped; if upstream reviewer verdicts can use `comment` as an abstention, this will inflate disagreement instead of dropping it, so either remove that alias or document why `comment` is a definite warning verdict here.
• nit: packages/gittensory-engine/test/reviewer-consensus-calibration.test.ts:291 says the all-zero custom-weight case falls back to objective-only, but the assertion only checks a loose range and would not catch the current default-weight behavior; assert the exact score/weights for that branch.
• packages/gittensory-engine/src/reviewer-consensus-calibration.ts:307 should let all-zero normalized weights flow through as zeros, then rely on the existing objective-only fallback at line 482.
• packages/gittensory-engine/test/reviewer-consensus-calibration.test.ts:291 should assert `allZero.weights` equals `{ objectiveAnchor: 1, pairwiseJudge: 0, structuredReviewerConsensus: 0 }` and `allZero.compositeScore === 0.4` if objective-only is the intended contract.
• packages/gittensory-engine/src/reviewer-consensus-calibration.ts:202 should either treat `comment` as unrecognized/abstention or add a test proving that upstream emits it only for definite warn votes.

Why this is blocked

• packages/gittensory-engine/src/reviewer-consensus-calibration.ts:314 makes the later objective-only fallback at line 482 unreachable for `weights: { objectiveAnchor: 0, pairwiseJudge: 0, structuredReviewerConsensus: 0 }`, so a caller explicitly disabling every component gets the default 0.45/0.35/0.2 blend instead of the documented/test-named fallback; change the zero-total branch to preserve zero weights so line 482 can handle it, e.g. `if (total <= 0) return { objectiveAnchor: 0, pairwiseJudge: 0, structuredReviewerConsensus: 0 };`, or change the API/test/docs to explain why all-zero means defaults.

Signal	Result	Evidence
Code review	❌ 1 blocker	1 reviewer
Linked issue	✅ No-issue rationale	PR body explains why no issue is linked.
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (no linked issue context).
Validation posture	❌ 5/25	Preflight is holding this PR: the review lane is unavailable, so it is not ready for automated review.
Contributor workload	✅ 10/10	Author activity: 1888 registered-repo PR(s), 1249 merged, 59 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	galuis116; Gittensor profile; 1888 PR(s), 59 issue(s).
Gate result	❌ Blocking	Repo-configured hard blocker found.

Review context

• Author: galuis116
• Role context: outside_contributor
• Public audience mode: oss maintainer
• Lane context: Repository registration is not available in the local Gittensory cache.
• Public profile languages: not available
• Official Gittensor activity: 1888 PR(s), 59 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Await review-lane availability.
• Refresh registry data or choose a registered active repo.
• Link the issue being solved, or explicitly explain why this is a no-issue PR.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

• Re-run Gittensory review

[loopover-orb]

Gittensory is closing this pull request on the maintainer's behalf (AI reviewers agree on a likely critical defect: packages/gittensory-engine/src/reviewer-consensus-calibration.ts:314 makes the later objective-only fallback at line 482 unreachable for `weights: { objectiveAnchor: 0, pairwiseJudge: 0, structuredReviewerConsensus: 0 }`, so a caller explicitly disabling every component gets the default 0.45/0.35/0.2 blend instead of the documented/test-named fallback; change the zero-total branch to preserve zero weights so line 482 can handle it, e.g. `if (total <= 0) return { objectiveAnchor: 0, pairwiseJudge: 0, structuredReviewerConsensus: 0 };`, or change the API/test/docs to explain why all-zero means defaults.). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.