Skip to content

fix(engine): report below-baseline calibration delta as a signed negative - #3890

Closed
glorydavid03023 wants to merge 1 commit into
JSONbored:mainfrom
glorydavid03023:fix/phase7-below-baseline-delta
Closed

fix(engine): report below-baseline calibration delta as a signed negative#3890
glorydavid03023 wants to merge 1 commit into
JSONbored:mainfrom
glorydavid03023:fix/phase7-below-baseline-delta

Conversation

@glorydavid03023

Copy link
Copy Markdown
Contributor

Summary

computePhase7CalibrationLoop (packages/gittensory-engine/src/phase7-calibration-loop.ts) computed deltaFromBaseline — the signed deviation of the combined calibration accuracy from the documented 62% baseline — by rounding through roundScore, which clamps its input to [0, 1]:

const deltaFromBaseline =
  combinedAccuracy === null ? null : roundScore(combinedAccuracy - DOCUMENTED_CALIBRATION_BASELINE);

combinedAccuracy is in [0, 1] and the baseline is 0.62, so the true delta ranges over [-0.62, +0.38]. Every below-baseline run (a calibration regression) produced a negative delta that roundScore flattened to 0, so a regression was reported as "on baseline / no deviation."

deltaFromBaseline is a signed field (number | null), the module tracks accuracy "against the documented 62% baseline," and the audit renderer prints it as a signed magnitude (${(deltaFromBaseline * 100).toFixed(2)} percentage points) — so surfacing below-baseline regressions is exactly its purpose. The sibling combinedAccuracy is a genuine [0, 1] score and correctly uses roundScore; only the signed delta was mis-clamped.

The fix rounds the signed difference without the [0, 1] clamp via a dedicated roundDelta helper.

Example: two sources at 0.5 accuracy → combined 0.5 (0.12 below baseline) now yields deltaFromBaseline === -0.12 (rendered -12.00 percentage points) instead of 0.

No linked issue: a small, self-contained correctness fix in pure engine logic; per this repo's linkedIssuePolicy: preferred, the defect is described in full above.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked an issue, or this is small enough that the summary explains why an issue is not needed.

Validation

  • git diff --check
  • npm run typecheck (green)
  • npm run build:miner builds + typechecks the engine package with the change (green)
  • New behavior has regression tests covering below / at / above baseline, added in both conventions: a root test/unit/phase7-calibration-delta.test.ts (vitest, run by test:coverage) and the engine packages/gittensory-engine/test/phase7-calibration-loop.test.ts (node:test).
  • Verified empirically: below-baseline combined accuracy 0.5 now reports deltaFromBaseline === -0.12 (was 0).

If any required check was skipped, explain why:

  • The full npm run test:ci was not run in my local dev environment (several self-host suites require Linux/bash/Docker/Postgres and fail only on Windows). This change is confined to packages/gittensory-engine (a pure function, no src/ app, API/OpenAPI, MCP, binding, DB, or migration surface), so no generated artifacts change; typecheck, the engine build, and the new regression tests are green, and the remaining CI runs on this PR.

Safety

  • No secrets, wallets, hotkeys, coldkeys, PATs, private keys, trust scores, private rankings, or maintainer evidence are exposed.
  • Public GitHub text stays sanitized and low-noise; no compensation/optimization implications.
  • No auth/cookie/CORS/GitHub App/Cloudflare/session changes.
  • No API/OpenAPI/MCP behavior changes.

UI Evidence

N/A - pure backend engine logic; no UI, frontend, docs, or extension change.

…tive

computePhase7CalibrationLoop rounded deltaFromBaseline through roundScore, which clamps to [0,1]. Since combinedAccuracy is in [0,1] and the documented baseline is 0.62, the true delta ranges over [-0.62, 0.38], so every below-baseline calibration run (a regression) was flattened to 0 and reported as on-baseline. deltaFromBaseline is a signed field surfaced as signed percentage points in the audit, so it must round the difference without the score clamp; add a roundDelta helper and use it. Regression tests cover below/at/above baseline in both the engine node:test and a root vitest suite.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 7, 2026
@loopover-orb

loopover-orb Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review result - reject/close recommended

Review updated: 2026-07-07 05:47:46 UTC

3 files · 1 AI reviewer · 2 blockers · readiness 100/100 · CI pending · blocked

🛑 Suggested Action - Reject/Close

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
The diff fixes a real clamping bug: `deltaFromBaseline` in `phase7-calibration-loop.ts` previously ran through `roundScore`, which clamps to `[0,1]`, so any below-baseline (negative) delta was flattened to `0`, hiding calibration regressions. The new `roundDelta` helper (lines ~121-124) rounds without clamping and is correctly swapped in at the `deltaFromBaseline` computation (~line 473), leaving the unrelated `combinedAccuracy` clamp (which is a genuine `[0,1]` score) untouched. The added unit and integration tests exercise the real `computePhase7CalibrationLoop` path with actual `prOutcome`/`historicalReplay` inputs (not fabricated payloads) and assert the negative, positive, and exact-baseline-zero cases, matching the rendered Markdown output too.

Nits — 4 non-blocking
  • The `roundDelta` doc comment (phase7-calibration-loop.ts ~118-121) is a bit long for a one-line helper; could be trimmed to a single sentence per repo comment conventions.
  • The new top-level `test/unit/phase7-calibration-delta.test.ts` duplicates the same three scenarios already added to `packages/gittensory-engine/test/phase7-calibration-loop.test.ts` (only the negative case is duplicated exactly) — consider keeping the coverage in one location to avoid divergence over time.
  • Consider consolidating the two new test files (unit/ and package test/) since they test the same function with overlapping scenarios.
  • Since `roundDelta` and `roundScore` are now easy to confuse, a brief comment at `roundScore`'s definition noting 'do not use for signed deltas — see roundDelta' would help future readers.

Why this is blocked

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ✅ No-issue rationale PR body explains why no issue is linked.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 308 registered-repo PR(s), 198 merged, 15 issue(s).
Contributor context ✅ Confirmed Gittensor contributor glorydavid03023; Gittensor profile; 308 PR(s), 15 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Review context
  • Author: glorydavid03023
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 308 PR(s), 15 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb

loopover-orb Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Gittensory is closing this pull request on the maintainer's behalf (No linked issue detected; Maintainer requires a linked issue). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.

@loopover-orb loopover-orb Bot closed this Jul 7, 2026
@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.77%. Comparing base (bfe55f6) to head (895a0dd).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...s/gittensory-engine/src/phase7-calibration-loop.ts 50.00% 0 Missing and 1 partial ⚠️

❌ Your patch status has failed because the patch coverage (50.00%) is below the target coverage (99.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3890      +/-   ##
==========================================
+ Coverage   93.57%   93.77%   +0.19%     
==========================================
  Files         340      340              
  Lines       33516    33517       +1     
  Branches    12262    12262              
==========================================
+ Hits        31364    31431      +67     
+ Misses       1528     1389     -139     
- Partials      624      697      +73     
Files with missing lines Coverage Δ
...s/gittensory-engine/src/phase7-calibration-loop.ts 36.70% <50.00%> (+35.63%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant