Skip to content

fix(review): count owner reopen-then-merge as a real reversal signal - #8074

Merged
JSONbored merged 1 commit into
mainfrom
fix/owner-reopen-merge-reversal-signal
Jul 22, 2026
Merged

fix(review): count owner reopen-then-merge as a real reversal signal#8074
JSONbored merged 1 commit into
mainfrom
fix/owner-reopen-merge-reversal-signal

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Closes #7985.

Summary

  • recordReversalSignals previously excluded EVERY repo-owner reopen of a bot-closed PR from the reversal signal (treated as "administrative re-queue, not a dispute"). That blanket exclusion is right for a bare reopen — genuinely ambiguous on its own — but wrong once the owner actually merges the PR afterward: that's an unambiguous "the bot's close was wrong" correction, not routine administration.
  • This exact pattern (bot closes, owner reopens, owner merges within minutes) was every single one of the 2026-07-21/22 metagraphed false-positive incidents (fix(review): paginate preview-url.ts's PR-comment and check-run GitHub reads #7469, Epic: dual-path repo provisioning — BYOR + APR #7589, Design spec: idea-completion determination + customer-facing status for APR (no GitHub UI exposure) #7591, fix(ui): give four app-panel form controls a programmatic accessible name #7594) — and because none of them ever got recorded as a reversal, the accuracy/self-correction system had no signal that anything was wrong on the day it mattered most, part of the reason the homepage "Decision" accuracy metric looked misleadingly close to perfect.
  • Fix: an owner reopen of a bot-closed PR now writes a time-bounded owner_reopen_pending_reversal marker (via recordAuditEvent) instead of being silently dropped. If a merge for that same PR follows within OWNER_REOPEN_MERGE_WINDOW_MS (6h), it's promoted to a real reversal_reopened event in both review_audit (what reversalRate/calibration read) and audit_events. A bare reopen with no follow-up merge, or a merge outside the window, still records nothing extra — same "genuine correction, not noise" bar a contributor reopen already has to clear. A bot reopening itself is still never a human-disagreement signal.

Test plan

  • npm run typecheck
  • New tests in test/unit/outcomes-wire.test.ts (describe("owner reopen + merge within the window (#7985)")): promotes reopen+merge to a real reversal; a plain merge with no marker records nothing; a stale marker outside the window records nothing; an owner reopen when the last bot action wasn't a close writes no marker
  • Full unsharded npm run test:coverage: 1088/1088 files, 20312 tests, 0 failures

Closes #7985.

A bare owner reopen of a bot-closed PR stayed excluded from
reversalRate (still ambiguous — could be an administrative re-queue),
but an owner reopen followed by an approve/merge within 6h is
unambiguous: the owner looked again and decided the bot was wrong.
Every one of the 2026-07-21/22 metagraphed false-positive incidents
(#7469/#7589/#7591/#7594) was exactly this pattern, and the old
unconditional owner-reopen exclusion recorded nothing for any of them
— part of why the accuracy metric stayed misleadingly high that day.

Record a time-bounded owner_reopen_pending_reversal marker on reopen,
then promote it to a real reversal_reopened event if a merge follows
within OWNER_REOPEN_MERGE_WINDOW_MS. A bot reopening itself still
never counts.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 22, 2026
@JSONbored
JSONbored merged commit ddcf366 into main Jul 22, 2026
12 checks passed
@JSONbored
JSONbored deleted the fix/owner-reopen-merge-reversal-signal branch July 22, 2026 19:22
@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 22, 2026
@codecov

codecov Bot commented Jul 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.94737% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.67%. Comparing base (a3dc73e) to head (cffa2f7).
⚠️ Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
src/review/outcomes-wire.ts 78.94% 2 Missing and 2 partials ⚠️

❌ Your patch status has failed because the patch coverage (78.94%) is below the target coverage (99.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8074      +/-   ##
==========================================
- Coverage   91.83%   89.67%   -2.16%     
==========================================
  Files         734        1     -733     
  Lines       75086      184   -74902     
  Branches    23068       50   -23018     
==========================================
- Hits        68952      165   -68787     
+ Misses       5034       12    -5022     
+ Partials     1100        7    -1093     
Flag Coverage Δ
rees ?
shard-1 26.63% <36.84%> (-28.71%) ⬇️
shard-2 89.13% <78.94%> (+33.34%) ⬆️
shard-3 26.63% <36.84%> (-23.94%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/review/outcomes-wire.ts 89.67% <78.94%> (-1.29%) ⬇️

... and 733 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Count an owner reopen-then-merge as a real reversal signal, not administrative noise

1 participant