Skip to content

Commit 080065a

Browse files
authored
fix(review): guard recordPrOutcome's webhook path against double-counting (#10346)
recordPrOutcome (the inbound pull_request.closed webhook path) always incremented loopover_pr_outcomes_total and wrote a pr_outcome row, with no existence check. recordTerminalActionOutcome (the bot's own direct-action path) already probes for an existing pr_outcome row and skips both the metric increment and the write when one exists. In the realistic production ordering the direct write lands first (right after the merge/close mutation), then GitHub delivers the closed webhook for the same action, so the counter was incremented twice for one real PR outcome. The duplicate ROW is harmless (every downstream reader takes the LATEST row per target), but the counter had no such defense. Mirror recordTerminalActionOutcome's guard exactly: probe for an existing pr_outcome row before incrementing or writing, and return early when one exists. A probe-read failure logs and proceeds (fail-open) so a genuinely-new outcome is never dropped. No other decision logic in recordPrOutcome changes. Closes #10332
1 parent c353e9d commit 080065a

2 files changed

Lines changed: 60 additions & 1 deletion

File tree

src/review/outcomes-wire.ts

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -429,10 +429,28 @@ export async function recordPrOutcome(
429429
return;
430430

431431
const decision = merged ? "merged" : "closed";
432+
const targetId = reviewAuditTargetId(repoFullName, pr.number);
433+
// The bot's own recordTerminalActionOutcome writes the pr_outcome row first (right after the merge/close
434+
// mutation); GitHub then delivers the `closed` webhook for the SAME action, landing here second. Probe for
435+
// the existing row before touching the counter so loopover_pr_outcomes_total is incremented exactly once per
436+
// real outcome — mirroring recordTerminalActionOutcome's own guard (#10332).
437+
try {
438+
const existing = await env.DB.prepare(
439+
"SELECT 1 AS x FROM review_audit WHERE target_id = ? AND event_type = 'pr_outcome' LIMIT 1",
440+
)
441+
.bind(targetId)
442+
.first<{ x: number }>();
443+
if (existing) return;
444+
} catch (error) {
445+
// An unreadable ledger must not suppress a genuinely-new outcome — a duplicate row is strictly better than a
446+
// lost one (fleet export + computeGateEval both read the LATEST pr_outcome per target). Fail open, like the
447+
// direct path.
448+
console.warn(JSON.stringify({ event: "pr_outcome_webhook_probe_error", message: errorMessage(error).slice(0, 160) }));
449+
}
450+
432451
// Observability (#reviews-dashboard): realized human outcome (merged vs closed) for the Grafana panel + as the
433452
// ground truth to compare against the engine's gate verdicts.
434453
incr("loopover_pr_outcomes_total", { outcome: decision });
435-
const targetId = reviewAuditTargetId(repoFullName, pr.number);
436454

437455
await appendReviewAudit(env, {
438456
project: repoFullName.slice(0, 200),

test/unit/outcomes-wire.test.ts

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ import {
2222
type PlannedAgentAction,
2323
} from "../../src/settings/agent-actions";
2424
import { recordAuditEvent } from "../../src/db/repositories";
25+
import { renderMetrics, resetMetrics } from "../../src/selfhost/metrics";
2526
import { upsertRepoFocusManifest } from "../../src/signals/focus-manifest-loader";
2627
import type { GitHubPullRequestPayload } from "../../src/types";
2728
import { createTestEnv } from "../helpers/d1";
@@ -267,6 +268,46 @@ describe("recordTerminalActionOutcome — webhook-independent ground truth (#882
267268
expect(await reviewAuditRows(env, "pr_outcome")).toHaveLength(1);
268269
});
269270

271+
it("counts loopover_pr_outcomes_total exactly ONCE in the realistic terminal-action-first ordering (#10332)", async () => {
272+
// Production ordering: the bot's own recordTerminalActionOutcome writes first (right after the merge/close
273+
// mutation), THEN GitHub delivers the `closed` webhook and recordPrOutcome runs for the SAME PR. Before the
274+
// fix, the webhook path always incremented the counter, so one real outcome was counted twice.
275+
const env = createTestEnv();
276+
resetMetrics();
277+
await recordTerminalActionOutcome(env, "owner/repo", 42, "closed");
278+
await recordPrOutcome(env, "pull_request", {
279+
action: "closed",
280+
repository: { name: "repo", full_name: "owner/repo", owner: { login: "owner" } },
281+
pull_request: pullRequestPayload({ number: 42 }),
282+
sender: { login: "loopover[bot]", type: "Bot" },
283+
});
284+
const counter = (await renderMetrics())
285+
.split("\n")
286+
.find((l) => l.startsWith('loopover_pr_outcomes_total{outcome="closed"}'));
287+
expect(counter).toBe('loopover_pr_outcomes_total{outcome="closed"} 1'); // once total, not twice
288+
expect(await reviewAuditRows(env, "pr_outcome")).toHaveLength(1); // and the row stays deduplicated too
289+
});
290+
291+
it("recordPrOutcome fails OPEN when its idempotency probe throws — a lost outcome is worse than a duplicate (#10332)", async () => {
292+
const env = createTestEnv();
293+
const warn = vi.spyOn(console, "warn").mockImplementation(() => undefined);
294+
const realPrepare = env.DB.prepare.bind(env.DB);
295+
// Fail ONLY the probe SELECT; every subsequent write (appendReviewAudit/audit_events) still runs for real.
296+
vi.spyOn(env.DB, "prepare").mockImplementationOnce(() => {
297+
throw new Error("ledger unavailable");
298+
});
299+
await recordPrOutcome(env, "pull_request", {
300+
action: "closed",
301+
repository: { name: "repo", full_name: "owner/repo", owner: { login: "owner" } },
302+
pull_request: pullRequestPayload({ number: 77 }),
303+
sender: { login: "maintainer", type: "User" },
304+
});
305+
(env.DB.prepare as unknown as { mockRestore: () => void }).mockRestore?.();
306+
void realPrepare;
307+
expect((await reviewAuditRows(env, "pr_outcome"))[0]).toMatchObject({ target_id: "owner/repo#77" });
308+
expect(warn).toHaveBeenCalled();
309+
});
310+
270311
it("a failing audit_events mirror never breaks the canonical review_audit write", async () => {
271312
const env = createTestEnv();
272313
const warn = vi.spyOn(console, "warn").mockImplementation(() => undefined);

0 commit comments

Comments
 (0)