Skip to content

fix(review): keep changelog policy target-specific - #986

Draft
vincentkoc wants to merge 1 commit into
mainfrom
fix/review-target-changelog-policy-20260731
Draft

fix(review): keep changelog policy target-specific#986
vincentkoc wants to merge 1 commit into
mainfrom
fix/review-target-changelog-policy-20260731

Conversation

@vincentkoc

Copy link
Copy Markdown
Member

What Problem This Solves

Fixes an issue where ClawSweeper could apply OpenClaw's release-owned changelog rule to ClawSweeper or another target repository and incorrectly request removal of a valid release-note entry.

Why This Change Was Made

The review prompt now limits release-owned CHANGELOG.md handling to the exact openclaw/openclaw target and explicitly requires every other repository's own release-note policy to win. Focused prompt assertions preserve that boundary.

User Impact

Maintainers no longer receive false changelog-removal findings when reviewing ClawSweeper or another repository that intentionally tracks changes in its own changelog.

Evidence

  • ./node_modules/.bin/tsc -p tsconfig.json: passed.
  • node --test test/clawsweeper.test.ts: 76/76 passed.
  • git diff --check: clean.
  • Uncommitted autoreview: clean, 0.98 confidence.
  • Committed branch autoreview against origin/main: clean, 0.97 confidence.

Real Behavior Proof

  • Claim: the review prompt cannot describe a non-OpenClaw target changelog as release-owned based on the OpenClaw-only rule.
  • Exercised surface: the exact review prompt consumed by ClawSweeper review workers.
  • Scenario: inspect the rendered prompt contract and assert the exact openclaw/openclaw scope, explicit openclaw/clawsweeper exclusion, and prohibition on inferred changelog-removal findings.
  • Observed result: all 76 focused prompt and workflow assertions pass, including the new repository boundary.
  • Limits: this is a prompt-policy regression test; production review output changes after the prompt ships.

Finding Disposition

  • PR 981's repeated request to remove ClawSweeper's own changelog entry is rejected as a target-policy leak. AGENTS.md limits release ownership to the OpenClaw target, while CONTRIBUTING.md says other repositories follow their own release-note policy. This PR encodes that distinction directly in the worker prompt.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. labels Jul 31, 2026
@clawsweeper

clawsweeper Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Codex review: found issues before merge. Reviewed August 3, 2026, 5:06 AM ET / 09:06 UTC.

ClawSweeper review

What this changes

The PR limits the review prompt’s release-owned changelog rule to openclaw/openclaw, records that policy correction in ClawSweeper’s unreleased changelog, and adds regression assertions.

Merge readiness

⚠️ Ready for maintainer review - 4 items remain

The target-specific changelog correction remains necessary because current main still gives the release-owned rule without the exact repository boundary. Keep this member-authored draft open for a rebase and one focused P3 test-placement repair; it is not ready to merge while GitHub reports it dirty.

Priority: P3
Reviewed head: 099218ae83764515e1ad8178a5dfa11663faea0d

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) The policy direction is well supported, but the unresolved focused-test finding and three-file rebase conflict leave the patch below merge-ready quality.
Proof confidence 🌊 off-meta tidepool Not applicable: This member-authored prompt-policy PR is exempt from the external contributor proof gate; its body nevertheless documents focused terminal validation of the prompt contract.
Patch quality 🦐 gold shrimp (3/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: This member-authored prompt-policy PR is exempt from the external contributor proof gate; its body nevertheless documents focused terminal validation of the prompt contract.
Evidence reviewed 5 items Current prompt remains overbroad: Current main still describes CHANGELOG.md as release-owned for OpenClaw PR review without limiting that instruction to the exact openclaw/openclaw target; the PR supplies the missing boundary.
Established repository policy: The existing contribution policy already specifies that only an openclaw/openclaw target uses a release-owned changelog and that ClawSweeper or other targets follow their own release-note policy.
History provenance: git blame attributes the target-specific contribution-policy lines to commit f86bf91c2c5eade562a7d3f9814a5ddcf84591dd, tying the requested prompt change to existing repository direction.
Findings 1 actionable finding [P3] Move boundary assertions into the policy suite
Security None None.

How this fits together

ClawSweeper loads prompts/review-item.md into review workers that evaluate pull requests across many target repositories. The prompt influences the findings maintainers receive, so its changelog guidance must distinguish OpenClaw’s release-owned changelog from each other target’s own policy.

flowchart LR
  A[Target repository PR] --> B[ClawSweeper review worker]
  B --> C[Review prompt]
  C --> D[Target-specific changelog policy]
  D --> E[Codex review decision]
  E --> F[Maintainer-facing findings]
Loading

Before merge

  • Move boundary assertions into the policy suite (P3) - Move these prompt-policy assertions to test/review-prompt-policy.test.ts, which already owns prompts/review-item.md contract coverage. Root AGENTS.md requires the narrowest matching test file; extending this automerge-opt-in test makes the policy boundary harder to maintain.
  • Resolve merge risk (P1) - The current head is conflict-dirty against main in all three touched files, so it needs a rebase before its effective merge result can be reviewed.
  • Resolve merge risk (P1) - The new policy contract is asserted in a broad automerge test instead of the focused policy suite required by repository guidance.
  • Complete next step (P2) - This member-authored draft has a clear repair but is conflict-dirty and needs its PR owner to rebase and refresh validation, rather than an automated repair handoff.

Findings

  • [P3] Move boundary assertions into the policy suite — test/clawsweeper.test.ts:2547-2553
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Patch scope 3 files changed; +20, -10 The correction is narrow but spans the worker prompt, its regression coverage, and ClawSweeper’s unreleased release notes.
Merge conflicts 3 of 3 touched files conflict The branch must be rebased before maintainers can evaluate the actual combined patch.

Merge-risk options

Maintainer options:

  1. Decide the mitigation before merge
    Rebase onto current main, preserve the exact openclaw/openclaw boundary, and move the four changelog-boundary assertions into a dedicated case in test/review-prompt-policy.test.ts before requesting a fresh exact-head review.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Technical review

Best possible solution:

Rebase onto current main, preserve the exact openclaw/openclaw boundary, and move the four changelog-boundary assertions into a dedicated case in test/review-prompt-policy.test.ts before requesting a fresh exact-head review.

Do we have a high-confidence way to reproduce the issue?

Yes. Current main loads the overbroad prompt text from prompts/review-item.md, while CONTRIBUTING.md defines the conflicting exact-target policy; the review runtime loads that prompt for workers.

Is this the best way to solve the issue?

Yes, with one adjustment. The exact-target wording mirrors the established contribution policy, but its regression assertions should live in the dedicated prompt-policy suite rather than a broad automerge test.

Full review comments:

  • [P3] Move boundary assertions into the policy suite — test/clawsweeper.test.ts:2547-2553
    Move these prompt-policy assertions to test/review-prompt-policy.test.ts, which already owns prompts/review-item.md contract coverage. Root AGENTS.md requires the narrowest matching test file; extending this automerge-opt-in test makes the policy boundary harder to maintain.
    Confidence: 0.99

Overall correctness: patch is correct
Overall confidence: 0.96

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 2d1a40bfa643.

Labels

Label justifications:

  • P3: This is a narrow review-policy correction with no demonstrated outage, but focused test placement and a rebase are still needed before merge.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: This member-authored prompt-policy PR is exempt from the external contributor proof gate; its body nevertheless documents focused terminal validation of the prompt contract.

Evidence

What I checked:

Likely related people:

  • Peter Steinberger: Current contribution-policy provenance and recent main-branch work connect Peter Steinberger to the target-specific review-policy contract. (role: current prompt-policy contributor; confidence: high; commits: f86bf91c2c5e, 2d1a40bfa643; files: CONTRIBUTING.md, prompts/review-item.md, test/review-prompt-policy.test.ts)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Rebase and resolve the three current-main conflicts.
  • Move the boundary assertions to test/review-prompt-policy.test.ts.
  • Run focused policy validation and pnpm run check, then request a fresh exact-head review.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (14 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-02T08:53:56.217Z sha 099218a :: needs changes before merge. :: [P3] Move changelog assertions into the focused policy suite
  • reviewed 2026-08-02T11:14:27.012Z sha 099218a :: needs changes before merge. :: [P3] Move changelog assertions to the focused policy suite
  • reviewed 2026-08-02T12:47:04.110Z sha 099218a :: needs changes before merge. :: [P3] Move changelog assertions to the focused policy suite
  • reviewed 2026-08-02T15:26:00.392Z sha 099218a :: needs changes before merge. :: [P3] Move target-boundary checks into the policy suite
  • reviewed 2026-08-02T17:34:31.070Z sha 099218a :: needs changes before merge. :: [P3] Move changelog checks to the policy suite
  • reviewed 2026-08-02T21:04:50.470Z sha 099218a :: found issues before merge. :: [P3] Move changelog checks to the policy suite
  • reviewed 2026-08-03T01:07:42.331Z sha 099218a :: found issues before merge. :: [P3] Move changelog assertions to the policy suite
  • reviewed 2026-08-03T06:40:54.339Z sha 099218a :: found issues before merge. :: [P3] Move boundary assertions into the policy suite

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 1, 2026
@vincentkoc
vincentkoc force-pushed the fix/review-target-changelog-policy-20260731 branch from 099218a to af87627 Compare August 4, 2026 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant