feat: incorporate PR review comments into follow-up review context

[captainsafia]

Problem

When the review bot performs a follow-up review on a PR, it does not incorporate the author's responses to its initial review comments as context. This leads to the bot repeating findings that the author has already addressed or refuted in the comment thread.

Example

On warpdotdev/warp-server#10967, the bot left an initial review with a finding. The PR author replied that the finding was not valid. When a follow-up review was requested, the bot repeated the same invalid finding — indicating it was not aware of the prior conversation.

Expected Behavior

When performing a follow-up review, the bot should be given the existing PR review comments (both its own and the author's replies) as context. This would allow it to:

1. Avoid repeating findings that have already been discussed and dismissed
2. Incorporate feedback from the author to improve subsequent reviews
3. Focus on new or unresolved issues rather than re-raising resolved ones

Suggested Approach

Include the existing PR comment threads (review comments + replies) in the context provided to the review agent when it performs a follow-up review pass. This could be done by fetching the PR's review comments via the GitHub API and injecting them into the agent's prompt alongside the diff and PR description.

[oz-for-oss]

@captainsafia

Here's what I found while triaging this issue:

• This looks actionable without more reporter input; follow-up reviews currently appear to lack the prior review-thread context needed to avoid repeating resolved findings.

Maintainer details

I concluded that follow-up PR review runs appear to omit existing review comment threads from the review agent context, causing repeated findings that were already discussed..

Triage summary

This is an actionable workflow enhancement request. Follow-up PR reviews are routed through the same review-pull-request workflow as initial reviews, but the dispatched review context appears to contain only PR metadata, the annotated diff, spec context, and repo-local review guidance. It does not include existing PR review threads or replies, so the agent has no reliable way to know that a prior finding was refuted or resolved.

Evidence

• core/routing.py routes /oz-review on PR conversation or review comments back to WORKFLOW_REVIEW_PR, so follow-up review requests use the review workflow rather than the PR-comment response workflow.
• core/workflows/review_pr.py defines ReviewContext and gather_review_context() with fields for PR title/body, base/head branches, trigger source, annotated diff, spec context, diff maps, reviewer state, and progress comment id, but no field for prior PR comments or review comment threads.
• build_review_prompt_for_dispatch() in core/workflows/review_pr.py inlines the PR description, diff, and spec context, and explicitly tells the review agent not to make ad-hoc GitHub API calls. That makes the missing review-thread context a control-plane context-gathering gap rather than something the review agent can recover on its own.
• Existing tests around review dispatch and explicit retrigger throttling in tests/test_builders.py exercise counting /oz-review invocations from issue/review comments, but do not assert that prior review discussion is included in the prompt.

Suggested direction

Add a review-thread context gathering step for rereview triggers, likely in gather_review_context() / ReviewContext, and render it in build_review_prompt_for_dispatch() with the same untrusted-content caveats used for PR body and diff. The implementation should include both Oz-authored review comments and human replies so follow-up reviews can avoid repeating already-dismissed findings and focus on unresolved or new concerns.

Duplicate check

I checked all open non-PR issues and recent closed issues. Adjacent issues cover retriggering failures, review invocation behavior, comment/progress behavior, or review application errors, but I did not find two existing issues matching this specific request to include prior PR review comment threads in follow-up review context.

This is my automated analysis and may be incorrect. A maintainer will verify the details.

Powered by Oz