Track stale PR follow-up for requested-changes reviews

[vkodithala]

Problem

External PRs can receive requested-changes reviews from Oz or human reviewers and then sit indefinitely without author follow-up. This creates review backlog and leaves contributors without a clear expectation for when inactive requested-changes PRs will be closed.

This came up in discussion as a desired lifecycle:

1. Oz or a human reviewer requests changes.
2. If there are no commits, comments, or other meaningful PR activity for X days, automation posts a reminder asking the author to respond or push updates.
3. If there is still no activity for Y more days, automation closes the PR with an explanatory comment.

This issue is intended to track the problem and desired behavior. We have not decided on the implementation plan yet.

Current context

In oz-for-oss:

• /oz-review produces a review.json with verdict: "APPROVE" | "REJECT".
• For non-member PRs, an Oz REJECT verdict is applied back to GitHub as a real REQUEST_CHANGES review.
• Member/collaborator PRs currently keep comment-only review behavior even when Oz rejects.
• Oz also posts REQUEST_CHANGES for external PRs that fail the linked-ready-issue gate.
• Later Oz approvals already dismiss stale Oz-authored requested-changes reviews.
• Human REQUEST_CHANGES reviews are also possible, but current webhook routing only reacts to pull_request_review events when the review body mentions @oz-agent.

In warpdotdev/warp:

• Fork PRs are labeled external-contributor.
• There are existing scheduled/custom GitHub Actions that close specific stale automated fix PRs.
• There is not currently a general Stale or requested-changes-specific label for this flow.

Desired behavior

Add automation for open PRs with active requested-changes reviews, likely scoped initially to external-contributor PRs:

• After X days without PR activity, post a reminder comment tagging the author.
• The reminder should explain that changes were requested and that the PR may be closed after Y additional days without activity.
• After Y more inactive days, re-check that the PR still has active requested changes, then close it with an explanatory comment.
• Any new meaningful PR activity should reset the inactivity timer.
• Draft PRs and PRs with an explicit maintainer-applied exemption label should be skipped.

Implementation options to evaluate later

Option 1: actions/stale plus labels

Use actions/stale for the reminder/close lifecycle, with a companion workflow or webhook path that applies/removes a label such as changes-requested when the PR enters or leaves requested-changes state.

Pros:

• Reuses a common stale workflow.
• Built-in reminder and close timers.

Cons:

• Requires reliable label synchronization for both Oz and human reviews.
• Label drift could cause incorrect reminders or closures.
• actions/stale does not appear to directly filter on GitHub review state such as review:changes_requested.

Option 2: custom scheduled GitHub Action

Add a scheduled actions/github-script workflow in the target repo that searches for open PRs with active requested-changes reviews and manages reminder/close state directly.

Pros:

• Can use GitHub review state as source of truth.
• Avoids maintaining a separate request-changes label.
• Matches existing custom stale PR cleanup patterns.

Cons:

• More custom logic to maintain.
• Needs careful handling of reminder state, exemptions, and activity windows.

Open questions

• Should this apply only to external-contributor PRs, or all PRs?
• Should Oz-requested changes and human-requested changes be treated identically?
• What values should we use for X and Y?
• What exactly should count as activity: any GitHub updated_at change, only author comments/commits, or something else?
• What exemption label should maintainers use?
• Should the first rollout be dry-run or reminder-only to avoid surprise bulk closures?
• Should this live entirely in warpdotdev/warp, or should any generic reusable pieces live in oz-for-oss?

Acceptance criteria

• A clear policy exists for inactive PRs with active requested-changes reviews.
• Automation can identify eligible open PRs without relying on stale or incorrect state.
• Authors receive one clear reminder before closure.
• PRs are only closed after the follow-up window expires and requested changes are still active.
• Maintainers have an explicit exemption path.
• The initial rollout is conservative enough to avoid accidental bulk closures.

[oz-for-oss]

@vkodithala

Here's what I found while triaging this issue:

This looks like a new workflow automation request. The current Oz review flow can create or dismiss requested-changes reviews, but it does not yet implement the inactivity reminder and closure lifecycle described here.

I have a few follow-up questions before I can narrow this down:

1. Should the first version apply only to external-contributor/non-member PRs, or to all PRs with active requested-changes reviews?
2. Should Oz-authored and human-authored requested-changes reviews use the same reminder and closure policy?
3. What reminder and closure inactivity windows should be used for X and Y?
4. What events should reset inactivity: author commits only, author comments, any review/comment activity, label changes, or any PR updated_at change?
5. What maintainer exemption label and initial rollout mode should be used: dry-run, reminder-only, or full reminder-plus-close?

Reply in-thread with those details and the triage workflow will automatically re-evaluate the issue and update the diagnosis, labels, and next steps.

Maintainer details

I concluded that add a requested-changes inactivity lifecycle for PRs so eligible stale external contributor PRs receive a reminder before possible closure..

Triage summary

This is an enhancement request for PR review lifecycle automation, not an observed regression in an existing stale-PR workflow. The current code supports creating real requested-changes reviews for non-member PRs and dismissing stale Oz-authored requested-changes reviews after a later Oz approval, but it does not maintain reminder/closure state for inactive PRs with active requested changes.

Relevant implementation areas:

• core/workflows/review_pr.py handles non-member review behavior, REQUEST_CHANGES application, linked-ready-issue enforcement reviews, and stale Oz-authored requested-changes dismissal after approval.
• core/routing.py currently routes pull_request_review events only when a non-bot review body mentions @oz-agent, so human requested-changes reviews are not otherwise a lifecycle trigger.
• api/cron.py and core/workflows/__init__.py implement the existing Vercel cron/run-draining flow, but there is no requested-changes inactivity scanner or reminder/closure workflow.
• Existing tests in tests/test_pr_issue_state_enforcement.py and tests/test_routing.py cover pieces of the current behavior and would be natural places to add coverage for any new policy/workflow logic.

Duplicate check: I scanned all open non-PR issues and recently closed issues. Related workflow issues exist, but none match this reminder-and-close lifecycle closely enough to mark this as a duplicate.

This needs maintainer policy decisions before implementation: scope, timers, activity definition, exemption label, rollout mode, and whether the reusable logic belongs in oz-for-oss versus target-repo workflows.

Question reasoning

1. Should the first version apply only to external-contributor/non-member PRs, or to all PRs with active requested-changes reviews? — The current Oz behavior treats external and member PRs differently, and the automation scope changes both risk and implementation shape.
2. Should Oz-authored and human-authored requested-changes reviews use the same reminder and closure policy? — The current code can identify Oz-authored stale reviews separately, but human requested-changes reviews may have different maintainer expectations.
3. What reminder and closure inactivity windows should be used for X and Y? — The issue intentionally leaves the timing policy undecided, and those values directly control user-facing reminders and closures.
4. What events should reset inactivity: author commits only, author comments, any review/comment activity, label changes, or any PR updated_at change? — GitHub exposes several activity signals, and selecting the wrong one could either close active PRs or keep inactive PRs open indefinitely.
5. What maintainer exemption label and initial rollout mode should be used: dry-run, reminder-only, or full reminder-plus-close? — The acceptance criteria require an exemption path and conservative rollout, both of which are policy choices maintainers need to name explicitly.

This is my automated analysis and may be incorrect. A maintainer will verify the details.

Powered by Oz