fix: bound validation report publication without losing verdicts - #48
olavurellefsen wants to merge 1 commit into
Conversation
🤖 Documentation ValidationAction result: ERROR · Critical issues: 0 · Grounding: not-required Full validated report artifact ( PR Validation ReportSummaryThe provider completed, but its final assistant response was not a valid validation report. Critical Violations ❌
Validation Outcome
📊 Validation Statistics
|
🤖 Comment Revalidation TestAction result: PASSED · Critical issues: 0 · Grounding: not-required Full validated report artifact ( PR Validation ReportSummaryThis is a comment revalidation test for the Usable PR Validator. The PR fixes large validation report publication by handling GitHub's 60,000-byte comment limit through a new formatter script that truncates safely while preserving action results and artifact links. The test includes comprehensive Python unit tests and system prompt improvements. Override Comment Test Results
Override AppliedA deviation from standards has been approved and documented:
This deviation has been acknowledged for this test run. The comment revalidation feature successfully detected the override marker (🔄) and parsed the deviation request. Critical Violations ❌None found (test scenario) Important Issues
|
🤖 Integration Test ValidationAction result: ERROR · Critical issues: 0 · Grounding: not-required Full validated report artifact ( PR Validation ReportSummaryThe provider completed, but its final assistant response was not a valid validation report. Critical Violations ❌
Validation Outcome
📊 Validation Statistics
|
Large validation reports can fail after a completed review because
cat | headtriggers SIGPIPE underpipefail, or because the PR comment exceeds GitHub’s limit. This patch reads the preview directly, uploads the complete validated report first, and posts a comment bounded to 60,000 UTF-8 bytes with the action result and full artifact link kept visible.The formatter changes presentation only. Strict final-assistant extraction, verdict parsing, grounding enforcement, tool permissions, model choices and release policy are unchanged. Missing artifacts and failed validation steps are explicit. Reviewer instructions use workspace reads and final stdout instead of temporary-file copies or permission workarounds.
Validation: all 40 Python tests pass, including six new regressions for large reports through actual orchestration, Unicode boundaries, retained failure status, artifact preservation and the actual GitHub comment script against a local mock. Pinned installer, MCP restriction, dependency, workflow, secret and release-policy checks pass, along with Bash/Node syntax, YAML parsing and
git diff --check.Based on released v2.2.7 (
f10856bcf763f94fbdf0b9edd01b3e168d8462ac). This ready-for-review candidate publishes no shared tag and remains unmerged. A reviewed consumer now pins this exact repair commit and its application CI plus both real standards/security reports passed with zero critical or important issues. Grounding remains NOT REQUIRED as configured, rather than claiming deterministic certification. The consumer source PR has merged; no other consumers changed. The action’s own hosted run passes deterministic checks, but two advisory AI cases produced safe infrastructure-failure reports while comment revalidation produced a structured PASS. Those limitations remain separate from the successful real consumer pilot.