Surface detector-authored warnings in the result contract (`warnings: []`)

[davidslater]

Created by GitHub Ace · View Session

Problem

When an artifact channel is present but cannot be inspected, nothing about that reaches the result files. The only trace is a ::warning:: annotation in the job log, which no host can react to programmatically.

Today pkg/artifacts records an ArtifactWarning for:

Field	Cause
prompt	aw-prompts/prompt.txt missing or empty
agent_output	agent_output.json missing, empty, or not valid JSON
patch	HAS_PATCH=true was set but no readable, non-empty aw-*.patch/aw-*.bundle was found
comment_memory	the comment-memory directory could not be Lstat'd or ReadDir'd

These are recorded on Artifacts.Warnings, emitted as annotations in run(), and then dropped. detection_result.json and detection_result_full.json carry only the three booleans and reasons.

The consequence is that a partially-inspectable bundle is indistinguishable from a fully-inspected clean one. The detector analyzed less than the full artifact set, reported clean, and exited 0 — and a host has no structured way to tell that apart from a genuine all-clear.

Proposal

Add a warnings: [] array to the result contract, alongside reasons.

{
  "prompt_injection": false,
  "secret_leak": false,
  "malicious_patch": false,
  "reasons": [],
  "warnings": [
    {
      "field": "comment_memory",
      "code": "ERR_VALIDATION",
      "message": "Unable to read comment-memory directory at /tmp/gh-aw/threat-detection/comment-memory: permission denied"
    }
  ]
}

Warnings belong in both result files, unlike reasons. reasons are confined to --full-output because they are model-authored open text (TD-10f). Warnings are detector-authored: fixed strings plus host-controlled paths, with no model input at any point. That makes them safe to publish in the uploaded detection_result.json, which is what lets gh-aw's own conclude_threat_detection.sh — used by the standalone smoke locks instead of threat-detect conclude — see them too.

threat-detect conclude should render them with a ⚠️ marker, distinct from both the verdict block and the reasons block, so a misconfigured job is visible in the job log without reading annotations.

Constraints

• Warnings MUST NOT affect the verdict or the exit code. A warning says "the detector could not inspect everything", not "a threat was found". Conflating the two would reintroduce false positives, which is the failure mode feat(detector): enforce structural eligibility for threat verdicts #916 exists to reduce. Gating on warnings is tracked separately in the CONTINUE_ON_WARNING issue.
• The verdict remains sourced solely from the sink (TD-06a). Warnings are assembled by the detector from Artifacts.Warnings and attached on write; the model never authors or influences them.
• Field values must be sanitized on the way out — messages embed host-controlled paths.

Compatibility

Additive and backward compatible. The parser in pkg/detector/result.go does not set DisallowUnknownFields, and the JSON Schema does not set additionalProperties: false, so an older consumer ignores the field and a newer one reading an older result sees it absent. reasons stays a required array, so validation bounds and replay comparisons are unaffected.

Work

• Add Warnings []ResultWarning to detector.Result + schema
• Populate from Artifacts.Warnings in writeResult (both destinations)
• Render with ⚠️ in threat-detect conclude
• Spec: extend the result contract section; state explicitly that warnings never affect the verdict
• README: document the field and the reasons-vs-warnings distinction
• Tests: warnings appear in both files; a warning never changes exit code; sanitization of host-controlled paths

Context

Split out of #916, which added structural eligibility. That change introduced the uninspectable signal on a channel — an artifact that may hold content the detector could not read — and deliberately fails open on it so a staging failure cannot suppress a real finding. This issue is the other half: making that condition visible to the host rather than only to the eligibility calculation.