fix: make Context Tree value audits evidence-safe

[Gandy2025]

What changed

• bind audits to the immutable runtime Agent slug and require runtime, workspace, and local binding UUIDs to agree
• fetch and filter historical Chat messages against the exact acquisition window
• classify every in-window bound-Tree read attempt into a conserving four-state grammar
• recover statically closed multi-file, composite, continuation, and provider-orchestrated reads while keeping dynamic or unsafe shapes fail-closed
• recognize the two additional historical shapes proven safe in the pilot: semicolonless direct .output forwarding with literal inner arguments, and single-branch literal filesystem guards
• keep unseparated auxiliary output without a read ID, so exposure remains pending
• gate effect reporting on Task-level evidence readiness: unresolved exposure is N/A / pending, never zero effect
• retain an optional hash-anchored reviewed positive baseline separately from the current sample and saturation calculation
• release the current Draft payload as 0.2.3

Why

The original pilot exposed two independent failures. First, mutable display names and Chat summary timestamps could prevent a valid historical export. Second, the collector accepted only isolated single-file reads even though real Agents commonly read several Tree nodes through one read-only shell or orchestration call.

Those false negatives must not erase previously reviewed value evidence or turn missing exposure into a numeric zero. The collector now recovers only shapes whose commands, Tree paths, and output forwarding can be proved. Everything else stays unresolved.

Validation

• python3 scripts/validate_skill.py
• python3 -m compileall -q skills tests scripts
• python3 -m unittest discover -s tests -v (27/27)
• git diff --check
• GitHub Actions on Python 3.11 and 3.12
• prior fixed-window private pilot at exact head 00711a4: 123 clear Tasks, 40 confirmed exposure, 83 pending, and 24 current-sample effect Tasks; the reviewed 37-effect baseline remained separate
• deterministic collector, evidence, and report hashes on repeated prior-pilot runs

The new exact head is ed0ece52c14bec0579de1deeb78707758e5a3572. A same-window private rerun and Task-level re-judgment are in progress. This PR remains Draft until that rerun passes the evidence and release gates; pending Tasks will not be interpreted as no effect, and saturation will not be claimed without the required no-new-findings batches.

[yuezengwu]

Blocking: AGENT_SLUG_PATTERN at skills/context-tree-insights/scripts/context_tree_insights.py:82 does not cover the full grandfathered producer grammar that this patch promises to preserve.

Before the current 1–64 rule, First Tree created names with ^[a-z0-9_-]+$ up to 100 characters. Current First Tree explicitly grandfathers rows outside the tightened regex, so an existing runtime slug such as _legacy or -legacy can still have a valid local binding and can still be passed to chat --agent. This new gate rejects both before verify_cli_agent_identity() can cross-check the binding, leaving those still-runnable Agents unable to run the audit.

Please accept the complete historical 1–100 producer grammar in addition to the current grammar (the local binding UUID check still provides the fail-closed identity guarantee), and add leading _ and - cases to the grandfathered compatibility matrix. The current test that classifies -fixture-agent as invalid should be updated accordingly.

[Gandy2025]

Fixed in the new head.

• AGENT_SLUG_PATTERN now accepts the complete historical
  [a-z0-9_-]{1,100} producer grammar, including leading - / _.
• The producer-owned local binding lookup and UUID equality check remain the
  fail-closed identity authority; this only removes a compatibility false
  rejection for still-runnable grandfathered Agents.
• Added positive regressions for -fixture-agent, _fixture-agent, -, and
  _. Option-looking legacy names such as --json and --help are passed as
  --agent=<slug> so the CLI cannot reinterpret them as global options.
  101 characters, uppercase, dots, and other non-producer characters remain
  rejected.
• Updated the evidence schema and manual checklist to state the exact current
  plus historical boundary.

Validation: repository Skill validator, compileall, 19/19 deterministic tests,
and git diff --check all pass.

[yuezengwu]

The requested compatibility fix is complete at af3bd18.

The collector now accepts the full historical [a-z0-9_-]{1,100} producer grammar, including leading separators, while retaining the runtime/workspace/local-binding UUID equality check as the fail-closed identity authority. Agent selectors are consistently passed as --agent=<slug>, so option-looking grandfathered names remain data rather than CLI flags. The updated regression matrix covers the reported cases, and I found no new blocking issue in the incremental or full diff.

[yuezengwu]

The runtime adapters and Task-level pending semantics are moving in the right direction, but the new evidence grammar still has three fail-open / non-conserving cases that block an evidence-safe release:

1. safe_tree_cli() only searches for adjacent tree tree tokens anywhere in the argument vector (skills/context-tree-value-audit/scripts/context_tree_value_audit.py:1943). As a result, a different First Tree namespace such as first-tree chat send tree tree is accepted as a read-only diagnostic; when it appears in a compound command with a real Tree Markdown read, the whole call can be classified accepted_read_only_composite even though it performs an external mutation. Please parse/whitelist the actual tree tree command path (and only its safe selector arguments), rather than accepting those words at arbitrary positions, and add a regression for a mutating namespace containing the same token pair.

2. The rg path treats every unknown option as harmless and only validates bare path-shaped tokens (...py:2028, ...py:2167). Options that read their values from files therefore bypass Tree isolation: for example rg --file=/etc/passwd pattern <tree>/NODE.md (and similarly --ignore-file=/...) can recover the Tree operand and be accepted while the original process also reads outside the bound Tree. This contradicts the documented rejected_unsafe boundary for proven Tree-external reads. Please whitelist the supported rg grammar and validate/reject every file-valued option, including --option=value forms, with external-path regressions.

3. Claude call/result pairing drops incomplete calls before classification (...py:4171-4177). An in-window tool_use whose literal payload names bound-Tree Markdown but whose tool_result is missing or duplicated produces a coverage-gap string, yet it never reaches provider_read_rows(), so it contributes zero to in_window_tree_read_attempts and none of the four statuses. The report can then claim that all such calls are conserved when the denominator omitted them. Please carry attributable unpaired/invalid calls through as unresolved_opaque (or otherwise make the denominator truly include them) and add missing/duplicate-result conservation regressions.

These are core collector/schema correctness issues; there is no database change. Per reviewer policy I reviewed the diff statically and did not run the test or QA suites.

[Gandy2025]

Addressed all three blocking findings in e7baf37.

1. First Tree command parsing now accepts only the exact bare first-tree tree tree / first-tree-staging tree tree command path with a closed selector grammar and Tree-contained path operands. Other namespaces, unknown options, Tree-external selectors, and path-qualified executable lookalikes fail closed.
2. rg now uses a closed option grammar. File-valued or external-program options (--file, --ignore-file, --pre, --hostname-bin, including --option=value forms), unknown options, stdin operands, Tree-external paths, and path-qualified executable lookalikes cannot become accepted read evidence.
3. Claude pairing now retains every attributable in-window Tree-reading tool_use. Missing results, duplicate results, duplicate call IDs, missing timestamps, out-of-order results, and results completing after the acquisition end are conserved as unresolved_opaque. Pairing anomalies wholly outside the acquisition window no longer pollute current-window gaps.

I also added regression coverage for the exact reported examples plus path-qualified reader spoofing, rg stdin, acquisition-end crossing, and pre-window pairing anomalies. The evidence schema and manual checklist now state the same boundaries.

Validation:

• python3 scripts/validate_skill.py
• python3 -m compileall -q skills tests scripts projections
• python3 -m unittest discover -s tests -v (40/40)
• git diff --check
• two independent post-fix reviews: no blocking findings

[yuezengwu]

The previous three blockers are fixed at e7baf37: First Tree commands now require the actual tree tree namespace, file-valued/unknown rg options are closed, and incomplete Claude pairs remain in the four-state denominator. Two implicit side-effect paths still keep the new “read-only” grammars from being statically closed:

1. safe_tree_cli() allows first-tree tree tree without --no-pull (skills/context-tree-value-audit/scripts/context_tree_value_audit.py:2038-2105). The CLI’s own help states that tree tree performs an automatic git pull --ff-only unless --no-pull is supplied. Therefore cd <tree> && first-tree tree tree && cat NODE.md is currently accepted as accepted_read_only_composite even though the original call can perform network I/O and mutate the Tree checkout. Please require --no-pull for selector executions (while handling a standalone help invocation separately) and add a regression proving the default-refresh form is rejected/unresolved.

2. The closed rg argv grammar still makes --no-config optional (...py:2179-2242, with the flag merely listed at line 243). Ripgrep reads the file named by RIPGREP_CONFIG_PATH unless --no-config is present, and that config can inject options such as --pre or --file; historical tool records here do not bind the process environment or config contents. Consequently even a bare rg Decision <tree>/NODE.md cannot prove “no external program / no Tree-external file” from the recorded command alone. Please require --no-config for accepted rg calls (or persist and validate the exact historical config/environment) and cover the implicit-config boundary in the regression matrix.

Both cases directly contradict the schema’s promise that accepted composites are statically closed and that network, external-program, and external-file shapes are rejected. These remain collector/evidence-grammar issues; there is no database change. Per reviewer policy I did not run tests or QA.

[Gandy2025]

Resolved the two follow-up blockers in 4d087cd.

1. first-tree tree tree selector executions now require explicit --no-pull. Only the exact standalone -h / --help forms are handled separately; the default-refresh form is rejected. Regression coverage includes accepted --no-pull, accepted exact help, and rejected implicit refresh.
2. Every accepted rg call now requires explicit --no-config, so historical command evidence cannot inherit behavior from RIPGREP_CONFIG_PATH. Regression coverage includes an accepted closed option grammar with --no-config and a rejected implicit-config form.

The evidence schema and manual behavior checklist now state both requirements.

Validation:

• python3 scripts/validate_skill.py
• python3 -m compileall -q skills tests scripts projections
• python3 -m unittest discover -s tests -v — 40/40 passed
• git diff --check
• two independent focused reviews — no blocking findings

[yuezengwu]

The two requested fixes are correctly implemented at 4d087cd: selector execution now requires --no-pull (with exact help isolated), and accepted rg calls require --no-config. One equivalent implicit-configuration execution path remains in the existing Git diagnostic whitelist:

safe_git_diagnostic() accepts git diff, git log, and git show without proving that external diff/text-conversion helpers are disabled (skills/context-tree-value-audit/scripts/context_tree_value_audit.py:2146-2183). Git’s own diff-options documentation states that textconv helpers are enabled by default for git diff and git log family commands, and repository attributes/config can name an external helper; --no-ext-diff and --no-textconv are the controls that disable those paths: https://git-scm.com/docs/diff-options. Historical tool rows do not bind the Git config, attributes, or process environment, so a command such as git -C <tree> diff && cat <tree>/NODE.md can still be classified accepted_read_only_composite while the original Git process executed an external program.

Please close the Git grammar the same way as rg: for commands capable of diff rendering, require the explicit no-helper controls (or otherwise persist and validate the exact historical Git configuration/environment), and add a regression with configured textconv/external-diff potential. Please also review the remaining whitelisted Git subcommands for other config-driven process hooks before treating the whitelist as statically closed.

This is the same collector/evidence-safety boundary, not a database change. Per reviewer policy I reviewed statically and did not run tests or QA.

[Gandy2025]

Resolved the Git configured-helper blocker in 99a75e2.

The collector no longer accepts any Git command as a statically closed read-only diagnostic. This is intentionally stricter than maintaining a partial subcommand grammar: unbound system/global/repository/environment configuration can activate external helpers during rendering, status, and even index-backed revision resolution.

The unsafe classifier now scans the full Git argv, including global -c, --config-env, and --exec-path forms, before subcommand handling. Regression coverage includes:

• diff, log, show, and status configured-helper potential;
• rev-parse with an index-backed revision;
• rev-parse --resolve-git-dir with an external path;
• global --exec-path and inline -c diff.external=...;
• remote get-url;
• exact unsafe reason/status conservation.

The evidence schema and manual checklist now consistently state the all-Git fail-closed policy.

Validation:

• python3 scripts/validate_skill.py
• python3 -m compileall -q skills tests scripts projections
• python3 -m unittest discover -s tests -v — 40/40 passed
• git diff --check
• two independent final reviews — no blocking findings

[yuezengwu]

The Git blocker is fully closed at 99a75e2: all Git commands now fail closed instead of depending on unbound config/attributes. The same external-helper issue remains in the bat content-reader whitelist.

bat is accepted as a pure reader (skills/context-tree-value-audit/scripts/context_tree_value_audit.py:69, :1664-1803), and the parser explicitly accepts any --paging=... value, including --paging=always, without requiring --no-config or no-paging mode. Bat’s own documentation states that it loads system/user or BAT_CONFIG_PATH configuration, that config may set --pager, and that paging launches the command from BAT_PAGER/PAGER; a TTY or forced --paging=always can therefore execute an external pager: https://github.com/sharkdp/bat#automatic-paging and https://github.com/sharkdp/bat#configuration-file. The historical shell assessment does not bind the TTY flag, Bat config, or pager environment, so bat <tree>/NODE.md (and explicitly bat --paging=always <tree>/NODE.md) can still be classified as an exact/read-only Tree read while the original process launched an unbound external program.

Please either remove bat from the accepted reader grammar, or require explicit CLI controls that override both paths (at minimum --no-config plus documented never-page mode) and reject every other paging form. Add regressions for bare Bat, forced paging, and the fully closed accepted form. Please apply the same final implicit-helper scan to the remaining accepted executables before claiming the grammar is statically closed.

This is a collector/evidence-grammar safety issue, not a database change. Per reviewer policy I reviewed statically and did not run tests or QA.

[Gandy2025]

Addressed at exact head 99a75e218fb03a80102cf33ebca75b66a5fff486.

The follow-up takes the conservative Skill-only route: historical tool rows do not seal Git's system/global/repository configuration, attributes, environment, hooks, or optional-lock behavior, so no Git diagnostic is now eligible for accepted_read_only_composite.

• diff, log, show, and status fail closed as configured-helper-capable shapes.
• -c, --config-env, --exec-path, external diff/textconv, --no-index, and output options retain explicit unsafe reasons.
• Other Git invocations fail closed as unsafe_git_unbound_configuration instead of being treated as statically closed diagnostics.
• Deterministic coverage includes diff/log/show/status, inline/global config, index-backed revision resolution, an external repository path, and the formerly accepted remote get-url shape.
• The evidence schema and manual checklist now state this boundary explicitly.

Validation on the immutable commit:

• repository validator: PASS
• canonical Skill Creator validator: PASS
• compileall: PASS
• deterministic suite: 40/40 PASS
• range diff check: PASS
• GitHub Actions Python 3.11 / 3.12: PASS

Two independent exact-range reviews are in progress. The PR remains blocked; this comment does not request merge or release yet.

[Gandy2025]

Resolved the remaining implicit-helper and open-grammar paths in d9491d5.

• Removed bat from accepted readers; bare, forced-paging, and closed-looking forms now fail closed.
• Removed find and ls from accepted diagnostics; file-valued Find forms and recursive/dereferencing LS forms now fail closed.
• Accepted rg reads now require both --no-config and --no-ignore, closing external config, parent/global/VCS ignore, and helper-capable option paths.
• Pipeline head, tail, and nl now use exact per-command option/value grammars; help, version, follow, unknown, and output-replacing forms are unresolved instead of being attributed to Tree content.
• Removed the open compact-option fallback from direct nl reads.

I also performed a final broad scan of every remaining accepted executable. Two independent reviewers found no remaining pager, preprocessor, external-helper, file-valued-option, recursive-dereference, or open-argv blocker.

Validation:

• python3 scripts/validate_skill.py
• python3 -m compileall -q skills tests scripts projections
• python3 -m unittest discover -s tests -v — 40/40 passed
• git diff --check
• two independent final broad reviews — no blocking findings

[yuezengwu]

Approved at exact head d9491d529b7823defb7ac0731ab48df7fb65ae2f.

The prior Bat blocker is fully resolved: Bat is no longer eligible for accepted read evidence, so unbound config and pager execution cannot be misclassified as a statically closed read. The same incremental range conservatively removes find and ls, requires rg --no-config --no-ignore, and closes the accepted head / tail / nl pipeline-filter grammars.

Together with the earlier fixes, the collector now preserves the four-state read-attempt denominator while failing closed on runtime identity mismatch, provider pairing anomalies, mutable/config-driven command behavior, external paths, and helper-capable Git commands. I found no new blocking issue in this incremental range or the previously reviewed full diff.

This remains a core collector/evidence-schema change, but there is no database change. Per reviewer policy, I did not run tests or QA; the approval is based on static diff review (GitHub Actions is currently green independently).