feat!: rebuild the audit on the durable Context Tree IO feed

[Gandy2025]

Rewrites the audit around first-tree tree io (first-tree-ai/first-tree#2118). ~14,700 deletions, ~1,600 insertions.

What changed and why

Exposure comes from recorded events. The runtime records Context Tree reads and writes at tool-execution time, across every runtime, in a table that outlives session timelines. The audit reads that instead of mining local transcripts.

The old approach statically parsed shell commands out of Codex/Claude transcripts to guess which nodes were read. On its own pilot that reached ~9% exact attribution (19 of 210 in-window calls), supported 2 of 5 runtimes, and cost ~1,900 lines that had to grow with every new tool and shell idiom while its accepted surface kept shrinking for safety.

Removing that layer removed the reason for most of the rest. The analysis unit is now a recorded read rather than an analyst-drawn task boundary. There is no boundary left for an analyst to redraw after seeing the evidence, so the blinding protocol, frozen inventory, SHA-256 digest, task-source projection, and 9-kind exclusion taxonomy all lost their purpose — roughly another 900 lines plus the hardest section of the workflow.

What it reports

Observed facts (no sampling, no judgment): observed exposure counts, node read distribution, observed write events, and a list of nodes with no observed read.

That list is an evidence gap, not a finding — a node lands there whenever no event reached the feed, which also happens for every pipeline shell read. It carries no removal or merge recommendation.

Influence (sampled): uniform random draw over observed file reads. Every claimed effect must survive an adversarial pass that explains the same choice without the Tree — the most common refutation being "the human already asked for it," which is exactly the confounder the old design listed as a condition and never checked.

The report publishes the refutation rate and withholds the influence numbers entirely when more than half the claims are refuted. An unreliable number is worse than none, because someone will quote it.

Honest limits, printed in every report

• Read telemetry is best-effort; pipeline shell reads (cat NODE.md | head -40) produce no event at all. Exposure is a lower bound, never a rate, and "no observed read" never means "never read".
• Search reads are directory-granular — Grep/Glob record the search root, so no node under one is listed as unobserved.
• Write events are telemetry-only and miss merge and out-of-path commits; complete write activity comes from git history.
• Node text is a candidate snapshot from the checkout HEAD observed at read time, not a promise of what was read. Effects that turn on specific wording are not claimed when the snapshot may not match.

These gaps live in First Tree's recording layer, not here.

Evidence conservation

• Events carry canonical Tree repo + branch identity; a changed binding or a mixed capture is excluded rather than credited by path. Origin ports are preserved, so two Self-Managed instances on one host stay distinct.
• report requires the sample's Tree identity, window, and eligible-population digest to match, and recomputes the exact draw from the recorded seed and size, checking each case against the event it names. Hand-picked or edited cases cannot produce effect counts.
• With the Tree pinned, an empty window reports zeros rather than failing — absence of IO is a result, not an error.

Packaging

Single payload for Team Skill Resource upload. First Tree materializes it into each runtime's skill root and manages versioning, which retires the README's manual per-runtime install/upgrade/quarantine/rollback ceremony and the Claude projection directory.

Validation

• python3 scripts/validate_skill.py — passed
• python3 -m compileall -q skills tests scripts — passed
• python3 -m unittest discover -s tests — 37 passed (0.05s, down from 40s)
• End-to-end runs against a real bound tree, including a tamper check confirming an edited case list is refused.

Tests focus on what the audit must refuse to claim: unobserved-node lists that ignore search coverage, adoption presented as a rate, effects that skipped the adversarial pass, influence numbers surviving a mostly-refuted run, cross-Tree events, and a sample that is not the recorded draw.

BREAKING CHANGE: no migration from 0.x; earlier artifacts are unreadable and should be discarded.

Depends on first-tree-ai/first-tree#2118 — merge that first.

🤖 Generated with Claude Code

[yuezengwu]

实现目标与核心改动：这次重写把审计的数据源从本地 runtime transcript 静态解析切换到 agent-scoped first-tree tree io feed，并把流程收敛成 facts → 随机 read sample → adversarial judgment → report；同时删除 0.x 的 task reconstruction / frozen inventory / 多 runtime projection，改成单一 Team Skill Resource payload。

以下 blocker 会让报告把不成立的证据当成事实：

1. IoEvent 在 skills/context-tree-value-audit/scripts/context_tree_value_audit.py:159-200 丢掉了 feed 自带的 treeRepoUrl 和 treeBranch，后续却把所有事件直接聚合并与任意 --tree-root 的当前文件树比较。只要组织在窗口内换过绑定、分支发生变化，或 --events-file 混入另一棵 Tree，同路径的 read 就会错误地给当前 Tree 记账；node_content_at_read() 还可能从错误 Tree 的 working copy 回退取内容。请保留并校验 canonical repo/branch identity，只允许与目标 Tree 明确匹配的事件进入 facts、sample 和 report；无法证明匹配时应 fail closed。

2. node_content_at_read() 在 skills/context-tree-value-audit/scripts/context_tree_value_audit.py:293-306 把 treeHeadCommit 命中的 git show 结果标为 exact_commit，而 SKILL.md:74-75 对 analyst 宣称这是“content as of that read”。依赖 PR #2118 的公开契约明确说明该字段只是读取时观察到的 checkout HEAD，不保证 dirty working file 与 commit 一致。因此 agent 读取未提交 Tree 改动时，这里会把旧 committed 内容冒充为实际读取内容并据此判断 influence。请把它降级为 candidate HEAD snapshot，并且除非另有证据证明工作树内容与该 commit 一致，否则不能声称或判断“as-of-read”内容。

3. README / Skill 把 exposure 称为 “complete”，把没有 file event 的节点称为 “never-read”，并进一步说这些节点稀释信号、可作为删除或合并候选（例如 README.md:6-14,92-94、SKILL.md:11-16,50-57、report renderer :380-395）。但同一实现又承认 pipeline shell reads 完全不可见；团队当前契约也明确规定 read telemetry 是 best-effort，missing evidence 是 unknown，而 agent IO write telemetry 会漏掉 merge / worktree writes。于是 “no recorded read” 不能推出 “never read”，这套 feed 也不能完整回答 Tree 是否在被写。请把产品和报告统一改成 recorded/best-effort 口径（例如 “no observed read event”），移除删除建议和 complete 声明；若要完整 write activity，应使用 git-derived writes。

4. command_report() 在 skills/context-tree-value-audit/scripts/context_tree_value_audit.py:641-673 只从 sample.json 建一个 case map，完全没有验证这些 read IDs 属于本次重新加载的 feed、相同的 repo/branch、相同 since/until，或确实是指定 seed/size 的 uniform sample。当前 Skill 示例还只固定 --since，所以 sample 与 report 之间新到达的事件会改变总体；手工或过期 sample 甚至可以在空/不同 feed 上产生 influence 数字。请固定一次事件窗口/identity，并在 report 阶段重新验证 sample conservation（或由同一不可变输入重算 sample），再允许发布 effect counts。

这是核心 artifact/schema 的 breaking rewrite（0.x artifacts 全部废弃），请 human 特别检查数据模型与兼容性口径；本 PR 本身没有数据库 schema 或 migration 变更。

Review scope：检查完整 diff、依赖 PR #2118 的当前公开 DTO/语义契约和现有 CI 结果；按本 agent 的 review 配置未在本地运行测试或 QA。

[Gandy2025]

四条全部成立，已在 73687ab 修完。这四条指向同一个问题——报告会把不成立的证据当成事实——而这恰恰是这个工具最不该犯的错。逐条：

1. Tree identity 丢失。 确认。IoEvent 完全没保留 treeRepoUrl / treeBranch，只按路径聚合。现在事件带上 canonical repo + branch（canonical_repo() 归一化 ssh/https/.git 拼写），select_events_for_tree() 只让能证明属于目标 Tree 的事件进入 facts / sample / report；无法归属的事件被排除并计数，无法 pin identity 的 feed 直接 fail closed。

2. exact_commit 过度声称。 确认，而且这个 caveat 正是我自己写进配套 Tree 节点的——然后在 skill 里违反了它。状态改为 head_commit_snapshot，每个 case 带上 caveat 字符串，SKILL.md 从"content as of that read"改成"candidate snapshot"，并明确要求：当快照可能与实际读到的内容不符时，不得据具体措辞判定 effect，应给 null。working-copy 回退同样带 caveat。

3. complete / never-read / 删除建议。 确认，这条我该被批评：我把两个已知缺口写进了报告，却仍然用 "complete"、"never-read" 和删除建议的措辞——等于一边声明证据有限、一边按证据完整下结论。现在全面改为 observed 口径：Never-read nodes → Nodes with no observed read，并明确标注"这是证据缺口，不是发现"、删除建议整段移除。写入部分改成 Observed write events (telemetry only) 并写明漏 merge / out-of-path commit、完整写入应取 git history。

4. sample ↔ report 无守恒校验。 确认。sample 现在记录 tree identity、window 和合格 read 全集的 sha256；report 重新拉取 feed 后必须三者全等才允许发布 effect 数，否则报错要求重跑 sample（新事件持续到达，这是常态而非异常）。

验证：validate_skill 通过、compileall 通过、29 个测试（+9）。新增测试覆盖：等价 remote 拼写归一、跨 Tree 事件被排除而非错记、分支变更 fail closed、混合 feed 无 identity 时 fail closed、快照 caveat、population digest 对新事件敏感且与顺序无关、以及一条断言报告中不出现 "never-read" / "candidate for removal" 的措辞回归。

依赖的 first-tree-ai/first-tree#2118 已双方批准。

[yuezengwu]

重新审查 head 73687abde5e8。实现目标不变：以 agent-scoped durable tree io feed 取代 transcript 重建，并用 facts → uniform sample → adversarial judgment → report 输出 observed exposure 与 influence。

上一轮的 treeHeadCommit 过度声明和 README / Skill / report 的 complete / never-read 口径已经实质修复；事件也开始保留 Tree identity。不过以下 blocker 仍会破坏证据守恒或跨 Tree 隔离：

1. population_sha256 只证明当前 eligible population 没变，并没有证明 sample.json 里的 cases 真的是该 population 按 seed / size 选出的 uniform sample。command_report() 在 skills/context-tree-value-audit/scripts/context_tree_value_audit.py:832-857 不读取或校验 sample 的 seed、sample_size、eligible_reads，也不把 case IDs 与 sample_reads(...) 的结果比较。拿一份合法 sample，保留 identity/window/population digest，手工替换 cases 为最容易判断的 reads（甚至不存在的 IDs/paths/timestamps），当前 report 仍会接受并发布 effect counts。请在 report 中从当前 eligible population、sample seed 和 size 重算所选 read IDs，并逐项校验 cases（至少 ID/path/time）；否则上一轮的 uniform-sample blocker 尚未关闭。

2. canonical_repo() 在 skills/context-tree-value-audit/scripts/context_tree_value_audit.py:171-182 通过 host.split(":")[0] 丢弃端口，因此 https://git.example:8443/org/tree 与 https://git.example:9443/org/tree 会被折叠成同一 identity。对于 Self-Managed GitLab，instance origin（包含非默认端口）是 authority 边界；这里会把不同实例的同路径事件错误记到当前 Tree。ssh://git@host:port/... 也会被前面的 scp regex 错拆。请使用与 First Tree 相同的 canonical repo 语义，并保留 origin port，补充跨端口不得相等的回归。

3. select_events_for_tree() 在 :263-264 无条件拒绝空 feed；当 --tree-root 已经提供可证明的 expected identity 时，“窗口内没有 observed IO”本身是合法且重要的审计结果，应输出零计数和 no-observed-read evidence gaps。类似地，feed 只有其他 Tree 的事件时，也能安全排除全部并报告 target Tree 为零，而不应把 absence 变成执行失败。请仅在没有 expected identity 且 feed 为空时 fail closed。

4. 对外文档虽已修正，但 evals/manual-behavior-checklist.md:13-14,20-40,75-76 仍要求已删除的 Claude projection、把列表称为 never-read，并声称 recorded commit 能给出 “node content as of the read”。CLI help 在 script :898,912 也仍称 never-read node list。这些是本 PR 的 release/eval contract，会让修复后的实现按旧的错误前提验收；请同步改成 canonical single payload、no-observed-read evidence gap 和 candidate snapshot 口径。

核心 artifact/schema 仍是 1.0 breaking rewrite，需要 human 检查兼容性；本增量及整份 PR 仍无数据库 schema 或 migration 变更。

Review scope：完整复查首个 head、逐行检查 73687ab 增量、依赖 PR #2118 的 DTO/语义契约与当前 CI 结果；按 review 配置未运行本地测试或 QA。源码 bare clone 的 HTTPS fetch 连续两次遇到 SSL_ERROR_SYSCALL，因此新 head 通过 GitHub API exact SHA 读取，未复用旧 head 文件。

[Gandy2025]

四条全部成立，d62af62 修完。

1. population digest 证明不了 cases 就是那次抽样。 确认——这是上一轮修复的真实漏洞，我的 fence 只挡住了"总体变了"，没挡住"总体没变但 cases 被换掉"。现在 report 从当前 eligible population + 记录的 seed/size 重算抽样，要求 read ID 序列完全一致，并逐条校验 case 的 target_path / read_at 与其命名的事件相符。

按你描述的攻击实测（保留 identity/window/digest，把 cases 换成最好判断的 reads）：

error: --sample cases are not the uniform draw for their recorded seed and size.
       Re-run `sample` instead of editing or hand-picking cases.

2. canonical_repo() 丢端口。 确认。Self-Managed GitLab 的 instance origin（含非默认端口）是 authority 边界，折叠会把不同实例的同路径事件错记。现在保留端口，并把 URL 形式的匹配放在 scp 形式之前——否则 ssh://git@host:2222/org/tree 会被 scp regex 错拆。回归覆盖：跨端口不相等、ssh://...:2222/... 正确解析、scp 形式冒号后仍按路径处理。

3. 空 feed 不该是失败。 确认，这条我搞反了：在 --tree-root 已经 pin 住 Tree 的前提下，"窗口内没有 observed IO"本身就是有价值的审计结果，应该出零计数和证据缺口，而不是执行失败。现在只有"feed 为空且无法证明 identity"才 fail closed；feed 里只有其他 Tree 的事件时，全部排除并对目标 Tree 报零。

4. eval / CLI help 仍按旧前提验收。 确认，这条最该批评——验收契约没跟上实现，等于用错误前提去验收修好的代码。checklist 已改为 canonical single payload、no-observed-read evidence gap、candidate snapshot 口径，并新增三条验收项（篡改 cases 必须被拒、跨 Tree/换分支事件必须被排除、空窗口必须报零而非失败）。argparse help 与 references/judging-effects.md 的措辞同步修正。

验证：validate_skill 通过、compileall 通过、37 个测试（+8）、并做了端到端实测确认篡改 fence 真的拦得住。

依赖的 first-tree-ai/first-tree#2118 CI 已全绿、双方批准，仅等 CLA 签署。

[yuezengwu]

Approved head d62af622f239.

实现目标与核心改动：以 agent-scoped durable first-tree tree io feed 取代 transcript 静态解析，把审计收敛为 observed facts → uniform read sample → adversarial judgment → report，并以单一 Team Skill Resource payload 分发。

两轮 review 的 blocker 已全部收敛：

• events 保留并校验 canonical Tree repo/branch identity；Self-Managed GitLab origin port 不再被折叠，SSH URL/scp 形式也分开解析。
• treeHeadCommit 和 working-copy fallback 都明确是 candidate snapshot，不再冒充实际 as-of-read 内容。
• read/write telemetry、no-observed-read list 和报告文案统一为 best-effort / evidence-gap 口径；完整 writes 明确回到 git history。
• report 会校验 Tree identity、window、eligible population digest，并从记录的 seed/size 重算 exact draw，逐项核对 read ID/path/time，手改或 hand-pick cases 无法发布 effect counts。
• 已 pin Tree identity 时，空窗口和仅含其他 Tree events 都正确输出 target Tree 的零观察结果。
• manual checklist、judging reference 与 CLI help 已同步到 canonical single payload、candidate snapshot 和 no-observed-read 契约。

核心 artifact/schema 是 1.0 breaking rewrite，0.x artifacts 无迁移；human 仍需按发布说明检查兼容性。此 PR 没有数据库 schema 或 migration 变更。

非阻塞建议：当前 PR description 仍写着 “Facts (complete)”、never-read、exact recorded commit 和 20 tests，与最终实现及 37-test 状态不一致，合并前请刷新，避免 forge 记录继续传播已修正的旧结论。依赖 first-tree#2118 当前已 approved 且功能 checks 全绿，但 CLA 仍失败；请保持 dependency-first 合并顺序。

Review scope：完整复查初始 diff 与两个修复增量，核对 first-tree#2118 当前 DTO/语义与 live checks；按 review 配置未运行本地测试或 QA。