agnosticism-audit: GL-005 Rules 1-2 unenforced, PKM/ wrongly in scope (root cause of #22)

[CodingEbert]

Summary

validation-script.sh's agnosticism-audit exists to enforce GL-005-llm-agnostic-portable-core. It enforces 2 of the Guideline's 4 rules, and it audits the wrong file set. The combination produces a false assurance: a file that violates GL-005 as blatantly as possible passes clean, and the audit affirmatively prints "portable core is host-agnostic."

The scope problem is also the root cause of #22.

Reproduction

A minimal 5.0.0 fixture with one contract, Team/Foo - Tester/AGENTS.md, using GL-005's own examples verbatim:

# Foo - Tester

This agent runs in Claude Code. On Codex and Cursor, behavior differs.
To dispatch a specialist, use the `Agent` tool. To modify a file, use the `Edit` tool.
Run the script via Bash. Read the file with the Read tool, then Glob for matches.
Gemini CLI users should use the equivalent tool instead.

Every line violates GL-005 Rule 1 (harness names) or Rule 2 (host tool names). Result:

--- agnosticism-audit (v4 tool-agnostic core) ---
ok:   no '.claude/' references in portable core
ok:   no 'subagent_type' references in portable core
...
ok:   agnosticism-audit: portable core is host-agnostic
  Failures: 0
  Warnings: 0

0 failures, 0 warnings, and an explicit certification of host-agnosticism for a maximally host-coupled file.

Rule coverage

GL-005 defines four rules. The audit's core_grep calls map to them as:

GL-005 rule	Audit check	Enforced?
Rule 1 — no harness names	\.claude/ only	No. Catches a path, not the brands the rule names ("Claude Code", "Codex", "Cursor", "Gemini CLI").
Rule 2 — no host-specific tool names	subagent_type only	No. One key. "the Edit tool", "via Bash", "the Agent tool" — GL-005's own examples — are unchecked.
Rule 3 — no slash-command-only trigger	slash-command check	Yes
Rule 4 — no hardcoded model	model-id regex	Yes, but broken — see #22

Real examples that pass silently today, from a live vault:

• Team/Echo - Voice Transcription Specialist/AGENTS.md: "Run it via Bash." — textbook Rule 2.
• Team/Nolan - HR/AGENTS.md: names Claude Code, Codex CLI, Gemini CLI — Rule 1, six times.
• Team/Mack - Automation Specialist/AGENTS.md: ".mcp.json for Claude Code, .cursor/mcp.json for Cursor" — Rule 1; only a WARN via the unrelated MCP check.

The scope bug — and why it is #22's root cause

CORE_FILES is built from all of PKM/**, all of Team Knowledge/** (including session-logs/), plus Team/*/AGENTS.md.

PKM/ is the member's own knowledge. A member who uses Claude Code will legitimately have PKM/My Life/Topics/claude-code.md, and journal entries mentioning it. That is a note about a tool, not host-coupling. No harness executes PKM/; it is data, not instructions.

This is exactly where #22's false positives live — all 9 are [[claude-code]] / [[claude-cowork]] in PKM journals and Topics.

Measured on a real vault: restricting the audit to governance docs + contracts (Team Knowledge/{SOPs,Workstreams,Guidelines,Templates} + Team/*/AGENTS.md), and dropping PKM/ and session-logs/:

• validation-script.sh: model-id check false-positives on claude-code and any claude-<word> slug #22's model-id false positives: 9 → 0, without touching the regex.
• Rule 1 candidate check: 9 real hits (Nolan ×6, Mack, WS-003, SOP-002) — all genuine.
• Rule 2 candidate check: 3 real hits (Echo, SOP-011, WS-004) — all genuine.

Session logs are historical record; v1.10.0's own recipe says to leave them untouched. Auditing them means a log that accurately records "we wrote the shim to .claude/agents/" is a hard failure forever.

The doctrine problem underneath

GL-005 §"Layer 1" declares the portable core to be PKM/ + Team Knowledge/ + the body of every Team/*/AGENTS.md. Rule 1 then says:

No file in the portable core may name a specific harness. No "Claude Code", "Codex", "Cursor", "Gemini CLI", or any other product name appears in PKM/, in Team Knowledge/, or in the body of any Team/*/AGENTS.md.

Read literally, GL-005 forbids a member from keeping a note about the tool they use. That cannot be the intent. The portable-core rule makes sense for things a harness executes — contracts, SOPs, Workstreams — and no sense for the member's personal knowledge, which is data.

Suggest GL-005's Layer 1 exclude PKM/ from the harness-naming rules (or scope Rule 1 to instructions, not data), and the audit follow.

Suggested fix

1. Fix the scope first. Audit Team Knowledge/{SOPs,Workstreams,Guidelines,Templates} + Team/*/AGENTS.md. Drop PKM/ and session-logs/. This alone resolves validation-script.sh: model-id check false-positives on claude-code and any claude-<word> slug #22 and makes the rest viable — adding Rule 1/2 checks before this would flood every vault with false positives on members' own notes.
2. Then add Rule 1 and Rule 2 checks. Candidates, tested against a real vault (0 false positives under the corrected scope):

# Rule 1 - harness product names. Multi-word anchors avoid "cursor position" etc.
core_grep '(Claude Code|Claude Desktop|Codex CLI|Gemini CLI|Cursor IDE|GitHub Copilot|Windsurf)'

# Rule 2 - host tool names.
core_grep '\b(Agent|Edit|Read|Write|Glob|Grep|Bash|MultiEdit|NotebookEdit|WebFetch|WebSearch) tool\b|\bvia `?(Bash|Agent|Edit)`?\b'

3. Legitimate host-naming needs an escape hatch, not an exemption list that grows. Some contracts must name hosts — Nolan generates per-host shims, Mack registers MCP servers, SOP-001 documents the shim formats. META_ALLOWLIST_RE already exempts GL-005/GL-002/SOP-001 by filename, which does not scale to member-authored contracts. The existing per-line agnosticism-audit:allow marker, or the "harness-config caveat" pattern the MCP check already uses, both fit better.

Environment

Scaffold 5.0.0; validation-script.sh byte-identical to the release archive. macOS, bash/BSD grep.

Related

• validation-script.sh: model-id check false-positives on claude-code and any claude-<word> slug #22 — the model-id false positive. Fixing the scope here resolves it at the root; the regex fix in validation-script.sh: model-id check false-positives on claude-code and any claude-<word> slug #22 is still worth doing as defense in depth, but scope is the deeper cause.