feat(parser): schema-carried is_test flag on chunks — v33, PARSER_VERSION 18 (closes #2249)

[jamie8johnson]

Closes #2249 — a parse-time, schema-carried is_test flag on every chunk, so rank-time consumers can identify #[cfg(test)]-in-src/ test code that path/name detection can't see. This is the fix the CQS_TESTCODE_DEMOTE A/B (#2251) proved necessary: the audit's dominant near-miss crowders are #[cfg(test)] functions inside src/ files (e.g. src/parser/chunk.rs tests beating src/schema.sql golds), invisible to is_test_chunk (src/ path, no test_ name) and to content sniffing (chunk content starts at fn).

What lands

• v33 chunks.is_test column (additive migration, default 0; v10→v33 full chain + is_test=0 legacy-default guard tested) + PARSER_VERSION 17→18 so a plain reindex re-parses and populates the flag.
• Parse-time detection (src/parser/chunk.rs): Rust #[test]-family attributes + #[cfg(test)]/cfg(all(test,…)) gates on the item or any ancestor mod (any(test,…)/not(test)/feature="test" correctly excluded) + the is_test_chunk registry patterns (all languages). Windowed chunks inherit the parent's flag.
• Knob + signal rewired to the stored flag: apply_testcode_demote AND the test_demote rank-signal both read chunk.is_test (same field on the same SearchResult — divergence structurally impossible, preserving the feat(search): CQS_TESTCODE_DEMOTE — demote test-origin chunks on non-test-seeking queries (default-inert) #2251 coherence pin).
• The (d) decision: %/tests.rs + %/test.rs added to the Rust registry test_path_patterns (leading-/ anchored so contest.rs/latest.rs don't match). A deliberate, tested is_test_chunk behavior change (single source of truth — the same classifier feeds chunk_importance/scout/dead-code); the shared tests/common/mod.rs fixture was moved off the test.rs filename to accommodate it.

Review

Fable code-review: APPROVE, no blockers, MEDIUM risk (schema+parser blast radius). Verified: fan-out completeness — all ~130 is_test: construction sites mapped; every parse-path site computes or inherits, every false literal is a mod tests/fixture/ChunkRow::empty() (not the build_batched_with_dim silent-failure class); migration ordinal — is_test pinned at column ordinal 16, both appended-column readers moved 16→17, no other ordinal read ≥14, try_get(16) false-fallback; knob/signal coherence — structural; registry blast radius — intended, /-anchor pins the substring traps, tests.rs reclassification is more-accurate not a regression. Three LOW/INFO non-blocking residuals (chunk_importance still uses the registry sniff since CandidateRow lacks the flag; cross-file #[cfg(test)] mod x;/inner #![cfg(test)] under-detection; post-merge ranking shift) → tracked on #2253; the audit's dominant AST case is fully caught.

Gates

Full --tests sweep 4211 passed (only the pre-existing umap timing flake red — 5/5 solo, unrelated). Post-merge re-gate on current main: is_test_flag 6/0, provenance 15/0, migration 47/0, eval 8/0; fmt/clippy(all-targets, cuda-index)/provenance clean. Migration: v32→v33 default + fresh-vs-migrated parity + v10→v33 chain + legacy is_test=0 guard.

After merge (required)

Daemon redeploy + reindex — PARSER_VERSION 18 drives the full re-parse that populates the flag (no --force needed). Then /recall-gate (default ranking shifts: chunk_importance now hits tests.rs/test.rs chunks even at the knob's inert default) and a re-run of the CQS_TESTCODE_DEMOTE A/B — this is the change that should let the knob move the self-corpus R@5 the earlier A/B found flat.

🤖 Generated with Claude Code