What
On real (non-test-fixture, non-drift-flagged) production source files, Python function
extraction (_slice_by_indentation, Mode C) loses contiguous ranges of otherwise ordinary
functions while functions immediately before and after the range are found correctly. This is
distinct from #1183 (mid-file language misrouting) — none of the three files below trigger
_partition_segments's language switch; every segment stays python throughout.
Root cause not yet isolated — filing this as a confirmed, reproducible pattern with concrete
evidence, not a diagnosed fix. It needs real debugging time in _slice_by_indentation /
_build_indentation_safe_stream.
Concrete repro (ast ground truth vs. extracted, in source order)
gitgalaxy/core/guidestar_lens.py — first 11 functions all missing, last 4 all found:
line 54 MISSED __init__
line 92 MISSED scan_project_config
... (9 more missed) ...
line 306 MISSED _parse_toml_style_manifest
line 321 FOUND _extract_execution_triggers
line 345 FOUND _scan_gitattributes
line 403 FOUND _scan_gitignore_evasion
line 452 FOUND _calculate_documentation_coverage
gitgalaxy/galaxyscope.py — first 3 found, then everything from line 682 to EOF (20 functions,
including main) missing:
line 88 FOUND execution_timeout_failsafe
line 99 FOUND _init_worker
line 178 FOUND _process_file_worker
line 682 MISSED __init__
... (19 more missed, through line 3013) ...
gitgalaxy/core/prism.py — a missing block sandwiched between two found blocks (lines 68-313
found, 425-718 missing, 731 found again) — rules out a simple "everything after one bad match
cascades to EOF" theory; whatever the mechanism is, it can apparently recover.
None of these are naming collisions or truncation (#1182) — the missed names are short,
ordinary, and unique within their file (__init__, main, cleanup, unmask, callback).
Precision stays high throughout (~99.7% corpus-wide once #1182's truncation is corrected for) —
this is a pure recall/loss problem, not a hallucination problem.
Scale
Measured against GitGalaxy's own 228-file Python self-scan corpus, restricted to the 217 files
that do not trigger #1183's language-drift bug: 2,183 real functions (ast ground truth) vs.
1,375 extracted (name-matched, truncation-normalized) — 63% recall on files with no other
known confounding bug. Class extraction on the same corpus is 100% (61/61), and the dedicated
per-rule unit test suite (tests/extraction/languages/test_python.py,
tests/extraction/languages/test_python_strict.py) already proves func_start's regex is
correct on isolated snippets — so this is specifically a scope/boundary-resolution problem in
_slice_by_indentation's dedent-scan, not a regex-correctness problem.
Related observations, likely downstream of the same root cause (not separately filed)
- Args-count accuracy: for functions that are found,
args count matches ast ground
truth exactly only ~46% of the time (944 compared, 433 exact matches). Some mismatches are
clean 0-counts on multi-line typed signatures that verify correctly when the args regex is
tested against the captured block in isolation (e.g. _slice_by_braces's own 6-parameter
signature) — suggesting the block text handed to the args regex in production differs from
what a standalone reproduction receives, possibly the same underlying mechanism as this issue.
file_data.struct_branch (the self-scan DB's per-file keyword-hit aggregate) undercounts
badly for some files (e.g. gitgalaxy/security/manifest_parser.py reports 0, vs. 271 real
if/for/while/... occurrences confirmed via tokenize and via running the branch regex
directly against the file's shielded code_stream, which found 301). The raw regex is fine in
isolation; the aggregation layer that rolls hits up appears to only credit hits attributed to
successfully-extracted functions, so it inherits this issue's loss.
How this was found
Diffing GitGalaxy's own self-scan of its own repo against Python's ast module as ground
truth, file by file, then confirming specific missed names weren't a naming/truncation artifact
by checking exact (non-truncated) name matches and manually reading the source at the
found/missed boundary lines.
What
On real (non-test-fixture, non-drift-flagged) production source files, Python function
extraction (
_slice_by_indentation, Mode C) loses contiguous ranges of otherwise ordinaryfunctions while functions immediately before and after the range are found correctly. This is
distinct from #1183 (mid-file language misrouting) — none of the three files below trigger
_partition_segments's language switch; every segment stayspythonthroughout.Root cause not yet isolated — filing this as a confirmed, reproducible pattern with concrete
evidence, not a diagnosed fix. It needs real debugging time in
_slice_by_indentation/_build_indentation_safe_stream.Concrete repro (ast ground truth vs. extracted, in source order)
gitgalaxy/core/guidestar_lens.py— first 11 functions all missing, last 4 all found:gitgalaxy/galaxyscope.py— first 3 found, then everything from line 682 to EOF (20 functions,including
main) missing:gitgalaxy/core/prism.py— a missing block sandwiched between two found blocks (lines 68-313found, 425-718 missing, 731 found again) — rules out a simple "everything after one bad match
cascades to EOF" theory; whatever the mechanism is, it can apparently recover.
None of these are naming collisions or truncation (#1182) — the missed names are short,
ordinary, and unique within their file (
__init__,main,cleanup,unmask,callback).Precision stays high throughout (~99.7% corpus-wide once #1182's truncation is corrected for) —
this is a pure recall/loss problem, not a hallucination problem.
Scale
Measured against GitGalaxy's own 228-file Python self-scan corpus, restricted to the 217 files
that do not trigger #1183's language-drift bug: 2,183 real functions (ast ground truth) vs.
1,375 extracted (name-matched, truncation-normalized) — 63% recall on files with no other
known confounding bug. Class extraction on the same corpus is 100% (61/61), and the dedicated
per-rule unit test suite (
tests/extraction/languages/test_python.py,tests/extraction/languages/test_python_strict.py) already provesfunc_start's regex iscorrect on isolated snippets — so this is specifically a scope/boundary-resolution problem in
_slice_by_indentation's dedent-scan, not a regex-correctness problem.Related observations, likely downstream of the same root cause (not separately filed)
argscount matchesastgroundtruth exactly only ~46% of the time (944 compared, 433 exact matches). Some mismatches are
clean 0-counts on multi-line typed signatures that verify correctly when the
argsregex istested against the captured block in isolation (e.g.
_slice_by_braces's own 6-parametersignature) — suggesting the block text handed to the args regex in production differs from
what a standalone reproduction receives, possibly the same underlying mechanism as this issue.
file_data.struct_branch(the self-scan DB's per-file keyword-hit aggregate) undercountsbadly for some files (e.g.
gitgalaxy/security/manifest_parser.pyreports 0, vs. 271 realif/for/while/...occurrences confirmed viatokenizeand via running thebranchregexdirectly against the file's shielded
code_stream, which found 301). The raw regex is fine inisolation; the aggregation layer that rolls hits up appears to only credit hits attributed to
successfully-extracted functions, so it inherits this issue's loss.
How this was found
Diffing GitGalaxy's own self-scan of its own repo against Python's
astmodule as groundtruth, file by file, then confirming specific missed names weren't a naming/truncation artifact
by checking exact (non-truncated) name matches and manually reading the source at the
found/missed boundary lines.