#10 add pycrefine coherency analyzer

[sahebbiswas]

Summary by CodeRabbit

• New Features
  • Added decompilation coherency assessment tool to measure bytecode decompilation quality across multiple dimensions including imports, functions, classes, tokens, and lines.
  • Supports human-readable and JSON output formats with progress reporting and CI integration.
  • Includes CLI with configurable thresholds and diff sampling to identify decompilation gaps.

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

Introduce a coherency scoring system, new script does. Evaluate how well Python bytecode decompilation works, it shall. Compile source code, decompile with pycrefine, score across multiple dimensions, the tool can. Normalize text, extract features, render reports human-readable and JSON, it must.

Changes

Cohort / File(s)

Summary

New Coherency Scoring Tool debug/check_coherency.py

Comprehensive end-to-end decompilation fidelity measurement. Includes normalization pipeline, multi-dimensional scoring (imports, functions, classes, tokens, lines, keywords, cleanliness), fuzzy matching via trigram index, diff sampling, progress reporting, and dual output modes (human-readable and JSON). Exports public API: CoherencyReport, DimensionResult, score(), sample_diff(), print_report(), and main().

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Intricate logic throughout, much there is. Understand the feature extraction, normalization pipeline, scoring dimensions, and fuzzy matching algorithms, one must. Verify the weighted calculations and threshold logic, important it is. Multiple public APIs and their correctness, check carefully you should.

Poem

🔍 Decompile and score, this new tool will,
Coherency measure, dimensions fulfill,
From bytecode to source, the journey complete,
With fuzzy match and grade, precision so sweet,
Yoda approves this validation feat! ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly and concisely describes the main change: introducing a new coherency analyzer tool for pycrefine decompilation, which aligns with the substantial new debug/check_coherency.py file.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 10-add-a-decompilation-coherency-checker

📝 Coding Plan

• Generate coding plan for human review comments

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}