Skip to content

22 exception chain decomposition is not correct - #34

Merged
sahebbiswas merged 7 commits into
mainfrom
22-exception-chain-decomposition-is-not-correct
Mar 29, 2026
Merged

22 exception chain decomposition is not correct#34
sahebbiswas merged 7 commits into
mainfrom
22-exception-chain-decomposition-is-not-correct

Conversation

@sahebbiswas

@sahebbiswas sahebbiswas commented Mar 28, 2026

Copy link
Copy Markdown
Owner

Summary by CodeRabbit

  • New Features

    • Much-improved reconstruction of try/except/finally and with ... as ... blocks, including emitting "with as :" when present and better Python 3.9/3.11/3.14 handling.
  • Bug Fixes

    • Cleaner trailing output: implicit compiler-added returns are suppressed or replaced with pass.
    • Suppressed internal exception/cleanup artifacts (e.g., removed sentinel/noisy cleanup calls) and fixed handler/finally merge detection for accurate control flow.
  • Tests

    • Added extensive tests covering nested/sequential try/except/finally and with-statement scenarios.

@sahebbiswas sahebbiswas linked an issue Mar 28, 2026 that may be closed by this pull request
@sahebbiswas sahebbiswas self-assigned this Mar 28, 2026
@coderabbitai

coderabbitai Bot commented Mar 28, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This PR adds a prescan-driven reconstruction pass for try/except/finally and with-statements, extends the instruction model with argrepr, introduces helpers to detect compiler-generated trailing returns, refactors block-closing in decompile(), and implements deferred rendering and suppression of compiler/epilogue artifacts across version-specific handlers.

Changes

Cohort / File(s) Summary
Core decompiler & instruction model
pycrefine.py
Added BytecodeInstruction.argrepr; added DecompilerBase.is_effectively_last() and DecompilerBase.is_compiler_generated_return(); refactored DecompilerGeneric.decompile() to use _close_blocks() and perform a final cleanup pass to remove or replace compiler-added return None.
Prescan & deferred rendering for exceptions
pycrefine.pyDecompilerGeneric._prescan_try_structure(), _deferred_* maps/sets
New prescan populates try-entry NOP offsets, finally-merge offsets, suppressed PUSH_EXC_INFO offsets, handler/finally deferred instruction ranges, and NOP→PUSH_EXC_INFO maps for deferred emission and suppression.
Instruction dispatch & suppression
pycrefine.pyDecompilerGeneric._handle_instruction(), updated dispatch order
Dispatch now consults prescan results: skips suppressed with-exit epilogues, emits deferred except/finally at merge labels, suppresses inlined finally/handler/wrapper ranges, treats some backward jumps as handler exits, and avoids emitting compiler-generated trailing return None.
With-statement handling & NOP generalization
pycrefine.pySETUP_WITH, BEFORE_WITH, NOP handling
SETUP_WITH/BEFORE_WITH optionally consume an immediately following STORE_* to emit with <ctx> as <var>:; NOP try-entry detection generalized to prescanned offsets and integrates NOP→PUSH_EXC_INFO/finally-merge boundaries.
Version-specific logic (3.9 / 3.11+ / 3.14)
pycrefine.pyDecompiler39, Decompiler314, DecompilerGeneric tweaks
3.9/3.11+: improved POP_BLOCK/SETUP_FINALLY handling to distinguish wrapper merges and emit finally: correctly; 3.14: added LOAD_SPECIAL/COPY-based with preamble parsing and suppressed normal-path with cleanup sequences.
Tests
tests/test_pycrefine.py
Added tests asserting reconstruction of try/except/finally order, with ... as ... bindings, suppression of sentinel/cleanup artifacts and wrapper/re-raise machinery; minor whitespace/CR normalization edits.

Sequence Diagram(s)

sequenceDiagram
    participant Decompiler as Decompiler.decompile()
    participant Prescan as _prescan_try_structure()
    participant Dispatcher as _handle_instruction()
    participant Deferred as DeferredRenderers
    participant Output as OutputBuffer

    Decompiler->>Prescan: analyze bytecode for try/except/finally and with structure
    Prescan-->>Decompiler: return suppression ranges, merge labels, mappings

    loop per-instruction
        Decompiler->>Dispatcher: process instruction at pc
        Dispatcher->>Dispatcher: consult prescan maps
        alt pc is finally/except merge
            Dispatcher->>Deferred: emit deferred except/finally blocks
            Deferred->>Output: append reconstructed handler/finally
        else pc in suppression range
            Dispatcher->>Dispatcher: skip inlined epilogue/wrapper instrs
        else Normal
            Dispatcher->>Output: emit decompiled line
        end
    end

    Decompiler->>Decompiler: _close_blocks(0x7fffffff)
    Decompiler->>Output: post-pass cleanup (remove/replace compiler return)
Loading
sequenceDiagram
    participant Code as Bytecode
    participant Decompiler as DecompilerGeneric
    participant WithHandler as SETUP_WITH/BEFORE_WITH logic
    participant Output as OutputBuffer

    Code->>Decompiler: load bytecode with with/try/finally
    Decompiler->>Prescan: prescan structure
    Prescan-->>Decompiler: suppression & merge info

    Decompiler->>WithHandler: encounter SETUP_WITH or BEFORE_WITH
    WithHandler->>Decompiler: peek for STORE_* binding
    WithHandler->>Output: emit "with ... as var:" and track block extents
    Decompiler->>Output: later emit deferred finally/except at merge points
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~65 minutes

Possibly related issues

Possibly related PRs

Poem

In quiet bytecode I gently trace,
Each try, each with, each hidden place.
I fold away the needless art,
And leave the structure, clean and heart.
So code may speak, uncluttered, face to face.

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 64.44% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title references issue #22 about exception chain decomposition, yet the changeset encompasses comprehensive decompiler refactoring affecting try/except/finally handling, with/context managers, structural reconstruction, and extensive test coverage—far broader than the singular issue framing suggests. Consider a more descriptive title reflecting the substantial scope: e.g., 'Refactor exception handling and context manager decompilation with comprehensive try/except/finally structural reconstruction' to better represent the extensive changes and improvements across multiple decompiler classes.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • ✅ Generated successfully - (🔄 Check to regenerate)
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 22-exception-chain-decomposition-is-not-correct

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

@sahebbiswas
sahebbiswas merged commit 15f7e77 into main Mar 29, 2026
3 of 4 checks passed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pycrefine.py`:
- Line 593: The unpacked variables from self.blocks.pop() (currently written as
block_end, block_type) are unused and triggering lint warnings; rename them to
underscore-prefixed placeholders (e.g., _block_end, _block_type) to indicate
intentionally unused values and silence RUF059/B007; apply the same
underscore-prefixed rename pattern to the other identical unpackings in this
module (the other occurrences that unpack two values but don't use them).
- Around line 1669-1672: The blanket except Exception around
_dis.Bytecode(self.code_obj).exception_entries hides real bugs; restrict the
fallback to only expected metadata-shape failures (e.g., AttributeError,
TypeError, ValueError) when computing _exc_entries: catch those specific
exceptions and set _exc_entries = [], but allow any other exception to propagate
(do not swallow). Locate the try block using _dis.Bytecode and
.exception_entries and replace the broad except with a narrow tuple of exception
types (or re-raise unexpected ones) so only metadata-related failures are
handled.
- Around line 410-414: The detection currently reads the mutable runtime stack
(self.stack) to decide if a RETURN_VALUE is returning None, which can
misclassify instructions; instead, stop inspecting self.stack and use
instruction-local evidence: for RETURN_CONST keep the existing instr.argval
check (e.g., instr.argval is None) and for RETURN_VALUE examine the immediately
preceding bytecode instruction (e.g., prev_instr = instructions[i-1]) to see if
it is a LOAD_CONST whose argval is None (prev_instr.opname == "LOAD_CONST" and
prev_instr.argval is None); remove the branch that reads self.stack and rely
only on instr, op, instr.argval and the previous instruction to decide to
continue.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 244bf41e-c53e-47b0-b8ee-68e58b395738

📥 Commits

Reviewing files that changed from the base of the PR and between a11bbc5 and 16da51a.

📒 Files selected for processing (2)
  • pycrefine.py
  • tests/test_pycrefine.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/test_pycrefine.py

Comment thread pycrefine.py
Comment on lines +410 to +414
stack = getattr(self, "stack", None)
if op == "RETURN_VALUE" and stack and str(stack[-1]) == "None":
continue
if op == "RETURN_CONST" and str(instr.argval) == "None":
continue

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Detach trailing-return detection from mutable runtime stack state.

With humility, this check can misclassify later RETURN_VALUE instructions because it consults the current self.stack instead of instruction-local evidence. That can silently suppress a real return None in edge control-flow.

Suggested fix
-            # Skip subsequent None returns (common in 3.14 multiple exit paths)
-            stack = getattr(self, "stack", None)
-            if op == "RETURN_VALUE" and stack and str(stack[-1]) == "None":
-                continue
+            # Skip subsequent implicit None returns using instruction-local evidence.
+            if op == "RETURN_VALUE":
+                prev = self.instructions[i - 1] if i > 0 else None
+                if prev and prev.opname in ("LOAD_CONST", "LOAD_SMALL_INT") and prev.argval is None:
+                    continue
             if op == "RETURN_CONST" and str(instr.argval) == "None":
                 continue
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pycrefine.py` around lines 410 - 414, The detection currently reads the
mutable runtime stack (self.stack) to decide if a RETURN_VALUE is returning
None, which can misclassify instructions; instead, stop inspecting self.stack
and use instruction-local evidence: for RETURN_CONST keep the existing
instr.argval check (e.g., instr.argval is None) and for RETURN_VALUE examine the
immediately preceding bytecode instruction (e.g., prev_instr =
instructions[i-1]) to see if it is a LOAD_CONST whose argval is None
(prev_instr.opname == "LOAD_CONST" and prev_instr.argval is None); remove the
branch that reads self.stack and rely only on instr, op, instr.argval and the
previous instruction to decide to continue.

Comment thread pycrefine.py
_NO_INDENT_TYPES = frozenset({"finally_wrapper"})

while self.blocks and offset >= self.blocks[-1][0]:
block_end, block_type = self.blocks.pop()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Clean unused loop/unpack variables to keep lint gates peaceful.

These names are currently unused and trigger Ruff warnings (RUF059 / B007). A tiny rename to underscore-prefixed placeholders keeps intent clear and CI quiet.

Suggested fix
-            block_end, block_type = self.blocks.pop()
+            _block_end, block_type = self.blocks.pop()
@@
-        for push_off, merge_off in self._push_exc_to_finally_merge.items():
+        for _push_off, merge_off in self._push_exc_to_finally_merge.items():
@@
-                boff, btype = sub.blocks.pop()
+                _boff, _btype = sub.blocks.pop()
@@
-        for i, ins in enumerate(self.instructions):
+        for _i, ins in enumerate(self.instructions):

Also applies to: 1729-1729, 2044-2044, 3590-3590

🧰 Tools
🪛 Ruff (0.15.7)

[warning] 593-593: Unpacked variable block_end is never used

Prefix it with an underscore or any other dummy variable pattern

(RUF059)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pycrefine.py` at line 593, The unpacked variables from self.blocks.pop()
(currently written as block_end, block_type) are unused and triggering lint
warnings; rename them to underscore-prefixed placeholders (e.g., _block_end,
_block_type) to indicate intentionally unused values and silence RUF059/B007;
apply the same underscore-prefixed rename pattern to the other identical
unpackings in this module (the other occurrences that unpack two values but
don't use them).

Comment thread pycrefine.py
Comment on lines +1669 to +1672
try:
_exc_entries = _dis.Bytecode(self.code_obj).exception_entries # type: ignore[name-defined]
except Exception:
_exc_entries = []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Narrow this fallback catch to expected metadata failures.

With respect, except Exception here can bury genuine reconstruction defects. Please catch only expected metadata-shape failures and let unexpected faults surface.

Suggested fix
-        except Exception:
+        except (AttributeError, IndexError, TypeError, ValueError, NameError):
             _exc_entries = []
🧰 Tools
🪛 Ruff (0.15.7)

[warning] 1671-1671: Do not catch blind exception: Exception

(BLE001)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pycrefine.py` around lines 1669 - 1672, The blanket except Exception around
_dis.Bytecode(self.code_obj).exception_entries hides real bugs; restrict the
fallback to only expected metadata-shape failures (e.g., AttributeError,
TypeError, ValueError) when computing _exc_entries: catch those specific
exceptions and set _exc_entries = [], but allow any other exception to propagate
(do not swallow). Locate the try block using _dis.Bytecode and
.exception_entries and replace the broad except with a narrow tuple of exception
types (or re-raise unexpected ones) so only metadata-related failures are
handled.

@sahebbiswas
sahebbiswas deleted the 22-exception-chain-decomposition-is-not-correct branch March 29, 2026 03:36
@coderabbitai

coderabbitai Bot commented Mar 29, 2026

Copy link
Copy Markdown
Contributor

Note

Docstrings generation - SUCCESS
Generated docstrings for this pull request at #35

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exception chain decomposition is not correct

1 participant