RuntimeError on async generator cleanup due to task group context mismatch

[deleon626]

Bug Report: RuntimeError on async generator cleanup due to task group context mismatch

Summary

The Claude Agent SDK throws a RuntimeError: Attempted to exit cancel scope in a different task than it was entered in during cleanup when using the query() async generator. The functionality works correctly, but the error appears during shutdown.

Environment

• Python Version: 3.13
• claude-agent-sdk Version: (latest from pip)
• anyio Version: (dependency of SDK)
• OS: macOS Darwin 24.6.0

Reproduction

Minimal Example

import asyncio
from claude_agent_sdk import query, ClaudeAgentOptions, ResultMessage

async def main():
    options = ClaudeAgentOptions(
        model="claude-sonnet-4-20250514",
        output_format={"type": "json_schema", "schema": {"type": "object"}}
    )

    async for message in query(prompt="Say hello", options=options):
        if isinstance(message, ResultMessage):
            print(message.result)
            break  # Exit early after getting result

if __name__ == "__main__":
    asyncio.run(main())

Error Output

Task exception was never retrieved
future: <Task finished name='Task-5' coro=<<async_generator_athrow without __name__>()> exception=RuntimeError('Attempted to exit cancel scope in a different task than it was entered in')>
Traceback (most recent call last):
  File ".../claude_agent_sdk/_internal/client.py", line 121, in process_query
    yield parse_message(data)
GeneratorExit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../claude_agent_sdk/_internal/client.py", line 124, in process_query
    await query.close()
  File ".../claude_agent_sdk/_internal/query.py", line 609, in close
    await self._tg.__aexit__(None, None, None)
  File ".../anyio/_backends/_asyncio.py", line 783, in __aexit__
    return self.cancel_scope.__exit__(exc_type, exc_val, exc_tb)
  File ".../anyio/_backends/_asyncio.py", line 457, in __exit__
    raise RuntimeError(
        "Attempted to exit cancel scope in a different task than it was entered in"
    )
RuntimeError: Attempted to exit cancel scope in a different task than it was entered in

Root Cause Analysis

The issue is in the manual management of the anyio task group lifecycle in query.py:

Current Implementation

In query.py:160-165 (start method):

async def start(self) -> None:
    """Start reading messages from transport."""
    if self._tg is None:
        self._tg = anyio.create_task_group()
        await self._tg.__aenter__()  # Entered in context A
        self._tg.start_soon(self._read_messages)

In query.py:602-610 (close method):

async def close(self) -> None:
    """Close the query and transport."""
    self._closed = True
    if self._tg:
        self._tg.cancel_scope.cancel()
        with suppress(anyio.get_cancelled_exc_class()):
            await self._tg.__aexit__(None, None, None)  # Exited in context B
    await self.transport.close()

Why It Fails

1. __aenter__() is called in the process_query() generator context when query.start() is invoked
2. When the user breaks out of the async for loop early (or when asyncio.run() shuts down), Python cleans up the generator
3. The finally block in client.py:123-124 calls query.close()
4. This cleanup happens in a different async task context than where __aenter__() was called
5. AnyIO's cancel scopes require enter/exit to happen in the same task, hence the RuntimeError

Execution Flow

client.py:106  →  query.start()
                      ↓
query.py:164   →  self._tg.__aenter__()  [Task Context A]
                      ↓
client.py:120  →  yield messages (generator running)
                      ↓
                  [User breaks loop or asyncio.run() exits]
                      ↓
client.py:124  →  query.close()  [Task Context B - cleanup task]
                      ↓
query.py:609   →  self._tg.__aexit__()  [FAILS - wrong context]

Suggested Fix

Option 1: Use proper async context manager pattern

Restructure the code to use async with instead of manual __aenter__/__aexit__:

async def start(self) -> None:
    """Start reading messages from transport."""
    if self._tg is None:
        # Don't manually call __aenter__, instead manage lifecycle differently
        pass

# Use async with in the calling code
async with anyio.create_task_group() as tg:
    query._tg = tg
    tg.start_soon(query._read_messages)
    # ... rest of logic

Option 2: Handle cleanup in the same context

Ensure close() is called within the same async context as start():

async def close(self) -> None:
    """Close the query and transport."""
    self._closed = True
    if self._tg:
        self._tg.cancel_scope.cancel()
        # Don't call __aexit__ manually - let the context manager handle it
        # Or use a flag to signal the task group to exit gracefully
    await self.transport.close()

Option 3: Use anyio.from_thread or task-safe cleanup

async def close(self) -> None:
    """Close the query and transport."""
    self._closed = True
    if self._tg:
        self._tg.cancel_scope.cancel()
        # Skip __aexit__ if we're in a different context
        try:
            await self._tg.__aexit__(None, None, None)
        except RuntimeError as e:
            if "different task" in str(e):
                pass  # Expected during generator cleanup
            else:
                raise
    await self.transport.close()

Impact

• Severity: Low (functionality works, just produces warning)
• Frequency: Every time an async generator exits before natural completion
• User Impact: Confusing error messages, potential for masking real errors

Additional Context

This issue is specific to:

• Using the query() function as an async generator
• Breaking out of the loop early (e.g., after getting ResultMessage)
• Python 3.13 with anyio's asyncio backend

The error does not occur when:

• The generator naturally exhausts all messages
• Using synchronous wrappers that handle cleanup differently

Related Issues

• Similar pattern issues in other async libraries: https://github.com/agronholm/anyio/issues/XXX
• Python async generator cleanup behavior: https://docs.python.org/3/reference/expressions.html#asynchronous-generator-functions