RuntimeError: CurrentThreadExecutor already quit or is broken

[andersk]

This code ping-pongs between async_to_sync → sync_to_async → async_to_sync → loop.call_later → create_task → sync_to_async (although this sounds like a bizarre thing to do, it was minimized from a real problem in the Zulip server). It works with asgiref 3.7.2 and 3.8.1, but breaks with current main (abc69a0) with RuntimeError: CurrentThreadExecutor already quit or is broken. A Git bisection implicates 85d2445 as the first broken commit:

• Fix Local behaviour with asyncio Task.  #478, cc @spanezz

import asyncio

from asgiref.sync import async_to_sync, sync_to_async


def later(fut: asyncio.Future[asyncio.Task[None]]) -> None:
    task = asyncio.create_task(sync_to_async(print)("later"))
    fut.set_result(task)


async def inner() -> asyncio.Future[asyncio.Task[None]]:
    loop = asyncio.get_running_loop()
    fut = loop.create_future()
    loop.call_later(1, later, fut)
    return fut


async def main() -> None:
    fut = await sync_to_async(async_to_sync(inner))()
    task = await fut
    await task


async_to_sync(main)()

Traceback (most recent call last):
  File "/tmp/asgiref-bug/test.py", line 24, in <module>
    async_to_sync(main)()
  File "/tmp/asgiref-bug/.venv/lib/python3.12/site-packages/asgiref/sync.py", line 256, in __call__
    return call_result.result()
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/anders/.local/share/uv/python/cpython-3.12.5-linux-x86_64-gnu/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/anders/.local/share/uv/python/cpython-3.12.5-linux-x86_64-gnu/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/tmp/asgiref-bug/.venv/lib/python3.12/site-packages/asgiref/sync.py", line 332, in main_wrap
    result = await awaitable
             ^^^^^^^^^^^^^^^
  File "/tmp/asgiref-bug/test.py", line 21, in main
    await task
  File "/tmp/asgiref-bug/.venv/lib/python3.12/site-packages/asgiref/sync.py", line 456, in __call__
    exec_coro = loop.run_in_executor(
                ^^^^^^^^^^^^^^^^^^^^^
  File "/home/anders/.local/share/uv/python/cpython-3.12.5-linux-x86_64-gnu/lib/python3.12/asyncio/base_events.py", line 863, in run_in_executor
    executor.submit(func, *args), loop=self)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/asgiref-bug/.venv/lib/python3.12/site-packages/asgiref/current_thread_executor.py", line 115, in submit
    return self._submit(fn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/asgiref-bug/.venv/lib/python3.12/site-packages/asgiref/current_thread_executor.py", line 99, in _submit
    raise RuntimeError("CurrentThreadExecutor already quit or is broken")
RuntimeError: CurrentThreadExecutor already quit or is broken

[andrewgodwin]

Yup, this is why it's so hard to release changes to that code - I had been holding off releasing it until we could test it, and well, there you go.

I don't have any immediate good ideas on how to solve this except by rolling back the original commit, but I'd rather not, so hopefully there's something more nuanced that's broken here.

[andersk]

This is possibly related, although introduced earlier:

• [3.8 regression] Deadlock with async_to_sync → sync_to_async → async_to_sync → create_task → sync_to_async #492