Skip to content

Complete PJRT copy futures via event callbacks instead of awaiter threads - #681

Merged
copybara-service[bot] merged 1 commit into
mainfrom
test_965614436
Aug 19, 2026
Merged

Complete PJRT copy futures via event callbacks instead of awaiter threads#681
copybara-service[bot] merged 1 commit into
mainfrom
test_965614436

Conversation

@copybara-service

@copybara-service copybara-service Bot commented Aug 16, 2026

Copy link
Copy Markdown

Complete PJRT copy futures via event callbacks instead of awaiter threads

PjRtCopyFuture::OnReady parked a detached thread in PJRT_Event_Await per
in-flight copy; on the transfer receive path that is one short-lived
thread per layer per transfer, and sustained decode-side load crashed
schedulers with a segfault in the detach on the push-handler path.
Register a PJRT_Event_OnReady callback per event instead, joined by a
countdown that preserves the first error; completion callbacks now run
on PJRT completion threads rather than a dedicated awaiter.

Core dump before the CL:
https://github.com/sgl-project/sglang-torchtpu/actions/runs/31962643994

!!!!!!! Segfault encountered !!!!!!!
  File "<unknown>", line 0, in raise
  File "<unknown>", line 0, in pthread_detach
  File "<unknown>", line 0, in std::thread::detach()
  File "<unknown>", line 0, in tpu_raiden::KVCacheManagerWithTransfer::OnLayerReceived(unsigned long, unsigned long)
  File "<unknown>", line 0, in tpu_raiden::transport::BlockTransport::HandleIncomingPush(int, tpu_raiden::transport::lib::ChunkHeader const&)
  File "<unknown>", line 0, in tpu_raiden::transport::BlockTransport::HandleCustomRequest(int, tpu_raiden::transport::lib::ChunkHeader const&)
  File "<unknown>", line 0, in tpu_raiden::transport::lib::RawBufferTransport::ProcessPeerRequest(int)
  File "<unknown>", line 0, in tpu_raiden::transport::lib::RawBufferTransport::ConnectionWorker(int)
  File "<unknown>", line 0, in 0xffffffffffffffff

@copybara-service
copybara-service Bot force-pushed the test_965614436 branch 10 times, most recently from a3b0e7e to c6cd39b Compare August 19, 2026 08:46
…eads

PjRtCopyFuture::OnReady parked a detached thread in PJRT_Event_Await per
in-flight copy; on the transfer receive path that is one short-lived
thread per layer per transfer, and sustained decode-side load crashed
schedulers with a segfault in the detach on the push-handler path.
Register a PJRT_Event_OnReady callback per event instead, joined by a
countdown that preserves the first error; completion callbacks now run
on PJRT completion threads rather than a dedicated awaiter.

Core dump before the CL:
https://github.com/sgl-project/sglang-torchtpu/actions/runs/31962643994
```
!!!!!!! Segfault encountered !!!!!!!
  File "<unknown>", line 0, in raise
  File "<unknown>", line 0, in pthread_detach
  File "<unknown>", line 0, in std::thread::detach()
  File "<unknown>", line 0, in tpu_raiden::KVCacheManagerWithTransfer::OnLayerReceived(unsigned long, unsigned long)
  File "<unknown>", line 0, in tpu_raiden::transport::BlockTransport::HandleIncomingPush(int, tpu_raiden::transport::lib::ChunkHeader const&)
  File "<unknown>", line 0, in tpu_raiden::transport::BlockTransport::HandleCustomRequest(int, tpu_raiden::transport::lib::ChunkHeader const&)
  File "<unknown>", line 0, in tpu_raiden::transport::lib::RawBufferTransport::ProcessPeerRequest(int)
  File "<unknown>", line 0, in tpu_raiden::transport::lib::RawBufferTransport::ConnectionWorker(int)
  File "<unknown>", line 0, in 0xffffffffffffffff
```

PiperOrigin-RevId: 967085230
@copybara-service
copybara-service Bot merged commit fd4754d into main Aug 19, 2026
@copybara-service
copybara-service Bot deleted the test_965614436 branch August 19, 2026 09:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant