liger-kernels: add device context manager#1029
Conversation
|
/kernel-bot build-and-stage |
|
No kernels provided. Use |
|
/kernel-bot build-and-stage liger-kernels |
|
Build request processed. Command: Dispatched (1):
|
|
/kernel-bot build-and-stage liger-kernels |
|
Build request processed. Command: Dispatched (1):
|
sayakpaul
left a comment
There was a problem hiding this comment.
I think it applies to all kernels that don't have this fix as you also pointed out in Slack. But I guess we can treat on a case by case basis?
|
Let me hunt them all down as best as I can, I think it's low effort for big impact tbh so no need to delay on case by case imo |
|
Added it to all triton launches I could find |
|
/kernel-bot merge-and-upload liger-kernels |
|
Build request processed. Command: Merge result: PR is already merged. Continuing with build/upload. Dispatched (1):
|
It's affecting models that use device map auto (under multi gpu setting) and
use_kernels=True, e.g. RMS Norm gets corrupted. But this goes for any triton kernel launch so it is propogated to all of themFor more details, see linkedin/Liger-Kernel#1303