Fix pre-compile on cpu-only machines

[AlongWY]

• Fix pre-compile on cpu-only machines

[tjruwase]

Suggested change

	if not torch.cuda.is_available():
	if not self.is_available():

[tjruwase]

@AlongWY, thanks for the PR. Can you please share the motivation to help us understand your use case?

[AlongWY]

@AlongWY, thanks for the PR. Can you please share the motivation to help us understand your use case?

I used GitHub Actions to precompile DeepSpeed, but after upgrading to v0.16.4, the precompilation failed. The repository is here: https://github.com/AlongWY/deepspeed_wheels

And the compile result here show that tranformer inference is not support cuda 12.6, which is also shown in #6709.

[loadams]

@AlongWY, thanks for the PR. Can you please share the motivation to help us understand your use case?

I used GitHub Actions to precompile DeepSpeed, but after upgrading to v0.16.4, the precompilation failed. The repository is here: https://github.com/AlongWY/deepspeed_wheels

And the compile result here show that tranformer inference is not support cuda 12.6, which is also shown in #6709.

@AlongWY - just so I follow, you're compiling in a GitHub action that has cuda installed but no GPU (so that's why you are hitting the CPU only path here) but with DS_ACCELERATOR=cuda? Just wanted to make sure I understood the repo correctly?

Also the error you shared is only on Windows (that we've observed) but your action appears to be running on Ubuntu, is that right?

[AlongWY]

Yes, the GitHub Action is compiling in an Ubuntu environment with CUDA installed but without a GPU.
The GitHub Action runs only on Ubuntu, but the error still occurs.

[loadams]

Yes, the GitHub Action is compiling in an Ubuntu environment with CUDA installed but without a GPU. The GitHub Action runs only on Ubuntu, but the error still occurs.

Thanks @AlongWY - can you share the error that you were seeing in this environment? Also if you have triton installed in the environment or not?

[AlongWY]

Yes, the triton has been installed in the environment.

Error for triton

Error for type