upgrade torch npu version #4433
Open
+60
−76
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wangxiyuan
added
ready
github-actions
bot
added
the
documentation
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request upgrades the required versions of torch and torch-npu from 2.7.1 to 2.8.0. The changes are applied consistently across all relevant files, including the CMake build configuration (CMakeLists.txt), Python dependency files (pyproject.toml, requirements.txt), and documentation (README.md, README.zh.md). The changes are correct and align with the goal of the pull request. I have not found any issues of high or critical severity.
wangxiyuan
force-pushed
the
upgrade_torch_npu
branch
2 times, most recently
from
b3935b7 to
708ebbc
Compare
wangxiyuan
force-pushed
the
upgrade_torch_npu
branch
from
708ebbc to
9d131fa
Compare
wangxiyuan
force-pushed
the
upgrade_torch_npu
branch
2 times, most recently
from
c61e8cb to
bf448d0
Compare
Contributor
|
torchair + v1 scheduler + torch2.8 test failed command: llm serve vllm-ascend/DeepSeek-V3-W8A8 \
--no-enable-prefix-caching \
--max-num-seqs 16 \
--tensor-parallel-size 16 \
--max-model-len 36864 \
--max-num-batched-tokens 6000 \
--enable-expert-parallel \
--trust-remote-code \
--quantization ascend \
--gpu-memory-utilization 0.9 \
--speculative-config '{"num_speculative_tokens": 1, "method":"deepseek_mtp"}' \
--additional-config '{"ascend_scheduler_config":{"enabled":false},"torchair_graph_config":{"enabled":true,"enable_multistream_moe":true},"chunked_prefill_for_mla":true,"enable_weight_nz_layout":true}' & error log |
wangxiyuan
force-pushed
the
upgrade_torch_npu
branch
2 times, most recently
from
300d5c4 to
7024251
Compare
wangxiyuan
force-pushed
the
upgrade_torch_npu
branch
from
7024251 to
9b88dd7
Compare
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
wangxiyuan
force-pushed
the
upgrade_torch_npu
branch
from
4d23315 to
9186dc2
Compare
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
wangxiyuan
force-pushed
the
upgrade_torch_npu
branch
2 times, most recently
from
ebbdffa to
acfac40
Compare
Signed-off-by: wangxiyuan <[email protected]>
wangxiyuan
force-pushed
the
upgrade_torch_npu
branch
from
acfac40 to
5dc99cf
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
vLLM graph feature now rely on torch >=2.8. To make graph mode work, we need upgrade torch version as well. For long term support, upgrade torch to a newer one is good to go as well.
Related vLLM change: vllm-project/vllm#25110