Skip to content

Commit b8b22b9

Browse files
committed
Merge branch 'pr_ds32_mtp' of https://github.com/rjg-lyh/vllm-ascend into pr_ds32_mtp
2 parents f410a81 + d252e36 commit b8b22b9

File tree

3 files changed

+13
-7
lines changed

3 files changed

+13
-7
lines changed

docs/source/installation.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -261,8 +261,14 @@ for output in outputs:
261261
Then run:
262262

263263
```bash
264-
# Try `export VLLM_USE_MODELSCOPE=true` and `pip install modelscope`
265-
# to speed up download if huggingface is not reachable.
264+
python example.py
265+
```
266+
267+
If you encounter a connection error with Hugging Face (e.g., `We couldn't connect to 'https://huggingface.co' to load the files, and couldn't find them in the cached files.`), run the following commands to use ModelScope as an alternative:
268+
269+
```bash
270+
export VLLM_USE_MODELSCOPE = true
271+
pip install modelscope
266272
python example.py
267273
```
268274

docs/source/tutorials/multi_node_kimi.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
Refer to [multi_node.md](https://vllm-ascend.readthedocs.io/en/latest/tutorials/multi_node.html#verification-process).
66

77
## Run with Docker
8-
Assume you have two Atlas 800 A3 (64G*16) or four A2 nodes, and want to deploy the `Kimi-K2-Instruct-W8A8` quantitative model across multiple nodes.
8+
Assume you have two Atlas 800 A3 (64G*16) or four A2 nodes, and want to deploy the `Kimi-K2-Instruct-W8A8` quantitative model across multiple nodes.
99

1010
```{code-block} bash
1111
:substitutions:

docs/source/user_guide/feature_guide/kv_pool_mooncake.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,10 @@
2121
Also, you need to set environment variables to point to them `export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib64/python3.11/site-packages/mooncake`, or copy the .so files to the `/usr/local/lib64` directory after compilation
2222

2323
### KV Pooling Parameter Description
24-
**kv_connector_extra_config**:Additional Configurable Parameters for Pooling.
25-
**mooncake_rpc_port**:Port for RPC Communication Between Pooling Scheduler Process and Worker Process: Each Instance Requires a Unique Port Configuration.
26-
**load_async**:Whether to Enable Asynchronous Loading. The default value is false.
27-
**register_buffer**:Whether to Register Video Memory with the Backend. Registration is Not Required When Used with MooncakeConnectorV1; It is Required in All Other Cases. The Default Value is false.
24+
**kv_connector_extra_config**: Additional Configurable Parameters for Pooling.
25+
**mooncake_rpc_port**: Port for RPC Communication Between Pooling Scheduler Process and Worker Process: Each Instance Requires a Unique Port Configuration.
26+
**load_async**: Whether to Enable Asynchronous Loading. The default value is false.
27+
**register_buffer**: Whether to Register Video Memory with the Backend. Registration is Not Required When Used with MooncakeConnectorV1; It is Required in All Other Cases. The Default Value is false.
2828

2929
## Run Mooncake Master
3030

0 commit comments

Comments
 (0)