Skip to content

docs : clarify tensor split context sizing - #26453

Open
zcxGGmu wants to merge 1 commit into
ggml-org:masterfrom
zcxGGmu:docs/tensor-parallel-ctx-size
Open

docs : clarify tensor split context sizing#26453
zcxGGmu wants to merge 1 commit into
ggml-org:masterfrom
zcxGGmu:docs/tensor-parallel-ctx-size

Conversation

@zcxGGmu

@zcxGGmu zcxGGmu commented Aug 2, 2026

Copy link
Copy Markdown

Summary

  • clarify that --split-mode tensor shards KV cache and can allow larger context sizes on the same hardware
  • note that MTP/speculative decoding should be validated separately when using tensor split
  • fix a small spacing typo in the tensor split note

Test Plan

  • Documentation-only change; checked with git diff --check

Fixes #25732

Review Complexity : Low

Document that tensor split shards the KV cache and can allow larger context sizes, while MTP/speculative decoding should be validated separately.
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 2, 2026
@ggml-gh-bot

ggml-gh-bot Bot commented Aug 2, 2026

Copy link
Copy Markdown

Hi @zcxGGmu, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

  • PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

  • Multiple open PRs from a new contributor: We limit new contributors (those without a previously merged PR) to 1 open PR at a time. You currently have 2 open PRs.


Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: double ctx-size with "split-mode: tensor" clarification.

1 participant