Add DSpark algorithm page

docs/.nav.yml

@@ -8,6 +8,7 @@ nav:
                 - user_guide/algorithms/index.md
                 - Eagle3: user_guide/algorithms/eagle3.md
                 - Dflash: user_guide/algorithms/dflash.md
+                - DSpark: user_guide/algorithms/dspark.md
                 - P-Eagle: user_guide/algorithms/peagle.md
                 - MTP: user_guide/algorithms/mtp.md
                 - Decision Guide: user_guide/algorithms/decision_guide.md

docs/user_guide/algorithms/decision_guide.md

@@ -1,6 +1,6 @@
 # Algorithm Decision Guide
 
-Speculators currently supports four speculative decoding algorithms: **Eagle-3**, **P-EAGLE**, **DFlash**, and **MTP**. All are lossless -- they produce output from the same distribution as the target model.
+Speculators currently supports five speculative decoding algorithms: **Eagle-3**, **P-EAGLE**, **DFlash**, **DSpark**, and **MTP**. All are lossless -- they produce output from the same distribution as the target model.
 
 ## How They Differ
 
@@ -10,29 +10,32 @@ Speculators currently supports four speculative decoding algorithms: **Eagle-3**
 
 **DFlash** predicts all draft tokens in a single forward pass using block-based prediction with anchor points.
 
+**DSpark** extends DFlash with a Markov head, so positions inside a block depend on earlier ones, plus a confidence head that predicts per-position acceptance.
+
 **MTP** finetunes the model's native multi-token prediction head on domain-specific data. Unlike the other algorithms, MTP does not train from scratch -- it starts from pre-existing MTP layers and is only available for models with native MTP support.
 
-Eagle-3, P-EAGLE, and DFlash can be paired with any supported verifier model (including quantized variants) -- the draft architecture is independent of the verifier architecture. MTP requires a model with native MTP layers (e.g. Qwen3-Next, Qwen3.5).
+Eagle-3, P-EAGLE, DFlash, and DSpark can be paired with any supported verifier model (including quantized variants) -- the draft architecture is independent of the verifier architecture. MTP requires a model with native MTP layers (e.g. Qwen3-Next, Qwen3.5).
 
 ## Current Support
 
-|                     | Eagle-3       | P-EAGLE             | DFlash              | MTP                         |
-| ------------------- | ------------- | ------------------- | ------------------- | --------------------------- |
-| **Draft layers**    | Llama-style   | Llama-style         | Qwen3-style         | Native MTP layers           |
-| **Verifier models** | Any supported | Any supported       | Any supported       | Models with native MTP only |
-| **Training mode**   | From scratch  | From scratch        | From scratch        | Finetune existing MTP head  |
-| **Speculators**     | Mature        | Newer, growing fast | Newer, growing fast | Newer, growing fast         |
-| **vLLM**            | Mature        | Newer, growing fast | Newer, growing fast | Newer, growing fast         |
+|                     | Eagle-3       | P-EAGLE             | DFlash              | DSpark              | MTP                         |
+| ------------------- | ------------- | ------------------- | ------------------- | ------------------- | --------------------------- |
+| **Draft layers**    | Llama-style   | Llama-style         | Qwen3-style         | Qwen3-style         | Native MTP layers           |
+| **Verifier models** | Any supported | Any supported       | Any supported       | Any supported       | Models with native MTP only |
+| **Training mode**   | From scratch  | From scratch        | From scratch        | From scratch        | Finetune existing MTP head  |
+| **Speculators**     | Mature        | Newer, growing fast | Newer, growing fast | Newer, growing fast | Newer, growing fast         |
+| **vLLM**            | Mature        | Newer, growing fast | Newer, growing fast | Newer, growing fast | Newer, growing fast         |
 
-Eagle-3 has been available longer and has broader support in both Speculators and vLLM. P-EAGLE, DFlash, and MTP were added more recently and support is improving rapidly.
+Eagle-3 has been available longer and has broader support in both Speculators and vLLM. P-EAGLE, DFlash, DSpark, and MTP were added more recently and support is improving rapidly.
 
 ## Which Should I Use?
 
-If you're unsure, start with Eagle-3 -- it has the most mature tooling and documentation. If you want parallel multi-token prediction with an Eagle-3-based architecture, try P-EAGLE. If you want to experiment with DFlash's single-forward-pass block prediction approach, the training workflow is the same. If your model already has native MTP layers (e.g. Qwen3-Next, Qwen3.5), MTP finetuning lets you improve the existing MTP head on domain-specific data without training a separate draft model.
+If you're unsure, start with Eagle-3 -- it has the most mature tooling and documentation. If you want parallel multi-token prediction with an Eagle-3-based architecture, try P-EAGLE. If you want to experiment with DFlash's single-forward-pass block prediction approach, the training workflow is the same, and DSpark adds intra-block dependencies and confidence scheduling on top of it. If your model already has native MTP layers (e.g. Qwen3-Next, Qwen3.5), MTP finetuning lets you improve the existing MTP head on domain-specific data without training a separate draft model.
 
 For more details on each algorithm, see:
 
 - [Eagle-3](eagle3.md)
 - [P-EAGLE](peagle.md)
 - [DFlash](dflash.md)
+- [DSpark](dspark.md)
 - [MTP](mtp.md)

docs/user_guide/algorithms/dspark.md

@@ -0,0 +1,63 @@
+# DSpark
+
+DSpark extends [DFlash](dflash.md) with two heads on top of the same block-parallel draft backbone: a low-rank Markov head that biases each draft position by the token before it, and a confidence head that predicts per-position acceptance probability. Pure block-parallel drafting has no dependency between tokens inside a block, so acceptance decays toward the end of the block -- the Markov head restores that dependency, and the confidence signal indicates how far a block is worth verifying. The draft model subclasses DFlash, so the architecture and training pipeline are otherwise unchanged, and it can be paired with any supported verifier. Serving uses vLLM's own `dspark` method (`"method": "dspark"` in `--speculative-config`).
+
+## How It Works
+
+### Markov Head
+
+The head adds a low-rank logit bias `B = W1 @ W2` to the DFlash logits: `W1` embeds the previous block token (verifier vocabulary) into `markov_rank` dimensions and `W2` projects to the draft vocabulary. Three variants are available:
+
+- **`vanilla` (default)**: bias from the previous token alone
+- **`gated`**: bias gated by the backbone hidden state
+- **`rnn`**: recurrent state carried across positions within the block
+
+Setting `--markov-rank 0` disables the head, leaving pure DFlash drafting. It must be paired with `--no-confidence-head-with-markov`, since that option requires a Markov head.
+
+### Confidence Head
+
+A linear head predicts each position's acceptance probability from the backbone hidden state, concatenated with the Markov previous-token embedding when `--confidence-head-with-markov` is set. It is trained with a BCE term weighted by `--confidence-head-alpha`.
+
+### Sample From Anchor
+
+DSpark defaults to `sample_from_anchor: True` -- the anchor and all mask positions predict future tokens, producing `block_size` speculative tokens. See [DFlash](dflash.md#sample-from-anchor) for details.
+
+## Key Parameters
+
+| Parameter                       | Default   | Description                                                       |
+| ------------------------------- | --------- | ----------------------------------------------------------------- |
+| `--markov-rank`                 | 256       | Low-rank dimension of the Markov logit bias (0 disables the head) |
+| `--markov-head-type`            | `vanilla` | Sequential head variant: `vanilla`, `gated`, or `rnn`             |
+| `--enable-confidence-head`      | enabled   | Attach the per-position acceptance head                           |
+| `--confidence-head-with-markov` | enabled   | Feed the Markov previous-token embedding into the confidence head |
+| `--confidence-head-alpha`       | 1.0       | Weight of the confidence-head BCE term                            |
+
+All DFlash parameters (`--block-size`, `--max-anchors`, `--num-layers`, ...) apply unchanged.
+
+## Pretrained Models
+
+Pretrained DSpark speculator models are available on HuggingFace from the [RedHatAI speculator models collection](https://huggingface.co/collections/RedHatAI/speculator-models):
+
+| Verifier              | Speculator                                                                                                        |
+| --------------------- | ----------------------------------------------------------------------------------------------------------------- |
+| `zai-org/GLM-5.2-FP8` | [`RedHatAI/GLM-5.2-speculator.dspark-preview`](https://huggingface.co/RedHatAI/GLM-5.2-speculator.dspark-preview) |
+
+To train your own, see `examples/train/dspark_qwen3_0_6b_sharegpt_online.sh`.
+
+## Research & Citation
+
+DSpark is based on research from DeepSeek: [arXiv Paper](https://arxiv.org/abs/2607.05147)
+
+```bibtex
+@article{cheng2026dspark,
+  title={DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation},
+  author={Cheng, Xin and Yu, Xingkai and Shao, Chenze and Li, Jiashi and Xiong, Yunfan and Qian, Yi and Zhu, Jiaqi and Ma, Shirong and Zhang, Xiaokang and Ye, Jiasheng and others},
+  journal={arXiv preprint arXiv:2607.05147},
+  year={2026}
+}
+```
+
+## See Also
+
+- [DFlash](dflash.md) -- The base algorithm DSpark extends
+- [Train DFlash Tutorial](../tutorials/train_dflash_online.md) -- Step-by-step training guide; the DSpark pipeline is the same plus the flags above

docs/user_guide/algorithms/index.md

@@ -1,6 +1,6 @@
 # Algorithms
 
-Speculators supports three speculative decoding algorithms. All are lossless -- they produce output from the same distribution as the target model.
+Speculators supports five speculative decoding algorithms. All are lossless -- they produce output from the same distribution as the target model.
 
 ## [Eagle-3](eagle3.md)
 
@@ -14,6 +14,10 @@ Extends Eagle-3 with parallel multi-token prediction across multiple depths, usi
 
 Predicts all draft tokens in a single forward pass using block-based prediction with Qwen3-style draft layers. Newer, with support improving rapidly.
 
+## [DSpark](dspark.md)
+
+Extends DFlash with a Markov head for intra-block token dependencies and a confidence head predicting per-position acceptance. Newest, with support improving rapidly.
+
 ## [MTP](mtp.md)
 
 Finetunes the model's native multi-token prediction head on domain-specific data. Available for models with built-in MTP support (e.g. Qwen3-Next, Qwen3.5).