Does the DFlash acceleration effect on Qwen3.5 2B or 0.8B show significant improvement?

[xinanjiao]

Thank you for your amazing work！
I trained DFlash in the reproduced open-source code. When trying 0.8B, the acceptance length and speedup were unsatisfactory. Have you tried experimenting with smaller models?
Also, I'm not sure if my DFlash configuration is incorrect. Could you please provide the DFlash configuration for Qwen 3.5 2B or 0.8B?
Thank you again for your excellent work.

[PlyMxt]

I have 6GB of VRAM, so I've also been looking for DFlash models in smaller sizes. Unfortunately, there aren't any below 4B in the public space yet. I'd really like to try this on such small models, although DFlash itself will take up about a gigabyte of VRAM, but that should be fine. I just wouldn't run DFlash on my card without tensor cores — it would be a bottleneck. Maybe that's why they're not training them, but on the other hand, training on small models seems like it would require less compute than on large ones. Although honestly, who knows — I'm just basing this on the compute required for token generation by the models themselves.

[PlyMxt]

Although Flash Attention 2 is only available on the RTX 30 series, and those cards already have only 8 GB of VRAM, so training 2B and 0.8B models probably really doesn't make sense, except maybe as a personal experiment.

Although this turned out to be my misunderstanding based on a discussion I saw on HF, sdqa can still be applied, though it will be roughly twice as slow.

[jmking1]

+1 on this. I'm also interested in seeing DFlash results / configurations on smaller models like Qwen 2B or 0.8B. Would really appreciate it if the authors could share the config or any reference numbers. Thanks!