when will add training of deepseek v3？ it`s a big update of llamas

[sankexin]

⚠️ Please check that this feature request hasn't been suggested before.

• I searched previous Ideas in Discussions didn't find any similar feature requests.
• I searched previous Issues didn't find any similar feature requests.

🔖 Feature description

This project is very good, and DeepSeekv3's algorithm will become the foundation of a new generation of universal models.
Joining early would be very beneficial for this project, and of course, it would also be easier for us to use.

✔️ Solution

for example：
1、FP8 mixed precision training
2、like medusa of axolot one years ago: multi-token prediction training
3、Latent 𝐜𝑡𝐾V：q_lora_rank、kv_lora_rank
4、Large model distillation small model

https://github.com/deepseek-ai/DeepSeek-V3/blob/main/inference/model.py

These technologies are very efficient in deepseek v3，and can be extended to other models.

❓ Alternatives

No response

📝 Additional Context

No response

Acknowledgements

• My issue title is concise, descriptive, and in title casing.
• I have searched the existing issues to make sure this feature has not been requested yet.
• I have provided enough information for the maintainers to understand and evaluate this request.

[NanoCode012]

Are you able to try #2230 ? I think this minimal change should allow it to work with packing.

[ehartford]

need fp8 native, I don't wanna train in 16bit