MTP FP16 for M1/M2

[beamivalice]

Problem

Just drop in to say that someone found out why M1/M2 is slower with MTP. here mlx-lm

Proposal

If you can provide a model in hugging face with fp16 mtp for m1/m2 selection, it will be a tremendous help.

[youssofal]

Great minds think alike. I saw the thread earlier today and have created an FP16 varient and have update the CLI to auto detect M1/M2 chips.

The next release will have this.

Model: https://huggingface.co/Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed-FP16

[beamivalice]

Neat, seems like real goal is to reduce sampling time as much as possible. I am very curious though if fp8 mtp can push this even further (maybe at the cost of acceptance rate, but the speed could possibly justify n+1 depth)

[youssofal]

I am already using calibrated INT4 MTP heads and achieving 96% acceptance rate. The model linked uses INT4 MTP heads in MLX FP16.

[zumpchke]

When will this be available for M2?

[youssofal]

15 mins

[youssofal]

Download V3.0 I have added M1/M2 FP16 support.