DeepSeek V3 FP16 Atten NaN
This is a minimal reproduceable sample to let the final layer of DeepSeek V3's attention output NaNs when using data type float16.
Run the run.py to see the NaNs.
Weights are converted to bfloat16 using the original float8 e4m3fn, then converted to float16, then extracted from the final layer's attention.
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for v2ray/DeepSeek-V3-FP16-Atten-NaN
Base model
deepseek-ai/DeepSeek-V3