unsloth
/

DeepSeek-V3-GGUF

Inference Endpoints

Model card Files Files and versions Community

Resources

View closed (1)

I loaded DeepSeek-V3-Q5_K_M up on my 10yrs old old Tesla M40 (Dell C4130)

#8 opened 1 day ago by

why use q5 for key cache?

#7 opened 1 day ago by

Advice on running llama-server with Q2_K_L quant

#6 opened 2 days ago by

What is the required GPU size to run Is a 4090 possible and does it support ollama

#5 opened 3 days ago by

I'm a newbie. How to use?

#4 opened 3 days ago by

Getting error with Q3-K-M

#2 opened 3 days ago by

Are these imatrix GGUF quants?

#1 opened 4 days ago by