File tree Expand file tree Collapse file tree 1 file changed +27
-17
lines changed Expand file tree Collapse file tree 1 file changed +27
-17
lines changed Original file line number Diff line number Diff line change @@ -20,20 +20,30 @@ Here's a simple example:
2020
2121NOTE: The other backends may have different support.
2222
23- | Quant / Type | CUDA |
24- | --------------| ------|
25- | F32 | ✔️ |
26- | F16 | ✔️ |
27- | BF16 | ✔️ |
28- | I32 | ✔️ |
29- | Q4_0 | ✔️ |
30- | Q4_1 | ✔️ |
31- | Q5_0 | ✔️ |
32- | Q5_1 | ✔️ |
33- | Q8_0 | ✔️ |
34- | Q2_K | ❌ |
35- | Q3_K | ❌ |
36- | Q4_K | ❌ |
37- | Q5_K | ❌ |
38- | Q6_K | ❌ |
39- | Q8_K | ❌ |
23+ | Quant / Type | CUDA | Vulkan |
24+ | --------------| ------| --------|
25+ | F32 | ✔️ | ✔️ |
26+ | F16 | ✔️ | ✔️ |
27+ | BF16 | ✔️ | ✔️ |
28+ | I32 | ✔️ | ❌ |
29+ | Q4_0 | ✔️ | ✔️ |
30+ | Q4_1 | ✔️ | ✔️ |
31+ | Q5_0 | ✔️ | ✔️ |
32+ | Q5_1 | ✔️ | ✔️ |
33+ | Q8_0 | ✔️ | ✔️ |
34+ | Q2_K | ❌ | ❌ |
35+ | Q3_K | ❌ | ❌ |
36+ | Q4_K | ❌ | ❌ |
37+ | Q5_K | ❌ | ❌ |
38+ | Q6_K | ❌ | ❌ |
39+ | Q8_K | ❌ | ❌ |
40+ | IQ1_S | ❌ | ✔️ |
41+ | IQ1_M | ❌ | ✔️ |
42+ | IQ2_XXS | ❌ | ✔️ |
43+ | IQ2_XS | ❌ | ✔️ |
44+ | IQ2_S | ❌ | ✔️ |
45+ | IQ3_XXS | ❌ | ✔️ |
46+ | IQ3_S | ❌ | ✔️ |
47+ | IQ4_XS | ❌ | ✔️ |
48+ | IQ4_NL | ❌ | ✔️ |
49+ | MXFP4 | ❌ | ✔️ |
You can’t perform that action at this time.
0 commit comments