Suggestion Description
Devs are starting to run this model in CPU @ 6tps in Llama CPP and 12tps in CUDA in a RTX 3060 12Gb. Please, could you bring this model to XDNA2?
More open models are coming and they can even run at decent tps in CPU. By the paper XDNA2 would be able to handle these new models at a good pace.
Operating System
Ubuntu 26.04
GPU
Ryzen AI 9 HX 370
ROCm Component
No response
Suggestion Description
Devs are starting to run this model in CPU @ 6tps in Llama CPP and 12tps in CUDA in a RTX 3060 12Gb. Please, could you bring this model to XDNA2?
More open models are coming and they can even run at decent tps in CPU. By the paper XDNA2 would be able to handle these new models at a good pace.
Operating System
Ubuntu 26.04
GPU
Ryzen AI 9 HX 370
ROCm Component
No response