Hello FastFlowLM team,
I would like to request NPU support for the Polish Llama-based model Bielik-11B-v3.0-Instruct-heretic-MPOA.
Model sources:
I converted the Q6_K GGUF to Q4NX_FLL successfully. The resulting model has 50 layers, hidden size 4096, intermediate size 14336, 32 attention heads, and 8 key/value heads.
System:
- Linux
- FastFlowLM 1.0.4
- AMD RyzenAI-npu6 / XDNA2
- XRT detects the NPU and
flm validate --json reports ready: true.
I tested the existing Llama-3.1-8B-NPU2 xclbins as a diagnostic only. The model loads and acquires the NPU, but generation fails with:
runlist failed execution (ERT_CMD_STATE_TIMEOUT)
Kernel Instance: MLIR_AIE
Could you please advise:
- Is there an existing compatible xclbin set for this Bielik/Llama architecture?
- If not, could support be added, including the required
attn.xclbin, dequant.xclbin, layer.xclbin, and mm.xclbin?
- Is there a supported public workflow for compiling FLM-compatible xclbins for a custom Q4NX_FLL model?
I can provide the Q4NX tensor metadata, model configuration, and reproducible logs if helpful.
Thank you.
Hello FastFlowLM team,
I would like to request NPU support for the Polish Llama-based model Bielik-11B-v3.0-Instruct-heretic-MPOA.
Model sources:
I converted the Q6_K GGUF to Q4NX_FLL successfully. The resulting model has 50 layers, hidden size 4096, intermediate size 14336, 32 attention heads, and 8 key/value heads.
System:
flm validate --jsonreports ready: true.I tested the existing
Llama-3.1-8B-NPU2xclbins as a diagnostic only. The model loads and acquires the NPU, but generation fails with:runlist failed execution (ERT_CMD_STATE_TIMEOUT)Kernel Instance: MLIR_AIECould you please advise:
attn.xclbin,dequant.xclbin,layer.xclbin, andmm.xclbin?I can provide the Q4NX tensor metadata, model configuration, and reproducible logs if helpful.
Thank you.