Skip to content

ggml: riscv: add riscv spacemit backend #15288

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

alex-spacemit
Copy link

This pull request adds spacemit backend. Specific optimizations have been made for the SpacemiT X60 CPU. The SpacemiT IME extended instructions are used to accelerate matrix calculations for Q4_0/Q4_1/Q4_K, along with general RVV optimizations.

model name      : Spacemit(R) X60
isa             : rv64imafdcv_zicbom_zicboz_zicntr_zicond_zicsr_zifencei_zihintpause_zihpm_zfh_zfhmin_zca_zcd_zba_zbb_zbc_zbs_zkt_zve32f_zve32x_zve64d_zve64f_zve64x_zvfh_zvfhmin_zvkt_sscofpmf_sstc_svinval_svnapot_svpbmt
mmu             : sv39
uarch           : spacemit,x60
mvendorid       : 0x710
marchid         : 0x8000000058000001

Q4_0

Model Size Params backend threads test t/s
Qwen2.5 0.5B 403.20 MiB 630.17 M cpu 4 pp512 64.12 ± 0.26
Qwen2.5 0.5B 403.20 MiB 630.17 M cpu 4 tg128 10.03 ± 0.01
Qwen2.5 1.5B 1011.16 MiB 1.78 B cpu 4 pp512 24.16 ± 0.02
Qwen2.5 1.5B 1011.16 MiB 1.78 B cpu 4 tg128 3.83 ± 0.06
Qwen2.5 3B 1.86 GiB 3.40 B cpu 4 pp512 12.08 ± 0.02
Qwen2.5 3B 1.86 GiB 3.40 B cpu 4 tg128 2.23 ± 0.02

more information at build-riscv64-spacemit.md

Change-Id: I249bdc043485d815a9c351867137bc1e27cc2e23
@github-actions github-actions bot added documentation Improvements or additions to documentation build Compilation issues ggml changes relating to the ggml tensor library for machine learning labels Aug 13, 2025
Change-Id: I889ed1c85fb45e62350ecde0c06f70450cadfbe2
Change-Id: I321eb200f859751727afe5cae13074dfce2bb0ce
@alex-spacemit
Copy link
Author

The file format and the limitations of the riscv zba extension have been resolved. @qnixsynapse

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Compilation issues documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants