Code

[Kernel][CPU] CPU MLA#14744

Code

Merged

LucasWilkinson merged 48 commits intovllm-project:mainvllm-project/vllm:mainfrom gau-nernst:cpu_mlagau-nernst/vllm:cpu_mlaCopy head branch name to clipboardMar 25, 2025

Commits

Commits on Mar 10, 2025

• Add concat_and_cache_mla for CPU

  Show description for ec93219
  gau-nernst

  committed
  ec93219

  Copy full SHA for ec93219

• Add OpenMP for concat and cache

  Show description for a614744
  gau-nernst

  committed
  a614744

  Copy full SHA for a614744

• naive MLA CPU

  Show description for ecccd26
  gau-nernst

  committed
  ecccd26

  Copy full SHA for ecccd26

Commits on Mar 11, 2025

• simplify naive implementation

  Show description for f20a8a1
  gau-nernst

  committed
  f20a8a1

  Copy full SHA for f20a8a1

• hook up MLA (still not working e2e)

  Show description for b1d2b6d
  gau-nernst

  committed
  b1d2b6d

  Copy full SHA for b1d2b6d

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  dabf061

  Copy full SHA for dabf061

Commits on Mar 12, 2025

• modify metadata from TorchSDPA. hook up IPEX for prefill

  Show description for 611afa6
  gau-nernst

  committed
  611afa6

  Copy full SHA for 611afa6

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  b0d2cc9

  Copy full SHA for b0d2cc9

• fix num_tokens in CPU rotary kernel

  Show description for 611fea0
  gau-nernst

  committed
  611fea0

  Copy full SHA for 611fea0

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  0f130fc

  Copy full SHA for 0f130fc

• reenable MLA flag. add import guards

  Show description for 737fb20
  gau-nernst

  committed
  737fb20

  Copy full SHA for 737fb20

• don't use prepack without support

  Show description for d8d4d5b
  gau-nernst

  committed
  d8d4d5b

  Copy full SHA for d8d4d5b

Commits on Mar 13, 2025

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  bc5c227

  Copy full SHA for bc5c227

• Merge branch 'ipex_moe_prepack' into cpu_mla

  gau-nernst

  committed
  ac4adab

  Copy full SHA for ac4adab

• Merge branch 'cpu_rotary' into cpu_mla

  gau-nernst

  committed
  ff3c569

  Copy full SHA for ff3c569

• working example, but very slow

  Show description for 250a7bf
  gau-nernst

  committed
  250a7bf

  Copy full SHA for 250a7bf

• ipex varlen_attention logits soft cap for XPU only

  Show description for c63011c
  gau-nernst

  committed
  c63011c

  Copy full SHA for c63011c

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  73f4a37

  Copy full SHA for 73f4a37

Commits on Mar 14, 2025

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  1500558

  Copy full SHA for 1500558

• fix issues with mypy

  Show description for ccec19e
  gau-nernst

  committed
  ccec19e

  Copy full SHA for ccec19e

• add another impl, but still slow

  Show description for 14b3448
  gau-nernst

  committed
  14b3448

  Copy full SHA for 14b3448

• Merge branch 'main' into cpu_mla

  Show description for 69f1693
  gau-nernst

  committed
  69f1693

  Copy full SHA for 69f1693

• use vectorized load/store

  Show description for a11b82f
  gau-nernst

  committed
  a11b82f

  Copy full SHA for a11b82f

• cache KV tile and tune AVX2 intrinsics

  Show description for 4cf1f37
  gau-nernst

  committed
  4cf1f37

  Copy full SHA for 4cf1f37

Commits on Mar 15, 2025

• revert modified FP32Vec layout. attempted to fix AVX512

  Show description for 72ac22f
  gau-nernst

  committed
  72ac22f

  Copy full SHA for 72ac22f

• move mla_decode to a separate file. attempt to fix AVX512

  Show description for 9570f95
  gau-nernst

  committed
  9570f95

  Copy full SHA for 9570f95

• fix AVX512 (yay)

  Show description for 0826ceb
  gau-nernst

  committed
  0826ceb

  Copy full SHA for 0826ceb

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  0725661

  Copy full SHA for 0725661

• add avx2 specialization

  Show description for af7747f
  gau-nernst

  committed
  af7747f

  Copy full SHA for af7747f

• add context parallel and remove custom AVX2 optimization

  Show description for e57f716
  gau-nernst

  committed
  e57f716

  Copy full SHA for e57f716

• remove unused header

  Show description for abfb2c2
  gau-nernst

  committed
  abfb2c2

  Copy full SHA for abfb2c2

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  6898f33

  Copy full SHA for 6898f33

Commits on Mar 16, 2025

• revert fp8_utils changes

  Show description for 65a4771
  gau-nernst

  committed
  65a4771

  Copy full SHA for 65a4771

Commits on Mar 17, 2025

• silence mypy

  Show description for dc2b2ed
  gau-nernst

  committed
  dc2b2ed

  Copy full SHA for dc2b2ed

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  ce2677c

  Copy full SHA for ce2677c

Commits on Mar 19, 2025

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  1972d59

  Copy full SHA for 1972d59

• fix issues

  Show description for 5c9fdb0
  gau-nernst

  committed
  5c9fdb0

  Copy full SHA for 5c9fdb0

• unroll heads

  Show description for 632e9e4
  gau-nernst

  committed
  632e9e4

  Copy full SHA for 632e9e4

• fix avx512

  Show description for 4daa919
  gau-nernst

  committed
  4daa919

  Copy full SHA for 4daa919

Commits on Mar 21, 2025

• Merge branch 'main' into cpu_mla

  Show description for 312829c
  gau-nernst

  committed
  312829c

  Copy full SHA for 312829c

• Add test for concat_and_cache_mla

  Show description for dfac73b
  gau-nernst

  committed
  dfac73b

  Copy full SHA for dfac73b

• fix out of bounds read

  Show description for 8147c55
  gau-nernst

  committed
  8147c55

  Copy full SHA for 8147c55

• add test for mla_decode_cpu

  Show description for 351d203
  gau-nernst

  committed
  351d203

  Copy full SHA for 351d203

• add CPU test to CI

  Show description for 8cdb1db
  gau-nernst

  committed
  8cdb1db

  Copy full SHA for 8cdb1db

Commits on Mar 22, 2025

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  e63363f

  Copy full SHA for e63363f

• remove fp8 kvcache for test. make HEAD_UNROLL a constexpr

  Show description for 86e8396
  gau-nernst

  committed
  86e8396

  Copy full SHA for 86e8396

Commits on Mar 25, 2025

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  a9bb7a3

  Copy full SHA for a9bb7a3

• Merge branch 'main' into cpu_mla

  gau-nernst

  committed
  e8706da

  Copy full SHA for e8706da