Main w8a8 fp8#2

Open

HandH1998 wants to merge 259 commits intoyychfrom

This pull request is big! We're only showing the most recent 250 commits

Commits on Jan 9, 2025

[Doc]: Deepseek reference docs (#2787)
XiaotongJiang
authored

Commits on Jan 13, 2025

Commits on Jan 15, 2025

Commits on Jan 16, 2025

Commits on Jan 17, 2025

Commits on Jan 18, 2025

Commits on Jan 19, 2025

Commits on Jan 20, 2025

Commits on Jan 21, 2025

[Doc] Update doc of custom logit processor (#3021)
hongpeng-guo
authored
Fix flaky tests in test_programs.py (#3022)
merrymercy
authored
[EAGLE] Fix some boundary situation when retract reqs and req's max token = 1 (#2939)

josephydu
and
josephyou
authored
Enable Cohere2 Models (#3018)
hliuca
authored
Misc fixes for eagle (flush_cache, CPU overhead) (#3014)
merrymercy
committed
minor: update Makefile for sgl-kernel (#3025)
zhyncs
authored
upgrade torch version for sgl-kernel (#3026)
zhyncs
authored
fp8 dispatch change

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
clean code

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
fix

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
clean code

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
Add accuracy and latency tests of eagle into CI (#3027)
merrymercy
authored
feat: add flashinfer as 3rdparty and use rmsnorm as example (#3033)
zhyncs
authored
Support sm90 Int8 gemm (#3035)
ispobock
authored
fix pr-test-sgl-kernel (#3036)
zhyncs
authored

Commits on Jan 22, 2025

Add performance and accuracy test code for FP8 GEMM operations

yych0745
authored and
HandH1998
committed
support w8a8 fp8
HandH1998
committed
Use int64 as indices for set_kv_buffer (#3039)
merrymercy
authored
support bias
HandH1998
committed
fix compilation
HandH1998
committed
clean code

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
clean code

yych0745
authored and
hadoop-hmart-waimai-rank hadoop-hmart-waimai-rank
committed
Merge pull request #6 from HandH1998/tmptmp
HandH1998
authored
Merge remote-tracking branch 'origin/main' into main_w8a8_fp8
HandH1998
committed
format
HandH1998
committed
format
HandH1998
committed
Fix sgl-kernel compile for sm80 (#3046)
ispobock
authored
Merge branch 'main' into main_w8a8_fp8
zhyncs
authored
upd
zhyncs
authored
update norm cu (#3048)
zhyncs
authored
sync the upstream updates of flashinfer (#3051)
zhyncs
authored
Merge branch 'main' into main_w8a8_fp8
zhyncs
authored
feat: integrate norm kernels into sgl-kernel (#3052)
zhyncs
authored
feat: integrate activation kernels into sgl-kernel (#3053)
zhyncs
authored
minor: update header and use pytest (#3054)
zhyncs
authored
feat: integrate bmm_fp8 kernel into sgl-kernel (#3056)
zhyncs
authored
fix rotary_embedding rope_scaling for phi (#3055)
sudo-root-ns
authored
add notice about flashinfer in sgl-kernel (#3057)
zhyncs
authored
disable custom allreduce on HIP (#3058)
hliuca
authored
[Doc] Update doc of profiling with PyTorch Profiler (#3038)
Fridge003
authored
Fix the FP8 E4M3 parsing offline scales failure bug (#3045)
sleepcoo
authored
Add some flags to allow sync token ids across TP ranks (#3060)
merrymercy
authored

Commits on Jan 23, 2025

Commits on Jan 24, 2025

Commits on Jan 26, 2025