Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
297 commits
Select commit Hold shift + click to select a range
c8d849e
[cute_dsl] add gemm + all reduce (two_shot) (#1695)
Amir-19 Sep 18, 2025
ea2dfcf
misc: remove unused `torch.utils.cpp_extension` dependencies (#1711)
yzh119 Sep 18, 2025
ea322ab
test: skip unsupported (non-SM90) test cases for xqa (#1715)
jimmyzho Sep 18, 2025
05a882a
Fix DeepSeek quality for TRTLLM fused MoE routing (#1723)
GordonGustafson Sep 19, 2025
ed1d8c1
perf: Port the separate reduce kernel mode from trtllm. (#1685)
weireweire Sep 19, 2025
ea61569
Super tiny fix typo (#1730)
fzyzcjy Sep 19, 2025
ea43cb1
fix: put sampling kernel launch into macro (#1727)
ir1ka Sep 19, 2025
1e95001
bugfix: Fix flashinfer download-cubin (#1729)
tiran Sep 19, 2025
c1ffbd0
Fix missing namespace qualifier (#1731)
joker-eph Sep 19, 2025
7ee54c7
ci/cd: bring up flashinfer-cubin package (#1718)
yzh119 Sep 19, 2025
f7a0be4
disable optimization and add more debug information during verbose mo…
rainj-me Sep 19, 2025
9d523f8
ci/cd: add github workflows to publish flashinfer-cubin wheel to pypi…
yzh119 Sep 19, 2025
6832b46
Bump base container image from 13.0.0 to 13.0.1 for cu130 container (…
bkryu Sep 20, 2025
7d7aa87
fix: CI containers install nvidia-cudnn-cu12 vs. nvidia-cudnn-cu13 ba…
bkryu Sep 20, 2025
b6cfc2c
Test refactoring and fixes (#1736)
nvmbreughe Sep 20, 2025
a6e9c9f
TVM: support TVM binding for GroupedGemm (#1725)
neurusL Sep 20, 2025
7bac369
ci: enable tests for sm75 (G4) (#1705)
yongwww Sep 21, 2025
8ddfd28
doc: Super tiny fix doc math (#1747)
fzyzcjy Sep 21, 2025
1dd4af6
hotfix: Fix parsing pytorch verison (#1749)
sunghyunp-nvidia Sep 22, 2025
175fc73
feat: port fast_decode_plan from sgl (#1745)
zihaoye Sep 22, 2025
5afeac1
hotfix: slightly bump up `atol` to `3e-3` to pass `test_cudnn_prefill…
sunghyunp-nvidia Sep 22, 2025
7c8bcdb
tests: xfail moe quantization classes mxfp8_bf16 UTs on sm103 (#1754)
jimmyzho Sep 22, 2025
9573c71
ci: complete the list of modules in aot.py (#1746)
yzh119 Sep 22, 2025
eb25f07
tests: xfail attention sink UT for sliding window + non causal case (…
yzh119 Sep 23, 2025
50319b2
feat: Add compute capability checks to flashinfer_benchmark (#1756)
bkryu Sep 23, 2025
905f755
test: minor update on trtllm-gen attn speculative-decoding test (#1760)
yyihuang Sep 24, 2025
60dec90
fix: should pass global_override_indptr_cpu in fast_decode_plan param…
yyihuang Sep 24, 2025
951d354
fix(cleanup): ensure repository URL has no trailing slash (#1759)
tarukumar Sep 24, 2025
f2b455b
Fix tests/test_trtllm_gen_attention.py::test_trtllm_batch_prefill, ::…
kahyunnam Sep 24, 2025
84aaa3d
ci: add apache-tvm-ffi to ci docker container (#1763)
yzh119 Sep 24, 2025
3475e4b
fix: fix cannot import name 'cuda' from 'cuda' in CUDA13 (#1764)
LuYanFCP Sep 24, 2025
c721fb7
bugfix: partially fix tests/test_trtllm_gen_fused_moe.py unit test fa…
nv-yunzheq Sep 24, 2025
4fe837f
Fix sink attention accuracy regression, add sink test and cleanup. (#…
weireweire Sep 25, 2025
4e7969e
tests: skip non SM100/103 for grouped deepgemm (#1767)
jimmyzho Sep 25, 2025
b7be894
Added xfail for mx_fp4 matmul on SM120 (#1766)
nvmbreughe Sep 25, 2025
1bb7259
add test case for trtllm gen fused moe with kimi k2 problem sizes (#1…
nv-yunzheq Sep 25, 2025
08ed2b2
chore: label new issues with 'needs-triage' via GH Action (#1765)
sricketts Sep 25, 2025
ecd6648
Small fix on an exception (#1775)
nvmbreughe Sep 25, 2025
59a0c51
Waive / disable test_mla_decode_kernel.py::test_mla_decode_kernel for…
kahyunnam Sep 25, 2025
8ea06cd
bugfix: fix version tag validation in release_pypi_sdist.yml workflow…
yzh119 Sep 26, 2025
3e60597
bugfix: remove the filelock cleanup logic in cubin_loader.py (#1779)
yzh119 Sep 26, 2025
c305150
Docker updates (#1784)
nvmbreughe Sep 26, 2025
aca1e41
fix: add _check_tensor_params to check correct sampling parameters an…
raayandhar Sep 26, 2025
f2f7d6e
Set Trition path for cuda-13.0 (#1786)
nvmbreughe Sep 26, 2025
dada75b
ci: upgrade apache-tvm-ffi version in ci containers (#1788)
yzh119 Sep 26, 2025
db7af92
Fix autotune profile min shape bigger than max shape. (#1783)
weireweire Sep 26, 2025
4f884bf
hotfix: make aot wheel work without nvcc (#1782)
yzh119 Sep 27, 2025
ea56964
fix error message (#1789)
kahyunnam Sep 27, 2025
86d3e13
refactor: using tvm-ffi for multi-platform bindings (#1641)
yzh119 Sep 27, 2025
08b8da3
Upgrade CUTLASS to 4.2.1 (#1787)
jasl Sep 27, 2025
8438ec0
doc: fix documentation build error (#1797)
yzh119 Sep 28, 2025
63aff25
jit: defer ninja generation (#1792)
yzh119 Sep 28, 2025
d161b56
refactor: cleanup codebase after tvm-ffi refactor (#1795)
yzh119 Sep 29, 2025
41a1926
Update devcontainer.json and reuse ci docker images (#1791)
cyx-6 Sep 29, 2025
3697b39
fix: new issue workflow (#1773)
sricketts Sep 29, 2025
0f8ecab
fix: compilation failure in fp4Op.cpp (#1800)
jdebache Sep 29, 2025
b1b4549
fix: missing header include in decode kernel jit binding (#1802)
MasterJH5574 Sep 29, 2025
fb4293f
refactor: Test reorganization phase 2 (#1778)
nvmbreughe Sep 29, 2025
e19926b
fix: test_blackwell_kernels.sh script to no longer update dependencie…
bkryu Sep 30, 2025
1340443
fix: fp4 moe on sm120 (#1817)
ReinForce-II Sep 30, 2025
b9b0a2d
bugfix: fix devcontainer docker file cuda version (#1807)
cyx-6 Sep 30, 2025
0acd7ff
Enable support for CFLAGS as well as LDFLAGS when building (#1801)
directhex Sep 30, 2025
d252ed7
chore: improved URL handling for CUBIN/artifacts downloads (#1794)
jdebache Sep 30, 2025
bb42a8c
Remove `-isystem /usr/include` (#1821)
chelsea0x3b Oct 1, 2025
9c58b9d
Pytest flags and regression fix (#1822)
nvmbreughe Oct 1, 2025
4d26e4b
bugfix: Fixing variable name conflict bug introduced by PR1801 (#1823)
bkryu Oct 1, 2025
d50cfbc
Masked batch nvfp4 quantization (#1774)
wenscarl Oct 1, 2025
2931569
[Perf] Cache device property functions to avoid recomputation (#1824)
Jialin Oct 1, 2025
075775e
bugfix: remove the append "a" logic if user specifies cuda arch expli…
yzh119 Oct 1, 2025
ba2b4aa
Bugfix: Fix data hazard in persistent reduce (#1826)
Edenzzzz Oct 1, 2025
40df947
tests: upgrade cutlass, fix import and skip non-SM100 cutedsl two sho…
jimmyzho Oct 1, 2025
f765a2a
[Quantization] Add per-expert global scaling factor for fp4 batched q…
wenscarl Oct 1, 2025
35e099e
tests: Update support for tgv_gemm to SM100 only and add to ut (#1810)
jimmyzho Oct 1, 2025
85445d5
jit: add `get_object_paths` to JitSpec (#1836)
MasterJH5574 Oct 2, 2025
52089b5
docker: add image tags with date-SHA suffix (#1839)
yongwww Oct 2, 2025
398dc26
bugfix: Change module path in test_pod_kernels.py (#1842)
bkryu Oct 2, 2025
5f78377
bugfix: deep_gemm artifact load path (#1838)
jimmyzho Oct 2, 2025
d3346f6
bugfix: fix synchronize logic error in tests/comm/test_trtllm_alltoal…
nv-yunzheq Oct 2, 2025
b6e4da8
bugfix: show-config command (#1846)
sricketts Oct 2, 2025
989e82c
Run tests individually (#1847)
nvmbreughe Oct 3, 2025
2b753a5
unittest: remove debug-print jit examples from unittest (#1851)
yzh119 Oct 3, 2025
be130a7
jit: add `-lcuda` to default ldflags (#1825)
yzh119 Oct 3, 2025
ec4fc2c
feat: add warp-level persistent qk norm (#1843)
happierpig Oct 3, 2025
f3ea938
Add head_dim=64 for blackwell cutlass fmha implementation (#1850)
kahyunnam Oct 3, 2025
5c7ad5b
ci/cd: bringup flashinfer-jit-cache package (#1726)
yzh119 Oct 4, 2025
c87f962
ci: add docker-tags.yml to specify the docker image tag used in CI (#…
yzh119 Oct 4, 2025
ef9dd60
docker: upgrade apache-tvm-ffi==0.1.0b15 in docker container (#1857)
yzh119 Oct 4, 2025
1595175
ci: upgrade apache-tvm-ffi==0.1.0b15 (#1860)
cyx-6 Oct 4, 2025
c691768
raise error for group_gemm_fp8_nt_groupwise then num_groups > 1 on sm…
yongwww Oct 4, 2025
d3e9b44
bugfix: add check for empty MoE tactics and allow sm121 to use sm120 …
yongwww Oct 4, 2025
0b1d8a6
Bugfix: fix o_strides in persistent kernel (#1865)
Edenzzzz Oct 5, 2025
83af9a8
Improve dev container conda consistency (#1873)
cyx-6 Oct 6, 2025
91e6140
xfail the cute dsl tests for `l=1` (#1868)
cyx-6 Oct 6, 2025
74ee9d0
Added click package (#1875)
nvmbreughe Oct 6, 2025
792dcb1
Tune kernel compilation parameters for https://github.com/flashinfer-…
kahyunnam Oct 7, 2025
a4ddf26
PDL patch for TGV GEMM (#1877)
yangs75 Oct 7, 2025
ebea4bd
Moved common requirements from docker and setup.py to file (#1880)
nvmbreughe Oct 8, 2025
9c6c265
misc: fix some B200 GEMM bench (#1883)
Edenzzzz Oct 8, 2025
68826ac
ci/cd: add nightly build and CI for `flashinfer-python`, `flashinfer-…
yzh119 Oct 8, 2025
0522a32
fix: fix the failed sampling unittest on 5090 (#1886)
yzh119 Oct 8, 2025
da18466
Updated to latest docker tag (#1889)
nvmbreughe Oct 8, 2025
c08b529
Fix: Prevent race condition in cubin loader when file is being consum…
yzh119 Oct 9, 2025
d910f9a
Improve graph caching of cudnn graph (#1887)
Anerudhan Oct 9, 2025
fdc8338
misc: Various Updates to Attention Microbenchmark Suite (#1891)
bkryu Oct 9, 2025
4f7b549
docs: Fix installation instructions for CUDA-specific package URLs (#…
yzh119 Oct 9, 2025
674843f
docker image improvements (#1890)
nvmbreughe Oct 9, 2025
40d3fea
tests: Add batch size 1 cases to test_trtllm_gen_attention.py that fa…
bkryu Oct 9, 2025
c3ff7e7
Ensure docker installs the torch version we need (#1901)
nvmbreughe Oct 9, 2025
9fd6f67
bugfix: exclude `tests/utils/test_load_cubin_compile_race_condition.p…
yzh119 Oct 10, 2025
8d072e6
ci: use self-hosted runner for building docker containers (#1908)
yzh119 Oct 10, 2025
30314c3
feat: Add FP4 TRTLLM-Gen throughput MOE batched gemms (#1882)
jiahanc Oct 11, 2025
e8addbf
Update Docker CI tags to 20251010-8d072e6 (#1915)
github-actions[bot] Oct 11, 2025
940b2f1
ci/cd: consolidate release workflow (#1910)
yzh119 Oct 11, 2025
fd03820
bugfix: fix cli error when cuda toolkit is not installed (#1905)
yzh119 Oct 11, 2025
bbb57ad
feat: trtrllm-gen global scaled FP8 GEMMs (#1829)
jdebache Oct 11, 2025
8276d03
feat:enable fp8 blockscale moe for fused cultass for sm90 (#1819)
djmmoss Oct 12, 2025
4b55b26
use `ffi::TensorView` instead of `ffi::Tensor` (#1844)
cyx-6 Oct 12, 2025
9cd08af
Minor updates to cubin_loader.py download_file to avoid race conditio…
nvjullin Oct 13, 2025
80bdea5
chore: make cache directory flashinfer-version specific (#1920)
yzh119 Oct 13, 2025
2750c6c
misc: checksum check when downloading artifacts (#1761)
jimmyzho Oct 14, 2025
a88349f
release: bump version v0.4.1 (#1921)
yzh119 Oct 14, 2025
bea5949
fix vllm graph register and add test (#1894)
NVShreyas Oct 14, 2025
d728bcd
Support checks PoC (#1809)
nvmbreughe Oct 14, 2025
b01715b
chore: Restore FLASHINFER_LOCAL_VERSION environment variable (#1934)
yzh119 Oct 15, 2025
8d708b2
chore: fix wheel license packaging issues (#1932)
yzh119 Oct 15, 2025
9f25eee
Add layernorm op for inputs of mixed dtype (#1926)
akhilg-nv Oct 15, 2025
bd98dac
fixbug: fix devcontainer context (#1938)
cyx-6 Oct 16, 2025
8a88b0e
MLA RoPE + quantization fused kernel: shape generalization for MHA / …
kahyunnam Oct 18, 2025
ca49d01
ci: limit max_jobs for arm64 jit wheel cache build on CI (#1943)
yzh119 Oct 18, 2025
a0d9cfc
Add realistic bench for persistent kernel (#1942)
Edenzzzz Oct 18, 2025
3bbc47a
Add junit xml flags back to reorganized test script (#1940)
dierksen Oct 18, 2025
57c4f90
fix get max_q_len in page prefill plan (#1930)
ZhuJiaqi9905 Oct 18, 2025
33a6597
ci: Create Github Action to Automate CODEOWNER update (#1870)
yzh119 Oct 18, 2025
3e41094
chore: Update CODEOWNERS (#1871)
github-actions[bot] Oct 18, 2025
dbdf533
chore: use flashinfer-bot account to create auto pull requests in `re…
yzh119 Oct 18, 2025
8ffab81
Update Docker CI tags to 20251018-dbdf533 (#1945)
flashinfer-bot Oct 18, 2025
9ee58ac
Fix #1641: Use `/usr/local/cuda` as default `CUDA_HOME` if possible, …
netanel-haber Oct 20, 2025
d84e1d5
Fix bias dtype in cutlass_moe (#1876)
wenscarl Oct 20, 2025
dad0682
chore: rename FLASHINFER_JIT_VERBOSE to FLASHINFER_JIT_DEBUG for clar…
yzh119 Oct 20, 2025
c3f2596
fix: Fix trtllm-gen prefill IMA when batch_size==1 (#1912)
bkryu Oct 21, 2025
0c8f234
Feature: Support Relu2 activation in fused MoE (#1954)
amirkl94 Oct 21, 2025
ffcc5f4
fix: Add cutlass as an mm_fp4 backend in compute capability 12.0 in b…
bkryu Oct 21, 2025
979b759
Update the routing for TRTLLMGEN to support kimi k2 and qwen (#1831)
ChristinaZ Oct 22, 2025
0260ab3
unittest: fix deepgemm sha256 (#1953)
yzh119 Oct 22, 2025
b73f04c
misc: Update artifacts docstring and MetaInfoHash (#1967)
jimmyzho Oct 23, 2025
739df61
silu_and_mul nvfp4 quanization fusion rework (#1927)
wenscarl Oct 23, 2025
8b220a5
unittest: fix test_artifacts.py (#1950)
yzh119 Oct 23, 2025
1eebf02
chore: update the list of authorized codeowners (#1970)
yzh119 Oct 24, 2025
4512a6c
Added heuristic for trtllm_allreduce_fusion (#1972)
nvjullin Oct 24, 2025
0e48aaf
Bump tvm ffi to stable version 0.1.0 (#1960)
cyx-6 Oct 24, 2025
fac69ee
Update Docker CI tags to 20251024-0e48aaf (#1975)
flashinfer-bot Oct 24, 2025
a565294
fix: Make attention microbenchmark correctly use page table (#1976)
bkryu Oct 24, 2025
77091d4
fix: Skipping attention sink Blackwell test outside of Blackwell (#1978)
bkryu Oct 24, 2025
bf03ad4
feat: enable deepgemm jit for fp8 block-scale on SM90 (#1969)
djmmoss Oct 26, 2025
d225332
chore: Update CODEOWNERS (#1949)
flashinfer-bot Oct 26, 2025
d4a3ff4
fix: correct PDL parameter handling in RopeQuantize kernel (#1982)
cicirori Oct 26, 2025
99657ed
Fix: Verify scales are not None for Cutlass FP8 FusedMoE (#1961)
amirkl94 Oct 27, 2025
ef687e9
add xqa fp8 mha and fp8 kv cache (#1769)
qsang-nv Oct 27, 2025
28c8070
unittest: fix failed unittest on hopper (#1952)
yzh119 Oct 28, 2025
135d7bc
docs: Update documented versioning scheme to right-shifted semver (#1…
sricketts Oct 28, 2025
8ad64e2
Bugfix: Change get() -> GetDLTensorPtr() in cutlass FusedMoE validati…
amirkl94 Oct 28, 2025
c857f09
unittest: Add SM arch checks to skip unsupported tests on Hopper (#1998)
bkryu Oct 28, 2025
7d9d7af
Added workspace check and reflected this in test (#1991)
nvmbreughe Oct 28, 2025
9ce1af7
minor fix for xqa (#1994)
qsang-nv Oct 28, 2025
159d0a0
Feature: Add support for L40 FusedMoE in cutlass path (#1973)
amirkl94 Oct 29, 2025
ebb610c
unittest: Add head dim 256 test cases and mark as xfail (#1999)
bkryu Oct 29, 2025
bb6b620
feat: autotune tile_tokens_dim in trtllm-gen MOE (#1980)
jiahanc Oct 29, 2025
6a962ef
Fix trtllm-gen attention illegal memory access (#2002)
Tom-Zheng Oct 29, 2025
3c07921
release: Bump version for v0.5.0rc1 release; (#2008)
bkryu Oct 30, 2025
b9287c9
bugfix: fix regex in update wheel index script (#2009)
yzh119 Oct 30, 2025
a5ff033
fix: Enable SM121 for mm_fp4 (#2012)
bkryu Oct 30, 2025
de4c701
fix: ensure SM120/121 SFA/SFB contiguity (#1963)
yongwww Oct 31, 2025
1181c5d
More realistic bench for POD Attn (#2013)
Edenzzzz Oct 31, 2025
f9cd034
Feature: Support non-gated activation in cutlass fused MoE nvfp4 (#2011)
omera-nv Oct 31, 2025
5854494
feat: add xqa backend and completes NHD/HND coverage for trtllm-gen/x…
qsang-nv Nov 2, 2025
da01b1b
test: Enable xfailed trtllm decode long seqlen tests and update micro…
bkryu Nov 2, 2025
1e75bff
Updated decorator to support unspecified default (#2026)
nvmbreughe Nov 3, 2025
2d68a6b
release: Bump version for v0.5.1 release (#2031)
bkryu Nov 4, 2025
d528f0c
ci: Update cudnn version requirements in CI container (#2039)
bkryu Nov 4, 2025
f2cc526
test: Mark test_fp8_prefill.py as xfail on SM90 (#2038)
bkryu Nov 4, 2025
e1c1e2a
Update Docker CI tags to 20251104-d528f0c (#2041)
flashinfer-bot Nov 5, 2025
9bc5bd5
bugfix: fix failed unittest `test_green_ctx` and `test_jit_example` o…
yzh119 Nov 5, 2025
2580610
perf: Speed up fp4 quantization for small batch with swizzling for cu…
bkryu Nov 5, 2025
579012b
Support cc common check decorator for empty backends (#2015)
jimmyzho Nov 5, 2025
6d19a75
use scalar for kv_scale in xqa (#2033)
qsang-nv Nov 5, 2025
9721ff7
fix: support both pip and uv pip for finding flashinfer-python packag…
djmmoss Nov 5, 2025
747b4e2
test: Fix test_sampling.py on Spark (#2042)
bkryu Nov 6, 2025
adb0e89
Fix dtype of output scales from mnnvl_moe_alltoallv_prepare_without_a…
trevor-m Nov 6, 2025
b211926
Update trtllm-gen fused moe routing kernel and add more kernels (#1955)
jiahanc Nov 6, 2025
26d587a
chore: Update CODEOWNERS (#1984)
flashinfer-bot Nov 6, 2025
aacc8df
Add support for topkPacked input in block-level renormalize (#2051)
ChristinaZ Nov 6, 2025
f25929f
test: Skip test_fp8_quantize.py on Hopper (#2052)
bkryu Nov 6, 2025
55ea787
[BUG] Fix trtllm-gen fp4 moe renormalize routing (#2049)
IwakuraRein Nov 6, 2025
63cf562
release: Bump version for v0.5.2 release (#2057)
bkryu Nov 7, 2025
adcc5dd
perf: improve sampling/mask/softmax performance (part 1/2) (#2044)
yzh119 Nov 7, 2025
f566d49
misc: Add XQA decode to microbenchmark for sm90 and sm120 (#2055)
bkryu Nov 7, 2025
36d2463
test: Skip unsupported SM Archs for newly added trtllm MoE test (#2060)
bkryu Nov 7, 2025
3cb8f9a
feat: suitable_auto_backends to prune auto backends, bmm_fp8 refactor…
jimmyzho Nov 7, 2025
20435b4
update trtllm cutlass moe (#2020)
nv-yunzheq Nov 7, 2025
f588d96
perf: Optimize helper max/minmax function in sampling.cuh (#2058)
bkryu Nov 7, 2025
c8f2b03
[DSV3] Optimized Router Gemm (#2019)
nvmbreughe Nov 7, 2025
e450c7d
Fix moe fp8 failure for sm121 (#2061)
yongwww Nov 7, 2025
ba011d1
perf: TRT-LLM MoE Block-FP8 activation optimization (#2063)
nekorobov Nov 8, 2025
74281ed
[feat] Refactor trtllmgen MOE and add Bf16 trtllmgen moe (#2014)
jiahanc Nov 8, 2025
d56748f
Fix: several bugs/issues with trtllm-gen attention kernels. (#2062)
PerkzZheng Nov 9, 2025
8d7d0bc
refactor: remove MetaInfoHash class (#2064)
yzh119 Nov 9, 2025
f5a06a4
chore: Update CODEOWNERS (#2067)
flashinfer-bot Nov 10, 2025
d42fb90
feat: add xqa mla backend (#2053)
qsang-nv Nov 10, 2025
fbdb439
Enable renormalize(naive) routing for fp8 per-tensor (#2030)
IwakuraRein Nov 11, 2025
11177e8
unittest: improve the efficiency of xqa unittests (#2075)
yzh119 Nov 11, 2025
eccbdde
minor: canonicalize TFLOPS calculation (#2069)
Edenzzzz Nov 12, 2025
96e73b8
fix: fix test_trtllm_gen_attention when max_seq_len < page_size (#2076)
dongjiyingdjy Nov 12, 2025
53a6da4
enable xqa fp8 output (#2081)
qsang-nv Nov 12, 2025
abf6a14
chore: update requires-python in pyproject.toml (#2080)
raayandhar Nov 12, 2025
6765cad
[Test] Optimize test_trtllm_gen_fused_moe.py (#2072)
jiahanc Nov 12, 2025
b433fc7
test: Change incorrect inputs in test_hopper.py (#2083)
bkryu Nov 13, 2025
54101e9
[NVIDIA] Thor & Spark Support (#2028)
johnnynunez Nov 13, 2025
9a79b78
[API change] deprecate tile_token_dim in trtllm_moe (#2086)
jiahanc Nov 14, 2025
636a3ab
[Feature] Support batch prefill for POD Attention (#2079)
AKKamath Nov 14, 2025
37434ed
feat: patch sm103 for 3xfp4 moe generation (#2082)
aleozlx Nov 14, 2025
ba8f3ed
MNNVL All Reduce for large number of tokens (#2074)
nvmbreughe Nov 14, 2025
cce4952
perf: TRT-LLM Gen finalize kernel optimization (#2092)
nekorobov Nov 14, 2025
4ddf71d
refactor: update dpsk fused_moe test [1] (#2088)
yyihuang Nov 16, 2025
d42b71f
chore: update thor cuda arch (from 110f to 110a) (#2096)
yzh119 Nov 16, 2025
4aed50c
perf: enable pdl for cutlass fp4 gemm (#2095)
yzh119 Nov 16, 2025
0a36050
chore: Update CODEOWNERS (#2098)
flashinfer-bot Nov 18, 2025
3b07247
feat: Add flashinfer.rope.rope_quantize_fp8_append_paged_kv_cache (fu…
kahyunnam Nov 18, 2025
a9f71bd
[API change] Allow using torch.Tensor for scales for trtllm-gen atten…
IwakuraRein Nov 18, 2025
875403e
refactor: update dpsk fused_moe test [2] (#2097)
yyihuang Nov 18, 2025
1c4b522
hotfix: rename moe/test_utils.py to moe/utils.py (#2106)
yzh119 Nov 18, 2025
219592b
[DSR1] Added MLA test (#2100)
nvmbreughe Nov 19, 2025
b9964cc
test: Enable testing for trtllm-gen decode bs1 (#2103)
bkryu Nov 19, 2025
3a23405
[DSV3] Optimized routing kernels dsv3 (#2099)
nv-yunzheq Nov 19, 2025
0753095
feature: make the LSE returned by MLA support base 2 or e #2113 (#2114)
staugust Nov 20, 2025
76eea79
update xqa license (#2117)
qsang-nv Nov 20, 2025
af25b45
add tensor scale input for xqa (#2110)
qsang-nv Nov 20, 2025
049e8db
hotfix: add 9.0a to README and installation doc (#2112)
yzh119 Nov 20, 2025
450bf90
Revert "[API change] deprecate tile_token_dim in trtllm_moe (#2086)"
yzh119 Nov 20, 2025
a70d31c
bump-version
yzh119 Nov 20, 2025
2628beb
ci/cd: add nvidia-ml-py to requirments of build-system of flashinfer-…
yzh119 Nov 20, 2025
0e68a2f
Merge remote-tracking branch 'origin/main' into HEAD
yzh119 Nov 20, 2025
6d50048
Merge v0.5.3 into upgrade/amd-integration+v0.5.3
diptorupd Feb 18, 2026
b402485
jit/env.py: author IS_HIP block with amd-jit-cache integration
diptorupd Feb 18, 2026
48e7453
jit/core.py: author IS_HIP block with cpp_ext_hip and JitSpecRegistry
diptorupd Feb 18, 2026
1ca2b60
amd-flashinfer-jit-cache: align version with v0.5.3
diptorupd Feb 18, 2026
2034abd
Update decode/prefill_rocm to v0.5.3 API; expand HIP exports
diptorupd Feb 18, 2026
64cd387
fix HIP import errors blocking canary test on ROCm
diptorupd Feb 18, 2026
9feb175
Update and reorganize HIP tests
diptorupd Feb 19, 2026
c32d912
Re-HIPify sampling kernels
diptorupd Feb 19, 2026
61c11fd
Merge branch 'amd-integration' into upgrade/amd-integration+v0.5.3
diptorupd Feb 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"name": "CUDA Development Container",
"build": {
"dockerfile": "Dockerfile",
"context": "."
"dockerfile": "../../docker/Dockerfile.cu126.dev",
"context": "../../"
},
"runArgs": [
"--gpus=all"
Expand All @@ -12,8 +12,10 @@
"extensions": [
"llvm-vs-code-extensions.vscode-clangd",
"ms-python.python",
"ms-python.black-formatter",
"nvidia.nsight-vscode-edition"
"nvidia.nsight-vscode-edition",
"xaver.clang-format",
"charliermarsh.ruff",
"matangover.mypy"
]
}
},
Expand Down
26 changes: 26 additions & 0 deletions .devcontainer/cu128/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"name": "CUDA Development Container",
"build": {
"dockerfile": "../../docker/Dockerfile.cu128.dev",
"context": "../../"
},
"runArgs": [
"--gpus=all"
],
"customizations": {
"vscode": {
"extensions": [
"llvm-vs-code-extensions.vscode-clangd",
"ms-python.python",
"nvidia.nsight-vscode-edition",
"xaver.clang-format",
"charliermarsh.ruff",
"matangover.mypy"
]
}
},
"mounts": [
"type=bind,source=${localEnv:HOME}/.ssh,target=/home/devuser/.ssh,readonly"
],
"remoteUser": "devuser"
}
26 changes: 26 additions & 0 deletions .devcontainer/cu129/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"name": "CUDA Development Container",
"build": {
"dockerfile": "../../docker/Dockerfile.cu129.dev",
"context": "../../"
},
"runArgs": [
"--gpus=all"
],
"customizations": {
"vscode": {
"extensions": [
"llvm-vs-code-extensions.vscode-clangd",
"ms-python.python",
"nvidia.nsight-vscode-edition",
"xaver.clang-format",
"charliermarsh.ruff",
"matangover.mypy"
]
}
},
"mounts": [
"type=bind,source=${localEnv:HOME}/.ssh,target=/home/devuser/.ssh,readonly"
],
"remoteUser": "devuser"
}
26 changes: 26 additions & 0 deletions .devcontainer/cu130/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"name": "CUDA Development Container",
"build": {
"dockerfile": "../../docker/Dockerfile.cu130.dev",
"context": "../../"
},
"runArgs": [
"--gpus=all"
],
"customizations": {
"vscode": {
"extensions": [
"llvm-vs-code-extensions.vscode-clangd",
"ms-python.python",
"nvidia.nsight-vscode-edition",
"xaver.clang-format",
"charliermarsh.ruff",
"matangover.mypy"
]
}
},
"mounts": [
"type=bind,source=${localEnv:HOME}/.ssh,target=/home/devuser/.ssh,readonly"
],
"remoteUser": "devuser"
}
28 changes: 0 additions & 28 deletions .devcontainer/cuda/install/install_python.sh

This file was deleted.

26 changes: 0 additions & 26 deletions .devcontainer/cuda/install/install_python_packages.sh

This file was deleted.

45 changes: 45 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Code Owners File
# Generated automatically from git history analysis
# Analysis period: 180 days
# Minimum commits threshold: 1

benchmarks/ @bkryu @jiahanc @cyx-6 @yzh119 @nv-yunzheq
benchmarks/routines/ @bkryu @nv-yunzheq @jiahanc @cyx-6 @nvmbreughe
ci/ @cyx-6 @yzh119 @nvmbreughe
ci/scripts/ @cyx-6
ci/scripts/jenkins/ @cyx-6
csrc/ @wenscarl @yzh119 @cyx-6 @djmmoss @nv-yunzheq
csrc/fused_moe/ @nv-yunzheq @yzh119 @yongwww @djmmoss @cyx-6
csrc/fused_moe/cutlass_backend/ @nv-yunzheq @yzh119 @yongwww @djmmoss @cyx-6
csrc/nv_internal/ @wenscarl @djmmoss @nv-yunzheq @yongwww @cyx-6
csrc/nv_internal/cpp/ @wenscarl @bkryu @yongwww @djmmoss @joker-eph
csrc/nv_internal/include/ @wenscarl @nv-yunzheq
csrc/nv_internal/tensorrt_llm/ @wenscarl @djmmoss @nv-yunzheq @yongwww @cyx-6
csrc/xqa/ @cyx-6 @yzh119
docs/ @yzh119 @cyx-6 @wenscarl @nv-yunzheq @aleozlx
flashinfer/ @yzh119 @cyx-6 @nvmbreughe @aleozlx @wenscarl
flashinfer-cubin/ @yzh119 @cyx-6
flashinfer-cubin/flashinfer_cubin/ @yzh119
flashinfer-jit-cache/ @yzh119 @cyx-6
flashinfer-jit-cache/flashinfer_jit_cache/ @yzh119
flashinfer/comm/ @yzh119 @cyx-6 @nvmbreughe @wenscarl @djmmoss
flashinfer/cudnn/ @Anerudhan @yzh119 @cyx-6 @Anerudhan
flashinfer/cute_dsl/ @yzh119 @kaixih @Amir-19 @aleozlx
flashinfer/dsv3_ops/ @nvmbreughe
flashinfer/fused_moe/ @djmmoss @jiahanc @yzh119 @cyx-6 @aleozlx
flashinfer/gemm/ @nvmbreughe
flashinfer/jit/ @yzh119 @cyx-6 @aleozlx @jiahanc @nvmbreughe
flashinfer/jit/attention/ @yzh119 @cyx-6 @Anerudhan @joker-eph
flashinfer/jit/gemm/ @yzh119 @nv-yunzheq @jiahanc
flashinfer/logits_processor/ @cyx-6 @yzh119
flashinfer/profiler/ @cyx-6
flashinfer/triton/ @nvmbreughe @cyx-6
flashinfer/tuning_configs/ @kaixih
include/ @yzh119 @jiahanc @nvmbreughe @IwakuraRein @bkryu
include/flashinfer/ @yzh119 @jiahanc @nvmbreughe @IwakuraRein @bkryu
include/flashinfer/attention/ @yzh119 @kahyunnam @joker-eph
include/flashinfer/comm/ @yongwww @nvmbreughe @djmmoss @yzh119 @cyx-6
include/flashinfer/gemm/ @ttyio @yongwww @yzh119 @nvmbreughe @aleozlx
include/flashinfer/trtllm/ @jiahanc @joker-eph @aleozlx @yzh119 @wenscarl
profiler/ @cyx-6
scripts/ @yzh119 @nvmbreughe @dierksen @yongwww @bkryu
7 changes: 6 additions & 1 deletion .github/workflows/build-doc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ on:
branches:
- main

pull_request:
branches:
- main

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
Expand Down Expand Up @@ -42,17 +46,18 @@ jobs:
python -m pip install -r docs/requirements.txt

- name: Bulid Documentation
if: github.ref == 'refs/heads/main'
run: |
cd docs
./build_docs.sh

- name: Upload artifact
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
uses: actions/upload-pages-artifact@v3
with:
# Upload entire repository
path: 'docs/_build/html'

- name: Deploy to GitHub Pages
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
id: deployment
uses: actions/deploy-pages@v4
20 changes: 20 additions & 0 deletions .github/workflows/new-issue.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
name: Triage new issue

on:
issues:
types:
- opened

jobs:
label_issues:
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- name: Add needs-triage label
run: gh issue edit "$NUMBER" --add-label "$LABELS"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_REPO: ${{ github.repository }}
NUMBER: ${{ github.event.issue.number }}
LABELS: needs-triage
Loading
Loading