 Brayden Zhong
|
b42b9f6e1a
|
Support CuteDSL mm_fp4 backend (#18801)
|
2026-03-19 14:20:01 -07:00 |
|
|
Brayden Zhong
|
88c40ec16d
|
Use Flashinfer for target_verify in GDN model for SM120 (#20604)
|
2026-03-17 22:40:56 -07:00 |
|
|
Brayden Zhong
|
97d5386a21
|
Use TRTLLM allreduce fusion for Qwen 3.5 (#19889)
|
2026-03-17 22:40:22 -07:00 |
|
|
Brayden Zhong
|
591e61245a
|
[Doc] Add smal table for GEMM backends (#20213)
|
2026-03-09 22:19:57 -07:00 |
|
|
Brayden Zhong
|
e2af840c3d
|
Various SM120 improvements (#19721)
|
2026-03-03 16:46:13 -08:00 |
|
|
Brayden Zhongandamittell 1388680+amittell@users.noreply.github.com
|
9305f0e58d
|
Support triton_kernels for GPT-OSS on SM120 (#19718)
Co-authored-by: amittell 1388680+amittell@users.noreply.github.com
|
2026-03-03 14:14:01 -08:00 |
|
|
|
80a6b32703
|
[Perf] Optimize NSA backend metadata under MTP (#19536)
Co-authored-by: Baidu-AIAK <Baidu_AIAK@163.com>
Co-authored-by: zengpai <zengpai@baidu.com>
|
2026-03-01 01:59:26 -08:00 |
|
|
b8zhongandCheng Wan
|
78bf13db44
|
MoE Refactor: Refactor modelopt_quant.py -> flashinfer_trllm.py (#16685)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2026-02-02 20:45:14 -08:00 |
|
|
b8zhongandBrayden Zhong
|
398d13a189
|
[Perf] Add Flashinfer DeepGEMM SM90 for SwapAB Optimization (#15514)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2026-02-01 08:56:23 +08:00 |
|
|
b8zhong
|
ef134d407d
|
[Fix] Revert back to using CUTLASS mm_fp4 backend (#17369)
|
2026-01-31 23:01:29 +08:00 |
|
|
b8zhong
|
22498e10c0
|
[Fix] Triton TP MoE Dpsk V3/Qwen3 Coder with SwapAB (#17965)
|
2026-01-31 15:56:26 +08:00 |
|
|
b8zhongandBrayden Zhong
|
22df62d586
|
add weightless qk norm to RMSNorm interface for Llama 4 (#12813)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2026-01-29 19:09:55 -08:00 |
|
|
b8zhong
|
2ff0880a0e
|
[Fix] GLM 4.7 + NVFP4 + MTP (#17166)
|
2026-01-21 21:34:18 +08:00 |
|
|
b8zhong
|
7dc3cbe7ca
|
[Docker] Fix CUDA 13 installing wrong nvidia-nccl-cu13 due to nixl-cu13 not breaking system package (#17370)
|
2026-01-20 09:52:30 +08:00 |
|
|
b8zhong
|
f374623fa9
|
[Refactor] Set fp4-gemm-backend=auto on SM100 and rename fp4-gemm-backend with flashinfer_ prefix (#17309)
|
2026-01-19 20:09:07 +08:00 |
|
|
b8zhongandVincent Zhong
|
4df74eb576
|
[Refactor] Add -fp4-gemm-backend to replace SGLANG_FLASHINFER_FP4_GEMM_BACKEND (#16534)
Co-authored-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
|
2026-01-18 23:25:46 +08:00 |
|
|
b8zhong
|
d36f6f043c
|
[Fix] flashinfer_trtllm intermediate_size assertion with Qwen3 + TP=8 (#16824)
|
2026-01-17 09:24:05 +08:00 |
|
|
b8zhong
|
d44f09ad98
|
[Benchmark] Add GSM8K Platinum Eval (#14565)
|
2026-01-16 11:06:14 +08:00 |
|
|
b8zhong
|
3d72944fb8
|
[Doc] Add tip on how to use Spec V2 (#15455)
|
2026-01-16 05:30:18 +08:00 |
|
|
b8zhong
|
4346db5faf
|
[Fix] Remove assertion for padding for NVFP4 weight scales to fix GLM 4.5 NVFP4 (#12497)
|
2026-01-15 08:57:14 +08:00 |
|
|
b8zhongandBrayden Zhong
|
24b30f7757
|
MoE Refactor: Refactor fp8.py -> flashinfer_trllm.py (#15151)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2026-01-07 15:35:00 -08:00 |
|
|
b8zhong
|
74a166cb86
|
[Fix] Only add SM90 and SM100 to check for auto-enabling TRT Allreduce Fusion (#16283)
|
2026-01-03 11:43:17 +08:00 |
|
|
b8zhongandEven Zhou
|
a36142aa71
|
Fix Mindspore model import warning (#15287)
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
|
2025-12-18 23:15:33 -05:00 |
|
|
b8zhong
|
e88e75a9df
|
[Deepseek V3.2] Fix Deepseek MTP in V1 mode (#15429)
|
2025-12-18 17:56:07 -08:00 |
|
|
b8zhong
|
e72b02db28
|
tiny fix lint on main (#15424)
|
2025-12-18 15:55:02 -08:00 |
|
|
b8zhongandBrayden Zhong
|
d20699a33c
|
[Deepseek V3.2] Support Overlap Spec + NSA (#15307)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-17 13:35:39 -08:00 |
|
|
b8zhong
|
ffa7e03506
|
[Piecewise CUDA Graph] Support INT8 (#14918)
|
2025-12-17 18:20:57 +08:00 |
|
|
b8zhong
|
79ab57bd7a
|
Revert "direct register custom op for mm_fp4 (#13699)" (#15284)
|
2025-12-16 23:09:44 -08:00 |
|
|
b8zhong
|
4b8901ac0f
|
Update FP4 GEMM Benchmark (#14449)
|
2025-12-16 23:04:56 -08:00 |
|
|
b8zhongandBrayden Zhong
|
1ab9b8e0a3
|
Enable TRT AllReduce Fusion by default (#14764)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-14 20:01:49 -08:00 |
|
|
b8zhongandBrayden Zhong
|
fe6d38d2fa
|
fix: trtllm mha attention auto-selection on sm120 (#14842)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-11 22:35:25 -08:00 |
|
|
b8zhongandBrayden Zhong
|
dab31e4cc7
|
direct register custom op for mm_fp4 (#13699)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-11 22:33:17 -08:00 |
|
|
b8zhongandBrayden Zhong
|
6107268fe7
|
extend timeout for b200 test (#14925)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-11 15:43:53 -08:00 |
|
|
b8zhongandBrayden Zhong
|
c51efb8b84
|
fix fp8 gemm nightly CI (#14844)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-10 15:57:51 -08:00 |
|
|
b8zhong
|
da9b801eb7
|
fix lora target all + csgmv backend (#14796)
|
2025-12-10 11:42:51 -08:00 |
|
|
b8zhong
|
56e5c07424
|
fix b200 fa4 ci (#14788)
|
2025-12-10 00:03:43 -08:00 |
|
|
b8zhong
|
b0a25d0913
|
fix b200 ci (#14786)
|
2025-12-09 23:08:41 -08:00 |
|
|
b8zhong
|
55504df2f7
|
Add FP8 Blockwise GEMM Backend Flag --fp8-gemm-backend (#14379)
|
2025-12-09 12:05:56 -08:00 |
|
|
b8zhong
|
0f8bd55f3e
|
[CI] Fix Llama 3.1 8B FP4 CI (#14699)
|
2025-12-08 22:27:15 -08:00 |
|
|
b8zhong
|
32f8b6064e
|
improve default glm mtp setting (#14457)
Signed-off-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-08 13:27:13 -08:00 |
|
|
b8zhongandBrayden Zhong
|
3b47973af8
|
[CI] Tiny speed up VLM CI (#14517)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-07 13:30:41 -08:00 |
|
|
b8zhong
|
6d5d76ad97
|
remove unecessary dual stream token threshold from the rest of models (qwen moe, kimi linear, etc.) (#14337)
|
2025-12-06 19:57:26 -08:00 |
|
|
b8zhong
|
dd91d38e6a
|
[Doc] Add short explanation on page size (#14557)
|
2025-12-06 17:26:30 -08:00 |
|
|
b8zhong
|
ec7b2c16d9
|
tiny remove deprecated endpoint call (#13607)
|
2025-12-05 09:54:49 -08:00 |
|
|
|
88d1bab537
|
add doc for quantized kv cache (#14348)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
Co-authored-by: Ho-Ren (Jack) Chuang <horenchuang@bytedance.com>
|
2025-12-04 13:01:05 -08:00 |
|
|
b8zhong
|
9d82340298
|
Revert "Revert "enable csgmv automatically on cuda"" (#14277)
|
2025-12-03 13:12:30 -08:00 |
|
|
b8zhongandBrayden Zhong
|
65c8568c4a
|
sync attention, deepseek doc (#14335)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-02 21:19:40 -08:00 |
|
|
b8zhongandBrayden Zhong
|
236a7c2370
|
fix trtllm mla spec (#13738)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-01 22:16:25 -08:00 |
|
|
b8zhongandBrayden Zhong
|
e6420100ee
|
sync attention doc and ep doc to doctree (#14257)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-12-01 21:15:22 -08:00 |
|
|
b8zhongandBrayden Zhong
|
a24aefe562
|
add some more labels (#13701)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-11-21 11:29:33 -08:00 |
|
|
|
85ffce30af
|
[Piecewise CUDA Graph] Support Kimi-K2 (non-Thinking) (#13466)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
|
2025-11-21 09:39:59 -08:00 |
|
|
b8zhong
|
42028af614
|
enable csgmv automatically on cuda (#13600)
|
2025-11-20 12:53:02 -08:00 |
|
|
b8zhong
|
92ad2ff9ce
|
Flashinfer TRTLLM-GEN-MoE + Qwen3 (#13489)
|
2025-11-18 14:18:29 -08:00 |
|
|
b8zhong
|
f33860777c
|
[Piecewise CUDA Graph] Support ModelOpt FP8 (#13094)
|
2025-11-18 12:46:24 +08:00 |
|
|
b8zhong
|
d5fa58c4dd
|
fix nightly docker build (#13386)
|
2025-11-16 11:21:09 -08:00 |
|
|
b8zhong
|
f35f7f1245
|
[Piecewise CUDA Graph] Support W4A8 (#13179)
|
2025-11-16 11:53:50 +08:00 |
|
|
b8zhong
|
24a25ffa20
|
[Piecewise CUDA Graph] Support ModelOpt FP4 (#13101)
|
2025-11-16 11:03:19 +08:00 |
|
|
b8zhong
|
7b79cc4fe2
|
ci: speed up b200 ci (#13237)
|
2025-11-13 23:57:33 -08:00 |
|
|
b8zhong
|
e523e2167a
|
remove deprecated tile_tokens_dim (#13186)
|
2025-11-13 17:09:27 -08:00 |
|
|
b8zhong
|
cc2e36c352
|
overlap shared + routed expert computation in kimi linear (#12660)
|
2025-11-11 14:52:58 -08:00 |
|
|
b8zhong
|
49653c8896
|
use fast stream instead of torch.cuda.current_stream in llama 4 shared experts overlap (#12811)
|
2025-11-08 15:04:37 -08:00 |
|
|
b8zhongandBrayden Zhong
|
55e8e3999c
|
add back flashinfer jit cache to dev docker (#12851)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-11-07 14:51:24 -08:00 |
|
|
b8zhong
|
32ec68faf9
|
keep attention backend document up to date (#12741)
|
2025-11-05 20:41:20 -08:00 |
|
|
b8zhong
|
d31d48b341
|
update usage of trtllm_fp8_per_tensor_scale_moe (#12569)
|
2025-11-03 14:25:32 -08:00 |
|
|
b8zhong
|
a076ec1a7a
|
Revert "fix llama4 kv cache layout" (#12437)
|
2025-10-30 22:33:37 -07:00 |
|
|
b8zhongandBrayden Zhong
|
34c286b811
|
Fix the shared expert & routed expert overlap in Llama 4 (#12405)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-10-30 20:14:49 -07:00 |
|
|
b8zhong
|
7ed8ba05cb
|
[CI] Add Llama 3.1 8B FP4 to B200 CI (#12182)
|
2025-10-29 14:56:41 -07:00 |
|
|
b8zhong
|
bacb3825fe
|
fix: llama 4 + trtllm gen + fp8 kv cache incompatibility (#12347)
|
2025-10-29 11:31:02 -07:00 |
|
|
b8zhong
|
83a7c89c3f
|
followup fix for llama 4 trtllm flashinfer backend (#12314)
|
2025-10-28 22:17:08 -07:00 |
|
|
b8zhong
|
c143f416ce
|
fix: Llama 4 BF16 load on Blackwell (#12308)
|
2025-10-28 18:59:01 -07:00 |
|
|
b8zhong
|
77225d602a
|
Use Flashinfer TRT-LLM as Llama 4 compatible MoE backend (#11928)
|
2025-10-28 10:39:43 -07:00 |
|
|
b8zhong
|
f80371ff8c
|
Use flashinfer_trtllm moe runner backend to gain around 10% perf on b200 fp8 dpsk (#11816)
|
2025-10-23 19:12:15 -07:00 |
|
|
b8zhong
|
47e12e082e
|
Enable Llama 4 + TRTLLM MHA (#12003)
|
2025-10-23 18:22:58 -07:00 |
|
|
b8zhong
|
8ae9d4bb41
|
Revert "[ROCm] Remove vLLM rope dependency & use AITER impl" (#12028)
|
2025-10-23 12:42:59 -07:00 |
|
|
b8zhong
|
4d4feccbb2
|
[ROCm] Remove vLLM rope dependency & use AITER impl (#11322)
|
2025-10-22 19:17:34 -07:00 |
|
|
b8zhong
|
d0a64c7e2c
|
vlm: enforce pybase64 for image and str encode/decode (#10700)
|
2025-10-21 19:05:32 +08:00 |
|
|
b8zhong
|
f4f8a1b4d8
|
ci: update lmms-eval to speed up multimodal CI (#11000)
|
2025-10-19 02:51:19 +08:00 |
|
|
b8zhong
|
f9a7d9b3dc
|
support server arg override KV cache to bf16 to avoid slow cases (#11749)
|
2025-10-19 02:49:48 +08:00 |
|
|
b8zhong
|
6bc503af73
|
[Doc] Update support matrix for attn and hybrid attn (#11293)
|
2025-10-14 22:43:11 -07:00 |
|
|
b8zhong
|
a2faf8940c
|
[1/n] Enable DCA CUDA graph capture (#9537)
|
2025-10-03 11:30:00 +08:00 |
|
|
b8zhong
|
948278f173
|
fix cpp JIT compilation issue of ngram speculative decoding (#10837)
|
2025-10-02 21:05:01 +08:00 |
|
|
b8zhong
|
b2435be682
|
Cache the result of is_blackwell platform check (#10498)
|
2025-09-15 22:30:28 -07:00 |
|
|
b8zhong
|
716e682721
|
[Fix] Add undefined update_tensor_inplace function (#6307)
|
2025-08-18 11:11:00 +08:00 |
|
|
Brayden Zhong
|
bc938ea13f
|
Fix DP load for embedding (#9165)
|
2025-08-15 23:58:44 -07:00 |
|
|
Brayden Zhong
|
2871eacc05
|
Add Triton Fused MoE kernel config for E=16 on B200 (#7004)
|
2025-08-13 23:12:27 -07:00 |
|
|
Brayden Zhong
|
4a9f3eef90
|
Tiny Llama4 type error in constructor (#6752)
|
2025-08-09 01:03:59 -07:00 |
|
|
Brayden Zhong
|
4acf690206
|
[Optimization][Perf] Disable the GC during CUDA graph capture to speed up by up to 3x (#8577)
|
2025-07-31 11:31:21 -07:00 |
|
|
Brayden Zhong
|
a37e1247c1
|
[Multimodal][Perf] Use pybase64 instead of base64 (#7724)
|
2025-07-08 14:00:58 -07:00 |
|
|
Brayden ZhongandCinjon Resnick
|
ca9291181d
|
[Feature] Add Logit Bias (#6579)
Co-authored-by: Cinjon Resnick <cinjon.resnick@gmail.com>
|
2025-06-10 15:39:25 -07:00 |
|
|
Brayden Zhong
|
1aa0fbf416
|
Add note to add supported model to documentation (#6640)
|
2025-05-27 13:18:46 +08:00 |
|
|
Brayden Zhong
|
006ead9dcb
|
[FA][Test] Fix Sparse FA test (#6306)
|
2025-05-26 01:27:48 -07:00 |
|
|
Brayden Zhong
|
9a91fa0ed1
|
docs: fix a bad redirect (#6300)
|
2025-05-14 10:27:19 -07:00 |
|
|
Brayden Zhong
|
3c32895cbe
|
[Llama4] Add docs note about enable multimodal (#6235)
|
2025-05-13 10:05:47 +08:00 |
|
|
Brayden Zhong
|
12319a6787
|
[Docs] Add docs for SGLANG_ and SGL_ environment variables (#6206)
|
2025-05-13 01:45:41 +08:00 |
|
|
Brayden Zhongandvincent-4
|
43fb95c2fa
|
[Model] Support ArcticForCausalLM architecture (Snowflake/snowflake-arctic-instruct) (#5078)
Co-authored-by: vincent-4 <vincentzhongy+githubvincent4@gmail.com>
|
2025-04-25 15:24:09 +08:00 |
|
|
Brayden Zhong
|
b868526d94
|
Fix one more issue reported by torchfix (#4859)
|
2025-04-20 17:49:27 -07:00 |
|
|
Brayden Zhong
|
e84f4ba0ab
|
[Misc] Fix issues reported by torchfix (#4837)
|
2025-03-27 20:10:32 -07:00 |
|
|
Brayden Zhong
|
b149b39353
|
[CI] Remove unused imports with Ruff to pre-commit config, only to benchmarks/docs/examples folder (#3969)
|
2025-03-27 19:45:02 -07:00 |
|