minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity
16,246 Commits 2 Branches 0 Tags
b3ee679467fbc0e423d893608fe5753f44e9dc67
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Brayden ZhongandBrayden Zhong b3ee679467 [MXFP8] Use FlashInfer CUTLASS for dense GEMM on SM120, delete Triton path (#33208) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-08-07 14:30:43 -07:00
Brayden ZhongandBrayden Zhong 0c3a76fa0a Clean GLM-5.2 NVFP4 cookbook (#33935) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-08-07 11:51:34 +08:00
Brayden Zhong dd7e4c91e2 Fix Mistral-Large-3 EAGLE draft skipping DeepseekV2Model.__init__ (#33785) 2026-08-06 13:59:28 -07:00
Brayden ZhongBrayden ZhongBaizhou Zhang
 		28848bfe7c Fix Nightly NV CI (#33564) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-08-05 18:06:47 -07:00
Brayden ZhongandBrayden Zhong a14c870886 Fix broken Nemotron DP attention (#33123) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-08-05 14:27:05 -07:00
Brayden Zhong e0ba311026 Disable breakable CUDA graph for NemotronH (#33130) 2026-08-01 20:48:58 +00:00
Brayden Zhong f94d2c5663 [Fix] Restore online MXFP8 quantization for linear layers (#32953) 2026-07-31 06:42:06 +00:00
Brayden ZhongandBrayden Zhong 7dcebca255 Fix nightly CI: NVFP4 cuda-graph crash, NVILA batching, CuTe paged-KV zero-size, Kimi-VL OOM (#32118) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-29 01:39:55 -07:00
Brayden Zhong f01a0c7f97 Fixing MXFP8 online quantization pipeline (#31510) 2026-07-28 21:26:13 -07:00
Brayden Zhong e7511141ea Support CuteDSL GEMM BF16 on SM100 on by default when allowed by heuristic (#30567) 2026-07-22 14:13:12 -07:00
Brayden ZhongBrayden Zhongroot
 		03342e7732 Delete sgl-kernel AOT router GEMM and fused A GEMM (#30280) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: root <root@sgl-b300-inference.datacrunch.io>
 2026-07-22 08:44:59 +08:00
Brayden Zhongandroot 2f4f2362fb Delete sgl-kernel AOT bmm_fp8, use flashinfer.bmm_fp8 (#31202) 
Co-authored-by: root <root@sgl-b300-inference.datacrunch.io>
 2026-07-22 07:44:47 +08:00
Brayden ZhongandBrayden Zhong 0eae9423d8 Change the FP8 per-tensor GEMM backend on SM120 to cuBLAS (#31961) 
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
 2026-07-21 14:09:01 -07:00
Brayden Zhong 238b2b2c9c Remove QServe and FBGEMM FP8 quantization (#31109) 2026-07-17 17:10:34 -07:00
Brayden ZhongandBrayden Zhong 7fc3fb9657 Remove deprecated Mamba flags from doc, wrong FP8 GEMM docstrings and change Nemotron image to 0.5.15 (#31094) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-17 14:34:29 -07:00
Brayden ZhongrootBrayden Zhong
 		e5f9804e26 Refining fused A GEMM dispatch (#31241) 
Co-authored-by: root <root@sgl-b300-inference.datacrunch.io>
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-16 16:57:01 +08:00
Brayden ZhongandBrayden Zhong 7647a9d260 Fuse the preprocess kernels of trtllm-gen attention (#29690) 
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
 2026-07-15 18:21:34 -07:00
Brayden ZhongandBrayden Zhong edb2059139 Support Flashinfer one-sided A2A + CuteDSL MoE for Nemotron Ultra (#28309) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-15 18:14:20 -07:00
Brayden ZhongandBrayden Zhong ac23be8d09 Skip MXFP8 autotune on dense GEMM, which causes IMA (#29669) 
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
 2026-07-15 18:06:55 -07:00
Brayden ZhongandBrayden Zhong 0b04e9da83 Use fused A GEMM for fc1_latent_proj in NemotronH (#29692) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-15 18:05:22 -07:00
Brayden ZhongBrayden ZhongClaude Sonnet 5rootBrayden Zhong
 		7431f35fd8 Delete CUTLASS FP8 blockwise for SM90 and SM100, move SM120 to JIT and add SwapAB (#30438) 
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: root <root@sgl-b300-inference.datacrunch.io>
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-14 09:31:32 +08:00
Brayden Zhongandroot 9756f768a6 Refactor FP4 quantization and remove deprecated JIT kernels (#30448) 
Co-authored-by: root <root@sgl-b300-inference.datacrunch.io>
 2026-07-14 09:22:07 +08:00
Brayden Zhong 0ee236ebdf more fixes for Nemotron 3 parser for tool call and force nonempty content (#30533) 2026-07-14 06:50:27 +08:00
Brayden ZhongandBrayden Zhong 455ab36eeb Increase the KV cache pool when using indexShare by 15% (#30310) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-07 20:59:13 -07:00
Brayden ZhongandBrayden Zhong 6875df3378 [Cherry pick to release/v0.5.15] Fix NVILA weight loading (#30400) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-07 13:21:55 -07:00
Brayden ZhongandBrayden Zhong e2ea7aafad [Cherry pick to release/v0.5.15] Fix NVFP4 online quantization (#30397) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-07 13:20:36 -07:00
Brayden ZhongMindy LiBrayden Zhong
 		fefc1743a9 Cute-DSL FP8 MQA logits (#25220) 
Co-authored-by: Mindy Li <11663212+limin2021@users.noreply.github.com>
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-06 23:34:07 -07:00
Brayden ZhongBrayden ZhongBaizhou Zhang
 		e85ef54877 Support Cutedsl BF16 GEMM JIT kernel (#30117) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-07-06 19:16:16 -07:00
Brayden ZhongandBrayden Zhong 267ff1b5f9 Fix LTX2 RoPE JIT kernel CI (#30278) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-07 10:15:21 +08:00
Brayden ZhongandBrayden Zhong d8462f4961 Fixes for NVFP4 numerical accuracy for router GEMM output and wrong correction bias cast (#29783) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-06 13:53:05 -07:00
Brayden ZhongandBrayden Zhong 8f40b5eb3f When attention TP for linear and full attention, use Flashinfer allreduce fusion (#29699) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-06 13:01:53 -07:00
Brayden ZhongandBrayden Zhong 1b6d1e9752 Fix wrong RMSNorm fallback to old Flashinfer CUDA kernel when in PCG (#29702) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-07-03 00:49:49 +08:00
Brayden Zhong 98d0e702c3 [GLM-5] Tune the threshold of router GEMM (#29470) 2026-06-29 14:19:24 -07:00
Brayden ZhongandBrayden Zhong c7b9b92d9a Fix CI caused by https://github.com/sgl-project/sglang/pull/29576 (#29596) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-28 23:46:18 -07:00
Brayden ZhongandBrayden Zhong b9b860652e Fix DSA indexer fusion bug causing excessive memory consumption. (#29576) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-28 14:11:07 -07:00
Brayden ZhongBrayden ZhongKaixi
 		073de15053 Fuse the DSA (V3.2, GLM-5.x) indexer Q/K paths into single kernels (#27705) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: Kaixi <kaiximatteoc@nvidia.com>
 2026-06-27 14:29:56 -07:00
Brayden ZhongandBrayden Zhong e4253b39e2 Support JIT fused A GEMM (MLA down projection) and support GLM-5 hidden size, SM120 (#27397) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-27 13:32:06 -07:00
Brayden ZhongandBrayden Zhong 9495737d82 Fix CI broken by #28450 (#29308) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-25 10:35:34 -07:00
Brayden ZhongandBrayden Zhong 5d4e63d49e Sync the changes in #23402 (#29063) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-24 17:01:02 -07:00
Brayden ZhongandBrayden Zhong f82addd4a8 Support online MXFP8 quantization for ungated MoE (#27939) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-24 16:58:48 -07:00
Brayden ZhongandBrayden Zhong 3c95a87b66 Fix the CuDNN failure on bmm_fp8 when two libcudart.so exists. (#29201) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-24 15:26:46 -07:00
Brayden ZhongandBrayden Zhong d46afbf8b4 Fix nightly CI test for GLM-4.6 + B200 (#28749) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-24 12:34:11 -07:00
Brayden ZhongandBrayden Zhong 2c697daf5f [Cookbook] Nemotron3-Ultra: align MTP draft depth with NVIDIA reference (num_steps 5) (#29200) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-24 19:12:34 +00:00
Brayden Zhong ba9d5aed98 Fix nightly CI test for Kimi K2.5 INT4 + H200 (#28746) 2026-06-23 00:27:33 +00:00
Brayden ZhongBrayden Zhongzijiexia
 		99c18cceec Sync server arguments and environment variables + update various documentation (#28674) 
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
 2026-06-22 10:23:46 -07:00
Brayden ZhongandBrayden Zhong 34e5e38604 [Cookbook] Nemotron3-Ultra: Add mamba-backend and SSM dtype flags (#28675) 
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
 2026-06-22 09:50:22 -07:00
Brayden ZhongandBrayden Zhong 05ee93c44f Remove redundant cast and copy in calling trtllm_fp8_block_scale_moe (#28555) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-18 18:11:29 -07:00
Brayden ZhongandBrayden Zhong ea407df4b0 Use Flashinfer allreduce fusion for MNNVL allreduce for Nemotron (#28346) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-18 17:30:12 -07:00
Brayden ZhongandBrayden Zhong b8a73bfba0 Call Flashinfer mm_fp8 for per-tensor FP8 GEMMs on SM100 (#28333) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-16 20:50:03 -07:00
Brayden ZhongBrayden Zhongshaunkotek
 		13537f8e20 Unskip Marlin NVFP4 tests (#27589) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: shaunkotek <shaunkotek@users.noreply.github.com>
 2026-06-16 11:58:22 -07:00
Brayden ZhongandBrayden Zhong 25e696aa8d Fix Stage B CUDA CI (#28367) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-16 02:39:17 -07:00
Brayden ZhongandBrayden Zhong 1a19f66acb Fix DP attention + EP mode of Nemotron (#28102) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-12 18:39:04 -07:00
Brayden ZhongandBrayden Zhong 95867f0932 [Doc] Fix some inconsistencies in the Nemotron Cookbook (#28087) 
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
 2026-06-12 14:51:58 -07:00
Brayden ZhongandBrayden Zhong 8bfcc0c39c Use the correct wrapper for fp4_quantize (#27956) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-11 20:23:09 -07:00
Brayden ZhongandBrayden Zhong 493f828bfa Add DeepGEMM prerelease wheel tests (#27075) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-11 15:14:50 -07:00
Brayden ZhongandBrayden Zhong 38ae22e08c Nemotron perf changes (#26733) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-05 22:31:46 -07:00
Brayden ZhongandBrayden Zhong d381ec7997 [CI] Fix Nemotron nightly mixed precision checkpoints test (#27284) 
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
 2026-06-05 13:51:26 -07:00
Brayden ZhongandBrayden Zhong 3b62286fca Reland "Support NextN = 2/4 in DSV32" (#27166) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-05 13:43:28 -07:00
Brayden ZhongMartin VitClaude Opus 4.7Brayden Zhong
 		e76d36214b Changes for SM120 perf and usability for NVFP4 (#26496) 
Co-authored-by: Martin Vit <martin@voipmonitor.org>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
 2026-06-04 15:29:25 -07:00
Brayden Zhongb8zhongBrayden Zhong
 		5bf90ad988 Enable DeepGEMM PDL on by default (#23979) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
 2026-06-04 14:13:45 -07:00
Brayden Zhongandb8zhong 13852d3f31 Support NextN = 2/4 in DSV32 (#24870) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-06-02 19:06:06 -07:00
Brayden Zhongandb8zhong 50e0b3b77f Support Flashinfer Cute-DSL MLA attention (#24737) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-28 00:21:32 -07:00
Brayden Zhongandb8zhong e31ea50df8 Remove DeepGEMM for indexer GEMM in piecewise NSA path (#26494) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-28 00:15:23 -07:00
Brayden Zhongandb8zhong b4808d44da Use Cute-DSL MXFP8 quantize kernels (#25486) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-28 00:01:31 -07:00
Brayden Zhong 97dd6aad60 Add a little env var for disabling Flashinfer autotune cache (#26193) 2026-05-27 23:59:59 -07:00
Brayden Zhongandb8zhong d523ae127f Fix Mistral Large 3 nightly test (#25407) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-15 17:28:35 -07:00
Brayden Zhongandb8zhong d5f3254ed1 [Dependency] Flashinfer 0.6.8post1 -> 0.6.11 (#24452) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-12 14:38:32 -07:00
Brayden Zhongandb8zhong 1df9edcd01 Use Torch torch.mm for Deepseek V3.2 Indexer GEMM (#23856) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-11 00:41:05 -07:00
Brayden Zhongandb8zhong 1d80a1a9fe Use Cute-DSL NVFP4 quantization kernels (#23745) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-11 00:40:02 -07:00
Brayden Zhongandb8zhong 8acb0270fd Followup fix for Custom AR V2 in non NVL scenarios (#24742) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-10 16:57:46 -07:00
Brayden Zhongandb8zhong 4b23f6bdc5 Fix performance regression on Deepseek V3 on moe-runner-backend=triton on SM90 (#24562) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-09 03:49:12 -07:00
Brayden Zhongandb8zhong 05d1ab51e8 Enable PDL for various kernels in DSV32/GLM5 (#23965) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-09 03:42:56 -07:00
Brayden Zhongandb8zhong f4b7e73699 Enable trtllm-gen BF16 MoE for MTP (#24260) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-09 03:14:17 -07:00
Brayden Zhongandb8zhong 8f33bee31b Reland Cute-DSL FP4 dense GEMM (#23590) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-09 02:20:58 -07:00
Brayden Zhongandb8zhong 9ee830346f Disable Custom AR V2 when in multi-node (#24729) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-08 17:50:05 -07:00
Brayden Zhongandb8zhong 80d0226b68 Turn on JIT custom AR implementation by default (#24363) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-08 02:05:31 -07:00
Brayden Zhongb8zhonggemini-code-assist[bot]
 		5fa3bb2eaf Enable flashinfer::trtllm_allreduce_fusion with PDL (#23765) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-05-08 10:41:10 +08:00
Brayden Zhongandb8zhong 3fe8bc987e Support Triton MLA FP8 KV cache (#20479) 
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
 2026-05-06 18:32:39 -07:00
Brayden ZhongKangyan ZhouClaude Opus 4.7Baizhou Zhangb8zhongMick
 		88bb5dffe4 [Dependency] Upgrade to Torch 2.11.0 (#21247) 
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: b8zhong <b8zhong@users.noreply.github.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-05-02 12:25:36 -07:00
Brayden Zhong 6aafe756b9 Revert "[Feature] NVFP4 Marlin fallback for non-Blackwell GPUs (SM75+… (#22047) 2026-04-03 13:12:30 -07:00
Brayden Zhong cb0c2cbfdb Enable multi-thread weight loading by default (#20289) 2026-04-01 21:27:20 -07:00
Brayden Zhong 6a9b09847c CUTLASS NVFP4 GEMM improvement of SM120 (#21314) 2026-04-01 09:04:34 +08:00
Brayden Zhong 009eee85a0 CUTLASS FP8 Blockwise GEMM improvement of SM120 (#20887) 2026-03-22 17:55:54 +08:00
Brayden Zhong b42b9f6e1a Support CuteDSL mm_fp4 backend (#18801) 2026-03-19 14:20:01 -07:00
Brayden Zhong 88c40ec16d Use Flashinfer for target_verify in GDN model for SM120 (#20604) 2026-03-17 22:40:56 -07:00
Brayden Zhong 97d5386a21 Use TRTLLM allreduce fusion for Qwen 3.5 (#19889) 2026-03-17 22:40:22 -07:00
Brayden Zhong 591e61245a [Doc] Add smal table for GEMM backends (#20213) 2026-03-09 22:19:57 -07:00
Brayden Zhong e2af840c3d Various SM120 improvements (#19721) 2026-03-03 16:46:13 -08:00
Brayden Zhongandamittell 1388680+amittell@users.noreply.github.com 9305f0e58d Support triton_kernels for GPT-OSS on SM120 (#19718) 
Co-authored-by: amittell 1388680+amittell@users.noreply.github.com
 2026-03-03 14:14:01 -08:00
Brayden ZhongBaidu-AIAKzengpai
 		80a6b32703 [Perf] Optimize NSA backend metadata under MTP (#19536) 
Co-authored-by: Baidu-AIAK <Baidu_AIAK@163.com>
Co-authored-by: zengpai <zengpai@baidu.com>
 2026-03-01 01:59:26 -08:00
b8zhongandCheng Wan 78bf13db44 MoE Refactor: Refactor modelopt_quant.py -> flashinfer_trllm.py (#16685) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2026-02-02 20:45:14 -08:00
b8zhongandBrayden Zhong 398d13a189 [Perf] Add Flashinfer DeepGEMM SM90 for SwapAB Optimization (#15514) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2026-02-01 08:56:23 +08:00
b8zhong ef134d407d [Fix] Revert back to using CUTLASS mm_fp4 backend (#17369) 2026-01-31 23:01:29 +08:00
b8zhong 22498e10c0 [Fix] Triton TP MoE Dpsk V3/Qwen3 Coder with SwapAB (#17965) 2026-01-31 15:56:26 +08:00
b8zhongandBrayden Zhong 22df62d586 add weightless qk norm to RMSNorm interface for Llama 4 (#12813) 
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
 2026-01-29 19:09:55 -08:00
b8zhong 2ff0880a0e [Fix] GLM 4.7 + NVFP4 + MTP (#17166) 2026-01-21 21:34:18 +08:00
b8zhong 7dc3cbe7ca [Docker] Fix CUDA 13 installing wrong nvidia-nccl-cu13 due to nixl-cu13 not breaking system package (#17370) 2026-01-20 09:52:30 +08:00
b8zhong f374623fa9 [Refactor] Set fp4-gemm-backend=auto on SM100 and rename fp4-gemm-backend with flashinfer_ prefix (#17309) 2026-01-19 20:09:07 +08:00
b8zhongandVincent Zhong 4df74eb576 [Refactor] Add -fp4-gemm-backend to replace SGLANG_FLASHINFER_FP4_GEMM_BACKEND (#16534) 
Co-authored-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com>
 2026-01-18 23:25:46 +08:00
b8zhong d36f6f043c [Fix] flashinfer_trtllm intermediate_size assertion with Qwen3 + TP=8 (#16824) 2026-01-17 09:24:05 +08:00