minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity
18,687 Commits 2 Branches 0 Tags
b081dd3d234b30029fabba7dc358d76834a40e3a
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Mohammad Miadh AngkadandMohammad Angkad 4c81cd1b09 [KDA] Fix missing beta sigmoid in PTX prefill (#40685) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-21 23:24:38 -07:00
Mohammad Miadh Angkadandmmangkad e332e1b84e [Fix] Don't write conv state from the fused KDA verify kernel (#39524) 
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
 2026-09-21 18:33:03 -07:00
Mohammad Miadh Angkadmmangkadkpham-sglClaude Fable 5.1
 		11e661fd45 [Fix] Don't free the multi-CTAs KV counter the decode graphs captured (#39175) 
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-21 12:58:48 -07:00
Mohammad Miadh AngkadandMohammad Angkad 8d08dfdab7 [Fix] Add gigachat35 to the tool-call and reasoning parser name lists (#40554) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-21 16:34:43 +08:00
Mohammad Miadh Angkadandmmangkad 2fa6b94e34 [Perf] Fuse the glm5_next mHC attn->MLP boundary (#39200) 
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
 2026-09-20 16:20:37 -07:00
Mohammad Miadh AngkadandMohammad Angkad 8923f4d779 [Test] Fix optimistic prefill disaggregation test after mamba radix cache removal (#40469) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-20 22:39:34 +08:00
Mohammad Miadh AngkadandMohammad Angkad 22f02cc339 [Test] Fix scheduler fixtures after prefill burst counting (#40411) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-20 00:08:04 -07:00
Mohammad Miadh AngkadandMohammad Angkad df0dc44931 [Fix] Forward SWA prealloc reclaim through the DSV4 HiSparse allocator (#40354) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-19 20:27:51 -07:00
Mohammad Miadh AngkadandMohammad Angkad c055dc6ff6 [CI] Check B200 NUMA mapping against sysfs numa_node (#40055) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-17 21:14:47 -07:00
Mohammad Miadh AngkadandMohammad Angkad 935cbf24ec Accept MXFP8 dispatch in FlashInfer A2A TRT-LLM MoE (#39613) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-15 19:23:15 -07:00
Mohammad Miadh AngkadMohammad AngkadLiangsheng Yinhnyls2002
 		ddd4600197 [Fix] HiCache startup ImportError on the pinned kernel wheel (#39516) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-09-14 22:31:19 -07:00
Mohammad Miadh AngkadBrayden ZhongMohammad AngkadXinyuan TongXinyuan Tong
 		6c514ab025 Force reasoning mode for GLM-5.3 chat templates (#39227) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-09-15 13:16:26 +08:00
Mohammad Miadh Angkadandmmangkad 2fd835b9c1 [Fix] Don't write conv state from the fused KDA verify kernel (#39219) 
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
 2026-09-13 21:26:57 -07:00
Mohammad Miadh AngkadandMohammad Angkad 4309c7ce19 Fix stale DSV4 indexer metadata names in the TopK v2 dispatch test (#39101) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-11 08:31:42 -07:00
Mohammad Miadh AngkadandMohammad Angkad ad5af539cd [CI] Trim DSV4 trtllm B200 tests (#39013) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-11 01:04:16 -07:00
Mohammad Miadh AngkadandMohammad Angkad 52c191da52 [Deps] Retire the CUDA 12 lane (#38404) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-10 16:58:09 -07:00
Mohammad Miadh AngkadandMohammad Angkad 613002281e [CI] Drop the GPTQ dynamic-config test for the deleted non-Marlin kernel (#38881) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-10 20:55:06 +08:00
Mohammad Miadh AngkadandMohammad Angkad dc2157dcd6 [JIT] Port the expert-pack MXFP4 kernels to load_jit and fix their launch limits (#38830) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-10 17:50:51 +08:00
Mohammad Miadh Angkad 8022505705 Revert "[CI] Temporarily disable GB300 tests" (#38842) 2026-09-10 02:41:15 -07:00
Mohammad Miadh AngkadMohammad AngkadAlison Shao
 		2092f6df05 [CI] Install helion 1.4.0 for the KDA Helion kernel tests (#38688) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
Co-authored-by: Alison Shao <54658187+alisonshao@users.noreply.github.com>
 2026-09-09 16:00:17 -07:00
Mohammad Miadh AngkadandMohammad Angkad 72d5c5bb73 [Kimi-K3] Accept fp32 routing weights in the fused MoE finalize (#38612) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-09 01:33:39 -07:00
Mohammad Miadh AngkadandMohammad Angkad a711785475 [Kernel] Drop the vendored dense BF16 GEMM port in favor of FlashInfer 0.6.18 (#38124) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-07 12:59:33 -07:00
Mohammad Miadh Angkad c5367fa964 [CI] Fix stale ServerArgs fake in chunked-SGMV LoRA test (#38315) 2026-09-07 04:25:12 -07:00
Mohammad Miadh Angkad cfa989b7af [CI] Fix stale CP-v1 mock patches in NextN mm-embed test (#38280) 2026-09-06 23:20:29 -07:00
Mohammad Miadh Angkad df2f34cca1 Drop stale test_mla_gluon_h12_fp8.py broken by aiter gluon rewrite (#38259) 2026-09-06 20:41:42 -07:00
Mohammad Miadh AngkadandMohammad Angkad 09daea94ac Support NoPE layers in the tokenspeed_mla FP8 prefill hook (#38152) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-05 16:51:04 -07:00
Mohammad Miadh AngkadandMohammad Angkad ca7b8efd87 [CI] Fix handle_platform_cp_compatibility reading legacy CP flags off the record (#37930) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-03 22:07:04 -07:00
Mohammad Miadh Angkad 3ad3f23ed5 [Comm] Drop the in-tree MNNVL CuTe DSL port in favor of FlashInfer 0.6.18 (#37206) 2026-09-03 21:49:48 -07:00
Mohammad Miadh AngkadandMohammad Angkad a5f07b1241 [CI] Build the Rust extensions for aarch64 too (#37820) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-03 19:54:08 -07:00
Mohammad Miadh AngkadandMohammad Angkad c1b4d535d7 [CI] Fix lint (#37895) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-03 18:37:50 -07:00
Mohammad Miadh Angkad 4372b8efa7 [1/N] Quantization Refactor: remove dead code and dedup the FP4 marlin helpers (#37552) 2026-09-03 14:22:31 -07:00
Mohammad Miadh Angkad 718bd39fe0 [Fix] DP attention: correct the decode->extend prefix off-by-one (#37505) 2026-09-02 16:14:16 -07:00
Mohammad Miadh Angkad d9ae1ebabd [CI] Preserve NCCL 2.30.7 after dependency installs (#37508) 2026-09-01 19:59:13 -07:00
Mohammad Miadh Angkad 03b33cbe5d [CI] Fix hybrid wrapper test fake missing kv_index_translator (#37374) 2026-09-01 00:55:05 -07:00
Mohammad Miadh Angkad e6f21cdadc [Cohere Command-A-Plus] Optimize decode and BCG capture on SM10X (#36624) 2026-08-31 19:37:47 -07:00
Mohammad Miadh Angkad 4f761e8649 [Deps] Bump FlashInfer to 0.6.18 (#36954) 2026-08-30 19:02:39 -07:00
Mohammad Miadh Angkad 8a4c517a60 [Docs] Restore the AIME25 label so GLM-5.3 FP8 and BF16 scores render again (#36950) 2026-08-29 11:38:26 +08:00
Mohammad Miadh Angkad 70088aa5db Fix KeyError on batch requests whose state is freed before it is read (#36638) 2026-08-28 12:57:07 -07:00
Mohammad Miadh AngkadandMohammad Angkad 2935bb8e79 Fix OOB read in mxfp4 MoE weight scales on Hopper (#36456) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-26 13:36:49 -07:00
Mohammad Miadh AngkadandMohammad Angkad ffc431cd4c [CI] Fix stale tool_call_parser key in Spark2.5 detector test (#36464) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-26 01:34:39 -07:00
Mohammad Miadh AngkadandMohammad Angkad a3c4936438 Sync FlashInfer autotune tactic choice across TP ranks (#35343) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-26 16:30:47 +08:00
Mohammad Miadh Angkad 90354326c7 [VLM] Split Pixtral multi-image features before the CUDA IPC wrap (#35463) 2026-08-22 15:17:55 +08:00
Mohammad Miadh AngkadandMohammad Angkad 3e096629cf [CI] Re-enable B300 jobs (#35607) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-22 12:34:21 +08:00
Mohammad Miadh AngkadandMohammad Angkad 61c2da42bb [Fix] Pass Anthropic thinking history as reasoning_content for custom chat encoders (#35480) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-22 02:22:11 +08:00
Mohammad Miadh Angkad aa3f766799 [CI] Temporarily disable B300 jobs (#35627) 2026-08-20 17:54:53 -07:00
Mohammad Miadh Angkad a4ffb996db [Fix] Keep deterministic GDN prefill on Triton (#35632) 2026-08-20 19:38:53 +08:00
Mohammad Miadh Angkad ba433bb462 [Docs] Update contribution guide (#35419) 2026-08-19 22:31:24 -07:00
Mohammad Miadh AngkadandMohammad Angkad 1beb805356 [Perf] Restore the 16-token router GEMM threshold on SM10X (#34953) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-18 19:53:52 -07:00
Mohammad Miadh Angkad 82995a001b Stabilize GB300 nightly tests (#35044) 2026-08-17 03:30:16 -07:00
Mohammad Miadh AngkadBrayden Zhongmmangkad
 		8cc112d486 [DSA] Skip indexer KV cache for skip-topk layers (#30531) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
 2026-08-17 02:02:23 -07:00
Mohammad Miadh AngkadandMohammad Angkad 5769b6d637 [JIT Kernel] Migrate causal_conv1d_fwd and causal_conv1d_update from AOT to JIT (#35031) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-17 15:22:37 +08:00
Mohammad Miadh Angkad 721e359ca7 Suppress expected FlashInfer TRT-LLM workspace warnings (#34921) 2026-08-17 15:16:16 +08:00
Mohammad Miadh AngkadandMohammad Angkad 6ab4b99bc2 [Quantization] Fix GPTQ scheme attachment broken by LinearBase.scheme default (#34962) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-16 00:48:02 -07:00
Mohammad Miadh AngkadandMohammad Angkad 3802a725ac [CI] Pin the allowed_media_domains supplied-instance reads in the step-12 ratchet (#34961) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-15 16:35:19 -07:00
Mohammad Miadh Angkad d7207be156 Fix startup weight load after TorchAO removal (#34869) 2026-08-14 13:18:53 -07:00
Mohammad Miadh Angkad 9d2f1584fa Fix MiniMax-H3 Cache-DiT BCG warning (#34848) 2026-08-14 22:07:50 +08:00
Mohammad Miadh Angkad 96db53ec70 [CI] Fix test_resolution_is_reproducible after cuda_ipc became opt-in (#34746) 2026-08-13 12:47:03 -07:00
Mohammad Miadh Angkad aefe2d0207 Revert "[Kimi K3] Fuse MLA gate projection into QKV-A GEMM" (#34642) 2026-08-13 08:35:38 +08:00
Mohammad Miadh Angkad 00e57d74f0 Bump FlashInfer to 0.6.17 and remove Kimi K3 workarounds (#33997) 2026-08-12 02:17:26 -07:00
Mohammad Miadh Angkad 0dab252ffc Fix DFlash sliding attention causality defaults (#34524) 2026-08-11 23:05:55 -07:00
Mohammad Miadh Angkad 7fb6e61b95 Fix CUDA 13.0 VMM handle type compatibility (#34431) 2026-08-11 15:16:34 -07:00
Mohammad Miadh Angkad a3bd7d9401 Bump CuTeDSL to 4.6.2 (#34372) 2026-08-12 00:35:35 +08:00
Mohammad Miadh Angkad 2d193077f7 [JIT Kernel] Migrate per-token FP8 quantization from AOT to JIT (#34257) 2026-08-11 20:40:40 +08:00
Mohammad Miadh Angkad 8c5d5f75bf Fix DSV4 DSpark shared expert loading (#33312) 2026-08-11 07:36:50 +08:00
Mohammad Miadh Angkad 77b8315b84 [CI] Fix GSM8K floating-point tolerance boundary (#34272) 2026-08-10 16:27:11 -07:00
Mohammad Miadh Angkad c2d90db1e3 [CI] Add Kimi-K3 low-latency performance check (#34089) 2026-08-08 13:42:59 -07:00
Mohammad Miadh AngkadandBBuf ec5199b906 [jit_kernel] Fix missing JIT kernel namespaces (#34106) 
Co-authored-by: BBuf <1182563586@qq.com>
 2026-08-08 22:15:05 +08:00
Mohammad Miadh AngkadandBrayden Zhong 434e646282 [Deps] Upgrade CUDA PyTorch stack to 2.13 (#28836) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-08-06 12:08:44 -07:00
Mohammad Miadh Angkad 8a1637a479 Fix serving benchmark post-warmup cache flush race (#33663) 2026-08-06 15:27:10 +00:00
Mohammad Miadh Angkad 1a15cf1536 Gate multimodal feature transport by model capability (#33653) 2026-08-06 06:48:32 -07:00
Mohammad Miadh Angkad 04374ba5e0 Update AOT kernels for Torch 2.13 (#33841) 2026-08-06 01:03:26 -07:00
Mohammad Miadh Angkad f33f6a522f Relax GDN ReplaySSM fold test for Triton 3.7 (#33780) 2026-08-05 19:44:14 -07:00
Mohammad Miadh Angkad 65d5a0ec25 Support ModelOpt MXFP8 checkpoints (#32538) 2026-08-05 17:44:32 -07:00
Mohammad Miadh Angkad 5424d2039c [CI] Fix GLM-Image usage unit tests (#33731) 2026-08-05 16:20:08 -07:00
Mohammad Miadh Angkad 98ed5554bb Stop testing cu129 DeepGEMM wheels (#33675) 2026-08-05 01:38:28 -07:00
Mohammad Miadh AngkadXinyuan TongDavid Orman
 		059269594c [DSV4] Add official DSV4 reasoning effort support (#33140) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: David Orman <ormandj@corenode.com>
 2026-08-05 12:50:41 +08:00
Mohammad Miadh Angkad 7dd8a3d5ea [CI] Fix scheduler max new tokens test fixture (#33467) 2026-08-03 20:12:21 -07:00
Mohammad Miadh Angkad d48ab2d386 Fix BCG circular import during server startup (#33371) 2026-08-03 01:53:16 -07:00
Mohammad Miadh Angkad ec741e4161 Fix DSpark loading for hybrid DSV4 NVFP4 (#33276) 2026-08-02 18:05:09 +00:00
Mohammad Miadh Angkad 27b15349e5 [CI] Fix stale CPU test fixtures (#33277) 2026-08-02 22:38:45 +08:00
Mohammad Miadh Angkadandhnyls2002 c0d06a6547 [CI] Fix runtime context setup in flat logprob tests (#33179) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-07-31 22:46:24 -07:00
Mohammad Miadh Angkad a1344fad4e Replace Kimi K3 DeepGEMM patch with 0.1.5.post1 (#33143) 2026-07-31 22:17:46 -07:00
Mohammad Miadh Angkad 3a53c26c27 [CI] Fix MoE compile and DSA indexer regressions (#32937) 2026-07-30 15:21:51 -07:00
Mohammad Miadh Angkad a55e1764a2 Enable GPT-OSS FlashInfer MXFP4 on SM120 (#32668) 2026-07-30 00:04:23 +00:00
Mohammad Miadh Angkad 1b9dfa14e6 Fix FlashInfer MNNVL workspace size check (#32318) 2026-07-29 02:12:18 -07:00
Mohammad Miadh Angkad 3005af0941 Fix compressed-tensors NVFP4 MoE W13 layout (#32430) 2026-07-27 14:56:42 -07:00
Mohammad Miadh AngkadandXinyuan Tong 3ebb7c2d07 docs: point Kimi-K3 references to public branch (#32547) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-07-27 16:23:43 +00:00
Mohammad Miadh Angkad 9791fc7090 Add configurable FlashInfer autotune skips (#31389) 2026-07-25 11:17:58 -07:00
Mohammad Miadh Angkad 953c587adf [Docs] Add Qwen3.6 35B NVFP4 to cookbook (#31413) 2026-07-25 11:16:53 -07:00
Mohammad Miadh Angkad a2ddf92e61 [CI] Fix Mamba ServerArgs namespace (#32211) 2026-07-23 13:15:50 -07:00
Mohammad Miadh AngkadandBrayden Zhong 0c29c8fece Bump FlashInfer to 0.6.15.post1 (#31927) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-07-22 14:21:59 -07:00
Mohammad Miadh Angkad 0bdd4730af [CI] Fix failures on main (#32091) 2026-07-23 00:11:25 +08:00
Mohammad Miadh Angkad b855efd9e6 Fix Inkling kernel imports after migration (#32076) 2026-07-22 22:09:01 +08:00
Mohammad Miadh Angkad d6690de961 [CI] Fix Marlin MoE test ServerArgs initialization (#32049) 2026-07-22 20:16:43 +08:00
Mohammad Miadh Angkad 4f88206393 [CI] Fix stale per-token group quant callers (#32047) 2026-07-22 19:45:29 +08:00
Mohammad Miadh Angkad a82ead53bd Make Q contiguous before TRT-LLM MHA decode (#31667) 2026-07-20 15:21:00 -07:00
Mohammad Miadh Angkad 3d82dacd58 Bump CuTe DSL to 4.6.0 (#31714) 2026-07-20 02:11:59 -07:00
Mohammad Miadh Angkad 35f2d4f761 Fix no-padding CUDA graph admission (#31273) 2026-07-19 19:28:35 -07:00
Mohammad Miadh Angkad 8bf2ab9be9 Enable GPT-OSS TinyGEMM on CUDA 13 (#31649) 2026-07-19 19:16:39 -07:00
Mohammad Miadh AngkadandBrayden Zhong d67aa05697 Bump FlashInfer to 0.6.15 and revert regressions (#31502) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-07-17 00:50:12 -07:00