minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity
17,727 Commits 2 Branches 0 Tags
c1b4d535d772d7bfa89e1f71f91ab65f2f8da876
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Mohammad Miadh AngkadandMohammad Angkad c1b4d535d7 [CI] Fix lint (#37895) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-03 18:37:50 -07:00
Mohammad Miadh Angkad 4372b8efa7 [1/N] Quantization Refactor: remove dead code and dedup the FP4 marlin helpers (#37552) 2026-09-03 14:22:31 -07:00
Mohammad Miadh Angkad 718bd39fe0 [Fix] DP attention: correct the decode->extend prefix off-by-one (#37505) 2026-09-02 16:14:16 -07:00
Mohammad Miadh Angkad d9ae1ebabd [CI] Preserve NCCL 2.30.7 after dependency installs (#37508) 2026-09-01 19:59:13 -07:00
Mohammad Miadh Angkad 03b33cbe5d [CI] Fix hybrid wrapper test fake missing kv_index_translator (#37374) 2026-09-01 00:55:05 -07:00
Mohammad Miadh Angkad e6f21cdadc [Cohere Command-A-Plus] Optimize decode and BCG capture on SM10X (#36624) 2026-08-31 19:37:47 -07:00
Mohammad Miadh Angkad 4f761e8649 [Deps] Bump FlashInfer to 0.6.18 (#36954) 2026-08-30 19:02:39 -07:00
Mohammad Miadh Angkad 8a4c517a60 [Docs] Restore the AIME25 label so GLM-5.3 FP8 and BF16 scores render again (#36950) 2026-08-29 11:38:26 +08:00
Mohammad Miadh Angkad 70088aa5db Fix KeyError on batch requests whose state is freed before it is read (#36638) 2026-08-28 12:57:07 -07:00
Mohammad Miadh AngkadandMohammad Angkad 2935bb8e79 Fix OOB read in mxfp4 MoE weight scales on Hopper (#36456) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-26 13:36:49 -07:00
Mohammad Miadh AngkadandMohammad Angkad ffc431cd4c [CI] Fix stale tool_call_parser key in Spark2.5 detector test (#36464) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-26 01:34:39 -07:00
Mohammad Miadh AngkadandMohammad Angkad a3c4936438 Sync FlashInfer autotune tactic choice across TP ranks (#35343) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-26 16:30:47 +08:00
Mohammad Miadh Angkad 90354326c7 [VLM] Split Pixtral multi-image features before the CUDA IPC wrap (#35463) 2026-08-22 15:17:55 +08:00
Mohammad Miadh AngkadandMohammad Angkad 3e096629cf [CI] Re-enable B300 jobs (#35607) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-22 12:34:21 +08:00
Mohammad Miadh AngkadandMohammad Angkad 61c2da42bb [Fix] Pass Anthropic thinking history as reasoning_content for custom chat encoders (#35480) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-22 02:22:11 +08:00
Mohammad Miadh Angkad aa3f766799 [CI] Temporarily disable B300 jobs (#35627) 2026-08-20 17:54:53 -07:00
Mohammad Miadh Angkad a4ffb996db [Fix] Keep deterministic GDN prefill on Triton (#35632) 2026-08-20 19:38:53 +08:00
Mohammad Miadh Angkad ba433bb462 [Docs] Update contribution guide (#35419) 2026-08-19 22:31:24 -07:00
Mohammad Miadh AngkadandMohammad Angkad 1beb805356 [Perf] Restore the 16-token router GEMM threshold on SM10X (#34953) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-18 19:53:52 -07:00
Mohammad Miadh Angkad 82995a001b Stabilize GB300 nightly tests (#35044) 2026-08-17 03:30:16 -07:00
Mohammad Miadh AngkadBrayden Zhongmmangkad
 		8cc112d486 [DSA] Skip indexer KV cache for skip-topk layers (#30531) 
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
 2026-08-17 02:02:23 -07:00
Mohammad Miadh AngkadandMohammad Angkad 5769b6d637 [JIT Kernel] Migrate causal_conv1d_fwd and causal_conv1d_update from AOT to JIT (#35031) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-17 15:22:37 +08:00
Mohammad Miadh Angkad 721e359ca7 Suppress expected FlashInfer TRT-LLM workspace warnings (#34921) 2026-08-17 15:16:16 +08:00
Mohammad Miadh AngkadandMohammad Angkad 6ab4b99bc2 [Quantization] Fix GPTQ scheme attachment broken by LinearBase.scheme default (#34962) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-16 00:48:02 -07:00
Mohammad Miadh AngkadandMohammad Angkad 3802a725ac [CI] Pin the allowed_media_domains supplied-instance reads in the step-12 ratchet (#34961) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-15 16:35:19 -07:00
Mohammad Miadh Angkad d7207be156 Fix startup weight load after TorchAO removal (#34869) 2026-08-14 13:18:53 -07:00
Mohammad Miadh Angkad 9d2f1584fa Fix MiniMax-H3 Cache-DiT BCG warning (#34848) 2026-08-14 22:07:50 +08:00
Mohammad Miadh Angkad 96db53ec70 [CI] Fix test_resolution_is_reproducible after cuda_ipc became opt-in (#34746) 2026-08-13 12:47:03 -07:00
Mohammad Miadh Angkad aefe2d0207 Revert "[Kimi K3] Fuse MLA gate projection into QKV-A GEMM" (#34642) 2026-08-13 08:35:38 +08:00
Mohammad Miadh Angkad 00e57d74f0 Bump FlashInfer to 0.6.17 and remove Kimi K3 workarounds (#33997) 2026-08-12 02:17:26 -07:00
Mohammad Miadh Angkad 0dab252ffc Fix DFlash sliding attention causality defaults (#34524) 2026-08-11 23:05:55 -07:00
Mohammad Miadh Angkad 7fb6e61b95 Fix CUDA 13.0 VMM handle type compatibility (#34431) 2026-08-11 15:16:34 -07:00
Mohammad Miadh Angkad a3bd7d9401 Bump CuTeDSL to 4.6.2 (#34372) 2026-08-12 00:35:35 +08:00
Mohammad Miadh Angkad 2d193077f7 [JIT Kernel] Migrate per-token FP8 quantization from AOT to JIT (#34257) 2026-08-11 20:40:40 +08:00
Mohammad Miadh Angkad 8c5d5f75bf Fix DSV4 DSpark shared expert loading (#33312) 2026-08-11 07:36:50 +08:00
Mohammad Miadh Angkad 77b8315b84 [CI] Fix GSM8K floating-point tolerance boundary (#34272) 2026-08-10 16:27:11 -07:00
Mohammad Miadh Angkad c2d90db1e3 [CI] Add Kimi-K3 low-latency performance check (#34089) 2026-08-08 13:42:59 -07:00
Mohammad Miadh AngkadandBBuf ec5199b906 [jit_kernel] Fix missing JIT kernel namespaces (#34106) 
Co-authored-by: BBuf <1182563586@qq.com>
 2026-08-08 22:15:05 +08:00
Mohammad Miadh AngkadandBrayden Zhong 434e646282 [Deps] Upgrade CUDA PyTorch stack to 2.13 (#28836) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-08-06 12:08:44 -07:00
Mohammad Miadh Angkad 8a1637a479 Fix serving benchmark post-warmup cache flush race (#33663) 2026-08-06 15:27:10 +00:00
Mohammad Miadh Angkad 1a15cf1536 Gate multimodal feature transport by model capability (#33653) 2026-08-06 06:48:32 -07:00
Mohammad Miadh Angkad 04374ba5e0 Update AOT kernels for Torch 2.13 (#33841) 2026-08-06 01:03:26 -07:00
Mohammad Miadh Angkad f33f6a522f Relax GDN ReplaySSM fold test for Triton 3.7 (#33780) 2026-08-05 19:44:14 -07:00
Mohammad Miadh Angkad 65d5a0ec25 Support ModelOpt MXFP8 checkpoints (#32538) 2026-08-05 17:44:32 -07:00
Mohammad Miadh Angkad 5424d2039c [CI] Fix GLM-Image usage unit tests (#33731) 2026-08-05 16:20:08 -07:00
Mohammad Miadh Angkad 98ed5554bb Stop testing cu129 DeepGEMM wheels (#33675) 2026-08-05 01:38:28 -07:00
Mohammad Miadh AngkadXinyuan TongDavid Orman
 		059269594c [DSV4] Add official DSV4 reasoning effort support (#33140) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: David Orman <ormandj@corenode.com>
 2026-08-05 12:50:41 +08:00
Mohammad Miadh Angkad 7dd8a3d5ea [CI] Fix scheduler max new tokens test fixture (#33467) 2026-08-03 20:12:21 -07:00
Mohammad Miadh Angkad d48ab2d386 Fix BCG circular import during server startup (#33371) 2026-08-03 01:53:16 -07:00
Mohammad Miadh Angkad ec741e4161 Fix DSpark loading for hybrid DSV4 NVFP4 (#33276) 2026-08-02 18:05:09 +00:00
Mohammad Miadh Angkad 27b15349e5 [CI] Fix stale CPU test fixtures (#33277) 2026-08-02 22:38:45 +08:00
Mohammad Miadh Angkadandhnyls2002 c0d06a6547 [CI] Fix runtime context setup in flat logprob tests (#33179) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-07-31 22:46:24 -07:00
Mohammad Miadh Angkad a1344fad4e Replace Kimi K3 DeepGEMM patch with 0.1.5.post1 (#33143) 2026-07-31 22:17:46 -07:00
Mohammad Miadh Angkad 3a53c26c27 [CI] Fix MoE compile and DSA indexer regressions (#32937) 2026-07-30 15:21:51 -07:00
Mohammad Miadh Angkad a55e1764a2 Enable GPT-OSS FlashInfer MXFP4 on SM120 (#32668) 2026-07-30 00:04:23 +00:00
Mohammad Miadh Angkad 1b9dfa14e6 Fix FlashInfer MNNVL workspace size check (#32318) 2026-07-29 02:12:18 -07:00
Mohammad Miadh Angkad 3005af0941 Fix compressed-tensors NVFP4 MoE W13 layout (#32430) 2026-07-27 14:56:42 -07:00
Mohammad Miadh AngkadandXinyuan Tong 3ebb7c2d07 docs: point Kimi-K3 references to public branch (#32547) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-07-27 16:23:43 +00:00
Mohammad Miadh Angkad 9791fc7090 Add configurable FlashInfer autotune skips (#31389) 2026-07-25 11:17:58 -07:00
Mohammad Miadh Angkad 953c587adf [Docs] Add Qwen3.6 35B NVFP4 to cookbook (#31413) 2026-07-25 11:16:53 -07:00
Mohammad Miadh Angkad a2ddf92e61 [CI] Fix Mamba ServerArgs namespace (#32211) 2026-07-23 13:15:50 -07:00
Mohammad Miadh AngkadandBrayden Zhong 0c29c8fece Bump FlashInfer to 0.6.15.post1 (#31927) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-07-22 14:21:59 -07:00
Mohammad Miadh Angkad 0bdd4730af [CI] Fix failures on main (#32091) 2026-07-23 00:11:25 +08:00
Mohammad Miadh Angkad b855efd9e6 Fix Inkling kernel imports after migration (#32076) 2026-07-22 22:09:01 +08:00
Mohammad Miadh Angkad d6690de961 [CI] Fix Marlin MoE test ServerArgs initialization (#32049) 2026-07-22 20:16:43 +08:00
Mohammad Miadh Angkad 4f88206393 [CI] Fix stale per-token group quant callers (#32047) 2026-07-22 19:45:29 +08:00
Mohammad Miadh Angkad a82ead53bd Make Q contiguous before TRT-LLM MHA decode (#31667) 2026-07-20 15:21:00 -07:00
Mohammad Miadh Angkad 3d82dacd58 Bump CuTe DSL to 4.6.0 (#31714) 2026-07-20 02:11:59 -07:00
Mohammad Miadh Angkad 35f2d4f761 Fix no-padding CUDA graph admission (#31273) 2026-07-19 19:28:35 -07:00
Mohammad Miadh Angkad 8bf2ab9be9 Enable GPT-OSS TinyGEMM on CUDA 13 (#31649) 2026-07-19 19:16:39 -07:00
Mohammad Miadh AngkadandBrayden Zhong d67aa05697 Bump FlashInfer to 0.6.15 and revert regressions (#31502) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
 2026-07-17 00:50:12 -07:00
Mohammad Miadh Angkad 18043aec20 [CI] Fix TRTLLM MHA graph metadata test fixture (#31332) 2026-07-15 12:44:51 -07:00
Mohammad Miadh Angkad 5af670284e [CI] Lower GLM-5.2 NVFP4 MTP speed threshold (#31289) 2026-07-15 00:39:58 -07:00
Mohammad Miadh Angkad 271e5ef5c3 [CI] Fix Flash MLA SM120 test import path (#31199) 2026-07-14 09:01:23 -07:00
Mohammad Miadh Angkad 2f79d334f2 [Bugfix] Fix DeepSeek ForwardFlags across custom op boundary (#30987) 2026-07-13 17:19:44 -07:00
Mohammad Miadh Angkad cbcbef6811 [Bugfix] Fix Nemotron ForwardFlags across custom op boundary (#30968) 2026-07-12 22:10:23 -07:00
Mohammad Miadh Angkad 24d59d8d74 Fix CUDA 12 Docker dependency resolution (#30858) 2026-07-12 13:50:41 -07:00
Mohammad Miadh Angkad 96a04cb13f Fix DeepEP CI test registration (#30873) 2026-07-12 05:54:50 -07:00
Mohammad Miadh Angkad 14bef7cd11 fix: lazy load TileLang MHC kernels (#30580) 2026-07-12 08:32:27 +08:00
Mohammad Miadh Angkad 7b99900980 Fix hybrid attention graph hook test fixture (#30790) 2026-07-10 23:37:04 +05:30
Mohammad Miadh Angkad 7045e0fdff Seed sgl-kernel topk sigmoid tests on all backends (#30754) 2026-07-10 03:08:15 -07:00
Mohammad Miadh Angkad ecb7fb3989 Fix CuTe DSL DSA paged MQA export (#30627) 2026-07-10 01:32:58 -07:00
Mohammad Miadh Angkad 295f85df08 Fix DFlash mamba verify init ordering (#30680) 2026-07-09 17:40:12 -07:00
Mohammad Miadh Angkad b0ecbceed9 [Bugfix] Migrate retired parallel accessors (#30653) 2026-07-09 11:22:34 -07:00
Mohammad Miadh Angkad 666a09fe2a Disable multi-threaded load by default when prefetch is on (#30146) 2026-07-09 00:28:53 -07:00
Mohammad Miadh Angkad 04e4fadff3 Use FP32 logits in MoEGate fallbacks (#30323) 2026-07-08 23:43:45 +08:00
Mohammad Miadh Angkad fda87173ab Revert "Increase the KV cache pool when using indexShare by 15% (#30310)" (#30472) 2026-07-07 23:48:35 -07:00
Mohammad Miadh Angkad f32b4ecd26 [Docs] Use trtllm_mha for Qwen3.6 B300 (#29964) 2026-07-07 01:44:01 -07:00
Mohammad Miadh Angkad b1942fc3ea [Model] Support Qwen3.6 ModelOpt mixed NVFP4 (#27906) 2026-07-05 21:31:15 -07:00
Mohammad Miadh Angkad cfb9c574d3 Fix UE8M0 scale rounding for DeepGEMM (#29956) 2026-07-02 13:42:17 -07:00
Mohammad Miadh Angkad 8f0d320d31 [Spec] Enable FlashInfer autotune for spec draft (#29595) 2026-07-01 13:36:07 -07:00
Mohammad Miadh Angkad a0d9791810 [CI] Fix fused EH norm CI registration (#29845) 2026-07-01 21:27:07 +08:00
Mohammad Miadh Angkad 07ca24372b Add fused EH norm for DeepSeek NextN (#29667) 2026-07-01 02:46:21 -07:00
Mohammad Miadh Angkad bae78a44da [Deps] Bump transformers to 5.12.1 (#29393) 2026-06-30 00:54:01 -07:00
Mohammad Miadh Angkad b6fceaa789 [DSA] Use cos_sin_cache for DSA indexer fusion (#29613) 2026-06-29 23:49:00 -07:00
Mohammad Miadh Angkad 3a72d02415 [DSA] Optimize DSA CUDA graph replay metadata generation (#29499) 2026-06-29 19:53:23 -07:00
Mohammad Miadh Angkad 6c018eb4d1 Fix bounded checkpoint prefetching and buffered drop-cache handling (#29156) 2026-06-29 21:49:16 +00:00
Mohammad Miadh Angkad ae09b8302f [CI] Fix GB300 DSV4 Pro FP4 nightly (#29502) 2026-06-27 21:06:17 -07:00
Mohammad Miadh Angkad 212c30d008 [MoE Refactor] Centralize FlashInfer CUTLASS MoE runner (#28211) 2026-06-25 13:40:33 -07:00
Mohammad Miadh Angkad e4976683f4 [Docs] Fix broken links in cookbook (#29261) 2026-06-25 02:31:55 -07:00