minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
18,691 Commits 2 Branches 0 Tags
dsv41-pd
Commit Graph
77 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
DarkSharpnessandBBuf d1acbe0746 [DSV4.1] Big fused wo_a quant (#39957) 
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-19 19:46:53 +08:00
DarkSharpnessClaude Opus 5BBuf
 		81363bf8cb [kernel] Share the warp vectorized copy and enforce its alignment (#36176) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-18 22:40:54 +08:00
DarkSharpnessandClaude Opus 5 73a2414289 [CI] Double the base-b-test-1-gpu-large timeout (#37452) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-02 08:47:51 +08:00
DarkSharpnessandClaude Opus 5 cb6dd58fbe [Kernel] Replace dsv3_router_gemm with the unified tiny GEMM (#34693) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-01 23:00:21 +08:00
DarkSharpnessandClaude Opus 5 b6c06e1efb [DSA] Drop the redundant 512 from the top-k transform entry-point names (#36831) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-01 22:37:03 +08:00
DarkSharpness 689ade69d1 [kernel] Split the custom all-reduce communicator into push/pull planes (#35735) 2026-08-26 17:40:28 +08:00
DarkSharpnessandClaude Opus 5 7fd5454335 [DSA] Route the ragged prefill top-k to the v2 kernel (#35175) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-21 16:59:41 -07:00
DarkSharpnessandClaude Opus 5 746418a1ec [DSA] Trim top-k v2 output modes and tighten its PDL waits (#35041) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-19 13:09:21 -07:00
DarkSharpnessClaudeBBuf
 		b784726863 [kernel] Content-addressed JIT build cache, generated from our own ninja (#34274) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2026-08-14 13:41:04 +08:00
DarkSharpnessYeu JaungClaude Opus 5
 		7331287c1c [JIT Kernel][DSv4] Optimize epilogue of c128 (#26671) 
Co-authored-by: Yeu Jaung <jpppppppppppppppppppppppp@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-10 10:32:43 +08:00
DarkSharpnessandClaude Opus 5 accc51c6db [DSA] Fix top-k v2 dropping non-primary ranks' output on CUDA 13.1+ (root cause for #33835) (#34167) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-10 10:31:47 +08:00
DarkSharpnessandClaude Fable 5 4ad5bb5d9a [jit_kernel] Move JIT kernels into namespace sglang (#33400) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-08 16:10:15 +08:00
DarkSharpnessClaude Fable 5Xiaoyu Zhang
 		1307968605 [JIT] Drop redundant per-kernel arch overrides (#32952) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-08-04 08:33:54 +08:00
DarkSharpnessandClaude Fable 5 3abbc565e4 [Docs] Add a Conventions section to the add-jit-kernel skill (#32956) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-31 09:11:49 +08:00
DarkSharpnessandClaude Fable 5 9402012f0f [Perf] Halve the non-finite sanitization overhead in per_token_group_quant (#32296) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-25 08:17:12 +08:00
DarkSharpnessandClaude Fable 5 8bb0d8d005 [JIT] Trait-driven per_token_group_quant: unify the quant kernel family (flat + masked) (#30924) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-22 08:46:52 +08:00
DarkSharpnessClaude Fable 5BBufjessiewei7root
 		67e7f8d13a [JIT] Refactor dtype traits into DTypeTrait and unify warp reductions (#30838) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: BBuf <xiaoyu.zhang@radixark.ai>
Co-authored-by: jessiewei7 <jessiewei747@gmail.com>
Co-authored-by: root <root@GPUC5A6.maas>
 2026-07-18 10:07:18 +08:00
DarkSharpnessClaude Fable 5root
 		132ade55cd [Kernel] Rewrite JIT custom all-reduce (v2) with a decoupled kernel/storage design (#31049) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: root <root@GPUC5A6.maas>
 2026-07-17 18:37:22 +08:00
DarkSharpnessandClaude Fable 5 bda1dc0d95 [DSA] Fix top-k v2 emitting invalid indices under tie overflow / inf scores (IMA in FA3 sparse decode) (#30645) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-09 14:28:21 -07:00
DarkSharpnessziyi.xuClaude Fable 5
 		bbc537035a [DSA] Re-enable fused top-k v2 for MTP: clamp padded-row seq_lens to >= 0 (#30378) 
Co-authored-by: ziyi.xu <ziyi.xu@radixark.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-07 13:44:01 -07:00
DarkSharpness be70bfbdbb [DSA] Fold page-table into fused top-k v2 (decode): drop page_size=1 expansion (#30274) 2026-07-06 21:28:05 -07:00
DarkSharpnessandClaude c016c6f355 [JIT Kernel] DeepSeek-V4 DSA indexer: faster top-k + page-table transform (runtime k <= 2048) (#26788) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-07-05 20:23:39 -07:00
DarkSharpnessClaudegemini-code-assist[bot]ziyi.xuXiaoyu Zhang
 		cf36dca6d4 [JIT Kernel] Triton moe fused gate (#25835) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: ziyi.xu <ziyi.xu@radixark.ai>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-06-30 23:26:18 +08:00
DarkSharpnessClaudeziyi.xu
 		d72314808f [JIT Kernel] Multi-GPU test/bench framework for custom all-reduce + TP QKNorm (#26706) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: ziyi.xu <ziyi.xu@radixark.ai>
 2026-06-14 17:20:36 +08:00
DarkSharpnessandClaude cc72e2bd8c [Docs] Fix outdated benchmark marker API in add-jit-kernel skill (#27773) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-13 23:47:30 -07:00
DarkSharpnessandClaude b40f365732 [CI] Move misplaced mhc kernel test into test/registered/kernels (#27781) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-06-10 01:49:18 -07:00
DarkSharpnessandClaude Opus 4.7 8f21b3e2ef [Refactor] JIT kernel benchmark (#25274) 
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-28 00:49:55 -07:00
DarkSharpnessandClaude 19f55c0e6d [Refactor] major JIT kernel clean up for dsv4 (#25884) 
Co-authored-by: Claude <noreply@anthropic.com>
 2026-05-21 01:14:31 -07:00
DarkSharpnessCheng WanCheng WanClaude Opus 4.7
 		82254bd9c5 [JIT Kernel] Reland JIT activation (#22094) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Cheng Wan <chwan@rice.edu>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-24 23:00:28 -07:00
DarkSharpnessandMingyang Jiang 314d6ecf08 [Feature][JIT Kernel] Fused TP QK norm For Minimax (#20673) 
Co-authored-by: Mingyang Jiang <13463932+jmydurant@users.noreply.github.com>
 2026-04-13 20:29:47 +08:00
DarkSharpnessandweiminc 44e5d35703 [Feature][JIT Kernel] JIT activation and update skills (by codex) (#21766) 
Co-authored-by: weiminc <tnwilly@gmail.com>
 2026-04-03 23:28:54 +08:00
DarkSharpness d1b7c3907d [Parallel State Refactor 2/n] Unify code path of AMD deterministic all reduce (#20871) 2026-04-03 12:33:17 +08:00
DarkSharpness df94cdcebb [Parallel State Refactor 1/n] Remove stream of PyNCCL (#20866) 2026-04-03 00:47:50 +08:00
DarkSharpness 20f4193589 [Feature] JIT rmsnorm update (with claude) (#21834) 2026-04-01 23:40:00 +08:00
DarkSharpness 4e480982fa [misc] multiprocess compilation to speed up test (#21483) 2026-03-31 08:56:37 +08:00
DarkSharpness bb29893689 [Fix] Try to fix nvcc compilation error (#21246) 2026-03-26 10:59:36 +08:00
DarkSharpness 3d2a61cbf6 [Chore] Clean up JIT compilation flags (#21022) 2026-03-25 18:08:40 +08:00
DarkSharpness dfc15b78b0 [misc] clean up kernel API (#21325) 2026-03-25 09:10:23 +08:00
DarkSharpnessandXiaoyu Zhang 2dd9196079 [JIT Kernel][Feature] Support JIT custom all reduce (rewrite as v2) (#19880) 
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2026-03-20 18:24:07 +08:00
DarkSharpnessandgemini-code-assist[bot] d8d0208c63 [Feature] rewrite rope kernel; remove flashinfer dependencies (#18844) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-21 21:32:40 +08:00
DarkSharpness 52852404c8 [Fix] DO NOT skip save_kv_cache for dllm (#19020) 2026-02-20 23:20:29 +08:00
DarkSharpness 9d138685c1 [Refactor] Fix test and clean up hicache code (#18555) 2026-02-18 14:37:46 +08:00
DarkSharpness 8e2e835c2f [Fix] Fix backend selection after flashinfer version update (#18364) 2026-02-08 11:20:41 +08:00
DarkSharpness 95f59c13fd [Chore] include all jit files in building packages (#17493) 2026-01-21 14:48:02 -08:00
DarkSharpnessgemini-code-assist[bot]Xiaoyu Zhang
 		ba9f6d8f26 [Refactor] Clean up JIT kernel utilites (#16884) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
 2026-01-13 17:54:16 +08:00
DarkSharpness d112f6a25b [Feature] Support JIT set kv cache (#16273) 2026-01-10 17:34:09 -08:00
DarkSharpness da2f8cc33f [minor] reduce 1 unnecessary add (#16474) 2026-01-05 01:00:56 -08:00
DarkSharpness f6f7af4068 [Refactor] Clean up custom op (#15995) 2026-01-01 21:41:56 +08:00
DarkSharpness a3b1e8ef3d [Feature] add aligned_vector type for JIT kernel (#16162) 2026-01-01 21:40:05 +08:00
DarkSharpness 45f3ad2f52 [Refactor] Rename CustomOp -> MultiPlatformOp (#16175) 2025-12-31 01:16:32 +08:00
DarkSharpness 8e43980ebb [Feature] JIT Fused QK norm + qk norm clean up (#15835) 2025-12-28 11:53:50 +08:00
DarkSharpness 291f11ae39 [Minor] Enhance JIT kernel and add dev docs (#14570) 2025-12-23 22:34:59 +08:00
DarkSharpness f03bfa4ce3 [Feature] Fuse mrope all in 1 kernel (#14906) 2025-12-15 02:50:55 -08:00
DarkSharpness c1dd9a9599 [Fix] JIT kernel dependencies in other platforms (#13889) 2025-11-24 23:19:17 -08:00
DarkSharpness ac5505b04c [Feature] HiCache JIT kernel (once again) (#13764) 2025-11-22 22:19:16 -08:00
DarkSharpness b51f9bbee7 [Feature] Introduce JIT Kernel in sglang (with hicache JIT kernel) (#13453) 2025-11-20 00:03:32 -08:00
DarkSharpness f969664172 [Performance] Move the contiguous to torch compile region (#13199) 2025-11-15 20:49:52 -08:00
DarkSharpness e8b71445c0 [Misc] Improve the error message of failed import (#12119) 2025-10-25 12:09:05 -07:00
DarkSharpness 276e7b3e4e [Feature] New structural tag support (#10691) 2025-10-20 18:25:58 +08:00
DarkSharpness e28c9e526f [Minor] Update xgrammar dependency (#11622) 2025-10-14 13:46:50 -07:00
DarkSharpness 832c84fba9 [Chore] Update xgrammar 0.1.24 -> 0.1.25 (#10710) 2025-10-07 18:22:28 -07:00
DarkSharpnessandBaizhou Zhang e0b2d3eebe [Feature] Add a fast-topk to sgl-kernel for DeepSeek v3.2 (#11194) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2025-10-05 10:19:03 -07:00
DarkSharpnessandZhiqiang Xie 948b01a04c [Refactor] Remove Hicache Load & Write threads (#10127) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-09-08 22:18:50 -07:00
DarkSharpness 067246830d [Minor] fix lint in main (#10128) 2025-09-07 17:36:46 +08:00
DarkSharpness 86a0be65d8 [Feature] Support custom set kv buffer kernel (#8884) 2025-08-12 16:56:51 -07:00
DarkSharpness b4ac2b9c0c [Fix] Fix dual chunk model default behavior (#9032) 2025-08-11 23:50:23 -07:00
DarkSharpness 7ba5ad5766 [Fix] Fix flashinfer cpu <-> gpu synchronization (#8340) 2025-08-10 03:11:40 +00:00
DarkSharpness 19bc77f05c [Fix] Fix hicache backend (#8991) 2025-08-09 17:16:25 -07:00
DarkSharpness fc42ff7b63 [Fix] Fix wrong backend chosen in hybrid backend (#8989) 2025-08-08 21:21:17 -07:00
DarkSharpness e273aa6dcf [Feature] Radix Tree in C++ (#7369) 2025-08-02 19:50:14 -07:00
DarkSharpnessandZhiqiang Xie 47367b768d [Refactor] Clean up radix cache related API (#7303) 
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
 2025-06-20 00:58:48 +08:00
DarkSharpness ac3fae8445 [Feature] Support "strict" in function calling (#4310) 2025-03-24 22:15:25 -07:00
DarkSharpness 19120f71f3 [Fix & Style] Refactor the grammar backend to reduce human errors and improve readability (#4030) 2025-03-04 03:56:45 -08:00
DarkSharpness 9c745d078e [Performance] Update xgrammar-related constrained decoding (#2056) 2024-11-17 16:58:49 -08:00
DarkSharpness 954f4e6bd6 benchmark json schema (#2030) 2024-11-15 05:06:19 -08:00
DarkSharpnessandLianmin Zheng 125b1199c5 support parallel grammar preprocessing (#1996) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2024-11-12 08:45:28 -08:00
DarkSharpness b77a02cdfd [Performance] Support both xgrammar and outlines for constrained decoding (#1752) 2024-10-25 21:47:02 +00:00