minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
18,691 Commits 2 Branches 0 Tags
dsv41-pd
Commit Graph
18691 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
ashwini rathi 292e3ccc0c [ci][xpu] Re-seed the wan2_1_t2v_1.3b perf baseline on Arc Pro B60 (#39955) 2026-09-21 13:04:35 +08:00
minke.yu c74a4037fb mooncake: probe pointers via cuda.bindings instead of ctypes libcudart 
The ctypes-based probing loaded a second libcudart copy into the process,
which corrupted CUDA/runtime state and segfaulted the scheduler processes
shortly after registration (reproducible on prefill ranks, intermittent on
decode). cuda.bindings.runtime.cudaPointerGetAttributes is a properly
typed binding and boots cleanly on both roles.
 2026-09-21 12:46:25 +08:00
minke.yu b963295489 mooncake: support host-memory payloads with intra-node NVLink transport 
IntraNodeNvlinkTransport can only register/reach device memory. sglang's
register list includes host regions (aux buffers, some state components);
one host region failed the whole registerLocalMemoryBatch and the engine
rolled back every region, leaving the segment descriptor empty so all KV
transfers failed with 'Requested address ... not found'.

When the intra-node NVLink transport is active (MC_INTRANODE_NVLINK etc.):
- register only device-memory regions (probe via cudaPointerGetAttributes)
- route transfer blocks whose local source is host memory over the ordered
  per-endpoint zmq channel (new STATE_DATA message) instead of the engine;
  the receiver validates and writes the block into its buffer, relying on
  the same ordering guarantee as the aux TCP path
- force aux over the TCP socket path automatically
 2026-09-21 11:53:26 +08:00
ZiruandNiu Ziru 1da8ac10e1 Update to the cookbook for XPU-supported models (#33649) 
Co-authored-by: Niu Ziru <niuziru@a4bf018d3341.jf.intel.com>
 2026-09-20 20:26:09 -07:00
jianzhao-xu 62ba964848 Fix: post-load staging regression breaks offload meta/sharded_gpu modes (#38779) 2026-09-21 11:19:07 +08:00
Xueshen Liu ab03a8e7eb [Perf] Fork-safe import: no CUDA context at import time, lighter argument parsing (#40201) 2026-09-21 10:48:35 +08:00
zhaozx-cn 176dbcb85d [npu]add chunk gdn kernel and unify ssm state layout for ascend gdn backend (#36187) 2026-09-21 09:51:19 +08:00
MickandMick Qian 6ad78f2281 [diffusion] docs: add verified DGX Spark recipe for Qwen-Image 2.1 (#40487) 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
 2026-09-21 08:49:35 +08:00
WenhaoZhang b912db67ea [diffusion] fix: keep Qwen-Image 2.1 prefix KV per layer under Cache-DiT (#40472) 2026-09-21 08:48:31 +08:00
MickandMick Qian 501b7851e4 [diffusion] CI: guard E2E/loading latency with runner-aware baselines (#39206) 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
 2026-09-21 08:43:22 +08:00
MickandMick Qian 3a0324fb9b [diffusion] optimization: reduce Qwen-Image 2.1 vae and graph warmup memory (#40481) 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
 2026-09-21 08:42:02 +08:00
Liangsheng Yin 76a9065bef [Fix] Raise on undelivered embeddings in send_with_url, fix broken tests (#40502) 2026-09-20 17:35:27 -07:00
Kan WuandClaude Fable 5.1 4027740569 [sgl-router] refactor - session-aware policy (#40379) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 17:22:36 -07:00
Kan WuandClaude Fable 5.1 aedda8377e [sgl-router] refactor - layout BucketResolver, Bucket, EngineGroup and implement PowerOfTwo (#40241) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 16:57:07 -07:00
Kan WuandClaude Fable 5.1 4a9dc5c4af [sgl-router] refactor - move policy-required states under src/state (#40272) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 16:49:23 -07:00
Liangsheng Yin acd20a516e [CI] Give the kernel lane a 5090 suite and move kernel-only tests off the general lane (#40496) 2026-09-20 16:46:53 -07:00
DivyClaude Opus 5Po-Han Huang
 		42875bcd2a fix(modelopt): dispatch NVFP4 MoE on the cached backend, not the live global (#38932) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
 2026-09-20 19:28:54 -04:00
Mohammad Miadh Angkadandmmangkad 2fa6b94e34 [Perf] Fuse the glm5_next mHC attn->MLP boundary (#39200) 
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
 2026-09-20 16:20:37 -07:00
ollybbmonster 983e643854 [Feature] support bf16 MoE router and mxfp4 MoE for MiMo V2 (#40448) 2026-09-20 16:13:59 -07:00
Liangsheng Yin 2e2d8a2fda [CI] Drive per-commit stage jobs from a runner table instead of copied job blocks (#40495) 2026-09-20 15:43:46 -07:00
luoroger37andHank Han d97aed2c90 Fix TopK v2 fallback when 16-block cluster capacity is zero (#40163) 
Co-authored-by: Hank Han <hanhan7630@outlook.com>
 2026-09-21 06:41:41 +08:00
Yuxuan ZhangXinyuan Tonghnyls2002
 		f31a7bd45c Use pinned memory for asynchronous sampling metadata transfers (#39777) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-09-20 15:31:31 -07:00
Harmya Bhatt 95521da18d [DeepSeek-V4.1] Bound dense prefill indexer memory (#40217) 2026-09-20 14:43:19 -07:00
JinYan SuClaude Fable 5.1Liangsheng Yinhnyls2002
 		c2c3629f2d [Kimi-K3] O(1) expert weight lookup in load_weights (#38805) 
Signed-off-by: JinYan Su <751080330@qq.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-09-20 14:39:45 -07:00
Liangsheng Yin f6483e479f [Test] Drop cause-less disabled tests, fix XPU lane, demote quality gates off base-c (#40288) 2026-09-20 14:36:02 -07:00
Aurick QiaoOasis-GitKe Bao
 		d229952e25 [Fix] Preserve model runner contracts in prefill CUDA graphs (#35452) 
Co-authored-by: Oasis-Git <ayw.sirius19@gmail.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2026-09-20 13:53:14 -07:00
Shangming Cai 745de73ba3 Add CODEOWNERS entry for sglang-renderer (#40483) 2026-09-21 01:37:50 +08:00
Khoa Pham b3e4d198af [PD] Bound cached-prefix DCP transfers by pack capacity (#40376) 2026-09-21 01:17:15 +08:00
Shuwen Wang 80da4432d0 [Simulator] Fix meta host memory budgets on constrained runners (#40440) 2026-09-21 00:42:55 +08:00
WenhaoZhang 3dbdd700e2 [CI] update CI permissions (#40474) 2026-09-21 00:07:37 +08:00
Jimmy ShongandYangmin Li e97614d10c [Qwen4-Exp] Build the offloaded PLE table on the meta device so --ple-offload-embedding never materialises it on the accelerator (#39928) 
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
 2026-09-20 08:28:25 -07:00
ZY Y 5f017ffabb Update test cases and performance testing framework (#40392) 2026-09-20 22:42:00 +08:00
chenyang08056032 404dee10c0 [NPU] add coverage-based precision test selection pipeline (#38339) 2026-09-20 22:40:55 +08:00
Mohammad Miadh AngkadandMohammad Angkad 8923f4d779 [Test] Fix optimistic prefill disaggregation test after mamba radix cache removal (#40469) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-09-20 22:39:34 +08:00
abing fa826e08b1 Reuse shared compressed KV dequantization in DeepSeek V4.1 CP prefill 
(cherry picked from commit 44eb378e94610e4ccc6b990bf09c105f8c8f9ee9)
 2026-09-20 22:12:47 +08:00
SYChen123 3810f531a8 Support V4.1 decode vision MegaMoE 
(cherry picked from commit 98aa79972d6d31f30a3c5bc83b9d7e3009ae638a)
 2026-09-20 22:12:43 +08:00
SYChen123 2580c24d1b Support V4.1 DP attention in DP-only DSpark PD 
(cherry picked from commit 6ef0ebba730b3fffde4b59b953c5c9b58a35c4ed)
 2026-09-20 22:11:48 +08:00
faceless voidandronnie_zheng 791c7850d0 [Diffusion] Enable shared RMSNorm dispatch for SenseNova-U1 (#39705) 
Signed-off-by: syd520zy <529477025@qq.com>
Co-authored-by: ronnie_zheng <zl19940307@163.com>
 2026-09-20 17:08:02 +03:00
abing 4f22146e51 Avoid host synchronization in DeepSeek V4.1 CP prefill 
(cherry picked from commit 510596019fd5782d494c9b5227bf784d46661f6f)
 2026-09-20 22:07:30 +08:00
abing 12e3b82e52 run pass cp+megamoe+bcg 
(cherry picked from commit 85b8a150ef69c35cbd8c3fe940e41b414b903757)
 2026-09-20 22:07:29 +08:00
abing 8305f66fc8 fix bug 
(cherry picked from commit 60d31cb0c268c4b733ab290e133e3763b65f090e)
 2026-09-20 22:07:24 +08:00
abing 21a4a16b4b update code 
(cherry picked from commit 1453150a0bdf2cce425e9e2da83ef16b8e8f2859)
 2026-09-20 22:07:24 +08:00
abing fc954b7e08 add test 
(cherry picked from commit 094abd5e90af77507acd7c91513f69b5b29b685b)
 2026-09-20 22:07:24 +08:00
abing c2059c4fb2 run pass llm cp 
(cherry picked from commit 1d85394563d96cf22d5d84f15c0e9043582419e4)
 2026-09-20 22:07:23 +08:00
SageShangming CaiLiangsheng YinRain Jiang
 		7b1c2ed0a4 [rust-renderer] Standalone preprocessing (#36718) 
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Rain Jiang <96632942+rainj-me@users.noreply.github.com>
 2026-09-20 22:03:12 +08:00
MickandMick Qian 6880a47955 [diffusion] docs: simplify Qwen-Image 2.1 cookbook (#40455) 
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
 2026-09-20 20:39:26 +08:00
Kan Wu c610c40399 [sgl-router] refactor - config and organize CLI options (#39867) 2026-09-20 19:40:04 +08:00
Ruiyan Ma efa7be2091 [Simulator][Compatibility] Adapt to latest KV cache pool interfaces (#40418) 2026-09-20 18:00:06 +08:00
Liangsheng Yin 0024efa0de [CI] Derive registered-test kind from the registry call instead of the path (#40294) 2026-09-20 02:01:22 -07:00
Kan WuandCursor 671630abf1 [sgl-router] refactor - main startup logic (#39861) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-20 16:55:34 +08:00