Xinyuan Tong
6833498646
model: prune comments and redundant tests in dsv41 vision CP
2026-09-23 14:36:01 +08:00
Xinyuan Tong
b48e2cb1eb
model: TP-wide single-owner image encoding for DeepSeek V4.1
...
ViT and Aligner are replicated per TP rank, encoding each image eight
times with TP8 on both CP1 and CP8. Elect one owner per image and use
ordered full-span broadcasts with a six-phase agreement protocol.
Both CP1 and CP8 benefit while local cache hits preserve collective order.
2026-09-23 14:36:01 +08:00
Xinyuan Tong
bfeb7cd9b2
model: support DeepSeek V4.1 vision with interleave prefill CP
...
The CP runner bypassed the vision merge and used bare text embeddings.
Merge image features before sharding so request-global offsets stay valid.
Canonicalize model IDs separately to preserve scheduler hash IDs.
Keep unsupported combinations guarded and isolate embedding overrides
from multimodal prefills without starving queued FCFS requests.
2026-09-23 14:36:00 +08:00
Yuwei An
ddf5207630
[Fix] Handle chunked paged MQA metadata in DSV4.1 eager forwards ( #40637 )
build-sglang-image / build (push) Successful in 32m15s
2026-09-23 13:35:55 +08:00
minke.yu
b081dd3d23
Merge branch 'main' into dsv41-pd
2026-09-22 14:56:35 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
4c81cd1b09
[KDA] Fix missing beta sigmoid in PTX prefill ( #40685 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-21 23:24:38 -07:00
Guangda Liu and Guangda Liu
04c0913434
[HiSparse] Add MHA hisparse support for MiniMax M3 ( #31446 )
...
Co-authored-by: Guangda Liu <bingps@users.noreply.github.com >
2026-09-22 13:28:03 +08:00
095e45100b
[AMD] [GLM-5.3-Flash Day 0] Route mHC through AITER on gfx950 ( #38545 )
...
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-21 22:24:29 -07:00
YAMY
9b59fc5db5
[ModelOpt][PP] Keep BF16 shared experts out of the NVFP4 fusion so TP1 pipeline stages can load ( #40628 )
2026-09-21 21:45:58 -07:00
e1daf68304
[AMD] [GLM-5.3-Flash Day 0] Honor fused and per-expert names in quark exclude ( #39317 )
...
Co-authored-by: Yikai Zhang <ykzhang12@gmail.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-21 21:26:42 -07:00
Khoa Pham and Qiaolin Yu
018b73c7a0
[PD] Pack draft KV head slices for DCP transfers ( #40500 )
...
Co-authored-by: Qiaolin Yu <liin1211@outlook.com >
2026-09-21 21:11:27 -07:00
b44e248682
[AMD] [GLM-5.3-Flash Day 0] Enable FP8 and Quark MXFP4 MoE on gfx950 ( #38546 )
...
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: andyluo7 <andy.luo@amd.com >
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-21 21:07:37 -07:00
jthomson04
15ba54bd5d
perf(engine): avoid timed waits for Engine responses ( #39486 )
...
Signed-off-by: jthomson04 <jwillthomson19@gmail.com >
2026-09-21 20:42:03 -07:00
Cheng Wan
1d025491f3
[Test] Set DP size in the mocked Metal profiler test ( #40667 )
2026-09-21 19:57:52 -07:00
90cf471723
[AMD] [GLM-5.3-Flash Day 0] Support non-2048 top-k widths in the DSA page-table transform ( #39340 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-21 19:50:17 -07:00
jacky.cheng
bc30fa1759
[AMD][Fix] AgentX HIP TPOT regression when SGLANG_SIMULATE_ACC_LEN is set ( #40598 )
2026-09-21 19:16:15 -07:00
Cheng Wan
9eda772a21
[Test] Handle tied top-k indices in graph-pool logprob regression ( #40661 )
2026-09-21 19:02:43 -07:00
Mohammad Miadh Angkad and mmangkad
e332e1b84e
[Fix] Don't write conv state from the fused KDA verify kernel ( #39524 )
...
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai >
2026-09-21 18:33:03 -07:00
Dayananda V and Claude Opus 5
35eb7cf8d6
[Intel][XPU][KVCanary] Enable KV Canary on Intel XPU ( #33520 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-22 09:19:01 +08:00
Ma Mingfei
046cd6f4ea
[XPU][ci]: disable XPU NIXL disaggregation test ( #40540 )
2026-09-22 09:02:22 +08:00
Khoa Pham
c4d3770a68
[Kimi K3] Fix CUDA graph stream explosion ( #40640 )
2026-09-21 17:49:50 -07:00
Cheng Wan
98c8dee23b
Fix lint failure from draft-decode window test location ( #40654 )
2026-09-21 17:41:00 -07:00
042b6a488f
[AMD] [GLM-5.3-Flash Day 0] Enable zero-RoPE MHA prefill on ROCm ( #39338 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
2026-09-21 17:38:07 -07:00
avalliappan-nvidia
61d0cf2074
[Spec] Windowed draft-decode attention for built-in EAGLE / MTP drafts ( #32673 )
2026-09-22 08:17:57 +08:00
Vedant V Jhaveri and Copilot
9fdb71732a
Avoid materializing GDN QKV tensors during target verification ( #33778 )
...
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-09-21 17:04:18 -07:00
Cheng Wan
506698761d
[unified-memory] Hierarchical cache for every unified pool shape ( #37507 )
2026-09-21 16:50:37 -07:00
Cheng Wan
22587fb15c
[Fix] Run KV canary hooks for context-parallel prefill ( #40642 )
2026-09-21 16:46:22 -07:00
Yuxuan Zhang and Xinyuan Tong
00986c81be
Support GLM-5.3-Flash hybrid attention CPU offload and PD index mapping ( #40310 )
...
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
2026-09-21 16:03:03 -07:00
YAMY
0229025127
[Spec][PP] Launch extend microbatches before the spec output exchange ( #40499 )
2026-09-21 15:47:03 -07:00
Liangsheng Yin
a5c2cc517c
[CI] Split the CI control labels into four axes and resolve them live ( #40527 )
2026-09-21 15:37:28 -07:00
8bde82c0ad
[AMD] [GLM-5.3-Flash Day 0] Build the fused DSA k-pool top-k JIT kernel on HIP ( #39339 )
...
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-21 15:24:44 -07:00
Cheng Wan
acac4dd9d9
[Refactor] Clean up parallel runtime comments ( #40632 )
2026-09-21 14:32:22 -07:00
jacky.cheng
e0c2e8dc4d
[AMD] Tune Qwen3.5 TP4 GDN recurrent launch on gfx950 ( #39987 )
2026-09-21 13:20:54 -07:00
Liangsheng Yin
1ed6822039
[Test] Anchor basic_perf thresholds to each metric's measured spread ( #40617 )
2026-09-21 13:02:10 -07:00
11e661fd45
[Fix] Don't free the multi-CTAs KV counter the decode graphs captured ( #39175 )
...
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai >
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-21 12:58:48 -07:00
Cheng Wan
44bdf225d8
Fix lint failure from MXFP8 reserved-slot test location ( #40618 )
2026-09-21 12:31:11 -07:00
Cheng Wan
bccf691b22
Bringing the parallel runtime up becomes a phase, not a side effect ( #40345 )
2026-09-21 12:29:50 -07:00
Cheng Wan
1d3243d05f
Take the parallel getters off the package's public surface ( #40344 )
2026-09-21 12:27:50 -07:00
Cheng Wan
970e946e4f
Retire the per-runner parallel record ( #40343 )
2026-09-21 12:26:40 -07:00
Cheng Wan
73f071db52
Deprecate the parallel getters the context answers, and ratchet them shut ( #40342 )
2026-09-21 12:25:32 -07:00
Cheng Wan
65be3fa71a
A runner and the objects it builds freeze the placement they describe ( #40341 )
2026-09-21 12:24:17 -07:00
Cheng Wan
2d0e94e3a3
Check the topology identities where the layout is written, and build at the published widths ( #40340 )
2026-09-21 12:22:59 -07:00
Cheng Wan
0db1a93adb
State the draft's whole topology in its scope, and read the rest from the context ( #40339 )
2026-09-21 12:19:38 -07:00
ae7a516ba7
feat: use XGrammar V4.1 DSML parameter constraints ( #39026 )
...
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
2026-09-21 12:12:28 -07:00
jacky.cheng
90b3f8544c
[AMD] Use Triton softmax routing for Qwen3.5 on gfx950 ( #39986 )
2026-09-21 12:02:29 -07:00
cctry
7a6191c4b9
Preallocate HiCache MHA staging before post-capture KV sizing ( #40256 )
2026-09-21 10:44:29 -07:00
cctry
7ad55e4386
[HiCache] TMA-staged host<->device KV transfer kernel (sm_90+) ( #40278 )
2026-09-21 10:38:23 -07:00
Eric.Chin.AMD and Thomas Wang
3c71bb018a
[AMD] Enable GLM DSA prefill top-k to the v2 kernel ( #37889 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
2026-09-21 10:31:07 -07:00
5a6a1bb883
[mxfp8-kv] Skip writes to the reserved CUDA-graph padding slot ( #35351 )
...
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: Ke Bao <ispobaoke@gmail.com >
Co-authored-by: Sam Shleifer <sam@thinkingmachines.ai >
2026-09-22 01:21:25 +08:00
Liangsheng Yin
800613a74b
[Test] Split the serving perf tests by topic into basic_perf/ and route their thresholds through a kit ( #40505 )
2026-09-21 10:05:57 -07:00
Xinyuan Tong
minke.yu