Khoa Pham
c4d3770a68
[Kimi K3] Fix CUDA graph stream explosion ( #40640 )
2026-09-21 17:49:50 -07:00
jacky.cheng
31b577bb08
[AMD] Pad QSA MQA decode Q-heads to 16 for ROCm MFMA ( #38875 )
2026-09-21 17:43:46 -07:00
042b6a488f
[AMD] [GLM-5.3-Flash Day 0] Enable zero-RoPE MHA prefill on ROCm ( #39338 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
2026-09-21 17:38:07 -07:00
Cheng Wan
582389cec5
[Fix] Keep diffusion encoder TP context bindings consistent ( #40646 )
2026-09-21 17:23:15 -07:00
Tao Li and Xiaoyu Zhang
c53cc8e1eb
[NPU][BugFix] Avoid M-RoPE recompilation for variable sequence lengths ( #40371 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-09-22 08:19:25 +08:00
avalliappan-nvidia
61d0cf2074
[Spec] Windowed draft-decode attention for built-in EAGLE / MTP drafts ( #32673 )
2026-09-22 08:17:57 +08:00
Vedant V Jhaveri and Copilot
9fdb71732a
Avoid materializing GDN QKV tensors during target verification ( #33778 )
...
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-09-21 17:04:18 -07:00
Cheng Wan
506698761d
[unified-memory] Hierarchical cache for every unified pool shape ( #37507 )
2026-09-21 16:50:37 -07:00
Cheng Wan
22587fb15c
[Fix] Run KV canary hooks for context-parallel prefill ( #40642 )
2026-09-21 16:46:22 -07:00
Yuxuan Zhang and Xinyuan Tong
00986c81be
Support GLM-5.3-Flash hybrid attention CPU offload and PD index mapping ( #40310 )
...
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
2026-09-21 16:03:03 -07:00
YAMY
0229025127
[Spec][PP] Launch extend microbatches before the spec output exchange ( #40499 )
2026-09-21 15:47:03 -07:00
RuibinCheung
0c53fec476
[ROCm] fix: remove extra bf16 -> fp32 cast in jit grouped topk kernel path ( #39775 )
2026-09-21 15:43:21 -07:00
8bde82c0ad
[AMD] [GLM-5.3-Flash Day 0] Build the fused DSA k-pool top-k JIT kernel on HIP ( #39339 )
...
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-09-21 15:24:44 -07:00
Cheng Wan
acac4dd9d9
[Refactor] Clean up parallel runtime comments ( #40632 )
2026-09-21 14:32:22 -07:00
William Hu
f532ad1f9a
Fix GLM-5.3 forget-gate shape for nvCUTEDSL verify ( #40607 )
2026-09-21 13:52:26 -07:00
jacky.cheng
e0c2e8dc4d
[AMD] Tune Qwen3.5 TP4 GDN recurrent launch on gfx950 ( #39987 )
2026-09-21 13:20:54 -07:00
Liangsheng Yin
b18ca9ca44
[CI] Bump sgl-eval to 0.1.2 ( #40620 )
2026-09-21 13:00:54 -07:00
11e661fd45
[Fix] Don't free the multi-CTAs KV counter the decode graphs captured ( #39175 )
...
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai >
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-21 12:58:48 -07:00
Cheng Wan
bccf691b22
Bringing the parallel runtime up becomes a phase, not a side effect ( #40345 )
2026-09-21 12:29:50 -07:00
Cheng Wan
1d3243d05f
Take the parallel getters off the package's public surface ( #40344 )
2026-09-21 12:27:50 -07:00
Cheng Wan
970e946e4f
Retire the per-runner parallel record ( #40343 )
2026-09-21 12:26:40 -07:00
Cheng Wan
73f071db52
Deprecate the parallel getters the context answers, and ratchet them shut ( #40342 )
2026-09-21 12:25:32 -07:00
Cheng Wan
65be3fa71a
A runner and the objects it builds freeze the placement they describe ( #40341 )
2026-09-21 12:24:17 -07:00
Cheng Wan
2d0e94e3a3
Check the topology identities where the layout is written, and build at the published widths ( #40340 )
2026-09-21 12:22:59 -07:00
Cheng Wan
0db1a93adb
State the draft's whole topology in its scope, and read the rest from the context ( #40339 )
2026-09-21 12:19:38 -07:00
ae7a516ba7
feat: use XGrammar V4.1 DSML parameter constraints ( #39026 )
...
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
2026-09-21 12:12:28 -07:00
jacky.cheng
90b3f8544c
[AMD] Use Triton softmax routing for Qwen3.5 on gfx950 ( #39986 )
2026-09-21 12:02:29 -07:00
ronnie_zheng
f702a0be29
Revert "[Diffusion] migrate the whole _register_configs from registry.py to the model own config file" ( #40611 )
2026-09-21 21:05:19 +03:00
Bingxu Chen
632919e498
[AMD] Fix DeepSeek-R1-MXFP4 accuracy with AITER FP8 ( #37762 )
2026-09-21 10:51:37 -07:00
ronnie_zheng
e6931ca889
[Diffusion] migrate the whole _register_configs from registry.py to the model own config file ( #40475 )
2026-09-21 20:45:53 +03:00
cctry
7a6191c4b9
Preallocate HiCache MHA staging before post-capture KV sizing ( #40256 )
2026-09-21 10:44:29 -07:00
cctry
7ad55e4386
[HiCache] TMA-staged host<->device KV transfer kernel (sm_90+) ( #40278 )
2026-09-21 10:38:23 -07:00
William Hu
0cb37c018c
[KDA] Enable ReplaySSM for GLM-5.3 Flash ( #40517 )
2026-09-21 10:34:08 -07:00
Eric.Chin.AMD and Thomas Wang
3c71bb018a
[AMD] Enable GLM DSA prefill top-k to the v2 kernel ( #37889 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
2026-09-21 10:31:07 -07:00
5a6a1bb883
[mxfp8-kv] Skip writes to the reserved CUDA-graph padding slot ( #35351 )
...
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
Co-authored-by: Ke Bao <ispobaoke@gmail.com >
Co-authored-by: Sam Shleifer <sam@thinkingmachines.ai >
2026-09-22 01:21:25 +08:00
Liangsheng Yin
800613a74b
[Test] Split the serving perf tests by topic into basic_perf/ and route their thresholds through a kit ( #40505 )
2026-09-21 10:05:57 -07:00
Sage
14e9c40a72
[Observability] Expose python/rust frontend identity in /server_info ( #39993 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-21 23:32:52 +08:00
hanwlax
8faa2d6731
[NPU][Diffusion] Disable loading latency checks in Ascend fixtures ( #40544 )
2026-09-21 16:56:35 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
8d08dfdab7
[Fix] Add gigachat35 to the tool-call and reasoning parser name lists ( #40554 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-21 16:34:43 +08:00
ashwini rathi
c2f860af1c
[ci][xpu] Record device time in the multimodal_gen perf lane ( #39956 )
2026-09-21 15:39:29 +08:00
b63f8416b3
[Feature] Gigachat 3.5 support ( #29189 )
...
Co-authored-by: Stanislav Petrov <stapetrov@sberbank.ru >
Co-authored-by: Viacheslav Barinov <vvadbarinov@sberbank.ru >
Co-authored-by: Viacheslav <viacheslav.teh@gmail.com >
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
2026-09-21 14:37:55 +08:00
kpjeeja
b54d5b7c7b
disaggregation: Fix FakeKVSender queue accumulation ( #28652 )
...
Signed-off-by: KP, Jeeja <jeeja.kp@intel.com >
2026-09-21 14:27:05 +08:00
skyler-apdx
f5f3c38aad
[Fix] Preserve YaRN scaling when extending rotary caches ( #38786 )
2026-09-21 14:14:05 +08:00
AMRUTHA M and Ma Mingfei
d20cd9d77f
[XPU]Enable HiSparse hierarchical sparse KV cache on Intel XPU ( #32792 )
...
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com >
2026-09-21 14:07:49 +08:00
Lianmin Zheng
11ecdbf39f
Clean up startup logging and streamline log audits ( #40526 )
2026-09-20 22:28:35 -07:00
ashwini rathi
292e3ccc0c
[ci][xpu] Re-seed the wan2_1_t2v_1.3b perf baseline on Arc Pro B60 ( #39955 )
2026-09-21 13:04:35 +08:00
minke.yu
c74a4037fb
mooncake: probe pointers via cuda.bindings instead of ctypes libcudart
...
The ctypes-based probing loaded a second libcudart copy into the process,
which corrupted CUDA/runtime state and segfaulted the scheduler processes
shortly after registration (reproducible on prefill ranks, intermittent on
decode). cuda.bindings.runtime.cudaPointerGetAttributes is a properly
typed binding and boots cleanly on both roles.
2026-09-21 12:46:25 +08:00
minke.yu
b963295489
mooncake: support host-memory payloads with intra-node NVLink transport
...
IntraNodeNvlinkTransport can only register/reach device memory. sglang's
register list includes host regions (aux buffers, some state components);
one host region failed the whole registerLocalMemoryBatch and the engine
rolled back every region, leaving the segment descriptor empty so all KV
transfers failed with 'Requested address ... not found'.
When the intra-node NVLink transport is active (MC_INTRANODE_NVLINK etc.):
- register only device-memory regions (probe via cudaPointerGetAttributes)
- route transfer blocks whose local source is host memory over the ordered
per-endpoint zmq channel (new STATE_DATA message) instead of the engine;
the receiver validates and writes the block into its buffer, relying on
the same ordering guarantee as the aux TCP path
- force aux over the TCP socket path automatically
2026-09-21 11:53:26 +08:00
jianzhao-xu
62ba964848
Fix: post-load staging regression breaks offload meta/sharded_gpu modes ( #38779 )
2026-09-21 11:19:07 +08:00
Xueshen Liu
ab03a8e7eb
[Perf] Fork-safe import: no CUDA context at import time, lighter argument parsing ( #40201 )
2026-09-21 10:48:35 +08:00