Commit Graph
13024 Commits
Author SHA1 Message Date
Khoa Pham c4d3770a68 [Kimi K3] Fix CUDA graph stream explosion (#40640) 2026-09-21 17:49:50 -07:00
jacky.cheng 31b577bb08 [AMD] Pad QSA MQA decode Q-heads to 16 for ROCm MFMA (#38875) 2026-09-21 17:43:46 -07:00
042b6a488f [AMD] [GLM-5.3-Flash Day 0] Enable zero-RoPE MHA prefill on ROCm (#39338)
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
2026-09-21 17:38:07 -07:00
Cheng Wan 582389cec5 [Fix] Keep diffusion encoder TP context bindings consistent (#40646) 2026-09-21 17:23:15 -07:00
Tao LiandXiaoyu Zhang c53cc8e1eb [NPU][BugFix] Avoid M-RoPE recompilation for variable sequence lengths (#40371)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-09-22 08:19:25 +08:00
avalliappan-nvidia 61d0cf2074 [Spec] Windowed draft-decode attention for built-in EAGLE / MTP drafts (#32673) 2026-09-22 08:17:57 +08:00
Vedant V JhaveriandCopilot 9fdb71732a Avoid materializing GDN QKV tensors during target verification (#33778)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-09-21 17:04:18 -07:00
Cheng Wan 506698761d [unified-memory] Hierarchical cache for every unified pool shape (#37507) 2026-09-21 16:50:37 -07:00
Cheng Wan 22587fb15c [Fix] Run KV canary hooks for context-parallel prefill (#40642) 2026-09-21 16:46:22 -07:00
Yuxuan ZhangandXinyuan Tong 00986c81be Support GLM-5.3-Flash hybrid attention CPU offload and PD index mapping (#40310)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-09-21 16:03:03 -07:00
YAMY 0229025127 [Spec][PP] Launch extend microbatches before the spec output exchange (#40499) 2026-09-21 15:47:03 -07:00
RuibinCheung 0c53fec476 [ROCm] fix: remove extra bf16 -> fp32 cast in jit grouped topk kernel path (#39775) 2026-09-21 15:43:21 -07:00
8bde82c0ad [AMD] [GLM-5.3-Flash Day 0] Build the fused DSA k-pool top-k JIT kernel on HIP (#39339)
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-21 15:24:44 -07:00
Cheng Wan acac4dd9d9 [Refactor] Clean up parallel runtime comments (#40632) 2026-09-21 14:32:22 -07:00
William Hu f532ad1f9a Fix GLM-5.3 forget-gate shape for nvCUTEDSL verify (#40607) 2026-09-21 13:52:26 -07:00
jacky.cheng e0c2e8dc4d [AMD] Tune Qwen3.5 TP4 GDN recurrent launch on gfx950 (#39987) 2026-09-21 13:20:54 -07:00
Liangsheng Yin b18ca9ca44 [CI] Bump sgl-eval to 0.1.2 (#40620) 2026-09-21 13:00:54 -07:00
11e661fd45 [Fix] Don't free the multi-CTAs KV counter the decode graphs captured (#39175)
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 12:58:48 -07:00
Cheng Wan bccf691b22 Bringing the parallel runtime up becomes a phase, not a side effect (#40345) 2026-09-21 12:29:50 -07:00
Cheng Wan 1d3243d05f Take the parallel getters off the package's public surface (#40344) 2026-09-21 12:27:50 -07:00
Cheng Wan 970e946e4f Retire the per-runner parallel record (#40343) 2026-09-21 12:26:40 -07:00
Cheng Wan 73f071db52 Deprecate the parallel getters the context answers, and ratchet them shut (#40342) 2026-09-21 12:25:32 -07:00
Cheng Wan 65be3fa71a A runner and the objects it builds freeze the placement they describe (#40341) 2026-09-21 12:24:17 -07:00
Cheng Wan 2d0e94e3a3 Check the topology identities where the layout is written, and build at the published widths (#40340) 2026-09-21 12:22:59 -07:00
Cheng Wan 0db1a93adb State the draft's whole topology in its scope, and read the rest from the context (#40339) 2026-09-21 12:19:38 -07:00
ae7a516ba7 feat: use XGrammar V4.1 DSML parameter constraints (#39026)
Co-authored-by: yuchuan <yuchuan.7streams@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-21 12:12:28 -07:00
jacky.cheng 90b3f8544c [AMD] Use Triton softmax routing for Qwen3.5 on gfx950 (#39986) 2026-09-21 12:02:29 -07:00
ronnie_zheng f702a0be29 Revert "[Diffusion] migrate the whole _register_configs from registry.py to the model own config file" (#40611) 2026-09-21 21:05:19 +03:00
Bingxu Chen 632919e498 [AMD] Fix DeepSeek-R1-MXFP4 accuracy with AITER FP8 (#37762) 2026-09-21 10:51:37 -07:00
ronnie_zheng e6931ca889 [Diffusion] migrate the whole _register_configs from registry.py to the model own config file (#40475) 2026-09-21 20:45:53 +03:00
cctry 7a6191c4b9 Preallocate HiCache MHA staging before post-capture KV sizing (#40256) 2026-09-21 10:44:29 -07:00
cctry 7ad55e4386 [HiCache] TMA-staged host<->device KV transfer kernel (sm_90+) (#40278) 2026-09-21 10:38:23 -07:00
William Hu 0cb37c018c [KDA] Enable ReplaySSM for GLM-5.3 Flash (#40517) 2026-09-21 10:34:08 -07:00
Eric.Chin.AMDandThomas Wang 3c71bb018a [AMD] Enable GLM DSA prefill top-k to the v2 kernel (#37889)
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-09-21 10:31:07 -07:00
5a6a1bb883 [mxfp8-kv] Skip writes to the reserved CUDA-graph padding slot (#35351)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Sam Shleifer <sam@thinkingmachines.ai>
2026-09-22 01:21:25 +08:00
Liangsheng Yin 800613a74b [Test] Split the serving perf tests by topic into basic_perf/ and route their thresholds through a kit (#40505) 2026-09-21 10:05:57 -07:00
Sage 14e9c40a72 [Observability] Expose python/rust frontend identity in /server_info (#39993)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
2026-09-21 23:32:52 +08:00
hanwlax 8faa2d6731 [NPU][Diffusion] Disable loading latency checks in Ascend fixtures (#40544) 2026-09-21 16:56:35 +08:00
Mohammad Miadh AngkadandMohammad Angkad 8d08dfdab7 [Fix] Add gigachat35 to the tool-call and reasoning parser name lists (#40554)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-21 16:34:43 +08:00
ashwini rathi c2f860af1c [ci][xpu] Record device time in the multimodal_gen perf lane (#39956) 2026-09-21 15:39:29 +08:00
b63f8416b3 [Feature] Gigachat 3.5 support (#29189)
Co-authored-by: Stanislav Petrov <stapetrov@sberbank.ru>
Co-authored-by: Viacheslav Barinov <vvadbarinov@sberbank.ru>
Co-authored-by: Viacheslav <viacheslav.teh@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-09-21 14:37:55 +08:00
kpjeeja b54d5b7c7b disaggregation: Fix FakeKVSender queue accumulation (#28652)
Signed-off-by: KP, Jeeja <jeeja.kp@intel.com>
2026-09-21 14:27:05 +08:00
skyler-apdx f5f3c38aad [Fix] Preserve YaRN scaling when extending rotary caches (#38786) 2026-09-21 14:14:05 +08:00
AMRUTHA MandMa Mingfei d20cd9d77f [XPU]Enable HiSparse hierarchical sparse KV cache on Intel XPU (#32792)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-09-21 14:07:49 +08:00
Lianmin Zheng 11ecdbf39f Clean up startup logging and streamline log audits (#40526) 2026-09-20 22:28:35 -07:00
ashwini rathi 292e3ccc0c [ci][xpu] Re-seed the wan2_1_t2v_1.3b perf baseline on Arc Pro B60 (#39955) 2026-09-21 13:04:35 +08:00
minke.yu c74a4037fb mooncake: probe pointers via cuda.bindings instead of ctypes libcudart
The ctypes-based probing loaded a second libcudart copy into the process,
which corrupted CUDA/runtime state and segfaulted the scheduler processes
shortly after registration (reproducible on prefill ranks, intermittent on
decode). cuda.bindings.runtime.cudaPointerGetAttributes is a properly
typed binding and boots cleanly on both roles.
2026-09-21 12:46:25 +08:00
minke.yu b963295489 mooncake: support host-memory payloads with intra-node NVLink transport
IntraNodeNvlinkTransport can only register/reach device memory. sglang's
register list includes host regions (aux buffers, some state components);
one host region failed the whole registerLocalMemoryBatch and the engine
rolled back every region, leaving the segment descriptor empty so all KV
transfers failed with 'Requested address ... not found'.

When the intra-node NVLink transport is active (MC_INTRANODE_NVLINK etc.):
- register only device-memory regions (probe via cudaPointerGetAttributes)
- route transfer blocks whose local source is host memory over the ordered
  per-endpoint zmq channel (new STATE_DATA message) instead of the engine;
  the receiver validates and writes the block into its buffer, relying on
  the same ordering guarantee as the aux TCP path
- force aux over the TCP socket path automatically
2026-09-21 11:53:26 +08:00
jianzhao-xu 62ba964848 Fix: post-load staging regression breaks offload meta/sharded_gpu modes (#38779) 2026-09-21 11:19:07 +08:00
Xueshen Liu ab03a8e7eb [Perf] Fork-safe import: no CUDA context at import time, lighter argument parsing (#40201) 2026-09-21 10:48:35 +08:00