Commit Graph
5366 Commits
Author SHA1 Message Date
BingjiaWang 4e60127fb4 [DSV4.1][PD][1/N] Support prefill node use cp parallel strategy (#40323) 2026-10-20 18:27:25 +08:00
minke.yu ef1a5955f3 PR #39306: H20-3e block-FP8 GEMM configs + adapt tables to NVIDIA H20
Net diff of PR #39306 vs upstream/main (kernel path already landed via #39657).
Also duplicated the six #41098 quantization tables and the six #39306 gemm
tables under device_name=NVIDIA_H20: config lookup is exact-filename and our
GPUs report 'NVIDIA H20' (96GB), not H20-3e. H20-3e shares the SM90 compute
configuration, so the tuned tables transfer.
2026-10-20 10:26:58 +08:00
冯骐 8c6cbd8c40 Support DSpark decoder bounded replay with DSV4.1 vision CP
Use tail-aware CP gathers and logits metadata, preserve original CP ownership of canonical routing IDs, and pass global token indices to DSpark. Adapt mHC state slicing and register CPU regression tests. Based on PR40219; tail handling informed by PR40161.
2026-10-20 10:20:57 +08:00
冯骐 c82aed9fef Fix DSV4.1 Torch prefill indexer request-pool rows under CP
Materialize actual request-pool IDs from per-rank real row counts. Preserve zero-row requests, bounded-replay tails and the non-CP fast path. Add CPU regression coverage.
2026-10-20 10:16:54 +08:00
minke.yu fd86688428 PR #40219: DeepSeek V4.1 vision CP and single-owner image encoding
Source: https://github.com/sgl-project/sglang/pull/40219 (applied via GitHub .diff, 3way)
Adaptations for v0.5.21 base:
- attn_dp_enabled_of(cfg) -> cfg.enable_dp_attention (#41818 not in base)
- dropped _validate_joint_schema_request call (#42721 Clef, not in base)
- moved Backend import before the TRT-LLM prefill-graph check
2026-10-19 10:45:10 +08:00
Vladislav Nosivskoy da343ba5d1 [SM90] Use native block32 DeepGEMM for DeepSeek-V4.1 (#42604)
Co-authored-by: BBuf <1182563586@qq.com>
(cherry picked from commit 2ff3e6dce1)
2026-10-19 10:38:40 +08:00
minke.yu 9ea02a7201 PR #40421: reuse shared compressed KV dequantization in PD prefill
Source: https://github.com/sgl-project/sglang/pull/40421 (applied via GitHub .diff, 3way)
2026-10-19 10:37:44 +08:00
minke.yu 51624199b4 PR #41294: use the fused low-ratio compressor on Hopper (sm90 gate)
Source: https://github.com/sgl-project/sglang/pull/41294 (applied via GitHub .diff, 3way)
Skipped: test_dsv41_fused_compress.py (file not in v0.5.21 base).
2026-10-19 10:35:29 +08:00
minke.yu 31b3758c99 PR #42365: fuse Hopper prefill indexer scoring into one Triton kernel
Source: https://github.com/sgl-project/sglang/pull/42365 (applied via GitHub .diff, 3way)
Conflicts resolved: unioned env flags with #40945 SM90 flags; merged fused
scoring loop with #40860 candidate_blocks path (candidate > fused > torch priority).
2026-10-19 10:33:13 +08:00
minke.yu 740372bef0 PR #40945: SM90 static verification indexing (grouped/length-aware indexer)
Source: https://github.com/sgl-project/sglang/pull/40945 (applied via GitHub .diff, 3way)
Adaptation: PR base predates #41659 which removed the Selection struct;
ported sm90_decode.py and call sites to the inputs.out_page_indices interface
(raw_indices=None on decode paths, reset via inputs.reset_outputs()).
2026-10-19 10:30:41 +08:00
minke.yu 19fe20baec PR #40860: SM90 candidate indexers (compact candidate dispatch)
Source: https://github.com/sgl-project/sglang/pull/40860 (applied via GitHub .diff, 3way)
Conflicts resolved: kept #41658 padded-return trim in fp4_index_logits_paged,
took PR side for candidate_blocks branches, SM90 graph gate, deep_select test param.
Skipped: test_dsv41_candidate_graph_variants.py (file not in v0.5.21 base).
2026-10-19 10:22:09 +08:00
Po-Han Huang (NVIDIA) 383df91fcc [Test] dsv4: Fix Hopper indexer replay test after DeepSelect decode (#41561) (#44193)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit d5f80aa3c3)
2026-10-19 10:16:18 +08:00
yuyu5333 534842ac08 [DeepSeek V4.1] Use DeepSelect for Hopper full top-k decode. (#41561)
Co-authored-by: Claude Code <noreply@anthropic.com>
(cherry picked from commit b50fdeb0ef)
2026-10-19 10:16:18 +08:00
Yuwei An 4dd95a8817 [DSV4.1] Reduce eager Engram extend preparation (#42159)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit d434ea444d)
2026-10-19 10:16:08 +08:00
Yuwei An 656e00dcd3 [DSv4.1] mxfp8 dispatch cache (#42156)
Co-authored-by: Yuwei An <yuwei.an@MacBook-Pro-yuweian.local>
(cherry picked from commit 2d38934493)
2026-10-19 10:15:46 +08:00
DarkSharpness a4b1363bf4 [DSv4.1] Prefill indexer: raw selection, row-pair schedule ids, no host syncs (#41659)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
(cherry picked from commit 520d6d0229)
2026-10-19 10:15:46 +08:00
DarkSharpness ad2acaa2df [DSv4.1] Fold q_rope_store into fused_q_norm_rope (#41657)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 379ec90fb2)
2026-10-19 10:15:45 +08:00
Yuwei An ecaaa37cb0 [Fix][DSV4.1] SWA page size with bounded replay (#42128)
(cherry picked from commit ef867fa40d)
2026-10-19 10:14:30 +08:00
DarkSharpness 3ebbb2d536 [DSv4.1] Fused c1/c2 compress for eager extend, faster c2 decode (#41660)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
(cherry picked from commit 5916999afb)
2026-10-19 10:14:29 +08:00
DarkSharpness 5d52cccbee [DSv4.1] Faster fp4 index-K gather and combine_topk_swa_indices (#41658)
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: BBuf <1182563586@qq.com>
(cherry picked from commit d75d5b33f7)
2026-10-19 10:14:28 +08:00
Xiaoyu Zhang 2b192aade6 [Perf] Optimize DeepSeek V4.1 Flash Hopper paths and Blackwell prefill selection (#41251)
(cherry picked from commit 391e665dab)
2026-10-19 10:13:35 +08:00
minke.yu 9247c97919 PR #40919: fix dspark folded result buffer reuse race
Source: https://github.com/sgl-project/sglang/pull/40919 (applied via GitHub .diff, 3way)
2026-10-19 10:12:23 +08:00
minke.yu 5e7190b87a PR #40758: escape literal DSV4.1 image placeholder in user text
Source: https://github.com/sgl-project/sglang/pull/40758 (applied via GitHub .diff, 3way)
2026-10-19 10:12:16 +08:00
Kangyan-ZhouandBaizhou Zhang d7ff74b984 [Cherry-pick to release/v0.5.21] [Fix] Restore deferred layer dumps and pin SentencePiece for InternVL (#41777) (#41788)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2026-09-29 17:30:38 -07:00
Jiacong Fang bd78095030 [Fix] Add name mapping in load_weights of nvidia/LocateAnything-3B (#41059) 2026-09-29 00:38:58 -07:00
eeecho c7be3e935b [SM120] Add optional FlashInfer PCIe-IPC all-reduce for switch-free hosts (#34528) 2026-09-28 23:08:44 -07:00
Anupa SajikumarandMa Mingfei 2ff52e3ceb [Intel][XPU]Enable chunked prefill scnearios for XPU with UT (#33804)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-09-29 13:28:25 +08:00
9ae43b0eee [XPU] Support compressed-tensors W4A16 by reusing the torch int4pack path (#40828)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Singh <rohitsi2@iil-gnrap01.iind.intel.com>
Co-authored-by: Singh <rohitsi2@iil-login.iind.intel.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-09-29 13:16:03 +08:00
Cheng Wan 5781206833 [PD] Give FakeKVReceiver ensure_abort_notified (#41618) 2026-09-28 21:59:17 -07:00
Cheng Wan 6dad152490 [Refactor] Document layer boundary contracts and integration (#41557) 2026-09-28 20:18:27 -07:00
Cheng Wan 49e7bd179d [Refactor] Group layer boundary unit tests (#41556) 2026-09-28 20:18:08 -07:00
Cheng Wan de9f850b36 [Refactor] Rename the module to layer_boundary (#41555) 2026-09-28 20:17:50 -07:00
Cheng Wan 701213c867 [Refactor] Retire the layer facade and simplify boundary internals (#41554) 2026-09-28 20:17:33 -07:00
Cheng Wan 9b89b96230 [Refactor] Migrate specialized decoder and overlap boundaries (#41553) 2026-09-28 20:17:17 -07:00
Cheng Wan 49b5e76e45 [Refactor] Construct independent decoder stage boundaries (#41552) 2026-09-28 20:17:00 -07:00
Cheng Wan acffb0d20d [Refactor] Select reduction fusion at the consumer (#41551) 2026-09-28 20:16:44 -07:00
Cheng Wan cb1fa3c399 [Refactor] Capture auxiliary states at residual reads (#41550) 2026-09-28 20:16:26 -07:00
Cheng Wan 42f58b4da7 [Refactor] Carry residual state across stage boundaries (#41549) 2026-09-28 20:15:03 -07:00
Cheng Wan 6f72cc3b64 [Refactor] Centralize decoder output access (#41548) 2026-09-28 20:14:04 -07:00
Cheng Wan 0a8ca85565 [Refactor] Group communicator fusion and CP adapters (#41547) 2026-09-28 20:10:50 -07:00
Mohammad Miadh AngkadandMohammad Angkad e49d7fb170 Remove GLM-4.1V-9B-Thinking from encoder DP MMMU test (#41608)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-28 19:45:21 -07:00
77091cea68 [Radix Cache] Sync Rust TreeCore and make it the default (#39627)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com>
2026-09-29 10:18:57 +08:00
79cafec013 [Model] Add IQuest Q1 support and MTP draft (#41590)
Co-authored-by: zelong huang <yzhu@ubiquant.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: zelong518 <zelonghuang05@gmail.com>
Co-authored-by: chenwenxiaolive <cocoshirolive@gmail.com>
2026-09-28 17:21:31 -07:00
7889fbab9a [PD] Keep EAGLE DP graph and token metadata consistent (#32196)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
2026-09-28 17:15:51 -07:00
9654e5c4bc dsv4.1-amd: fused mHC boundary and all-reduce + mHC post kernels (#41021)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai>
2026-09-28 16:15:08 -07:00
Kevin MiandClaude Opus 5.5 096b066fb4 dsv4.1-amd: gfx950 sparse decode attention and sorted top-k (#41020)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 16:14:28 -07:00
Nan Jiang 6fa3fe69e2 [PD] Keep the sampling mask of a replayed rebootstrap token (#41235) 2026-09-28 12:02:27 -07:00
Shangming CaiandClaude Fable 5 2a1c477648 [PD] Defer decode KV release on every transfer failure, not only decode-initiated aborts (#41404)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-29 02:19:51 +08:00
YAMY f209e67767 [Spec] Model-agnostic last-stage draft embedding under pipeline parallelism (#39643) 2026-09-28 10:22:40 -07:00
Kurkur 6624999385 [Fix][NPU] fix dp-attn hang when pin_mem is True on NPU (#40446) 2026-09-29 00:41:43 +08:00