Commit Graph
157 Commits
Author SHA1 Message Date
Yuwei An d3f8a8f4f5 [Spec/Prefill Coordination] Dspark support (#41623) 2026-09-29 16:08:01 -07:00
Liangsheng Yin f88572a57c [Test] Add in-process sgl-eval adapter and move validated GSM8K tests to it (#41216) 2026-09-25 13:59:17 -07:00
Yuwei An 3ec86307f2 Fix mixed chunk prefill with DP speculative coordination (#41179) 2026-09-24 15:41:49 -07:00
Yuwei An 1417345f5f [Experimental] Preserve speculative decoding during prefill across DP ranks (#40118) 2026-09-24 00:01:12 -07:00
Liangsheng Yin acd20a516e [CI] Give the kernel lane a 5090 suite and move kernel-only tests off the general lane (#40496) 2026-09-20 16:46:53 -07:00
Liangsheng Yin d507accadc [Test] Drop dead and strictly-subsumed CI test registrations (#40264) 2026-09-18 17:49:13 -07:00
a6cf05817f dsv4.1: remaining model and runtime integration (#38798)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
2026-09-18 02:55:30 -07:00
iridiumine a984c78330 [NPU] Support DFlash speculative decoding for MiMo-V2.5-Pro (mxfp4) (#37565) 2026-09-12 11:59:23 +08:00
Cheng Wan db272201a2 [Config] Retire get_global_server_args, and clear the deprecated flags that have a replacement (#38375) 2026-09-08 16:42:12 -07:00
Shuwen Wang 755f97c622 [CI] Fix the DSpark dp-tier unit test fixture after #34919 (#38314) 2026-09-07 03:16:00 -07:00
Xiaoyu ZhangandMick Qian 4d23a4fa6d [Test] Consolidate test cleanup and CI taxonomy (net -11.4K lines) (#37436)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-07 15:13:59 +08:00
sglang-botandsglang-bot 6252993afe chore: update CI test est_time values (#38238)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-09-06 17:49:41 -07:00
44c786679f [sp] Make attention-TP sequence sharding a per-forward batch property (#37546)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
2026-09-04 02:11:03 -07:00
ashwini rathi e787de5478 [XPU][CI] Move XPU tests to nightly and add per-subclass server launch timeout (#37532) 2026-09-04 12:51:26 +08:00
eigenandYingyi Huang 516cfbd362 Fix UNO test adapter subdirectory resolution (#37872)
Co-authored-by: Yingyi Huang <averyh@nvidia.com>
2026-09-03 18:07:09 -07:00
2bb25dc18b [Speculative Decoding] Add native UNO serving support (#37667)
Co-authored-by: drproduck <drproduck@MacBook-Air-2.local>
Co-authored-by: BBuf <1182563586@qq.com>
2026-09-03 20:08:41 +08:00
Alex NailsandAlison Shao 28262c20df [CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
2026-09-02 19:46:08 -07:00
cctryandcctry 9a05b470fa [Memory] Size the CUDA graph pool from warmup measurements and fix graph-pool borrowing (#36911)
Co-authored-by: cctry <cctry@fb.com>
2026-09-01 09:32:38 -07:00
Yuwei AnandClaude Fable 5 07d84ebd6d [2/N][Mixed] Mixed chunk prefill with spec enabled (#36933)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-31 10:48:07 -07:00
Cheng WanandClaude Opus 5 1a3e152f03 config: six more runtime readers ask the bags (#36973)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 04:19:14 -07:00
Cheng WanandClaude Opus 5 6ff2a20ccf config: the record is not an object that gets passed around (#36622)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 12:57:10 -07:00
Cheng Wan 5b7fc61306 config: resolution reads the declarations, not the fields (#36253) 2026-08-26 05:05:28 -07:00
Aurick QiaoandYuwei An 58ecbba0bd [Feature] Coordinate FullCG prefill across DP-attention ranks (#35640)
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
2026-08-26 02:16:02 -07:00
cctryandcctry 37c09ff3d8 [Memory] Borrow CUDA graph pool storage for EAGLE sampling (#35375)
Co-authored-by: cctry <cctry@fb.com>
2026-08-18 16:19:41 -07:00
Cheng Wan 1ab713c334 config: the post-publish consumers of the supplied-instance surface read the bags
config: the speculative workers take page_size from the bags

Seven worker constructors stored `self.page_size = server_args.page_size` off
the handed record. They all run after publish and all keep a copy of a
process-level value, which is the first row of the plan doc's supplied-instance
disposition table -- so they read `get_schedule().page_size`, and a post-publish
override now reaches them like it reaches every other consumer.

The supplied-instance census named the seven pairs; the exposure ratchet in the
next member pins what remains after this batch of conversions.


config: the post-publish chunked_prefill_size consumers read the bags

Four of the ten supplied-instance `chunked_prefill_size` reads are plain
post-publish consumers -- the EPLB recorder's buffer sizing, the deep-gemm
compile warmup (five reads), the KV-cache builder's effective size, and the
ngram embedding manager's assert. All are reached from runner init, so they read
`get_schedule()`.

Two are deliberately left: `create_kt_config_from_server_args` builds a config
*from a supplied record* by name and contract, and `CanaryLaunchCapacities.from_args`
is the same shape. Converting those would change what the function is, not where
it reads -- the plan doc's disposition table says so per field.


config: the remaining post-publish graph/limit consumers read the bags

Three more of the census's supplied-instance debts are plain post-publish reads: the dspark worker's
cuda-graph decode sizes, the dspark planner's SPS table bound
(`max_running_requests`), and the LoRA manager's cuda-graph moe buffers. The
dspark worker is the clearest of them -- it already read
`get_exec().graph.cuda_graph_config.decode.bs` thirty lines below the instance
read, so the file disagreed with itself about where the same value comes from.

Left where the function's contract is "build a config from the record you are
handed" rather than "read this process's config":
`create_kt_config_from_server_args`, `DllmConfig.from_server_args`,
`CanaryLaunchCapacities.from_args`, `build_compilation_config`. Changing those
would change what the function is.


config: the runner, scheduler and offload manager take page_size from the bags

The same `self.page_size = server_args.page_size` shape as the speculative
workers, in the three remaining process-owned constructors: `ModelRunner`,
`Scheduler`, and the decode-side KV offload manager. The scheduler process
publishes before any of them run. The one path that did not is `ModelRunner`
constructed standalone -- `python -m sglang.benchmark.one_batch` and the manual
runner tests build it with no prior publish, and the constructor's own publish
sat below this read -- so that publish moves above the constructor's first bag
read instead of leaving a window where the runner half-exists unpublished.

Left where the read belongs to something else: `utils/common`'s predicates are
called only from the resolution pipeline with a `resolved_view`,
`allocation_sizing` takes the config its callers supply by contract, and
`CudaVmmFeatureTransport` is tokenizer-owned -- one per tokenizer worker, which
is the per-instance boundary.

The conversion left the offload manager parking a record it no longer
reads; the parked copy goes with the read (the constructor parameter stays
-- its hicache sizing still reads it directly).
2026-08-15 00:39:35 -07:00
weireweireandweireweire 6eb941a34c [Spec] Support MegaMoE for DSpark under dp attention (#34844)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-14 17:20:01 -07:00
fde9ad2531 [Feature] Add Muse Glimmer model support (#34262)
Co-authored-by: sglang-bot <232288953+sglang-bot@users.noreply.github.com>
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-08-11 15:41:52 -07:00
Liangsheng YinandBaizhou Zhang f6a6f5bf1e [CI] Trim redundant nightly test registrations (#34070)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2026-08-08 01:42:46 -07:00
Liangsheng Yin 9bd1461757 [CI] Bound the CUDA graph capture range in test launches and lift the spec fixture's admission cap (#33776) 2026-08-05 20:00:30 -07:00
Liangsheng Yin 3869fe556f [CI] Collapse the EAGLE launch matrix and the scoring engine boots on the per-commit runners (#33756) 2026-08-05 16:43:19 -07:00
Liangsheng Yin de34dd11e9 [CI] Fold duplicate-server suites and prune the retract matrix on 1-gpu-5090 (#33745) 2026-08-05 12:41:51 -07:00
Jason Mancuso 36853b8ffc [Spec] Support logprobs with DFlash (#33459) 2026-08-05 12:37:12 -07:00
Liangsheng Yin 0d99d91e49 [CI] Make B200 base-b suites single-GPU as prep for 1-gpu B200 runners (#33605) 2026-08-04 15:53:01 -07:00
Liangsheng Yin 0753663b8e [CI] Trim redundant B200 test registrations (#33586) 2026-08-04 13:22:00 -07:00
ashwini rathiandMa Mingfei 53804d609c [CI][XPU] Stabilize XPU CI: pin UMD/IGC, retry infra flakes, right-size EAGLE3 (#32438)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-08-04 16:28:52 +08:00
Vladislav NosivskoyandXinyuan Tong 154f0ac662 Fix DSpark and DP/EP (#33098)
Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-08-04 00:35:57 -07:00
Liangsheng YinandKaixi 6ab3231b97 [Perf] Skip the target-verify tree mask fill when the backend never reads it (#32886)
Co-authored-by: Kaixi <kaiximatteoc@nvidia.com>
2026-07-30 02:32:38 -07:00
Hồ Sỹ Thếandhnyls2002 d021990bf5 [DFLASH] Support grammar-constrained decoding in speculative verify (#30096)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-07-25 04:36:46 -07:00
Liangsheng Yin 3c5bf1f6d2 [Spec] Derive NGRAM grammar tree links on the host instead of reading back retrive_next_token (#32380) 2026-07-25 02:25:16 -07:00
ANSHUMAN TRIPATHY f0f78a6c93 Add deterministic inference for eagle parity test (#30026) 2026-07-24 12:48:21 +08:00
Xiaoyu ZhangandClaude Opus 4.8 62aa85d9aa [Kernel] Sweep missed dedicated kernels into kernels.ops (moe/quant siblings + dspark) (RFC #29630) (#32160)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 17:07:16 +08:00
Liangsheng Yin f35411ee81 [Spec] Enable grammar overlap scheduling for STANDALONE speculative decoding (#32110) 2026-07-23 01:47:27 -07:00
Liangsheng Yin 024639a372 [Perf] Stack dspark dense draft per-layer ctx KV projection into one GEMM (#31986) 2026-07-21 17:46:13 -07:00
Xiaoyu ZhangandClaude Opus 4.8 216b750c8f [Kernel] Sweep decoupled scattered kernels into sglang.kernels.ops (RFC #29630) (#31582)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 19:07:07 +08:00
Jorge António 26cb0fcdda Empty _REQ_TYPES_WITH_OPAQUE_FIELDS on the msgpack IPC path (#29465 Task 4) (#30182) 2026-07-15 14:55:06 -07:00
6cc9352dfe [Spec] Add DSpark: confidence-scheduled speculative decoding (#30261)
Co-authored-by: sglang-bot <232288953+sglang-bot@users.noreply.github.com>
Co-authored-by: Claude Code <noreply@anthropic.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-07-12 17:25:26 -05:00
ashwini rathiandarathi-hlab 850719ebd9 [XPU] Unbreak stage-b: re-add --disable-decode-cuda-graph, quarantine EAGLE3 parity (#30048)
Co-authored-by: arathi-hlab <arathi.rajanraghavendran@intel.com>
2026-07-06 12:25:46 +08:00
Cao E 9df16b5ba9 [XPU] Remove redundant xpu graph backend and make xpu graph opt-in by default (#29911) 2026-07-03 15:58:56 +08:00
ANSHUMAN TRIPATHY 4fffc6448b Speculative decoding support on XPU (#23180) 2026-07-02 13:23:24 +08:00
Liangsheng Yin 909123ddb8 [misc] Use --cuda-graph-max-bs-decode in tests, examples, and docs (#29591) 2026-06-28 18:38:28 -07:00