minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 14 Packages Projects Releases Wiki Activity
16,043 Commits 2 Branches 0 Tags
95d0e57e838b627d77031082c91095c200037fe5
Commit Graph
10978 Commits
Author SHA1 Message Date
Xiaoyu ZhangandClaude Fable 5 95d0e57e83 [diffusion] Fuse DiT FFN tanh-GELU into up-proj GEMM (cublasLt epilogue) behind quality=high (Qwen-Image 1024^2 denoise 12.36 -> 12.05 s on H200) (#33536) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 23:49:43 +08:00
Xiaoyu Zhang 0d0c7d853f [diffusion] FLUX.2 VAE decoder fast path behind quality=high (H200: 1024^2 97.6->29.2 ms, 2048^2 437.2->168.5 ms) (#33451) 2026-08-04 23:48:19 +08:00
Lianmin Zheng 7adf2f4a9a Inline _set_gc into _set_envs_and_config (#33538) 2026-08-04 07:31:56 -07:00
Lianmin ZhengandYinghai Lu d257b58e67 [Router] Report accelerator count in /v1/loads (#33548) 
Co-authored-by: Yinghai Lu <yinghai@meta.com>
 2026-08-04 05:55:35 -07:00
Lianmin Zheng 8f2a3ad6d7 [mem_cache] Label HiCache host pools and clarify post-capture KV sizing logs (#33445) 2026-08-04 04:21:36 -07:00
jacky.cheng 723c277640 [AMD] [Fix] Enable aiter hd256 FP8 prefill FMHA on gfx950 (#33399) 2026-08-04 02:40:33 -07:00
Xingyu Liu 5e6c37f2b4 [cuda_graph] Gate breakable-CG capture_inputs retention to DP-gather paths (#32678) 
Signed-off-by: xingyuliu <charlotteliu12x@gmail.com>
 2026-08-04 02:23:34 -07:00
Lianmin Zhengcctrycctry
 		b57721ccf7 Enable post-capture KV sizing with DP attention (#33427) 
Co-authored-by: cctry <cctry@meta.com>
Co-authored-by: cctry <cctry@fb.com>
 2026-08-04 02:20:24 -07:00
Lianmin Zheng 16d3b118a2 Reduce startup log noise and fix Dynamo / CUDA-graph edge cases (#33428) 2026-08-04 02:19:56 -07:00
Liangsheng Yin b6d548afd7 [Fix] Resolve VLM test image placeholders from the model's own chat template (#33509) 2026-08-04 02:01:39 -07:00
ashwini rathiandMa Mingfei 53804d609c [CI][XPU] Stabilize XPU CI: pin UMD/IGC, retry infra flakes, right-size EAGLE3 (#32438) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-08-04 16:28:52 +08:00
Kan WuandClaude Fable 5 17d19081d9 [mm] sglang-mm: server vision pipeline core (fetch/driver/pipeline) + Qwen VL (#32364) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 00:47:00 -07:00
Vladislav NosivskoyandXinyuan Tong 154f0ac662 Fix DSpark and DP/EP (#33098) 
Signed-off-by: Vladislav Nosivskoy <vladnosiv@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-08-04 00:35:57 -07:00
Alex NailsandClaude Opus 5 bfa4e4a57b [Nemotron] Hoist mamba track-mask host syncs out of the per-layer prefill path (#32589) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-03 23:58:28 -07:00
weireweireandweireweire 23ea7b6481 Prewarm DSV4 MHC post kernel at model load (#30741) 
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
 2026-08-03 23:41:56 -07:00
TobyMintandMick 101bb2327c [diffusion] fix: fix local-path detection for MiniMax-H3 and other non-diffusers models (#33365) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-08-04 14:03:35 +08:00
Liangsheng YinandMick afc868517b [Perf] Speed up the Kimi-K2.5 vision path and match PIL bicubic in the GPU resize (#33349) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-08-03 21:57:49 -07:00
Xiaoyu ZhangandClaude Fable 5 c6f2a9c1d4 [diffusion] Restrict request-level quality to two validated tiers: lossless (default) and high (#33453) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-04 11:43:16 +08:00
Jinchen HanandMick 614825fd38 [vla] fix: pi05 models does not apply scale factor for language embeddings (#33367) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-08-04 11:25:54 +08:00
Mick b058dc9106 [diffusion] fix: reject ring parallelism where it would silently miscompute (#33353) 2026-08-04 11:15:23 +08:00
EchO 1e64fc1563 [Fix] Honor FlashMLA natural-log LSE in DCP reduction (#33065) 2026-08-03 19:55:52 -07:00
Khoa Pham 91fae8a72c [DCP] Bound a request by the aggregate KV pool, not one rank's share (#33448) 2026-08-03 19:36:25 -07:00
Oguz UlgenandCheng Wan c113ead98a Bump helion version to 1.4 (#32562) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2026-08-03 19:10:31 -07:00
Yuwei AnandClaude Opus 5 92087ef4d2 fix(mem_cache): state the MLA KV bound in the DCP index space (#33432) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-03 19:07:49 -07:00
Baizhou Zhang eb31a53338 Revert "Add flashinfer rmsnorm + quant fusion support SM90, SM100, SM120" (#33455) 2026-08-03 19:03:56 -07:00
Xingyu Liu 572924634b [mem_cache] Build empty-prefix last_loc sentinel on-device to avoid per-call H2D sync (#32575) 
Signed-off-by: xingyuliu <charlotteliu12x@gmail.com>
 2026-08-03 18:55:03 -07:00
Baizhou Zhang 7f6a2e2b50 [Refactor] Clean up and split DSA indexer (#33443) 2026-08-03 18:05:01 -07:00
DevashishLal-CBDevashish LalXiaoyu Zhang
 		3960983753 Add flashinfer rmsnorm + quant fusion support SM90, SM100, SM120 (#32994) 
Signed-off-by: Devashish Lal <devcode@fb.com>
Co-authored-by: Devashish Lal <devcode@fb.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-08-04 08:37:42 +08:00
Xiaoyu Zhang f829fb3ff7 [diffusion] Fix component accuracy topology reuse (#33317) 2026-08-04 08:35:03 +08:00
DarkSharpnessClaude Fable 5Xiaoyu Zhang
 		1307968605 [JIT] Drop redundant per-kernel arch overrides (#32952) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-08-04 08:33:54 +08:00
Po-Han Huang (NVIDIA) 7eb27372b3 fix(server): capture legal multi-request prefill CUDA graph batches (#30206) 2026-08-03 16:56:59 -07:00
zijiexiaandClaude Opus 4.8 b819d2fb5b [Docs] Rename docs_new/ to docs/ (#32123) 
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
 2026-08-03 16:51:00 -07:00
Hanming Lu bc7e1a07c3 Bound prefill delayer all-branch delay and decay the max_prefill_bs high-watermark (#32880) 2026-08-03 14:43:02 -07:00
cctry db0fe370b7 [PD] Fix false health-503 during decode retraction re-admission (#33118) 2026-08-03 09:44:25 -07:00
cctry 6cf661117d [PD] Add a queues.prealloc_ready counter to the load snapshot (#33133) 2026-08-03 09:44:11 -07:00
Mick 0ba46c88e5 [diffusion] CI: add minimax-h3 2-gpu consistency coverage (#33281) 2026-08-03 22:40:08 +08:00
huangtingweiandjackyYang6 3953788596 [HiSparse]Fix DeepSeek V4 HiSparse PD Transfers with Separate Host and Device KV Indices (#31901) 
Co-authored-by: jackyYang6 <82102811+jackyYang6@users.noreply.github.com>
 2026-08-03 20:35:04 +08:00
Mohammad Miadh Angkad d48ab2d386 Fix BCG circular import during server startup (#33371) 2026-08-03 01:53:16 -07:00
Xun Sun a2d1003b18 [Mooncake] Fix ProcessGroup API imports (#32403) 2026-08-03 01:16:41 -07:00
amd-oshkarav 92999d84f4 [AMD]Qwen3.5 integration gfx950 fmha fp8 hd256 (#32046) 2026-08-03 00:25:31 -07:00
Liangsheng Yin 45c00daa1b [misc] Deep-merge nested config overrides and parse request bodies with orjson (#33351) 2026-08-02 23:55:06 -07:00
Lewis百麒David Orman
 		204e0fbac0 [SM120] Only split touched SWA pages in FlashMLA page-split kernel (#32320) 
Co-authored-by: 百麒 <yaozhong.lyz@alibaba-inc.com>
Co-authored-by: David Orman <ormandj@corenode.com>
 2026-08-02 23:45:07 -07:00
Wang, FangYuanandBingxu Chen 21d930aae3 [AMD] Fix JIT compile failure in sgl_kernel/warp.cuh (#33195) 
Co-authored-by: Bingxu Chen <bingxche@amd.com>
 2026-08-02 23:35:29 -07:00
Chengyu LinandRain Jiang e00f32ed4f feat: rust sglang server openai apis (#33103) 
Co-authored-by: Rain Jiang <rain-jiang@outlook.com>
 2026-08-02 23:13:44 -07:00
Baizhou Zhang 0bf0640b9d [CP] Support breakable CUDA graphs for zigzag strategy (#33136) 2026-08-02 23:11:57 -07:00
YAMYandSam Li 5fe97637df Support DeepGEMM for standard MoE dispatch (#33128) 
Co-authored-by: Sam Li <lsam@nvidia.com>
 2026-08-02 21:48:13 -07:00
Liangsheng Yin f5f021672a [Fix] Treat an empty grammar constraint as unset in SamplingParams (#33328) 2026-08-02 21:45:51 -07:00
Xiaoyu Zhang b64fd800d4 docs(diffusion): update skills for MiniMax-H3 (#33282) 2026-08-03 12:44:01 +08:00
Qiaolin Yu b1754a8f3b [gdn] fused replayssm ring write into flashinfer gdn mtp verify kernel (#33102) 2026-08-02 21:31:22 -07:00
Cheng Wan b8109b5d63 config: retire the last process-global config field reads (#33338) 
`get_server_args().<field>` reads one process's startup record. Nine sites still
did that for a value that has a namespace: the attention backend (5),
`skip_tokenizer_init` (2), the draft-aware `load_format`, and a chunked-prefill
size in `sglang.kernels`. They now read `get_exec().kernel` / `get_serving()` /
`get_model()` / `get_schedule()`, so they see the resolved value including
post-publish overrides.

The multimodal processor's device selection moves to the instance it was
constructed with rather than to a namespace: `base_gpu_id` differs per worker (the
encode-server DP workers each specialise their own copy), so no process-global
value can stand in for it, and engines sharing a tokenizer process each need their
own. Branch order, the NPU preprocess patches, and the case that leaves "device"
unset are unchanged.

What stays on `get_server_args()` is the derived API — `@property` and method
members computed from several fields plus the HF config
(`mamba_cache_chunk_size`, `get_model_config()`, `enable_mamba_extra_buffer*`) —
plus three config-intent reads of live-shadowed sizes, each of which needs an
answer the live topology property cannot give (the DSA indexer's PP gate must
short-circuit before touching the PP group, `allocation`'s DCP gate asks whether
DCP was configured at all, and the CUDA-IPC recycler runs where no group exists).

A new AST ratchet pins both shapes it can see — the direct call and an alias
bound from it in the same function — at 0 and 12 respectively, exempting the
derived APIs and those three sites by name. The alias-form baseline is not zero:
those reads are mostly per-runner fields in model code, and lowering them is the
next slice.

Two fixtures stopped faking config: `test_dllm_fdfo_kv_reuse` rebound
`allocation.get_server_args` to a SimpleNamespace, which silently stops
intercepting the moment a reader migrates; it publishes a real config instead.
 2026-08-02 21:24:42 -07:00