minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
15,932 Commits 2 Branches 0 Tags
df55e911d6212e7ebdd66cb6738b88fdd1040ef2
Commit Graph
10894 Commits
Author SHA1 Message Date
Cheng Wan df55e911d6 Fix the chunked-prefix-cache gate writing config the backends never read (#33168) 
The load-time gate (maybe_disable_chunked_prefix_cache) wrote its
ServerArgs instance while every reader has moved to the published
config: the attention backends assert / branch on
get_schedule().disable_chunked_prefix_cache when they initialize, so the
flip never reached them and a backend outside
CHUNKED_PREFIX_CACHE_SUPPORTED_ATTENTION_BACKENDS kept chunked prefix
enabled.

Reroute the writer through get_context().override (which writes the
published bags) and flip the two remaining instance reads — the gate's
own log check and the prefill cuda-graph runner's capture flag — to the
bag. A regression test pins the three contracts: the gate lands on the
bag, the pristine ServerArgs instance stays untouched, and the
draft-worker guard never writes.

The ServerArgs.override call-site ratchet drops 39 -> 38.
 2026-08-01 08:57:28 -07:00
JemmaFan 2fd78ec2d7 [AMD] Derive AITER verify tokens-per-req from input shape (#31221) 2026-08-01 00:29:25 -07:00
Sugar920Claudesglang-npu-botCherry_ming
 		33ecf4bcd8 Add pr tests (#31952) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: sglang-npu-bot <sglangnpu@163.com>
Co-authored-by: Cherry_ming <136634645@qq.com>
 2026-08-01 15:03:21 +08:00
Sugar920 fd96a35fb0 add NPU GSM8K accuracy tests for 7 models (#32649) 2026-08-01 14:22:57 +08:00
Mohammad Miadh Angkadandhnyls2002 c0d06a6547 [CI] Fix runtime context setup in flat logprob tests (#33179) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-07-31 22:46:24 -07:00
Xiaoyu ZhangClaude Opus 5hnyls2002zhangxiaolei
 		fb207b72b0 feat(kernels): port standalone Kimi K3 kernels (#32890) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: zhangxiaolei <zhangxiaolei.666@bytedance.com>
 2026-08-01 13:26:47 +08:00
jacky.cheng 0d186f49be [AMD][Fix] Restore aiter-padded MoE weight dims for serialized checkpoints (#33090) 
AMD only change
 2026-07-31 22:09:37 -07:00
alumkalqiuyuexuebiXiaoyu Zhang
 		bae8eb8d6c [minimax-m3] fp8 attention GEMMs on SM100 (fp8_e4m3 KV + trtllm_mha) (#30971) 
Co-authored-by: qiuyue <qiuyue@minimaxi.com>
Co-authored-by: xuebi <xuebi@minimaxi.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-08-01 09:39:52 +08:00
Andrew Tulloch e6a4cefc69 perf(startup): skip unused PyTorch headers for KV VMM allocator stub (#33126) 2026-07-31 18:32:36 -07:00
Cheng Wan 934a13ce3e [Inkling] Hold the short-conv per-step state on one metadata struct (#33116) 2026-07-31 18:11:35 -07:00
Sam Shleifer 58974ca16c [perf] Assemble flat prompt top logprobs scheduler-side as numpy arrays (#32223) 2026-07-31 18:00:51 -07:00
Liangsheng Yin ca07917c58 [Fix] Bound FULL_MASK verify-mask reuse by the captured max_bs (#33127) 2026-07-31 17:50:40 -07:00
Khoa Pham 1496bfee93 [Kimi] Support DCP + DSpark (ported from kimi-k3 branch) (#32828) 2026-07-31 17:39:00 -07:00
Yuwei AnandClaude Fable 5 3e0f7c3f30 [BCG][3/N] Enable bcg on dsa & deepep a2a backend (#31987) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-31 16:45:18 -07:00
Ethan (Yusheng) Su 3c5f115741 Split #32584 into 2/2: [LoRA] Shard attention LoRA by attn-TP and allow dynamic LoRA with dp attention (#32708) 2026-07-31 15:37:13 -07:00
Cheng Wan 55b6769b0e config: read resolved config via namespace accessors (#33013) 2026-07-31 15:06:59 -07:00
Cheng Wan 4862edc85f runtime_context: record the publishing process role (#33012) 2026-07-31 15:06:22 -07:00
Cheng Wan ab2553284a config: preserve resolved config across nested publishes + mutation ratchets (#33011) 2026-07-31 15:05:41 -07:00
Xinyuan Tong 4480e2a051 [Fix] Repair verify mask test fixture (#33087) 2026-07-31 14:48:30 -07:00
Ethan (Yusheng) Su 7e996a5d0d Split #32584 into 1/2: [LoRA] Guard DP-attention idle forwards against stale LoRA batch state (#32707) 2026-07-31 14:23:01 -07:00
Po-Han Huang (NVIDIA) 5df193b4ac [Speculative Decoding] Fix GPT-OSS EAGLE3 hidden states (#32334) 2026-07-31 11:58:49 -07:00
Rain Jiang 4af8ddb576 support rust sglang server (#29799) 2026-07-31 11:56:31 -07:00
Cheng Wan 77c77a3da8 feat(inkling): migrate short convs onto the ShortConv attention backend (#33023) 2026-07-31 11:52:12 -07:00
Cheng Wan d3222bcc3a [unified-memory] Support fa3, the default MLA backend on pre-Blackwell hosts (#33046) 2026-07-31 11:46:46 -07:00
luchangli 26486a957d Fix --hicache-size allocating ~2x host memory on hybrid Mamba (#32915) 2026-08-01 02:37:57 +08:00
Nan Jiang 89f4a80c1f Support fastsafetensors no-GDS loading and page-cache release (#31859) 2026-07-31 23:12:32 +08:00
Danila ShtanandDanila Shtan 5f9b0db18c Fix async loading of RunAI-streamed tensors (#32896) 
Co-authored-by: Danila Shtan <dan@nebius.com>
 2026-07-31 21:46:33 +08:00
luoroger37andXinyuan Tong 690de097c4 [fix]reject media input for text-only models (#32914) 
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-07-31 12:43:25 +00:00
Peng Wu e3d4f48e55 [Fix] missing max_context_len on HybridAttnBackend (#32690) 2026-07-31 19:43:09 +08:00
MickandClaude Fable 5 754b692afc [diffusion] optimization: support cuda-ipc zero-staging all-to-all for 2-rank Ulysses (#31854) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-31 19:35:48 +08:00
Khoa PhamandYangmin Li 2573190b93 feat: support Kimi Linear PD disaggregation with DCP (#32837) 
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
 2026-07-31 02:14:09 -07:00
Cheng Wan 33c27d8e7f [unified-memory] Let Kimi-Linear use the paged MLA attention backends (#32972) 2026-07-31 01:32:08 -07:00
Ziang Li 937c77cf50 [Fix] Clear stale FlashInfer BF16 MoE index cache (#33016) 2026-07-31 00:35:15 -07:00
Kangrui Du 585a7d05e3 [Diffusion] Return scheduler sigmas snapshot in rollout dit_trajectory (#32683) 2026-07-31 00:29:07 -07:00
Brayden Zhong f94d2c5663 [Fix] Restore online MXFP8 quantization for linear layers (#32953) 2026-07-31 06:42:06 +00:00
Liangsheng YinandKaixi Matteo Chen 5c6635d8f3 [Spec] Compact the target-verify mask when nothing reads it (#32920) 
Co-authored-by: Kaixi Matteo Chen <kaiximatteoc@nvidia.com>
 2026-07-30 23:13:21 -07:00
Shijin ZhangandXinyuan Tong 09193bf36f [Fix]: render tool_reference schema regardless of tool_result part order (#32522) 
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-07-31 13:57:52 +08:00
Cheng Wan e23ccb15f0 [unified-memory] Support MLA-hybrid-Mamba (Kimi-Linear) on the Triton backend (#32971) 2026-07-30 22:10:34 -07:00
Cheng Wan 06ccaef24a Fix silently wrong EPLB output with --moe-a2a-backend none (rank-invariant dispatch) (#32962) 2026-07-30 22:10:04 -07:00
Qiaolin Yu f3fd869494 [gdn] support replayssm with extra buffer (#32692) 2026-07-30 21:34:37 -07:00
DAI0818andXiaoyu Zhang afeaeccfa2 perf(hisparse): eliminate redundant swap output fill (#32483) 
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-07-31 12:09:44 +08:00
zky 425349b799 [Perf][DSA] Pass topk_length to flash_mla_sparse_fwd in the sparse attention path (#31128) 2026-07-30 20:25:59 -07:00
Ziang LiandParth Chadha 0aefba7283 fix(dsa): correct packed FlashInfer top-k and backend selection semantics (#32490) 
Co-authored-by: Parth Chadha <parth@humansand.ai>
 2026-07-30 20:20:16 -07:00
Mick a149717308 feat: log multimodal encoder DP tradeoffs (#30903) 2026-07-31 08:50:20 +08:00
weireweireandweireweire 55c1963df4 Remove unused draft-extend CUDA graph top-k (#31430) 
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
 2026-07-30 17:45:01 -07:00
Xinyuan Tong 68d442945f Flush dropped reasoning at stream end when stream_reasoning=False (#32225) 2026-07-31 08:25:54 +08:00
YAMY 48dbc24cbf [Qwen3.5][MTP] Support FlashInfer CuTe DSL for online NVFP4 draft MoE (#31382) 2026-07-30 17:19:48 -07:00
Trang DoandCheng Wan a1c30701aa Integrate pplx a2a backend (#30756) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2026-07-30 15:33:19 -07:00
Mohammad Miadh Angkad 3a53c26c27 [CI] Fix MoE compile and DSA indexer regressions (#32937) 2026-07-30 15:21:51 -07:00
Trevor Morris a6221d776f feat: Support nvidia/MiniMax-M3-NVFP4 (#31989) 2026-07-30 14:32:03 -07:00