This website requires JavaScript.
a50c17e979
[diffusion] perf: fuse the Wan VAE conv bias epilogue and tile the RMSNorm+SiLU kernel (#38650 )
Dayuxiaoshui
2026-10-15 13:39:27 +08:00
676c038aa5
[diffusion] fix: fix the load duration missing from subclassed server tests (#40657 )
2026-10-14 22:38:23 -07:00
7e20f0af79
gateway g9i: P2 T_D admission gates (off|shadow|enforce) + T_PREFER candidate fallback (T9)
minke.yu
2026-10-15 13:35:50 +08:00
f9bdb74a5a
[Fix] scheduler: Count a request's held req_to_token row once at admission (#44401 )
Liangsheng Yin
2026-10-14 22:29:10 -07:00
93842f52fe
[CPU]: Add group-aware CPU SHM collective kernels (#35435 )
Yuxingwang-intel
2026-10-15 13:22:20 +08:00
4dc9b5c68c
[Test] [AMD] diffusion: Expect CUDA-only fast paths to be off on ROCm in kernel tests (#44311 )
Michael
2026-10-14 22:18:01 -07:00
1d809bbbbd
[Fix] DSv4.1: Size the encoder-replay SWA pool by the DSpark draft's stages, and keep more prefix tails (#43502 )
2026-10-14 22:04:58 -07:00
e573d91275
[CI] tests: Fix stale fakes in the CP strategy and request-class override unit tests (#44380 )
kk and root
2026-10-15 12:35:20 +08:00
d4daf52369
[Fix] dsv41: Bound MoE routing to live SWA graph-tail rows (#44347 )
Khoa Pham
2026-10-14 21:14:28 -07:00
08f15e76bb
[Feat] sgl-router: Add --dp-rank-policy round_robin for --dp-aware (#43511 )
AMD-yanfeiwang
2026-10-15 12:03:01 +08:00
c92108bb93
[diffusion] comfyui: grouped SGLDOptions, native model loader and model-aware preflight (#44058 )
WenhaoZhang and Mick
2026-10-15 11:59:03 +08:00
3b5ae4ea86
[Fix] scheduler: Drop finished requests from the running batch before every admission (#44351 )
Liangsheng Yin
2026-10-14 20:40:05 -07:00
2528cd2b91
[PD] Send DSA indexer state only past decode_prefix_len (#44140 )
2026-10-14 20:25:18 -07:00
0cf683ea8d
[diffusion] comfyui: return several server-mode images as one IMAGE batch (#44088 )
Avaya Aggarwal and niehen6174
2026-10-15 08:08:37 +05:30
4851d182a1
[Doc][AMD] Update v41 amd cookbook 1015 (#44368 )
Thomas Wang
2026-10-15 10:33:08 +08:00
a12814c158
[Perf] [AMD] dsa: Enable the gfx950 fused DSA indexer for block-FP8 wq_b/wk (#43643 )
xiaofei-zheng
2026-10-15 09:36:16 +08:00
9fa9308e95
[Doc] cookbook: Add Step 5 Preview cookbook (#44357 )
Yuhao Yang and Claude Opus 5.5
2026-10-15 09:21:06 +08:00
9d68ecce2e
[DeepSeek-V4.1] DeepSelect cluster top-k for long rows and bf16 index logits (#44051 )
2026-10-15 09:00:19 +08:00
c5c4bbbd25
[Perf] [AMD] dsa: Run DSA prefill under breakable CUDA graphs on HIP (#43642 )
xiaofei-zheng
2026-10-15 08:59:34 +08:00
8333ccfccb
[Fix] [AMD] diffusion: Compare ReduceOp by identity so the compiled TP all-reduce stays one graph (#43770 )
Michael and jacky.cheng
2026-10-14 17:40:07 -07:00
64a0a08f0a
[DSv4.1] Support TRT-LLM decoder bounded replay with BCG (#44326 )
Yuwei An
2026-10-14 17:39:52 -07:00
c891c5c7ca
[DSv4.1] Fold encoder-SWA replay into the extend forward (#43166 )
YAMY and Yuwei An
2026-10-14 17:27:07 -07:00
3c28614f12
[Fix] hicache: Defer query beliefs until the rank-synchronized hit drain (#43108 )
Shuwen Wang
2026-10-15 08:25:08 +08:00
a6a8c56015
[Fix] detokenizer router: Route by rid without tokenizer worker ipcs and broadcast logging config (#33715 )
Aditya Singh and hnyls2002
2026-10-14 17:20:21 -07:00
0536cec73d
[Feat] metrics: Expose multi-detokenizer router CPU usage (#43323 )
2026-10-15 05:47:22 +05:30
0f36c8015c
[Bug] Fix multi-tokenizer BatchEmbeddingOutput repack dropping fields (#31892 )
Junnan Li and hnyls2002
2026-10-14 16:56:56 -07:00
209f8220d6
[Fix] sglang-server: Honour extra_key on the Rust frontend's /generate (#44207 )
2026-10-14 16:44:31 -07:00
e7d6a88bfc
[HiCache] Buffer-only L3 for an auxiliary SWA window; load back only missing window rows (#43932 )
2026-10-14 16:43:24 -07:00
aa8793f33b
[Rust] Serve scheduler metrics on the Rust frontend's /metrics (#43942 )
2026-10-14 16:43:06 -07:00
2a9f6a25a7
[Core] Add subclass hooks for the generation request class and optimistic prefill eligibility (#44113 )
2026-10-14 16:42:30 -07:00
5f2861bf59
[Perf] session: Cache multimodal prompt token counts across turns (#44112 )
2026-10-14 16:41:34 -07:00
9732188a58
[Rust] Serve /weights_checker from the Rust frontend (#43926 )
2026-10-14 16:41:10 -07:00
97d3150a6a
[Perf] Cache msgspec struct fields in the batch splitter (#43520 )
FREETRUMP and hnyls2002
2026-10-15 05:51:26 +08:00
4273814771
[Perf] detokenizer: Group return IPC sends by tokenizer worker (#44288 )
2026-10-15 05:26:51 +08:00
d3dc14eb38
[mem_cache][PD] Fix decode radix cache ownership and allow EAGLE/EAGLE3 with decode radix cache (#40857 )
Charles Chen and alphabetc1
2026-10-14 14:19:51 -07:00
12dc623102
[Chore] rust: Remove the sglang-renderer crate (#43451 )
Rain Jiang
2026-10-14 11:43:37 -07:00
b67785153a
[Fix] glm5.2: Align HiSparse warmup backends and move test to nightly (#44303 )
Shuwen Wang
2026-10-15 01:28:03 +08:00
87ed1b9526
[4/N] elastic-ep: Scale at DP-attention replica granularity (#33728 )
Yoray Zack
2026-10-14 20:17:01 +03:00
c964d9ba20
[Feat] sgl-router: Separate LoRA block IDs from prompt lookup keys (#42818 )
2026-10-14 09:50:19 -07:00
3a9cae27ed
[Doc] [NPU] docs: Update NPU CANN version to 9.1.0 (#44177 )
amote-i
2026-10-15 00:27:18 +08:00
1737251730
[Test] [AMD] diffusion: Run the IPC all-to-all transport tests only on CUDA (#44096 )
Michael
2026-10-14 08:04:23 -07:00
010776e94f
[KV events] Publish opaque LoRA block IDs and adapter names (#42817 )
2026-10-14 07:56:28 -07:00
ed40c7fc66
[Doc] cookbook: add Ling-3.1-flash (#44301 )
Xinyuan Tong
2026-10-14 22:40:01 +08:00
0b74ab2ac4
gateway g9i: shadow-consistent routing switch --g9i-shadow-route (P1 shadow T7)
minke.yu
2026-10-14 22:17:14 +08:00
3c7ec33d26
gateway g9i: est_T_D shadow metrics — est_tp/est_td histograms, residual by dominant component, pool gauges (P1 shadow T5)
minke.yu
2026-10-14 21:52:07 +08:00
09ca4957ee
gateway g9i: scrape P /metrics kv_transfer_bootstrap_ms as est_T_D ground truth (P1 shadow T4)
minke.yu
2026-10-14 21:50:11 +08:00
7921442a5c
gateway g9i: est_TTFT max(T_P, min_D T_D) composition with C4/C13 degradation (P1 shadow T3)
minke.yu
2026-10-14 21:44:17 +08:00
05ef104208
gateway g9i: pool_free_rate completion EWMA per decode node (P1 shadow T2)
minke.yu
2026-10-14 21:34:56 +08:00
d44620f285
gateway g9i: fetch disagg/queues sections in /v1/loads, extend NodeLoad for est_T_D (P1 shadow T1)
minke.yu
2026-10-14 21:21:11 +08:00
d2a436446c
[Feat] sgl-router: Fix cache-aware routing for MTP/EAGLE on hybrid (Mamba/KDA) models (#44252 )
Kan Wu and Claude Opus 5.5
2026-10-14 06:21:01 -07:00
204644c2dc
[Fix] sglang-processor: Follow Python's OpenAI layer through #43027 , #37577 , #43006 , #43026 , #43005 (#44243 )
2026-10-14 06:06:13 -07:00
5e90312eb1
[diffusion] perf: support bit-exact norm/bias fusions and channels_last decode for Qwen-Image 2.1(#41340 )
triple-mu
2026-10-14 20:47:46 +08:00
7b53a13401
[Fix] glm5.2: Align PD HiSparse prefill KV layout (#44256 )
Shuwen Wang
2026-10-14 20:40:28 +08:00
bab9c81fea
fix: FA4 paged-KV zero-size page-table registers on SM90 (#35757 )
2026-10-14 04:10:15 -07:00
3b72b70ef4
[CI] vlm: Pin baidu/Unlimited-OCR revision in test_unlimited_ocr_server (#44235 )
Po-Han Huang (NVIDIA) and Claude Opus 5.5
2026-10-14 17:50:18 +08:00
c82e8bd5fe
[Fix] DeepSeek-V4.1: Align paged indexer score rows for DeepSelect decode (#44210 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-10-14 17:31:19 +08:00
df1a089cd3
[Spec] Fix DFLASH verify mask policy for wrapped attention backends (#43054 )
2026-10-14 17:15:36 +08:00
817781b828
[Feat] sgl-router: Render Qwen3.8 (Max, 27B, Flash-Next) through sglang-processor (#43689 )
2026-10-14 02:09:35 -07:00
72730d6360
[Doc] cookbook: Mirror SA cluster NCCL_CUMEM_ENABLE in AgentX launches, fix GPU counts (#44228 )
Po-Han Huang (NVIDIA) and Claude Opus 5.5
2026-10-14 16:07:40 +08:00
c54ba66cd4
[Perf] Enable native MXFP4 routing and CuTeDSL fusion for MiMo (#43639 )
2026-10-14 15:39:44 +08:00
6daae9c98e
[Perf] Bind scheduler CPUs to the GPU's NUMA node when memory policy is not permitted (#44023 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-10-14 15:01:32 +08:00
58f2c923ed
[Feat] sgl-router: skip engines at an admission cap in power-of-two; forward x-sglext headers (#44218 )
2026-10-13 23:59:06 -07:00
53a24c448e
[Feat] sgl-router: Render Kimi-K3 through sglang-processor (#43608 )
Kan Wu and Claude Opus 5.5
2026-10-13 23:40:18 -07:00
6f646563a2
[AMD] diffusion: expect the residual-gate fast path to be off on ROCm in test_modulate (#40660 )
Michael
2026-10-13 23:34:31 -07:00
68c18a5701
Revert "[XPU] Support Gemma-3 bidirectional image attention in XPUAttentionBackend" (#44216 )
Ma Mingfei
2026-10-14 14:27:46 +08:00
b87d62e581
[XPU] Support Gemma-3 bidirectional image attention in XPUAttentionBackend (#39805 )
desmond-intel
2026-10-14 11:54:04 +05:30
0d130f1915
[Qwen3.8 CP 4/4] Prefill context parallel E2E support of Qwen3.8 (#39724 )
Shunkangz
2026-10-14 14:19:46 +08:00
7ca4cbfcef
[Disagg] Return prompt routed experts from the PD prefill server (#44002 )
2026-10-14 14:13:44 +08:00
05bb5afb62
[NPU][Bugfix] use CANN libdevice for DSpark Markov sampling (#43849 )
AndyLi429 and AndyLi429
2026-10-14 14:12:22 +08:00
e29eee18df
[AMD][DI][CI] nightly mi355x disagg: stage packed logs under RUNNER_TEMP, not /tmp (#43945 )
Zhaoyi Li
2026-10-14 00:53:40 -05:00
4560da58ee
[sgl-router] Render GLM-5.3 and GLM-5.3-Flash through sglang-processor (#43625 )
Kan Wu and Claude Opus 5.5
2026-10-13 22:19:44 -07:00
d5f80aa3c3
[Test] dsv4: Fix Hopper indexer replay test after DeepSelect decode (#41561 ) (#44193 )
Po-Han Huang (NVIDIA) and Claude Opus 5.5
2026-10-14 13:15:35 +08:00
30bdd09aed
DSPARK enable for XPU (#33739 )
ANSHUMAN TRIPATHY
2026-10-14 10:40:24 +05:30
721953c6f6
[Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main (GLM-5.2 NVFP4 B300 agg TP4, Dynamo frontend counts) (#44065 )
Po-Han Huang (NVIDIA) and Claude Opus 5.5
2026-10-14 13:03:51 +08:00
e00754bab1
[DSV4.1] Derive the late-layer DP geometry from the captured shape under the breakable prefill graph (#44144 )
Khoa Pham
2026-10-13 21:54:35 -07:00
ff8d964cf5
[CI] glm5.2: Temporarily disable PD HiSparse+MTP B300 test (#44192 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-10-14 12:53:58 +08:00
491d4ef178
[MLU][CI] Bootstrap MLU runner workflows (#41005 )
chenxb002 and chenxiaobing
2026-10-14 12:40:49 +08:00
2c1e85964e
[diffusion] cache-dit: respect CFG parallel for dual-transformer models (#43704 )
Chetan Kumar Verma and Claude Opus 5.5
2026-10-14 09:55:07 +05:30
2428d0f531
[XPU] Use the shared fused decode-metadata kernel in the intel_xpu ba… (#42122 )
xinchen9 and Ma Mingfei
2026-10-13 21:10:12 -07:00
16176278d0
[diffusion] perf: replay small Triton wrappers' launches per operand signature (#43859 )
2026-10-14 11:18:19 +08:00
4f751da975
[Agents] Add a rule to prefer sgl-eval for accuracy evals (#44040 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-10-14 11:17:04 +08:00
53fe21cdfd
[diffusion] perf: replace padded rows without a host sync for Z-Image (#44069 )
2026-10-14 11:15:28 +08:00
7e226b1b71
[AMD] Use the preallocated EAGLE topk=1 draft chain on HIP (#43640 )
xiaofei-zheng and Xiaoyu Zhang
2026-10-14 11:06:07 +08:00
5efb7384c1
[diffusion] comfyui: consolidate 6 unrelated plugin fixes (#43493 )
2026-10-14 08:26:09 +05:30
c053fc71d3
[CI] pr-title: Validate pull request title format (#44165 )
Cheng Wan
2026-10-13 19:53:52 -07:00
4f5fcbfef5
[JIT] Clean up kernel headers: move runtime helpers, add once.h (#41147 )
Ziyi Xu and Claude Opus 5.5
2026-10-14 10:46:51 +08:00
b50fdeb0ef
[DeepSeek V4.1] Use DeepSelect for Hopper full top-k decode. (#41561 )
yuyu5333 and Claude Code
2026-10-14 10:46:12 +08:00
87beb3a052
[CPU] Temporarily disable subblock sparse attention test on Xeon CI (#43468 )
Ma Mingfei and MingxuZh
2026-10-14 10:44:24 +08:00
13a6dee99b
[Fix] moe_sum_reduce: gate the bf16 vectorized path on hidden % 16 (#42837 )
2026-10-13 22:32:50 -04:00
7f0434e4af
[GLM-5.3-Flash] Drop the NextN layer from the KDA state pool (#44098 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-10-14 10:31:02 +08:00
ac187c0cd5
[diffusion] Bound frame_interpolation_exp to RIFE's supported range (#43014 )
Avaya Aggarwal
2026-10-14 07:59:28 +05:30
44b44774c3
[Bugfix] Keep rope_theta when --json-model-override-args overrides rope_scaling (#43067 )
Jin and Xinyuan Tong
2026-10-14 10:16:35 +08:00
536d065370
[CP] Automatically enable BCG for supported prefill CP configurations (#44022 )
Baizhou Zhang
2026-10-13 19:00:42 -07:00
684fea8af3
[DSV4.1] Skip Mega mHC fusion on 0-row (DP-idle) batches (#43993 )
shyeh25 and Claude Opus 5.5
2026-10-14 07:36:15 +08:00
2ebdf9360b
[Fix] Work around a CPython fork deadlock in the CPU preprocess pool (#44143 )
Liangsheng Yin
2026-10-13 16:31:57 -07:00
b3991b53da
GLM5.2 DSA Indexer Fallback (#32947 )
2026-10-13 18:27:33 -05:00
5917604a95
[DSV4.1] Support decoder SWA bounded replay with CUDA DP attention (#43812 )
Khoa Pham
2026-10-13 16:10:57 -07:00
a832609b92
[Feature] Return output top logprobs as flat base64 arrays on request (#42890 )
HEJIAN SANG and Claude Opus 5.5
2026-10-13 15:58:31 -07:00
baa632b049
[Score] Send stacked GPU embedding overrides through a CUDA IPC pool (#39730 )
2026-10-13 15:46:50 -07:00
08550acc28
Mark FP32 redux asm volatile (#44137 )
Andrew Briand
2026-10-13 17:08:52 -05:00