This website requires JavaScript.
c6e822c29b
[Fix] Adapt NPU Qwen-VL processor patch to transformers 5.19 resize() signature change (#43309 )
ming_wang
2026-10-12 19:24:05 +08:00
7953751bfa
[PD][Spec] Give a PD decode's first EAGLE draft step a proposal distribution (#41783 )
Sasha Sidorov and Shangming Cai
2026-10-12 03:49:52 -07:00
8afe107c24
[XPU] Upgrade XPU backend to PyTorch 2.14 (#42003 )
2026-10-12 15:54:48 +05:30
57db2a4ae9
gateway g9i: fail-closed 429 on probe failure / fast-P snapshot loss (p2 incident)
minke.yu
2026-10-12 10:16:09 +00:00
28621739fc
[Fix] Make tokenizer request-state initialization atomic (#41911 )
C and Cheng Wan
2026-10-12 18:10:57 +08:00
180933518e
[sgl-router] Serve the Anthropic Messages API on /v1/messages (#42316 )
2026-10-12 17:56:08 +08:00
f006fb49ec
[NPU] Stabilize HiCache and registered CI tests (#43308 )
Jinyan Yi
2026-10-12 17:10:26 +08:00
b0c84d4518
[Docs] Cookbook: refresh SemiAnalysis AgentX launches (GLM-5.2 B300, DeepSeek-V4.1 GB300) (#43799 )
Po-Han Huang (NVIDIA) and Claude Opus 5.5
2026-10-12 17:08:11 +08:00
450837e770
[Docs] Update GLM-5.3-Flash playground: B300 default, CP8, EP and PD (#43802 )
Baizhou Zhang
2026-10-12 01:38:12 -07:00
62ff466ac0
[DSV4.1][PD][1/N] Support prefill node use cp parallel strategy (#40323 )
BingjiaWang
2026-10-12 16:05:21 +08:00
54409c2fa0
[BCG][DSV4-Flash][CP] Support DeepSeek V4 prefill with interleave CP and breakable CUDA graphs (#38603 )
BingjiaWang
2026-10-12 16:03:10 +08:00
8d491fbc3e
[XPU] Support decode context parallelism (DCP) on Intel XPU (#34355 )
Anupa Sajikumar and Siju Samuel
2026-10-12 13:20:33 +05:30
a70a83bcd2
Keep NIXL Mamba checkpoint keys separate by TP rank (#43594 )
Ke Bao
2026-10-12 15:40:59 +08:00
b6ba148384
Capture KV host transfer buffers (#43711 )
Ke Bao
2026-10-12 15:40:06 +08:00
06887827c7
[CI] Move tests that do not need GB300 off the per-PR GB300 suite (#43784 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-10-12 15:38:10 +08:00
5ca0695aa6
[HiCache] Preserve all MTP KV and recurrent state (#40223 )
2026-10-12 03:37:59 -04:00
f5f5003323
[HiCache] Support Mamba in buffer-only host memory mode (#36345 )
Shuwen Wang
2026-10-12 15:35:16 +08:00
4195980f5a
[PD] Poison the deferred abort ack when a Mooncake send fails (#43719 )
Shangming Cai and Claude Fable 5.1
2026-10-12 20:33:09 +13:00
da4bf1ef8e
[Perf] Route BF16 mixed-batch decode tails through decode attention (#43238 )
2026-10-12 00:27:22 -07:00
a0e6effe50
[Fix] Link cuda_ipc JIT against CUDA driver stubs dir (#43764 )
Xinyuan Tong
2026-10-12 15:26:32 +08:00
62c853b654
docs: clarify prefill CP terminology and simplify wording (#43787 )
Baizhou Zhang
2026-10-12 00:24:57 -07:00
208df292e5
[diffusion][xpu] FLUX.2: enable the bit-exact Triton SwiGLU on XPU (#41940 )
2026-10-12 00:16:58 -07:00
166b4ae7fc
[Fix] Adapt to transformers 5.18 / 5.19 changes (#43688 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-10-12 14:24:57 +08:00
8ce76b88df
[XPU CI] Optimize PR CI per-file timeouts and fail hanging tests fast (#42048 )
ashwini rathi
2026-10-12 11:54:26 +05:30
130e11d50e
[Deps] Bump FlashInfer to 0.7.1 (#43494 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-10-12 14:23:11 +08:00
f9bd83e7a9
docs: sync LMSYS SGLang blog cards (#43462 )
sglang-bot and sglang-bot
2026-10-11 23:07:15 -07:00
9e0b001db4
[CI] Skip eval retries in the PD transfer-failure test (#43769 )
Liangsheng Yin
2026-10-11 22:37:40 -07:00
2240049194
[RL] Return a failed weight update when the cache flush is refused (#43758 )
Liangsheng Yin
2026-10-11 21:57:58 -07:00
a297cd3da1
[Fix] scheduler: stop latching batch_is_full when nothing can run (#41872 )
2026-10-12 12:56:19 +08:00
bf703013f2
[Test] Re-anchor the VLM flashinfer throughput floor (#43762 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-10-12 12:34:50 +08:00
d434ea444d
[DSV4.1] Reduce eager Engram extend preparation (#42159 )
2026-10-11 21:31:55 -07:00
a1001bcf3a
[PD] Retract transferred decode requests on a retract pause (#43757 )
Liangsheng Yin
2026-10-11 21:29:39 -07:00
0d0be8fc98
[AMD][MORI] Add MORI EP V2 support (#40666 )
ChengYM
2026-10-12 11:54:13 +08:00
20c49f12c9
Support kv cache sharding for DSA indexer (#40925 )
Shunkangz
2026-10-12 11:50:23 +08:00
37d2e5c34f
[AMD][V4.1][*/N] Fix DP attention + TP MoE decode NaN and support bounded replay (#43077 )
2026-10-12 11:47:13 +08:00
296774da37
[Tokenizer] Require fastokens>=0.3.2 and include it in the all extra (#43104 )
weireweire
2026-10-12 11:41:53 +08:00
cd8ed1bb55
[Bugfix] Preserve available host memory for integrated MUSA devices (#43398 )
adenzhou1350
2026-10-12 11:34:58 +08:00
6ac8602e8d
[CI] Fix TestQwenImageModulationCache after Qwen-Image modulation sharding (#43747 )
Po-Han Huang (NVIDIA) and Claude Opus 5.5
2026-10-12 11:31:29 +08:00
77a15cf4e2
[MemCache] Add shared component size and path APIs to the tree core interface (#43722 )
2026-10-11 20:14:07 -07:00
95ce9caaf2
[Rust] Lock hyper 1.12.0 and want 0.3.2 in the rust workspace (#43724 )
2026-10-11 20:12:55 -07:00
e4cfc0c42a
[MemCache] Split the SWA mapping move out of locked-full recovery as RecoverEvictedSWA (#43721 )
2026-10-11 20:11:52 -07:00
0c52cb1519
[MoE] Skip block-padding rows in the TMA down activation and quant (#42212 )
WangHemeng
2026-10-12 05:10:30 +02:00
1a7af3290d
Fix Mistral-native format detection when the Hub is unreachable (#43446 )
Alison Shao
2026-10-11 19:59:19 -07:00
3c74f68a9c
[Fix] Initialize BF16 GEMM dispatch in the one-batch benchmark (#43702 )
Chunan Zeng
2026-10-11 19:53:02 -07:00
7fd7fd58a5
gateway g9i: hit class always routes to fast-P pool (命中都在 fast-P 发生)
minke.yu
2026-10-12 02:20:12 +00:00
d0a098b2e0
[diffusion] perf: split and batch LTX-2.3 one-stage guidance passes under CFG parallel (#43684 )
WenhaoZhang and Mick Qian
2026-10-12 09:36:01 +08:00
a954b395e6
[diffusion] perf: layerwise offload for the LTX-2.5 Gemma-4 text encoder (#43683 )
WenhaoZhang
2026-10-12 09:34:35 +08:00
a67228aa43
[diffusion] fix: shard Qwen-Image adaLN modulation and text FFN under TP (#43691 )
WenhaoZhang
2026-10-12 09:32:52 +08:00
71a72b8fea
gateway g9i: keep large prefills off fast-P even without least-wait
minke.yu and Kimi
2026-10-12 08:58:03 +08:00
6c9518fcd8
gateway g9i: move ProbeHit to module scope (fix compile)
minke.yu and Kimi
2026-10-12 08:20:30 +08:00
3a4ab451ac
gateway g9i: classify on local (L1+L2) hits only, probe the whole pool
minke.yu and Kimi
2026-10-12 08:17:52 +08:00
5d38fc96a9
gateway g9i: route small prefills to the node that actually holds the prefix
minke.yu and Kimi
2026-10-12 08:01:10 +08:00
2ccceb5dcc
chore: update CI test est_time from recent run data
bot/update-est-time-20261012
sglang-bot
2026-10-12 00:00:31 +00:00
e6072935bf
[DSpark] Apply SGLANG_SIMULATE_ACC_LEN inside the accept instead of switching paths (#43623 )
Liangsheng Yin
2026-10-11 15:33:24 -07:00
a65b776963
[DSpark] Add SGLANG_DSPARK_USE_GREEDY_DRAFT, an argmax draft accepted as one-hot (#43670 )
Liangsheng Yin
2026-10-11 15:18:59 -07:00
3cc23b3b86
[DSpark] Merge the draft TP sync sites into dspark-draft (#43669 )
Liangsheng Yin
2026-10-11 15:17:47 -07:00
00ba90fd1a
[DSpark] Run the same markov block in the draft graph tail; drop SGLANG_DSPARK_FOLDED_SAMPLING (#43668 )
Liangsheng Yin
2026-10-11 15:15:59 -07:00
48abbdb845
[DSpark] Draft the eager markov block through MarkovSampleStep (#43622 )
Liangsheng Yin
2026-10-11 15:13:02 -07:00
467209e838
[DSpark] Add MarkovSampleStep, one op for a markov draft step's add and token pick (#43621 )
Liangsheng Yin
2026-10-11 15:10:58 -07:00
9943cf8eef
[DSpark] Run every markov head through one step interface (#43620 )
Liangsheng Yin
2026-10-11 15:07:51 -07:00
261ebbc55b
G9i: regular-P queue admission gate (est wait > 10s -> 429 regular_p_queue)
minke.yu
2026-10-12 00:08:34 +08:00
d62b9c2480
[KV Events] Publish LoRA adapter names and namespaced block hashes (#42455 )
Banruo Liu and Shangming Cai
2026-10-11 10:23:30 -05:00
97bfa5cc53
[diffusion] fix: select VSA-H3 for FastH3 when no transformer backend is given (#43690 )
2026-10-11 23:05:25 +08:00
46a97e5540
Preserve serialized page dtype in NIXL copy fallback (#43593 )
Ke Bao
2026-10-11 22:39:14 +08:00
b0a75c8196
Share MLA host capacity initialization (#43595 )
Ke Bao
2026-10-11 22:36:29 +08:00
c938b6e454
Build Mamba host pools from device buffer infos (#43596 )
Ke Bao
2026-10-11 22:35:52 +08:00
53fb9be7ee
G9i: TTFT p95/p99 statistics (observability only, no admission gate)
minke.yu
2026-10-11 21:42:21 +08:00
ef3e5c83c5
[HiCache] Keep storage workers alive on backend exceptions (#43550 )
Shuwen Wang
2026-10-11 22:26:56 +08:00
e5be84f795
G9i: probe classification across pool candidates (fast-P then regular)
minke.yu
2026-10-11 21:15:59 +08:00
11bfd868ac
[diffusion] perf: encode LTX video/audio through the shared bounded MP4 path (#43656 )
Mick and Mick Qian
2026-10-11 21:15:49 +08:00
0923bcd258
[Unified Radix] Invalidate lazy Mamba checkpoints before in-place tracking (#43567 )
Shuwen Wang
2026-10-11 21:12:51 +08:00
cf88ed4352
[BugFix][DP] Prevent health checks from perturbing user request routing (#35721 )
FeiHong Guo
2026-10-11 21:08:55 +08:00
d26b6a5a9c
fix: default Inkling to fa4 on Hopper (#35292 )
Shuwen Wang
2026-10-11 20:57:42 +08:00
586d98cc15
[Diffusion] Add ModelOpt FP8 checkpoint quantization for Pi0.5 (#43062 )
trebladev and Mick Qian
2026-10-11 20:55:59 +08:00
39ceba8a3d
G9i: router-side pool admission + fast-P routing (gateway)
minke.yu
2026-10-11 19:11:02 +08:00
04f740580c
[VLM] fix: repair multimodal VMM lease cleanup and source-stream ordering (#43475 )
Mick and Mick Qian
2026-10-11 19:13:39 +08:00
efd14cfb66
[DeepSeek-V4.1] Use num_stages=1 for the hc_mix stats partial kernel (#43070 )
2026-10-11 19:03:19 +08:00
b9d259441c
deploy: l3test stack image -> e7dc175e9 (G9 P0 fixes baked in)
minke.yu
2026-10-11 17:47:24 +08:00
e7dc175e94
G9 P0 fixes: v2 pool_transfers for storage hit query, disagg drain hook, query gate
dsv41-pd-visioncp
minke.yu
2026-10-11 17:37:21 +08:00
53295c6972
[kvevents] fix missing topics in replayed kv events (#43386 )
Sage
2026-10-11 12:25:43 +03:00
0a21a3b798
[Scheduler] Keep a DP-converted decode batch as the running batch (#43283 )
Liangsheng Yin
2026-10-11 01:23:51 -07:00
41652225d0
[Sampler] Sort only the max(top_k) leading columns in the torch sampler (#43346 )
jiaqiang-dot-liu
2026-10-11 16:00:59 +08:00
6612af9e0e
deploy: 归档 b300 实况——l3dummy 镜像升 baba18bc7d + D max-running-requests 512
minke.yu
2026-10-11 15:40:04 +08:00
71c6d8bc55
G9 P0 fix: /query_cache_hit 绕过 _tokenize_one_request 的 ReqState 依赖
minke.yu
2026-10-11 15:38:35 +08:00
dab108b9f3
[Scheduler] Hold back decode rows on a mixed step when they do not fit the KV pool (#43290 )
Liangsheng Yin
2026-10-11 00:18:27 -07:00
89e24e7a20
[KV canary] Expect sweep counters on the target runner only (#43286 )
Liangsheng Yin
2026-10-10 23:52:17 -07:00
ad8b1e1f85
[Test] Do not hang scripted-runtime teardown when the server died (#43285 )
Liangsheng Yin
2026-10-10 23:13:56 -07:00
448759b235
[Scheduler] Requeue preempted requests when admission admits nothing (#43298 )
Liangsheng Yin
2026-10-10 22:46:55 -07:00
047405f534
[qwen 3.8 next] Fuse QSA prefill KV gathering and dtype conversion (#41933 )
Qiaolin Yu
2026-10-10 22:40:23 -07:00
5678b71b6c
[Spec] Allow Qwen3.5 backends to opt into packed target verification (#43037 )
paulzhang-tm
2026-10-11 01:39:17 -04:00
22d63f5e43
[AMD]Qwen add Sparse attention to capturing hip graphs (#43383 )
Nikolai
2026-10-11 07:03:54 +02:00
260551ebe9
[KDA] Support Helion MTP verification and harden prefill, decode (#42857 )
Jzz1943
2026-10-11 12:47:57 +08:00
65e6bb6be7
[GDN] Fix CuTe DSL decode to read the [N, HV, V, K] state layout (#43332 )
SunskyXH and Xiaoyu Zhang
2026-10-11 13:43:16 +09:00
a1500b10e3
[diffusion] Don't report video jobs completed after their outputs are deleted (#43084 )
2026-10-11 10:12:38 +05:30
aa5c969dc3
[Perf] Fuse BF16 writes to native HND KV cache storage (#43237 )
Chunan Zeng and Claude Opus 5.5
2026-10-10 21:41:18 -07:00
58cb1a2be6
[Scheduler] Keep waiting-prefix LRU refresh consistent across TP ranks (#43479 )
Shuwen Wang
2026-10-11 12:18:41 +08:00
3458c94fad
[diffusion] comfyui: consolidate executor conditioning-cache fixes (#43492 )
Avaya Aggarwal and niehen6174
2026-10-11 09:12:00 +05:30
638330d975
[diffusion] comfyui: consolidate FluxAdapter correctness fixes (#43490 )
Avaya Aggarwal and niehen6174
2026-10-11 09:11:36 +05:30
6576e50cd5
[1/N][DSv41] Enable DP attention with MegaMoE (#43228 )
Khoa Pham
2026-10-10 20:36:35 -07:00
b931b2c8bd
[diffusion] perf: run the attention section eager under torch.compile for MiniMax-H3 (#43634 )
2026-10-11 11:02:54 +08:00