minke.yu
b419518774
deploy(benchb): bump new-stack image to g9i-est-td-148936bcb (hicache prefetch call fix)
2026-10-19 02:13:46 +08:00
minke.yu
148936bcb9
fix(hicache): drop stale completed_tokens arg in _check_hybrid_prefetch_result call
...
Merge artifact: upstream refactored the method to read
operation.completed_tokens internally (8 params), our call site kept
passing completed_tokens (9 args) -> TypeError at runtime on hybrid
prefetch completion, wedging the prefill engine under high L3 hit load.
2026-10-19 01:52:38 +08:00
minke.yu
6ceccac658
deploy: 阶段A/B 新旧基线 compose(4x2卡 standalone 并行 + 双 1P1D PD 并行)
...
- bencha: old/new × cp2/dp2 standalone,端口 30500-30800,新镜像独立 autotune cache
- benchb: 旧栈(31000/31200/31400+master 50051) 新栈(31100/31300/31500+master 50061),
NVLink-intra 四件套,新版 dp2 加 --skip-server-warmup(上游 dspark+dp-attention
warmup 死锁规避,真实流量路径不受影响)
2026-10-19 00:29:07 +08:00
minke.yu
1d2610bbc2
fix(mm): _collect_image_span_requests produce 3-tuple overlapping (idx,item,span_len)
...
Merged tree convention (PerImageRequestInfo.overlapping annotation and
upstream _assemble_per_image_chunk) is (idx, item, span_len); our span
collector still emitted 4-tuples (idx,item,start,end) from the pre-merge
version, crashing V4.1 warmup assembly: ValueError too many values to unpack.
2026-10-18 23:05:47 +08:00
minke.yu
e766b3a0e0
fix(mm): _owner_span_encoder pass data_embedding_func to _move_items_to_device
...
Upstream refactor added the third param (defer_move check); our local
owner-span encode path kept the old 2-arg call and crashed V4.1 warmup:
TypeError: _move_items_to_device() missing 1 required positional argument
2026-10-18 22:32:28 +08:00
minke.yu
e4391748e9
Merge upstream/main ( 6c3db0f520, 2026-10-18) into g9i-est-td
...
对齐上游 main 至 2026-10-18。回退基准 tag: 26.10.18。
上游官方版取代的本地 cherry-pick: #40323/#40335(Prefill CP+stream-sync)、
#40177(Decode DP-attn)、#40333->#38603(BCG 官方版)、#40178(MegaMoE 官方版)、
#42194/#41520(disagg prefill 官方重构, 删除本地 checkpoint_disagg_prefill 影子 shim)、
#42420/#42264/#41607/#42113/#41987/#40637/#41658/#41660/#42716 等。
保留的本地独有优化:
- g9i 网关集成 (sgl-model-gateway, query_cache_hit 经 TokenizerControlMixin 存活)
- hicache L3 io_aux 独立车道 + HC_TIMING 打点 (cache_controller.py)
- #38348 fail-soft hybrid prefetch 降级 (unified_radix_cache.py, 适配新
storage_existence_cache.pool().invalidate_beyond API)
- V4.1 vision CP 放行 (deepseek_v4.py / deepseek_v4_hook.py / eager_runner.py)
- NVLink-intra host 负载修复 (cuda.bindings 指针探测 + STATE_DATA 通道)
- mooncake standalone dummy 补丁 + rank 后缀主机名
- PD debug 仪表 (_dbg_phase/_sched_idled/record_prefill_drain)
- #40429 FlashMLA smem 崩溃修复 (_FLASHMLA_SCHED_MAX_SHARED_BYTES)
#42035 无需再 revert: 上游已默认关 (SGLANG_ENABLE_DISAGG_PREFILL_CONTINUOUS_
INPUT_POLLING=False) 且硬性要求 attn_dp==1 && attn_cp==1 && 非投机。
冲突 34 文件全部解决, srt 下 1038 个变更文件 py_compile 全过。
运行级验证未做, 待 b300 构建镜像 + 冒烟。
2026-10-18 18:13:02 +08:00
Ilia Yastrebov and ishandhanani
6c3db0f520
[Perf] NIXL: prepared dlists for hybrid-model state transfer ( #44740 )
...
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-10-18 16:59:18 +08:00
Xinyuan Tong
80fef63786
[Fix] Inkling: frame constrained output in native message tokens ( #44716 )
2026-10-18 16:47:56 +08:00
Huang Long
889658a5be
[Fix] disaggregation: Reuse startup decode graphs across PD role switches ( #43760 )
2026-10-18 16:16:46 +08:00
+14
d33dd0566a
[Feat] srt: Add muse spark runtime, multimodal support and tests (3/5) ( #44666 )
...
Co-authored-by: 1am9trash <1am9trash@gmail.com >
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
Co-authored-by: Chunan Zeng <zcnrex@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Ethan Che <eche@meta.com >
Co-authored-by: HAI <hixiao@gmail.com >
Co-authored-by: Jiajun Li <guapisolo@gmail.com >
Co-authored-by: Jiajun Li <jiajun.li@radixark.ai >
Co-authored-by: Jimmy Shiu <jimmysh341@gmail.com >
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com >
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com >
Co-authored-by: Wookje Han <52435620+wookjeHan@users.noreply.github.com >
Co-authored-by: Wookje Han <wookjeh@nvl72d079-T17.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d094-T17.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d157-T16.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d170-T04.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d214-T17.cm.cluster >
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com >
Co-authored-by: Yuhao Yang <yhyang201@users.noreply.github.com >
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com >
Co-authored-by: ethche <ethche@gmail.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
Co-authored-by: jianan-gu <jianan.gu@intel.com >
2026-10-18 01:00:24 -07:00
Shangming Cai
80914b7c9b
[Refactor] sgl-router: Rename --max-in-flight to --max-inflight-requests ( #44950 )
2026-10-18 15:33:58 +08:00
Yuwei An
3a6e1fcf77
[Perf] DSv4.1: Bound encoder BCG metadata memory ( #44963 )
2026-10-18 00:33:35 -07:00
Richard Wang and Mick Qian
de61ab53d4
[Perf] diffusion: Defer full GC collections while importing the runtime ( #44606 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-18 15:31:10 +08:00
Zhaoyi Li
f1cf2e1005
[Fix] [AMD] mori: Look up elided DSA state entries by the paired dst index ( #44412 )
2026-10-18 00:29:22 -07:00
Zhaoyi Li
396e97e5dd
[CI] [AMD] ci/slurm: Add a Mooncake KV-transfer test to the MI355X disaggregation nightly ( #44409 )
2026-10-18 00:28:34 -07:00
Zhaoyi Li
8a4239fbdc
[Fix] [AMD] mooncake: Bump Mooncake to v0.3.14-rc1 for cross-node KV transfer ( #44404 )
2026-10-18 00:22:55 -07:00
Shuwen Wang
ba3fc9441d
[Fix] mamba: Release consumed lazy Mamba prefix locks before donation retry ( #43559 )
2026-10-18 15:16:31 +08:00
jacky.cheng and Mick Qian
f96a7326e6
[Fix] [AMD] diffusion: Prevent ComponentResidencyError after compile warmup ( #43952 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-18 00:13:25 -07:00
Shuwen Wang
a11fa06be7
[Fix] hicache: Key MLA Mamba state per TP rank in the file backend ( #41907 )
2026-10-18 15:10:50 +08:00
Shuwen Wang
ffc4573c98
[Fix] hicache: Key MLA Mamba state per TP rank in the SeaweedFS backend ( #43816 )
2026-10-18 15:09:58 +08:00
Shuwen Wang
e6c2cd47a9
[Fix] hicache: Keep unified memory off the direct page_first_direct copies ( #43333 )
2026-10-18 15:09:26 +08:00
Liangsheng Yin
df966f2345
[Feat] deepseek-v4.1: Run encoder SWA bounded replay on PD prefill workers ( #44952 )
2026-10-18 00:07:55 -07:00
Ziang Li
520df8a491
[Perf] weight-sync: Allocate packed receive buffers from metadata ( #39264 )
2026-10-18 00:04:10 -07:00
Michael
00f6cdcd1b
[Fix] [AMD] kda: Reject match_cutedsl_decode on ROCm and skip its parity tests ( #44089 )
2026-10-18 00:01:40 -07:00
Michael
3626978a13
[Fix] [AMD] kda: Match the fused verify Q/K norm to its reference kernel ( #43991 )
2026-10-18 00:01:11 -07:00
Bowen Wang
3671503e44
[Feat] bench: report goodput and SLO attainment in bench_serving ( #43967 )
2026-10-18 13:31:48 +08:00
62d4261cf4
[Doc] sgl-router: Rewrite README and add architecture guide ( #43592 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-18 13:29:08 +08:00
Thomas Wang
e5dc10e73b
[Doc] [AMD] dsv41: Update amd cookbook 1016 ( #44903 )
2026-10-17 22:10:42 -07:00
Shangming Cai and Shangming Cai
1ff2c1d71e
[Fix] sgl-router: Fill the media fields when converting TextGenerate to Generate ( #44951 )
...
Co-authored-by: Shangming Cai <shangming.cai@radixark.ai >
2026-10-18 13:05:45 +08:00
Liangsheng Yin
d20d010574
[Fix] deepseek-v4.1: Exchange late-layer DP row counts before the forward ( #44948 )
2026-10-17 21:46:46 -07:00
Liangsheng Yin
6fe630ae14
[CI] deepseek-v4: Set the KV pool in the BCG replay contract test ( #44946 )
2026-10-17 20:59:03 -07:00
Baizhou Zhang
a8b939da43
Revert "[Doc] cookbook: Add Muse Spark Open SWA ratio calculator" ( #44947 )
2026-10-17 20:55:00 -07:00
140c57d5bf
[Perf] diffusion: classify residual_gate_add operands once per call ( #44478 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 11:51:52 +08:00
Liangsheng Yin and Vincent Zhong
ff73d63121
[Perf] deepseek-v4.1: Run encoder SWA bounded replay in the breakable prefill graph ( #44930 )
...
Co-authored-by: Vincent Zhong <207368749+vincentzed@users.noreply.github.com >
2026-10-17 20:51:18 -07:00
Chunan Zeng
a0b77db84d
[Perf] DFLASH: build the draft num_token_non_padded scalar with torch.full ( #44119 )
2026-10-17 20:40:16 -07:00
Kaushik Chatterjee
3acfa2efcc
[Feat] runtime: Support tokenized multimodal generation ( #42833 )
2026-10-18 12:35:51 +09:00
Richard Wang
01f3cba70e
[Fix] dLLM: Size decode graphs to the default max running requests ( #40941 )
2026-10-17 20:22:39 -07:00
531b426281
[Feat] diffusion/model: support Cosmos Dreams and Cosmos Multiview ( #39475 ) ( #39475 )
...
Co-authored-by: Kedi Wu <kediw@nvidia.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-18 11:15:57 +08:00
zijiexia
3b0a154e60
[Doc] cookbook: Add DeepSeek V4.1 Flash 1017 variant ( #44937 )
2026-10-17 20:04:41 -07:00
zijiexia and Claude Sonnet 5.5
df24bca4ba
[Doc] popular-models: Replace Kimi-K3 with Muse Spark Open ( #44693 )
...
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com >
2026-10-17 19:54:45 -07:00
14ad64c411
[Perf] custom_op: call register_custom_op ops directly in plain eager execution ( #44158 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 10:10:10 +08:00
Mick and Claude Opus 5
d748bede96
[Feat] diffusion: selectable sequence-parallel strategies for LingBot causal attention ( #34385 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-18 10:03:43 +08:00
8d02b69c11
[Perf] diffusion: Replay Triton launches without the runner's per-launch hook metadata ( #44471 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 09:58:54 +08:00
949e1a2080
[Perf] diffusion: Cache the fused linear-GELU verdict per linear ( #44643 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 09:55:29 +08:00
f53bd436c1
[Perf] distributed: cache the eager launch of custom all-reduce v2 per size ( #44642 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-18 09:54:58 +08:00
minke.yu
c8d06424fa
gateway metrics: T25b test fix — le buckets are cumulative
2026-10-18 09:53:17 +08:00
minke.yu
97c7eabb33
gateway metrics: T25b fixup — recorder.handle() for 0.17 API
2026-10-18 09:51:58 +08:00
minke.yu
e9b436dc2d
gateway metrics: explicit histogram buckets for all g9i/router histograms — kill zero-quantile summaries (P2 T25b)
2026-10-18 09:50:23 +08:00
minke.yu
fc63ae5392
gateway g9i: T25 test — relax timing-sensitive wait assertion
2026-10-18 08:34:56 +08:00
minke.yu
9df2d40324
gateway g9i: T25 cleanup — drop dead wrapper fns, cfg(test) the test-only ones
2026-10-18 08:31:31 +08:00
minke.yu
6a8a6e399f
gateway g9i: T25 fixup — shadow-mode reservoir eligibility is unreachable, not excluded
2026-10-18 08:27:15 +08:00
minke.yu
e6c0737541
gateway g9i: T25 fixup — class ttft reads ShadowResponseContext.class
2026-10-18 08:23:53 +08:00
minke.yu
9374e03f13
gateway g9i: hot-channel admission A/B — reservoir + hardlane policies (P2 T25)
2026-10-18 08:21:21 +08:00
Yuwei An
b234b5c32e
[Refactor] TCPCG: Consolidate attention eager regions and isolate Inkling policy (4/9) ( #42288 )
2026-10-17 16:06:24 -07:00
Yuwei An
e4512a5a17
[Perf] DSv4.1: Reduce BCG metadata memory ( #44920 )
2026-10-17 16:05:28 -07:00
Liangsheng Yin
563cd60345
[Fix] [AMD] deepseek-v4.1: Reject attention DP with encoder SWA bounded replay on ROCm ( #44934 )
2026-10-17 15:47:41 -07:00
Yi (Vincent) Zhong
05ce0e980f
[Perf] deepseek-v4.1: Skip wasted layer 20 work on earlier prompt tokens under decoder bounded replay ( #42901 )
2026-10-17 14:12:00 -07:00
05c1539098
[Feat] unified_cache: Support hybrid SWA + Mamba models in the Mooncake direct linker ( #41343 )
...
Co-authored-by: Zhangheng <hzh0425@apache.org >
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com >
2026-10-17 13:24:08 -07:00
YAMY and Claude Opus 5.5
3b34d9f8b6
[Perf] [NVIDIA] qsa: Keep sparse decode scratch in FP8 for FP8 KV pools ( #44114 )
...
With an FP8 KV pool on SM10x, keep the packed sparse-decode K/V scratch in FP8 instead of dequantizing to BF16 (BF16-q + FP8-KV TRTLLM-gen decode). Tests cover FP8 pool -> FP8 scratch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 11:50:38 -07:00
minke.yu
86883ae89f
gateway g9i: deep placeholder rings anchor classification (P2 T24)
...
v14 replay (c512, 32836 reqs): T23 chunking works (lookup_depth mean 41,
placeholder 6795 vs engine-hit 7183) but Hot never fired — placeholder
anchors kept suffix_start=0, so every shared-prefix request counted its
whole message list as new work -> ColdLarge. Now a placeholder ring at
depth >= MIN_PLACEHOLDER_DEPTH (2) anchors like a real hit: suffix_start
via ring_msg, matched_node filled (record writes the node on every ring),
anchor_tokens stays 0 (unknown, deliberate). depth==1 placeholders stay
misses (shared-greeting guard). ShadowDecision gains
anchor_is_placeholder; the shadow x engine agreement counter's shadow
axis widens to hit/placeholder/miss via ShadowAnchorKind. KNOWN
LIMITATION pinned in a test: a chunk-ring anchor mid-message hides an
unbounded remainder (wrong-Hot vector for >theta mid-message divergence
after a >=2048B shared prefix). No deploy.
2026-10-17 23:57:44 +08:00
minke.yu
43c2844c18
gateway g9i: intra-message 2048B chunk rings for long-prompt prefix hits (P2 T23)
...
A per-message chain is blind inside one long message (v13: 9034/9034
depth-0 misses on single-message long-prompt traffic). Text pieces
longer than 2048 bytes now roll into bare chunk rings
xxh3(prev || chunk) before the message-final ring, which carries only
the <=2048-byte tail; texts <=2048 hash byte-identically to pre-T23
(pinned by a verbatim legacy-chain equivalence test). Lookup itself is
unchanged, but classify's suffix_start now maps the hit ring back to
its message index via the new ring_msg sideband (the raw n-depth+1
formula overshoots the message arrays once chains outgrow messages).
No deploy.
2026-10-17 22:33:01 +08:00
4eb606b2ae
[Feat] diffusion: support opt-in pi0.5 completion-boundary action batching ( #44634 )
...
Co-authored-by: Jiahua Huang <35223412+dadiaokua@users.noreply.github.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-17 20:49:44 +08:00
b15a78a881
[Fix] diffusion/lora: revalidate FLUX.2 FP8/NVFP4 fast paths after LoRA attach ( #40171 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-17 20:48:14 +08:00
23f8e6f7ec
[Fix] diffusion: keep ZMQ ingress sockets on their configured port ( #43732 )
...
Co-authored-by: Sylvan Liu <syliu@cse-ai-16.amd.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-17 20:47:29 +08:00
+3
d4191a4624
[Fix] diffusion: stabilize FLUX.3 serving and reuse static conditioning ( #44493 )
...
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com >
Co-authored-by: Dom Brown <3886319+DomBrown@users.noreply.github.com >
Co-authored-by: niehen6174 <1639206518@qq.com >
Co-authored-by: Mick Qian <mickqian@radixark.ai >
Co-authored-by: Nikita Korobov <14355239+nekorobov@users.noreply.github.com >
Co-authored-by: Daniel Campora <961215+dcampora@users.noreply.github.com >
Co-authored-by: Richard Wang <11150595+rwang5203@users.noreply.github.com >
Co-authored-by: BBuf <1182563586@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Fabio Camaiora <fcamaiora@nvidia.com >
Co-authored-by: Linda-Stadter <lstadter@nvidia.com >
2026-10-17 20:46:13 +08:00
9771e5a4f0
[Feat] diffusion: Per-task prompt enhancer config and the FLUX.3 Klein upsampler protocol ( #44662 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 20:45:07 +08:00
Mick and Mick Qian
e01dfd3cbc
[Refactor] multimodal: Replace padding strategies with helpers ( #44845 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-17 20:43:02 +08:00
Shuwen Wang
86e67f7d93
[Fix] unified_cache: Prevent misaligned Mamba checkpoints after speculative truncation ( #43387 )
2026-10-17 20:20:49 +08:00
SuperSong
025655068c
[Fix] unified-memory: Avoid gather scratch during Mamba and Conv compaction ( #44097 )
2026-10-17 04:56:41 -07:00
SuperSong
ee30dc4d40
[Fix] unified-memory: Bound joint reclaim and preserve cached prefixes ( #39294 )
...
Bound joint FULL/SWA/Mamba reclaim so feasible allocations recover capacity without evicting unnecessary cached prefixes, and impossible demands preserve the cache. Use exact unsharded paged-extend demand and update allocation regression fixtures.
2026-10-17 04:46:48 -07:00
Shuwen Wang
ae28306a74
[Fix] cuda_graph: Wait for in-flight layer-wise cache loads before prefill graph replay ( #44220 )
2026-10-17 19:13:57 +08:00
Shuwen Wang and Zhangheng
5fcb07da34
[Fix] hicache: Key MLA Mamba state per TP rank in the HF3FS backend ( #43817 )
...
Co-authored-by: Zhangheng <hzh0425@apache.org >
2026-10-17 11:08:17 +00:00
Woody-Hu and Mick
465ddf9758
[Feat] diffusion/VSA-H3: support sm80 (A100) with a raw-pointer Triton load path ( #43823 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-17 18:47:19 +08:00
400c1dfe6e
[Perf] distributed: skip the DLPack re-wrap of custom all-reduce results ( #44251 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 18:44:31 +08:00
pllimax
eb5739b30e
[CI] pr-test-npu: Temporarily disable recommend-tests-from-coverage job ( #44881 )
2026-10-17 18:21:17 +08:00
Khoa Pham and Claude Fable 5.1
a126e8d640
[Perf] deepseek_v4: fold the MegaAttention query layout into wq_b for block-FP8-as-MXFP8 checkpoints ( #44838 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-17 02:45:53 -07:00
Mick and Mick Qian
80f40522cd
[UX] diffusion: log LoRA merge decisions only on first evaluation ( #44376 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-17 16:47:07 +08:00
56a20d7e34
[Perf] kernels: Enter a JIT kernel's device guard only when the device differs ( #44644 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 16:43:27 +08:00
f75232cb42
[Fix] hicache: Order write-back host copies after the forward that produced the KV ( #44800 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-17 01:31:40 -07:00
0d13efb2b8
[Feat] rust-server: Serve /v1/tokenize and /v1/detokenize from the Rust frontend ( #44767 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-17 01:30:50 -07:00
a2953d7b7f
[Fix] deepseek-v4.1: Serve prompt-logprob requests under decoder SWA bounded replay ( #44850 )
...
Co-authored-by: Pedram Ghazi <pedram.ghazi@amd.com >
Co-authored-by: David Orman <ormandj@corenode.com >
2026-10-17 01:25:33 -07:00
McZyWu
7a8804edb5
[Fix] [NPU] qwen: Fix shared-expert routing with Ascend FuseEP ( #40516 )
2026-10-17 16:19:44 +08:00
Liangsheng Yin
ef26f6ad49
[Fix] deepseek-v4.1: Keep the cached KV of folded encoder replay rows ( #44857 )
2026-10-17 00:56:18 -07:00
Khoa Pham and Claude Fable 5.1
bce7b27aa6
[Perf] deepseek_v4: run FlashInfer autotune for block-FP8 models served as MXFP8 ( #44842 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-17 00:19:35 -07:00
01dc08e23f
[Refactor] dsv41: Shard Engram tables on hash-column boundaries ( #42787 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-17 00:15:49 -07:00
minke.yu
9e288079e4
gateway g9i: fix non-stream CL truncation on shadow strip + pre-strip breakdown extraction + shadowxengine agreement metric (T22)
...
- Bug A (user-visible): non-stream passthrough forwarded the upstream
Content-Length after the shadow strip had shortened the body ->
cache-hit non-stream responses truncated (~60B, IncompleteRead).
non_stream_passthrough_response now pins CL to the actual body length.
- Bug B (order): strip_and_extract returned the post-strip JSON to
shadow_record, so extract_cached_breakdown always missed and
smg_g9i_shadow_cache_hit_tokens_total never fired. The breakdown is now
extracted pre-strip and threaded through record_response_json.
- T22 observability: extract_cached_total (standard
usage.prompt_tokens_details.cached_tokens first, tier breakdown as
fallback) + smg_g9i_shadow_engine_agreement_total{shadow,engine} 2x2
counter pairing the request-time lookup hit with the engine's actual
cached tokens; ShadowResponseContext carries shadow_hit.
2026-10-17 15:11:58 +08:00
Khoa Pham and Claude Fable 5.1
14c4dda24e
[Perf] deepseek_v4: fuse block-FP8 shared experts into MegaMoE for any 32-aligned block size ( #44837 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-16 23:47:41 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
9c7b338a19
[CI] jit-kernel: Honor continue_on_error on scheduled runs ( #44645 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-16 22:29:01 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
2d8a05e719
[Test] attention: Fix replicated attention test missing full_attention_interval ( #44602 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-16 22:28:37 -07:00
Liangsheng Yin
9259553f09
[Fix] megamoe: Support EPLB rebalance on SM100 ( #44847 )
2026-10-16 22:22:40 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
3fc08b939a
[Feat] server-args: Add --enable-hicache alias for --enable-hierarchical-cache ( #44849 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-17 13:22:34 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
4fc86f73c2
[CI] glm5.2: Keep EOS in HiSparse needle test output ( #44597 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-17 12:20:51 +08:00
Liangsheng Yin
4cbea9d3c7
[Doc] cookbook: Allow PD with DSpark for DeepSeek-V4.1 ( #44848 )
2026-10-16 21:17:32 -07:00
Xinyuan Tong
86cc2e02ec
[Fix] gateway: Drop unused npyz dev-dependency to unbreak builds after zip yank ( #44720 )
2026-10-16 20:56:31 -07:00
Ziang Li
29a7e19de1
[Refactor] nvfp4_online: Use explicit FlashInfer 4over6 config ( #41745 )
2026-10-16 20:56:10 -07:00
feixiangpeng and zijiexia
9590dc80cc
[Doc] hicache: Explain HiCache host pool sizing with a storage backend ( #44790 )
...
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com >
2026-10-17 11:28:37 +08:00
Shuwen Wang
6a39ddf1f2
[Fix] unified_cache: Resize test allocator before allocating backed pages ( #44843 )
2026-10-16 20:21:49 -07:00
3603824541
[Perf] DSv4.1: Enable decoder bounded replay by default ( #44403 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-10-16 19:52:02 -07:00
minke.yu
738d5a50d0
gateway g9i: shadow semantic gap tests (image-in-prefix/theta boundary/deep hit/shared-prefix repoint/try_consume/reasoning_content)
2026-10-17 10:15:38 +08:00
McZyWu
2d3dd3168d
[Fix] [NPU] attention: Use FIA TND for MLA prefix attention and add Gemma norm fallback ( #40131 )
2026-10-17 10:12:41 +08:00
Khoa Pham
ff979e0816
[Fix] megamoe: Respect request-pool limits in decode validation ( #44816 )
2026-10-16 18:56:38 -07:00
5cbe2371a5
[Feat] rust-server: Report Python's launch-time /server_info fields from the Rust frontend ( #44764 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-16 18:14:50 -07:00
Shuwen Wang
33b5d26fa6
[CI] lora: Use the canonical Llama model ID in eviction tests ( #44771 )
2026-10-16 18:13:37 -07:00
b62a48ee15
[Feat] sglang-server: Serve /flush_cache from the Rust frontend ( #44766 )
...
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-16 17:24:01 -07:00
d8dd49d21d
[Fix] srt: Apply --gc-threshold in every server process and freeze every tokenizer worker ( #44785 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com >
2026-10-16 16:55:54 -07:00
d230f2c35b
[Feat] profiler: Add the forward pass id to detailed step span labels ( #44775 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: charlotte12l <charlotte12l@users.noreply.github.com >
2026-10-16 16:51:20 -07:00
36cf028346
[Perf] unified-memory: Prewarm the compaction move at startup and copy whole page envelopes by page id ( #44592 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com >
2026-10-16 16:47:50 -07:00
Khoa Pham and Claude Fable 5.1
aaf5f397a5
[Feat] dsv4.1: Support encoder SWA bounded replay with CUDA DP attention ( #43813 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-16 15:38:33 -07:00
Yuwei An
8e3f11b219
[Perf] DSv4.1: Skip discarded KV rows during window commit ( #44783 )
2026-10-16 15:05:45 -07:00
a46fe83c70
[CI] tests: Fork the gloo exchange test's ranks from a preloaded forkserver ( #44679 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com >
2026-10-16 14:53:21 -07:00
Aarav and Aarav
d5d87899f9
[Fix] DSA: skip-topk pipeline handoff for owed outputs ( #43832 )
...
Co-authored-by: Aarav <211792885+AaravMehta-07@users.noreply.github.com >
2026-10-16 14:35:23 -07:00
Khoa Pham and Trevor Morris
71b60fa46c
[Fix] engram: Avoid blocking history copies during PD admission ( #44787 )
...
Co-authored-by: Trevor Morris <tmorris@nvidia.com >
2026-10-16 14:29:05 -07:00
2948662026
[Perf] dp-attention: Exchange the scheduler's MLP sync rows with gather + broadcast on gloo ( #44622 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: hanming-lu <hanming-lu@users.noreply.github.com >
2026-10-16 13:53:41 -07:00
metamergebot and metamergebot
f39c2693aa
[CI] rust-server: Raise the ChunkEvent frame-size cap to 152 bytes ( #44773 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
2026-10-16 13:47:52 -07:00
Chunan Zeng
07950be745
[Perf] [NVIDIA] custom-all-reduce: Add 2shot_push all-reduce and retune SM100 TP4/TP8 ( #42811 )
2026-10-16 13:43:18 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
21791e7023
[CI] [NVIDIA] DSv4.1-Flash: Split GB300 accuracy test into one file per config ( #44696 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-16 13:40:03 -07:00
metamergebot and metamergebot
a31a8c2bbd
[CI] tests: Let test_hicache_storage size its host cache to the runner ( #44729 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
2026-10-16 13:38:39 -07:00
Liangsheng Yin
a18a908066
[Test] mem_cache: Free SWA peers below the eviction floor in the SWA repair test ( #44777 )
2026-10-16 12:41:39 -07:00
+14
456231e251
[Feat] kernels: Add muse spark operators and collective GEMM (2/5) ( #44665 )
...
Co-authored-by: 1am9trash <1am9trash@gmail.com >
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
Co-authored-by: Chunan Zeng <zcnrex@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Ethan Che <eche@meta.com >
Co-authored-by: HAI <hixiao@gmail.com >
Co-authored-by: Jiajun Li <guapisolo@gmail.com >
Co-authored-by: Jiajun Li <jiajun.li@radixark.ai >
Co-authored-by: Jimmy Shiu <jimmysh341@gmail.com >
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com >
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com >
Co-authored-by: Wookje Han <52435620+wookjeHan@users.noreply.github.com >
Co-authored-by: Wookje Han <wookjeh@nvl72d079-T17.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d094-T17.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d157-T16.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d170-T04.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d214-T17.cm.cluster >
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com >
Co-authored-by: Yuhao Yang <yhyang201@users.noreply.github.com >
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com >
Co-authored-by: ethche <ethche@gmail.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
Co-authored-by: jianan-gu <jianan.gu@intel.com >
2026-10-16 10:49:35 -07:00
ormandj and hnyls2002
80c54e6ac5
[Fix] [AMD] hicache: Serve encoder-replay prefix hits with a paged DSpark draft SWA ( #42716 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-16 10:45:16 -07:00
553068af25
[Fix] rust-server: Honour cache_salt and require_reasoning on /generate ( #44533 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-16 10:38:09 -07:00
453b251dbd
[Fix] hicache: Bound the storage prefetch timeout from the first blocked admission poll ( #44614 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-16 10:37:11 -07:00
metamergebot and jiayisuse
989ccf4863
[Feat] disaggregation: Let a KV transfer backend skip the checksum for an incomparable peer ( #44605 )
...
Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com >
2026-10-16 10:36:46 -07:00
63a9b7a713
[Feat] http_server: Make the multi-worker ASGI app path configurable ( #44583 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com >
2026-10-16 10:36:35 -07:00
26a2f18c02
[Feat] sglang-server: serve /v1/loads from the Rust frontend ( #44532 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-16 10:35:37 -07:00
minke.yu
e80c8608dc
gateway g9i: --g9i-window-fixed pins the admission window, skipping all AIMD dynamics (P2 T21)
2026-10-17 01:24:23 +08:00
Xinyuan Tong
a038a0dca0
[Fix] dsv41: Preserve tool-result image order and current-turn thinking ( #44751 )
2026-10-16 23:39:50 +08:00
Duyi-Wang
ed2ebad342
[Doc] [AMD] cookbook: Refresh DeepSeek V4 AgentX PD recipe optimizations ( #44454 )
2026-10-16 23:34:01 +08:00
Jimmy Shong and Jimmy Shong
2c2986be2b
docs: add Beam 1.0 deployment cookbook ( #44694 )
...
Co-authored-by: Jimmy Shong <jimmy.shong@mac.attlocal.net >
2026-10-16 13:29:14 +00:00
minke.yu
5351d5491b
gateway g9i: AIMD warmup tuning (init 128, add 64, CLI flags) + prompt_too_long 400 front validation (P2 T20 A+B)
2026-10-16 20:07:49 +08:00
Benjamin Truong and Claude Opus 5.5
7c99ace127
[Fix] dsa: Fix ragged seed indices before MTP draft decode ( #42565 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 19:37:41 +08:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
5e240a200d
[Doc] cookbook: Refresh SemiAnalysis AgentX launches for Qwen3.5 and DeepSeek-V4.1-Flash GB300 ( #44712 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 19:31:08 +08:00
Xinyi Song and HAI
4afae5568b
[Perf] [AMD] dsv4: Add a separate MegaMoE decode instance ( #44657 )
...
Co-authored-by: HAI <hixiao@gmail.com >
2026-10-16 03:11:02 -07:00
2adc76ad61
[Perf] [AMD] dsv4: reuse the q_lora fp8 quant for the C4 indexer wq_b (4/8) ( #41903 )
...
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com >
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-10-16 02:55:25 -07:00
YC Yen-Ching Tseng
9246acd462
[CI] [AMD] aiter: pin aiter commit to 2bb549 for rocm10.1 gfx1250 ( #44620 )
2026-10-16 17:50:20 +08:00
kk and Duyi-Wang
baf451fcd6
[Perf] [AMD] dsv4: carry aiter DSv4 GEMM tuned rows and mHC tile_m=16 until aiter ships them (7/8) ( #44008 )
...
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-10-16 02:47:41 -07:00
f05da43d44
[Feat] [AMD] disagg: Support DSV4 decode radix cache on ROCm unified KV (bf16/fp8, DSpark) ( #30929 )
...
Co-authored-by: HAI <hixiao@gmail.com >
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-10-16 02:39:27 -07:00
42eb556e66
[Perf] [AMD] dsv4: optional e8m0 mxscale GEMM for dense 128x128 fp8 linears (6/8) ( #44007 )
...
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
Co-authored-by: HAI <hixiao@gmail.com >
2026-10-16 02:28:31 -07:00
caihuali95
8d229d96e0
[Fix] unified-memory: enabling trtllm_mha with decode graphs and speculative decoding ( #42930 )
2026-10-16 02:24:06 -07:00
kk and Duyi-Wang
6efc96abf9
[Perf] [AMD] dsv4: FP4 indexer on aiter's row-group kernel for decode and target-verify (5/8) ( #44006 )
...
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-10-16 02:16:06 -07:00
kk and Duyi-Wang
9fde8bcc46
[Perf] [AMD] moe: small-batch sort with histogram + bitonic sort for 32 < P <= 256 (3/8) ( #44004 )
...
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-10-16 02:09:32 -07:00
kk and Duyi-Wang
f0ff18353d
[Perf] [AMD] dsv4: fp8 verify writes the SWA ring in the fused norm/rope kernel (2/8) ( #44003 )
...
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-10-16 01:58:00 -07:00
Mingyi and Zijie Xia
56a3c4fa16
[Doc] cookbook: Add RTX Spark Qwen3.6 NVFP4 recipe ( #44707 )
...
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai >
2026-10-16 01:55:41 -07:00
Liangsheng Yin
81c9d804b0
[CI] pr-check-comment: Comment on PRs when lint or title check fails ( #44708 )
2026-10-16 01:54:38 -07:00
kk and Duyi-Wang
ab5858ac8a
[Perf] [AMD] dsv4: fp8 unified_kv decode/verify on FlyDSL v4 nm decode with a latency-model split plan (1/8) ( #44016 )
...
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-10-16 01:50:16 -07:00
Xinyuan Tong and zijiexia
9baace8456
[Doc] cookbook: refresh MiniCPM-V 4.7 DSPARK benchmarks at the final model PR head ( #44310 )
...
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com >
2026-10-16 01:48:25 -07:00
3d61bdd2c9
[Feat] sgl-router: Serve the remaining native gRPC inference RPCs ( #42681 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-16 16:23:17 +08:00
kk and wunhuang
536cdb4b04
[Feat] [AMD] docker: Install the sgl-amd-mono monokernel plugin in the ROCm image ( #44621 )
...
Co-authored-by: wunhuang <wunhuang@amd.com >
2026-10-16 01:20:40 -07:00
Liangsheng Yin
fad7d95a7f
[Fix] deepseek-v4.1: Keep the DSpark draft SWA out of the radix tree ( #44673 )
2026-10-16 01:18:58 -07:00
Baizhou Zhang
06b81b120e
[CI] deepgemm: Increase B200 release test timeout to 180 minutes ( #44706 )
2026-10-16 01:14:54 -07:00
kk and wunhuang
bdc4ec15ce
[Perf] [AMD] dspark: No per-step host sync in decode (1/4) ( #44405 )
...
Co-authored-by: wunhuang <wunhuang@amd.com >
2026-10-16 01:13:37 -07:00
Ziru and Niu Ziru
5dd37af412
[Doc] [XPU] Cookbook: Update the XPU Docker version to 0.5.22 ( #44647 )
...
Co-authored-by: Niu Ziru <niuziru@gnr799217.jf.intel.com >
2026-10-16 16:13:24 +08:00
minke.yu
8a2e1403a3
gateway g9i: blind-state timeout no longer blocks recovery when pipeline < 0.5W (P2 T19 P1)
2026-10-16 15:59:14 +08:00
minke.yu
1910eccb8e
gateway g9i: ledger reconcile clamp on sustained ledger>>engine divergence (subtract-only, 60s fuse) (P2 T19 P0-2)
2026-10-16 15:56:56 +08:00
jiaqiang-dot-liu
a448fdf316
[Perf] [AMD] qwen3: Fuse QK-norm + RoPE + KV cache write for dense Qwen3 decode ( #44267 )
2026-10-16 00:51:55 -07:00
minke.yu
6217b9de46
gateway g9i: TTL reaper for wedged D-ledger bookings (pipeline 120s / streaming 600s) (P2 T19 P0-1)
2026-10-16 15:51:40 +08:00
amote-i
9d394dfbd7
[Doc] [NPU] docs: refresh associated dependency versions in npu docs ( #44695 )
2026-10-16 15:48:37 +08:00
Ziang Li
7098ec387c
[Feat] FlashInfer: Add NVFP4 W4A16 and SiTU support to MegaMoE ( #44036 )
2026-10-16 00:38:50 -07:00
Liangsheng Yin and Jingshu Chen
3a5d46e946
[Perf] pd: Stage decode-side KV preallocation lengths through pinned memory ( #38955 )
...
Co-authored-by: Jingshu Chen <96519051+elmo2019@users.noreply.github.com >
2026-10-16 00:34:56 -07:00
Xiaoyu Zhang
51faf52305
[Fix] monokernels: Honor resolved CUDA graph configuration ( #44630 )
2026-10-16 15:06:14 +08:00
Baizhou Zhang
dfc89580f1
[Doc] cookbook: Add Muse Spark Open SWA ratio calculator ( #44687 )
2026-10-16 00:00:12 -07:00
amote-i
92580581da
[Doc] [NPU] docs: Freeze qwen3-235b-a22b recommended image to a fixed version ( #44650 )
2026-10-16 14:14:05 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
44be30d216
[Chore] deps: Bump FlashInfer to 0.7.2 ( #44591 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-15 22:51:21 -07:00
Shu Wang
83940471b4
[Fix] [NVIDIA] allreduce-fusion: Decline MNNVL fusion across NVLink cliques ( #44387 )
2026-10-15 22:50:33 -07:00
+14
e1d20574da
[Chore] docker: Add muse spark CUDA image and DeepGEMM patches (1/5) ( #44664 )
...
Co-authored-by: 1am9trash <1am9trash@gmail.com >
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
Co-authored-by: Chunan Zeng <zcnrex@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Ethan Che <eche@meta.com >
Co-authored-by: HAI <hixiao@gmail.com >
Co-authored-by: Jiajun Li <guapisolo@gmail.com >
Co-authored-by: Jiajun Li <jiajun.li@radixark.ai >
Co-authored-by: Jimmy Shiu <jimmysh341@gmail.com >
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com >
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com >
Co-authored-by: Wookje Han <52435620+wookjeHan@users.noreply.github.com >
Co-authored-by: Wookje Han <wookjeh@nvl72d079-T17.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d094-T17.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d157-T16.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d170-T04.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d214-T17.cm.cluster >
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com >
Co-authored-by: Yuhao Yang <yhyang201@users.noreply.github.com >
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com >
Co-authored-by: ethche <ethche@gmail.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
Co-authored-by: jianan-gu <jianan.gu@intel.com >
2026-10-15 22:46:22 -07:00
minke.yu
9dd446b5c1
gateway g9i: AIMD blind-state recovery threshold prealloc < max(1, max_running/4) (P2 T18 hole-4)
...
The blind-state (no TTFT samples — non-streaming traffic) slow-add
condition required D prealloc_queue == 0, which is unreachable in
production (prealloc sits at 180+ under load), so the window stayed
pinned at WINDOW_MIN=16 forever — the admission window never recovered
after a cut.
New rule: slow-add when prealloc_queue < max(1, max_running_total/4)
(pipeline < 0.8W condition kept; hold-timeout still forces hold). A
missing snapshot for the node is treated as busy (no blind recovery on
zero information).
Test: aimd_blind_holds_while_prealloc_busy replaced by
aimd_blind_recovery_prealloc_threshold (0/10 recover at limit 64,
boundary == limit holds, production-shape 180 holds, max(1, mr/4) floor
at mr=8, missing snapshot holds).
2026-10-16 13:42:48 +08:00
minke.yu
f76eaf0f92
gateway g9i: gates judge on last-known snapshots; blind fallback round-robins instead of cache_aware pass-through (P2 T18 hole-3)
...
The stale-exemption + blind-pass-through combination fed the 2026-10-16
p3 pileup: the most overloaded node answered /v1/loads slowest, its
snapshot went stale first, and staleness EXEMPTED it from every cap
judgment; with all snapshots stale, regular_plan returned
PrefillPlan::Default and the underlying cache_aware radix policy pinned
every same-prefix session onto that one node (p3: 477 queued, TTFT
~300s) while the drained node (p2) got zero traffic.
a) v1_over_cap / p_real_queue_over / least_wait / sorted_by_wait now use
last-known values: any snapshot that ever fetched OK participates at
any age; only a NEVER-answered node is unknown (unjudged, unpicked).
Stale judgments are counted: smg_g9i_stale_snapshot_judged_total{gate}.
b) All-blind (no node ever answered = true cold start) no longer returns
PrefillPlan::Default — regular_plan round-robins the pool via a new
rr_cursor (decision path regular_blind_round_robin). Partial blind
picks the known least-wait node as before. Prefix affinity lives in
the shadow table (shadow_route), not the underlying policy.
c) reqs_cap=10 / token_cap=16M / hold 1s/512 / HOT-first / stateless
per-request exclusion semantics unchanged; check_decode_pool freshness
(TTL) semantics intentionally untouched.
Tests: v1_over_cap_snapshot_semantics + shadow_path_real_queue_gate_filters
updated to the last-known contract; new t18_least_wait_uses_last_known,
t18_all_blind_round_robin_not_default, t18_partial_blind_least_wait_override.
2026-10-16 13:41:09 +08:00
minke.yu
45fdc97082
gateway g9i: concurrent /v1/loads fetch + detached bootstrap scrape loop + last-known snapshot survival (P2 T18 hole-2)
...
poll_once fetched 4x /v1/loads + 3x P /metrics SERIALLY, each with a 3s
timeout — one round cost up to ~21s against a 5s snapshot TTL, so one
slow node at peak aged the whole pool's snapshots and blinded every
gate (no_fresh_snapshot on ~30% of decisions, 2026-10-16).
- /v1/loads fetches now run concurrently (JoinSet); per-node 3s timeout
unchanged, a round costs the slowest node, not the sum. Poll interval
1s / TTL 5s unchanged.
- The per-P /metrics kv_transfer_bootstrap_ms scrape moves to its own
10s slow loop (concurrent scrapes); it only feeds observability gauges
and must never delay the snapshot round.
- A failed fetch no longer overwrites the snapshot with a zeroed
default: the last-known load survives (merge_poll_result) and keeps
aging via its original fetched_at; a node that never answered stays
truly unknown instead of fabricating '0 queue'.
- New gauges: smg_g9i_poll_duration_seconds (histogram),
smg_g9i_snapshot_age_seconds{worker}.
Tests: t18_poll_failure_preserves_last_known (merge semantics),
t18_poll_fetches_concurrently (two hanging nodes cost one timeout, stay
unknown but remain routing candidates).
2026-10-16 13:33:07 +08:00
Yuwei An
06934575b9
[Fix] DSv4.1: Fix BCG image handling and add multimodal bounded replay coverage ( #44131 )
2026-10-15 22:32:55 -07:00
minke.yu
f690861265
gateway g9i: book D pipeline ledger at decision time for deferred (miss) admissions (P2 T18 hole-1)
...
Root cause of the 2026-10-16 window-gate-blindness incident: for a
prefix-table MISS the D pipeline booking happened in shadow_admit at
response-headers time — and in PD mode headers arrive at ~first-token
time (the dispatch join waits on the prefill answer), so the booking
entered the ledger and was promoted to streaming within milliseconds.
The pipeline gauge pinned at 0 while 585 admitted requests awaited
their first token; the d_window_full gate (pipeline >= W) never fired.
Fix: AdmissionMark gains p_booked + a new_deferred() constructor; the
miss branch of shadow_classify_and_admit books the min-T_D D node into
the pipeline ledger at decision time (P side stays deferred — no node
known). shadow_admit consumes the mark by admitting P on the actual
landing node and rekeying the D booking; shadow_release_unconsumed
releases D (and P only when it was booked). Promote/release paths are
unchanged and conserve: admit = pipeline + streaming, complete/abort
= release exactly once.
Also adds the reconciliation gauge smg_g9i_d_engine_inflight{worker}
(running + prealloc + transfer queue, engine truth) beside the ledger
gauges, so ledger-vs-engine divergence is directly visible.
Tests: t11_miss_defers_admit_to_response rewritten as
t11_miss_books_d_pipeline_at_decision_time; new
t18_deferred_promote_release_conservation (promote idempotent,
mid-stream disconnect release, post-consume fallback no-op) and
t18_deferred_abort_pre_first_token_releases_pipeline (double-release
guard).
2026-10-16 13:28:08 +08:00
2a08e261ec
[Feat] MiniCPM-V-4.7: add model support ( #41993 )
...
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
2026-10-16 13:24:48 +08:00
a2981ab8ea
[Feat] sgl-router: Serve native gRPC ChatComplete ( #42674 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 13:11:36 +08:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
fd4b75c696
[Doc] cookbook: Add SemiAnalysis AgentX launches to Muse Spark Open, refresh DeepSeek-V4.1-Flash ( #44654 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 12:45:07 +08:00
Kolade Fajimi and Xiaoyu Zhang
5117bfbaeb
[Fix] compressed-tensors: Serve NVFP4 W4A4 linears via FP4 Marlin when --fp4-gemm-backend marlin is set on SM100 ( #44213 )
...
Signed-off-by: kolade fajimi <fajimikolade23@gmail.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-16 12:36:02 +08:00
HuangJi
6d70787398
[Fix] diffusion: Align Qwen-Image 2.1 resolution and handle RGBA JPEG output ( #44275 )
2026-10-16 12:35:04 +08:00
Mick and Mick Qian
e5274752a8
[Doc] diffusion: align PR title conventions ( #44646 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-16 11:29:20 +08:00
minke.yu
e4bfdc949b
gateway g9i: AIMD blind-state recovery on real engine signals + real-queue P gate + nonstream e2e metric (P2 T17)
2026-10-16 11:16:32 +08:00
Mick
3420baea8c
[diffusion] CI: batch performance failure-policy runner scenarios ( #44508 )
2026-10-16 11:04:05 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
6231a617d3
[Test] diffusion: Load NVFP4 block scales through the weight loader in tests ( #44595 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-16 10:56:27 +08:00
ashwini rathi and Claude Opus 5.5
75e25a21bd
[Fix] [XPU] nightly: Fix rerank, zimage, flux2 and EAGLE parity ( #44100 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 10:27:51 +08:00
Chetan Kumar Verma and Claude Sonnet 5
75e77fca25
[Fix] wan: fix opt-in Wan2.2 progressive mode and one-step transitions ( #43768 )
...
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com >
2026-10-16 10:22:09 +08:00
Shuwen Wang
f9df33494a
[Fix] scheduler: Keep PD decode non-idle while deferred KV releases are held ( #44386 )
2026-10-15 19:17:51 -07:00
Chetan Kumar Verma and Claude Opus 5.5
1e33d2c4bd
[Perf] [XPU] usp: Enable packed Ulysses QKV all-to-all ( #39989 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-16 10:08:50 +08:00
4e85e8ed37
[Fix] imports: Bind FlashInfer lazily at four import sites so the server starts without it ( #44430 )
...
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com >
Co-authored-by: yeqcharlotte <yeqcharlotte@users.noreply.github.com >
2026-10-15 18:52:33 -07:00
shodoco and metamergebot
2957ff3165
[Fix] sglang-server: Keep the matched stop token in output_ids ( #42637 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
2026-10-15 18:51:47 -07:00
minke.yu
59e3aee6d9
gateway g9i: pipeline-depth window (streaming excluded) + tiered hold queue + 1s hold budget (P2 T16)
2026-10-16 09:43:54 +08:00
sglang-bot and sglang-bot
c15edf314d
[Doc] blogs: Sync LMSYS SGLang blog cards ( #44616 )
...
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com >
2026-10-16 00:52:14 +00:00
e3961eb6fa
[Fix] utils: Allowlist torch.bcomplex32 in SafeUnpickler ( #44440 )
...
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com >
Co-authored-by: zx3xyy <zx3xyy@users.noreply.github.com >
2026-10-15 17:50:43 -07:00
f110b2f801
[Feat] rust-server: Report cached_tokens and reasoning_tokens in meta_info ( #44531 )
...
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-15 17:50:27 -07:00
d393298736
[Fix] tokenizer_manager: re-report only emitted customized_info keys on cumulative streams ( #44537 )
...
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-15 17:49:49 -07:00
metamergebot and Ye Qi
bda6ef7e01
[Perf] [NVIDIA] scheduler: Pageable host staging and inline hidden-state/DSpark readback under Confidential Computing ( #44568 )
...
Co-authored-by: Ye (Charlotte) Qi <7269575+yeqcharlotte@users.noreply.github.com >
2026-10-15 17:48:50 -07:00
ed8c3139f1
[Fix] deepseek-v4.1: Keep the full SWA window for short bounded-replay extends ( #41303 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-15 17:32:50 -07:00
minke.yu
406081d3e3
gateway g9i: admission hold queue (429 -> park) + AIMD window rebalance to 1x/0.25x capacity (P2 T15)
2026-10-16 08:27:52 +08:00
Khoa Pham
a93f16f0cf
[Perf] deepseek-v4.1: Integrate FlashMLA MegaAttention with NVFP4 KV ( #43922 )
2026-10-15 17:21:43 -07:00
Liangsheng Yin
d14fa5db9b
[Perf] pd: Copy prebuilt decode batch inputs to the GPU without host syncs ( #44598 )
2026-10-15 17:20:02 -07:00
Khoa Pham
4cca7fcd3e
chore: bump sgl-kernel version to 0.4.10 ( #44590 )
2026-10-15 15:50:51 -07:00
ec7ab9d0c2
[Feat] [AMD] monokernels: one runtime switch for vendor monokernel plugins ( #44353 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: wunhuang <wunhuang@amd.com >
2026-10-15 15:50:25 -07:00
0b1a73599a
[Feat] rust-server: Apply the x-override-* request headers ( #44312 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-15 15:50:04 -07:00
William Hu
4e0609ea27
[Fix] moe: Preserve negative routing sentinel in EP mapping ( #40498 )
2026-10-15 15:05:08 -07:00
Yuwei An
91f6ef2322
[DSv4.1] Fuse request-window KV copies ( #44555 )
2026-10-15 14:11:51 -07:00
Xinyuan Tong
877be2974a
[CI] pr-test: Let stage wait jobs reach their max-wait budget ( #44410 )
2026-10-15 13:34:49 -07:00
snadampal
aa5c7e887c
[Feat] docker: Add AWS EFA image variants built on published images ( #41006 )
...
Signed-off-by: Sunita Nadampalli <nadampal@amazon.com >
2026-10-15 13:19:22 -07:00
Ziyi Xu and Claude Opus 5.5
bc76a6b819
[DSv4] Sparse prefill: build compressed lengths without a host sync ( #44557 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 12:45:56 -07:00
87211088c9
perf: fix overlap scheduling for NVIDIA Confidential Computing(CC) on Blackwell ( #36810 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Po-Han Huang <pohanh@nvidia.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-15 12:09:09 -07:00
02a43eeef6
[Feat] rust-server: Attach configured metadata headers to a healthy /health_generate ( #44315 )
...
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-15 12:04:23 -07:00
173e0f2e9b
[Feat] hicache: Add opt-in tail-only SWA backups in buffer mode ( #44173 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-15 12:02:59 -07:00
Michael
bba535ebb4
[Fix] [AMD] docker: Repair the AITER build step after the 2bb549102 pin bump ( #44534 )
2026-10-15 11:18:41 -07:00
c5a5bf0d0b
[Fix] pp: Order output send/recv by rank parity to fix cross-node hang ( #44453 )
...
Co-authored-by: chuixue <chuixue@minimaxi.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-16 01:46:37 +08:00
minke.yu
1975265315
gateway g9i: AIMD congestion window replaces slot-estimate admission gate (P2 T14)
2026-10-16 01:33:18 +08:00
Shangming Cai
c7fb57d990
[Fix] sgl-router: Bump rustls to 0.23.46 for RUSTSEC-2026-0346..0349 ( #44509 )
2026-10-16 01:28:57 +08:00
a8aaf95b58
[Cookbook] Add cookbook for muse spark open ( #44527 )
...
Co-authored-by: thomawan <thomawan@amd.com >
Co-authored-by: HAI <hixiao@gmail.com >
Co-authored-by: jianan-gu <jianan.gu@intel.com >
Co-authored-by: Thomas Wang <1am9trash@gmail.com >
Co-authored-by: zijie <zijie.chen@amd.com >
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
2026-10-15 10:20:38 -07:00
Aleksi Vesanto and jacky.cheng
5ef8319f16
[Perf] [AMD] attention: Enable diffusion BF16 attention and MXFP4 compilation ( #27702 )
...
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-10-16 00:40:41 +08:00
Yihao Wang and Claude Opus 5.5
5b12ec163c
[Doc] cookbook: Expand the FLUX.3 Image Klein model description ( #44515 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 08:56:54 -07:00
Jimmy Shong and Jimmy Shong
05cbe241be
[Doc] cookbook: Add Nemotron 3.5 Super VL ( #44510 )
...
Co-authored-by: Jimmy Shong <jimmy.shong@mac.attlocal.net >
2026-10-15 08:39:14 -07:00
28b98e5d13
[Doc] cookbook: Add the FLUX.3 prompt upsampler section ( #44512 )
...
Co-authored-by: Sandro Cavallari <scavallari@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 08:38:33 -07:00
minke.yu
534417b82e
gateway g9i: D-side in-flight ledger + Little's-law slot release rate (P2 T13)
2026-10-15 23:21:42 +08:00
YC Yen-Ching Tseng
e87e32b311
[Feat] [AMD] docker: Release ROCm 10.1 for gfx1250 in parallel with ROCm 10.0 ( #42699 )
2026-10-15 22:59:54 +08:00
a57707377f
[Fix] sgl-router: Send disagg_prefill_dp_rank to decode under --dp-aware ( #44072 )
...
Co-authored-by: Shangming Cai <csmthu@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-15 22:42:33 +08:00
qiaonan02
c4870d98cf
[Fix] glm5_next: Correct num_experts_per_tok default to 8 ( #44093 )
2026-10-15 21:53:31 +08:00
09b7383624
[diffusion] model: support FLUX.3 image ( #44418 )
...
Co-authored-by: Dom Brown <3886319+DomBrown@users.noreply.github.com >
Co-authored-by: niehen6174 <1639206518@qq.com >
Co-authored-by: Mick Qian <mickqian@radixark.ai >
Co-authored-by: Nikita Korobov <14355239+nekorobov@users.noreply.github.com >
Co-authored-by: Daniel Campora <961215+dcampora@users.noreply.github.com >
Co-authored-by: Richard Wang <11150595+rwang5203@users.noreply.github.com >
Co-authored-by: BBuf <1182563586@qq.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-15 21:50:59 +08:00
elvischenv and Claude Opus 5.5
6fe5596ebb
Support EPLB for Nemotron-H ( #43685 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 13:50:09 +00:00
ormandj and Benjamin Oldenburg
c5bf3aafbc
Fix overflow in legacy and v1 DSA top-k selectors ( #37625 )
...
Co-authored-by: Benjamin Oldenburg <benjamin.oldenburg@ordis.co.th >
2026-10-15 21:44:58 +08:00
minke.yu
ed0a454730
gateway g9i: T_D slot (max_running) dimension — max(memory, slot) wait with dual EWMA (P2 T12)
2026-10-15 21:41:03 +08:00
jiaqiang-dot-liu and Cursor
2f5407da28
[Fix] [AMD] qwen3: Fix KV cache write in the fused QK-norm + mRoPE decode path ( #44266 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-15 21:22:46 +08:00
sogalin_codegen
2092160d42
[AMD][Diffusion] Route unquantized linears through aiter tgemm on gfx1250 ( #41988 )
2026-10-15 21:08:44 +08:00
amote-i
abf92826ee
[NPU] [DOC] update NPU best practices and model deployment docs to CANN 9.1.0 ( #44432 )
2026-10-15 20:22:54 +08:00
Zhang Yiyang (SII) and Mick Qian
23c2a026db
[diffusion] model: support MOSS-Image ( #44325 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-15 19:23:56 +08:00
minke.yu
04ee2d03d7
gateway g9i: atomic check-and-act admission at decision time (P2 T11)
2026-10-15 18:20:01 +08:00
minke.yu
9d5db13e53
gateway g9i: P-queue hard token cap with candidate fallback (P2 T10)
2026-10-15 17:38:17 +08:00
Cheng Wan
f7ad4345b8
[Refactor] deepseek-v4: Use explicit stage boundaries and row transport ( #43607 )
2026-10-15 02:05:28 -07:00
Cheng Wan
3a835cf417
[Refactor] inkling: Use short-convolution stage boundaries ( #43606 )
2026-10-15 02:04:46 -07:00
Cheng Wan
e1f9fce7b3
[Refactor] layer-boundary: Bind residual completion to producer updates ( #43605 )
2026-10-15 02:04:02 -07:00
Cheng Wan
d59c3448b0
[Fix] kimi-k3: Preserve pipeline graph bank inputs and bound chunk profiling ( #43604 )
2026-10-15 02:03:27 -07:00
Eddie Zhou
71a17dbc24
[AMD][CI] setup: Skip redundant pip-cache chown walk and MORI rebuild ( #42148 )
2026-10-15 16:21:46 +08:00
93dbe88183
Fuse MiMo verification RoPE and KV cache writes ( #43106 )
...
Co-authored-by: lianghougong <lianghouhong@xiaohongshu.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-15 16:15:24 +08:00
kangwangamd
28703575f9
[AMD] Update ROCm AITER pin to 2bb549102 ( #44379 )
2026-10-15 16:10:41 +08:00
GeorgeSherif and Xiaoyu Zhang
8f853b0041
[Doc] cookbook: Add Jais 3 deployment recipes ( #42766 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-15 16:09:44 +08:00
86f4ede4fb
[Fix] dsa: Make the fused top-k exact on an overflowing threshold bin ( #37941 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: DarkSharpness <2040703891@qq.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-15 15:52:48 +08:00
9ea5472833
[HiCache] Add the page_unified host KV layout and the unified L3 key ( #39727 )
...
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com >
Co-authored-by: Zhangheng <hzh0425@apache.org >
2026-10-15 15:46:13 +08:00
33618d4f00
[Feat] runtime: Include FlashInfer’s default tactic in startup tuning. ( #43927 )
...
Co-authored-by: Po-Han Huang <pohanh@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 15:30:45 +08:00
Liangsheng Yin
a776851c97
[Refactor] scheduler: Check the prefill budget at the admission loop head; drop a redundant PD gate ( #44420 )
2026-10-15 00:16:56 -07:00
fa66e7d816
[NPU] [Bugfix] Fix rope sin cos pre-calculation for Qwen3.5 during Pipeline Parallelism ( #29753 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Артем Савкин <58187114+OrangeRedeng@users.noreply.github.com >
2026-10-15 09:58:47 +03:00
ashwini rathi and Claude Opus 5.5
01ac2cf36d
[CI] [XPU] nightly: Split Intel Main and Wheel workflows ( #44108 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-15 14:56:27 +08:00
Mick and Mick Qian
f2b73d4f9b
[diffusion] doc: document reference resizing, LoRA auto mode and image profiling ( #44377 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-15 13:41:44 +08:00
Dayuxiaoshui
a50c17e979
[diffusion] perf: fuse the Wan VAE conv bias epilogue and tile the RMSNorm+SiLU kernel ( #38650 )
2026-10-15 13:39:27 +08:00
676c038aa5
[diffusion] fix: fix the load duration missing from subclassed server tests ( #40657 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
Co-authored-by: Michael <michaelzhang-ai@users.noreply.github.com >
Co-authored-by: quitenode <quitenode@users.noreply.github.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-10-15 13:38:23 +08:00
minke.yu
7e20f0af79
gateway g9i: P2 T_D admission gates (off|shadow|enforce) + T_PREFER candidate fallback (T9)
2026-10-15 13:35:50 +08:00
Liangsheng Yin
f9bdb74a5a
[Fix] scheduler: Count a request's held req_to_token row once at admission ( #44401 )
2026-10-14 22:29:10 -07:00
Yuxingwang-intel
93842f52fe
[CPU]: Add group-aware CPU SHM collective kernels ( #35435 )
2026-10-15 13:22:20 +08:00
Michael
4dc9b5c68c
[Test] [AMD] diffusion: Expect CUDA-only fast paths to be off on ROCm in kernel tests ( #44311 )
2026-10-15 13:18:01 +08:00
1d809bbbbd
[Fix] DSv4.1: Size the encoder-replay SWA pool by the DSpark draft's stages, and keep more prefix tails ( #43502 )
...
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
Co-authored-by: Po-Han Huang <pohanh@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com >
2026-10-14 22:04:58 -07:00
kk and root
e573d91275
[CI] tests: Fix stale fakes in the CP strategy and request-class override unit tests ( #44380 )
...
Co-authored-by: root <root@crsuse2-m2m-223.us-east2-a.compute.internal >
2026-10-14 21:35:20 -07:00
Khoa Pham
d4daf52369
[Fix] dsv41: Bound MoE routing to live SWA graph-tail rows ( #44347 )
2026-10-14 21:14:28 -07:00
AMD-yanfeiwang
08f15e76bb
[Feat] sgl-router: Add --dp-rank-policy round_robin for --dp-aware ( #43511 )
2026-10-15 12:03:01 +08:00
WenhaoZhang and Mick
c92108bb93
[diffusion] comfyui: grouped SGLDOptions, native model loader and model-aware preflight ( #44058 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-15 11:59:03 +08:00
Liangsheng Yin
3b5ae4ea86
[Fix] scheduler: Drop finished requests from the running batch before every admission ( #44351 )
2026-10-14 20:40:05 -07:00
2528cd2b91
[PD] Send DSA indexer state only past decode_prefix_len ( #44140 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-15 11:25:18 +08:00
Avaya Aggarwal and niehen6174
0cf683ea8d
[diffusion] comfyui: return several server-mode images as one IMAGE batch ( #44088 )
...
Co-authored-by: niehen6174 <1639206518@qq.com >
2026-10-15 10:38:37 +08:00
Thomas Wang
4851d182a1
[Doc][AMD] Update v41 amd cookbook 1015 ( #44368 )
2026-10-14 19:33:08 -07:00
xiaofei-zheng
a12814c158
[Perf] [AMD] dsa: Enable the gfx950 fused DSA indexer for block-FP8 wq_b/wk ( #43643 )
2026-10-15 09:36:16 +08:00
Yuhao Yang and Claude Opus 5.5
9fa9308e95
[Doc] cookbook: Add Step 5 Preview cookbook ( #44357 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-15 09:21:06 +08:00
9d68ecce2e
[DeepSeek-V4.1] DeepSelect cluster top-k for long rows and bf16 index logits ( #44051 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-15 09:00:19 +08:00
xiaofei-zheng
c5c4bbbd25
[Perf] [AMD] dsa: Run DSA prefill under breakable CUDA graphs on HIP ( #43642 )
2026-10-15 08:59:34 +08:00
Michael and jacky.cheng
8333ccfccb
[Fix] [AMD] diffusion: Compare ReduceOp by identity so the compiled TP all-reduce stays one graph ( #43770 )
...
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-10-14 17:40:07 -07:00
Yuwei An
64a0a08f0a
[DSv4.1] Support TRT-LLM decoder bounded replay with BCG ( #44326 )
2026-10-14 17:39:52 -07:00
YAMY and Yuwei An
c891c5c7ca
[DSv4.1] Fold encoder-SWA replay into the extend forward ( #43166 )
...
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com >
2026-10-14 17:27:07 -07:00
Shuwen Wang
3c28614f12
[Fix] hicache: Defer query beliefs until the rank-synchronized hit drain ( #43108 )
2026-10-14 17:25:08 -07:00
Aditya Singh and hnyls2002
a6a8c56015
[Fix] detokenizer router: Route by rid without tokenizer worker ipcs and broadcast logging config ( #33715 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-14 17:20:21 -07:00
0536cec73d
[Feat] metrics: Expose multi-detokenizer router CPU usage ( #43323 )
...
Co-authored-by: raghul chandramouli <raghul@c1xind0180-0243.local >
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-14 17:17:22 -07:00
Junnan Li and hnyls2002
0f36c8015c
[Bug] Fix multi-tokenizer BatchEmbeddingOutput repack dropping fields ( #31892 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-14 16:56:56 -07:00
209f8220d6
[Fix] sglang-server: Honour extra_key on the Rust frontend's /generate ( #44207 )
...
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-14 16:44:31 -07:00
e7d6a88bfc
[HiCache] Buffer-only L3 for an auxiliary SWA window; load back only missing window rows ( #43932 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: houseroad <houseroad@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-14 16:43:24 -07:00
aa8793f33b
[Rust] Serve scheduler metrics on the Rust frontend's /metrics ( #43942 )
...
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-14 16:43:06 -07:00
2a9f6a25a7
[Core] Add subclass hooks for the generation request class and optimistic prefill eligibility ( #44113 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: CuiCoco <CuiCoco@users.noreply.github.com >
2026-10-14 16:42:30 -07:00
5f2861bf59
[Perf] session: Cache multimodal prompt token counts across turns ( #44112 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com >
2026-10-14 16:41:34 -07:00
9732188a58
[Rust] Serve /weights_checker from the Rust frontend ( #43926 )
...
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-14 16:41:10 -07:00
FREETRUMP and hnyls2002
97d3150a6a
[Perf] Cache msgspec struct fields in the batch splitter ( #43520 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-14 14:51:26 -07:00
4273814771
[Perf] detokenizer: Group return IPC sends by tokenizer worker ( #44288 )
...
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-10-14 14:26:51 -07:00
Charles Chen and alphabetc1
d3dc14eb38
[mem_cache][PD] Fix decode radix cache ownership and allow EAGLE/EAGLE3 with decode radix cache ( #40857 )
...
Co-authored-by: alphabetc1 <2508695655@qq.com >
2026-10-14 14:19:51 -07:00
Rain Jiang
12dc623102
[Chore] rust: Remove the sglang-renderer crate ( #43451 )
2026-10-14 11:43:37 -07:00
Shuwen Wang
b67785153a
[Fix] glm5.2: Align HiSparse warmup backends and move test to nightly ( #44303 )
2026-10-15 01:28:03 +08:00
Yoray Zack
87ed1b9526
[4/N] elastic-ep: Scale at DP-attention replica granularity ( #33728 )
2026-10-15 01:17:01 +08:00
c964d9ba20
[Feat] sgl-router: Separate LoRA block IDs from prompt lookup keys ( #42818 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-15 00:50:19 +08:00
amote-i
3a9cae27ed
[Doc] [NPU] docs: Update NPU CANN version to 9.1.0 ( #44177 )
2026-10-15 00:27:18 +08:00
Michael
1737251730
[Test] [AMD] diffusion: Run the IPC all-to-all transport tests only on CUDA ( #44096 )
2026-10-14 23:04:23 +08:00
010776e94f
[KV events] Publish opaque LoRA block IDs and adapter names ( #42817 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-14 22:56:28 +08:00
Xinyuan Tong
ed40c7fc66
[Doc] cookbook: add Ling-3.1-flash ( #44301 )
2026-10-14 22:40:01 +08:00
minke.yu
0b74ab2ac4
gateway g9i: shadow-consistent routing switch --g9i-shadow-route (P1 shadow T7)
2026-10-14 22:22:58 +08:00
minke.yu
3c7ec33d26
gateway g9i: est_T_D shadow metrics — est_tp/est_td histograms, residual by dominant component, pool gauges (P1 shadow T5)
2026-10-14 21:52:07 +08:00
minke.yu
09ca4957ee
gateway g9i: scrape P /metrics kv_transfer_bootstrap_ms as est_T_D ground truth (P1 shadow T4)
2026-10-14 21:50:11 +08:00
minke.yu
7921442a5c
gateway g9i: est_TTFT max(T_P, min_D T_D) composition with C4/C13 degradation (P1 shadow T3)
2026-10-14 21:44:17 +08:00
minke.yu
05ef104208
gateway g9i: pool_free_rate completion EWMA per decode node (P1 shadow T2)
2026-10-14 21:34:56 +08:00
minke.yu
d44620f285
gateway g9i: fetch disagg/queues sections in /v1/loads, extend NodeLoad for est_T_D (P1 shadow T1)
2026-10-14 21:21:11 +08:00
Kan Wu and Claude Opus 5.5
d2a436446c
[Feat] sgl-router: Fix cache-aware routing for MTP/EAGLE on hybrid (Mamba/KDA) models ( #44252 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 21:21:01 +08:00
204644c2dc
[Fix] sglang-processor: Follow Python's OpenAI layer through #43027 , #37577 , #43006 , #43026 , #43005 ( #44243 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-14 21:06:13 +08:00
triple-mu
5e90312eb1
[diffusion] perf: support bit-exact norm/bias fusions and channels_last decode for Qwen-Image 2.1( #41340 )
2026-10-14 20:47:46 +08:00
Shuwen Wang
7b53a13401
[Fix] glm5.2: Align PD HiSparse prefill KV layout ( #44256 )
2026-10-14 20:40:28 +08:00
bab9c81fea
fix: FA4 paged-KV zero-size page-table registers on SM90 ( #35757 )
...
Co-authored-by: razorback16 <razorback16@users.noreply.github.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: BBuf <1182563586@qq.com >
2026-10-14 19:10:15 +08:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
3b72b70ef4
[CI] vlm: Pin baidu/Unlimited-OCR revision in test_unlimited_ocr_server ( #44235 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 17:50:18 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
c82e8bd5fe
[Fix] DeepSeek-V4.1: Align paged indexer score rows for DeepSelect decode ( #44210 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-14 17:31:19 +08:00
df1a089cd3
[Spec] Fix DFLASH verify mask policy for wrapped attention backends ( #43054 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-14 17:15:36 +08:00
817781b828
[Feat] sgl-router: Render Qwen3.8 (Max, 27B, Flash-Next) through sglang-processor ( #43689 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-14 17:09:35 +08:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
72730d6360
[Doc] cookbook: Mirror SA cluster NCCL_CUMEM_ENABLE in AgentX launches, fix GPU counts ( #44228 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 16:07:40 +08:00
c54ba66cd4
[Perf] Enable native MXFP4 routing and CuTeDSL fusion for MiMo ( #43639 )
...
Co-authored-by: lianghougong <lianghouhong@xiaohongshu.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-14 15:39:44 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
6daae9c98e
[Perf] Bind scheduler CPUs to the GPU's NUMA node when memory policy is not permitted ( #44023 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-14 00:01:32 -07:00
58f2c923ed
[Feat] sgl-router: skip engines at an admission cap in power-of-two; forward x-sglext headers ( #44218 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Codex <codex@openai.com >
2026-10-14 14:59:06 +08:00
Kan Wu and Claude Opus 5.5
53a24c448e
[Feat] sgl-router: Render Kimi-K3 through sglang-processor ( #43608 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 23:40:18 -07:00
Michael
6f646563a2
[AMD] diffusion: expect the residual-gate fast path to be off on ROCm in test_modulate ( #40660 )
2026-10-13 23:34:31 -07:00
Ma Mingfei
68c18a5701
Revert "[XPU] Support Gemma-3 bidirectional image attention in XPUAttentionBackend" ( #44216 )
2026-10-14 14:27:46 +08:00
desmond-intel
b87d62e581
[XPU] Support Gemma-3 bidirectional image attention in XPUAttentionBackend ( #39805 )
2026-10-14 14:24:04 +08:00
Shunkangz
0d130f1915
[Qwen3.8 CP 4/4] Prefill context parallel E2E support of Qwen3.8 ( #39724 )
2026-10-13 23:19:46 -07:00
7ca4cbfcef
[Disagg] Return prompt routed experts from the PD prefill server ( #44002 )
...
Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-10-13 23:13:44 -07:00
AndyLi429 and AndyLi429
05bb5afb62
[NPU][Bugfix] use CANN libdevice for DSpark Markov sampling ( #43849 )
...
Co-authored-by: AndyLi429 <AndyLi429@noreply.gitcode.com >
2026-10-14 14:12:22 +08:00
Zhaoyi Li
e29eee18df
[AMD][DI][CI] nightly mi355x disagg: stage packed logs under RUNNER_TEMP, not /tmp ( #43945 )
2026-10-14 13:53:40 +08:00
Kan Wu and Claude Opus 5.5
4560da58ee
[sgl-router] Render GLM-5.3 and GLM-5.3-Flash through sglang-processor ( #43625 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 13:19:44 +08:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
d5f80aa3c3
[Test] dsv4: Fix Hopper indexer replay test after DeepSelect decode ( #41561 ) ( #44193 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 13:15:35 +08:00
ANSHUMAN TRIPATHY
30bdd09aed
DSPARK enable for XPU ( #33739 )
2026-10-14 13:10:24 +08:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
721953c6f6
[Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX main (GLM-5.2 NVFP4 B300 agg TP4, Dynamo frontend counts) ( #44065 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 13:03:51 +08:00
Khoa Pham
e00754bab1
[DSV4.1] Derive the late-layer DP geometry from the captured shape under the breakable prefill graph ( #44144 )
2026-10-13 21:54:35 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
ff8d964cf5
[CI] glm5.2: Temporarily disable PD HiSparse+MTP B300 test ( #44192 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-13 21:53:58 -07:00
chenxb002 and chenxiaobing
491d4ef178
[MLU][CI] Bootstrap MLU runner workflows ( #41005 )
...
Co-authored-by: chenxiaobing <chenxiaobing@cambricon.com >
2026-10-13 21:40:49 -07:00
Chetan Kumar Verma and Claude Opus 5.5
2c1e85964e
[diffusion] cache-dit: respect CFG parallel for dual-transformer models ( #43704 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-14 12:25:07 +08:00
xinchen9 and Ma Mingfei
2428d0f531
[XPU] Use the shared fused decode-metadata kernel in the intel_xpu ba… ( #42122 )
...
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com >
2026-10-14 12:10:12 +08:00
16176278d0
[diffusion] perf: replay small Triton wrappers' launches per operand signature ( #43859 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 11:18:19 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
4f751da975
[Agents] Add a rule to prefer sgl-eval for accuracy evals ( #44040 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-13 20:17:04 -07:00
53fe21cdfd
[diffusion] perf: replace padded rows without a host sync for Z-Image ( #44069 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-14 11:15:28 +08:00
xiaofei-zheng and Xiaoyu Zhang
7e226b1b71
[AMD] Use the preallocated EAGLE topk=1 draft chain on HIP ( #43640 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-14 11:06:07 +08:00
5efb7384c1
[diffusion] comfyui: consolidate 6 unrelated plugin fixes ( #43493 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
Co-authored-by: niehen6174 <1639206518@qq.com >
2026-10-14 10:56:09 +08:00
Cheng Wan
c053fc71d3
[CI] pr-title: Validate pull request title format ( #44165 )
2026-10-13 19:53:52 -07:00
Ziyi Xu and Claude Opus 5.5
4f5fcbfef5
[JIT] Clean up kernel headers: move runtime helpers, add once.h ( #41147 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-14 10:46:51 +08:00
yuyu5333 and Claude Code
b50fdeb0ef
[DeepSeek V4.1] Use DeepSelect for Hopper full top-k decode. ( #41561 )
...
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-14 10:46:12 +08:00
Ma Mingfei and MingxuZh
87beb3a052
[CPU] Temporarily disable subblock sparse attention test on Xeon CI ( #43468 )
...
Co-authored-by: MingxuZh <109504044+MingxuZh@users.noreply.github.com >
2026-10-14 10:44:24 +08:00
13a6dee99b
[Fix] moe_sum_reduce: gate the bf16 vectorized path on hidden % 16 ( #42837 )
...
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-14 10:32:50 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
7f0434e4af
[GLM-5.3-Flash] Drop the NextN layer from the KDA state pool ( #44098 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-13 19:31:02 -07:00
Avaya Aggarwal
ac187c0cd5
[diffusion] Bound frame_interpolation_exp to RIFE's supported range ( #43014 )
...
Signed-off-by: onepunchmonk <aggarwal.avaya@yahoo.com >
2026-10-14 10:29:28 +08:00
Jin and Xinyuan Tong
44b44774c3
[Bugfix] Keep rope_theta when --json-model-override-args overrides rope_scaling ( #43067 )
...
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
2026-10-14 10:16:35 +08:00
Baizhou Zhang
536d065370
[CP] Automatically enable BCG for supported prefill CP configurations ( #44022 )
2026-10-13 19:00:42 -07:00
shyeh25 and Claude Opus 5.5
684fea8af3
[DSV4.1] Skip Mega mHC fusion on 0-row (DP-idle) batches ( #43993 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 16:36:15 -07:00
Liangsheng Yin
2ebdf9360b
[Fix] Work around a CPython fork deadlock in the CPU preprocess pool ( #44143 )
2026-10-13 16:31:57 -07:00
b3991b53da
GLM5.2 DSA Indexer Fallback ( #32947 )
...
Co-authored-by: Shu Wang <shuw@login-hecate03.hecate.clusters.nvidia.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-13 16:27:33 -07:00
Khoa Pham
5917604a95
[DSV4.1] Support decoder SWA bounded replay with CUDA DP attention ( #43812 )
2026-10-13 16:10:57 -07:00
HEJIAN SANG and Claude Opus 5.5
a832609b92
[Feature] Return output top logprobs as flat base64 arrays on request ( #42890 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 15:58:31 -07:00
baa632b049
[Score] Send stacked GPU embedding overrides through a CUDA IPC pool ( #39730 )
...
Co-authored-by: Sundara Raman Ramachandran <sundar24295@gmail.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-13 15:46:50 -07:00
Andrew Briand
08550acc28
Mark FP32 redux asm volatile ( #44137 )
2026-10-13 15:08:52 -07:00
Jiaqi Li and Zijie Xia
5569b14d26
fix(benchmark): request token IDs when counting OpenAI prompts ( #34511 )
...
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai >
2026-10-13 14:31:45 -07:00
Yizheng Jiao
2341e647d3
[DeepSeek-V4.1] Run decoder SWA bounded replay inside the breakable prefill CUDA graph ( #43736 )
2026-10-13 13:55:09 -07:00
79af3df346
[gRPC] Move runtime.v1 bindings to tonic 0.14, report OpenAI status codes, return text on request ( #42673 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 13:46:15 -07:00
huangtingwei
18210ed849
[HiSparse] Add speculative decoding support with multi-step KV cache swapping ( #37771 )
2026-10-13 13:38:31 -07:00
Xinyuan Tong and Nihar Patel
7dc2930585
[Feature] Add Aaronson-Gumbel text watermarking ( #37577 )
...
Co-authored-by: Nihar Patel <nihpat95@gmail.com >
2026-10-13 12:47:50 -07:00
Xinyuan Tong
5eda6b7e67
docs: add MiniCPM-V 4.7 cookbook ( #43829 )
2026-10-13 12:04:47 -07:00
ce5352a814
[Rust] Serve any model name on the OpenAI chat and completions routes ( #43923 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-13 11:58:57 -07:00
01f3ed18a5
[unified memory pool] Prefer SWA reclaim quotas with a HiCache storage backend or on PD decode ( #43919 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com >
2026-10-13 11:58:24 -07:00
7e3da1b784
[HiCache] Extract the buffer-mode prefetch and load-back lifecycle into pipeline operations ( #43870 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-13 11:51:44 -07:00
5bcaf0f8ed
[Core] Add request-level customized_info keys and mark Full prefill capture batches ( #43772 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com >
2026-10-13 11:45:52 -07:00
149ef7d1b7
[Rust] Let an embedding package wrap the frontend HTTP app ( #43924 )
...
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-13 11:44:43 -07:00
a8381d73be
[HiCache] Block-group pool hits, caller-owned load destinations, allocation-target FULL eviction ( #43868 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-13 11:39:48 -07:00
Khoa Pham and vincentzed
8bf145d119
[DCP] Decode context parallelism for DSA (GLM-5.2 / GLM-5.3) with MTP ( #43735 )
...
Co-authored-by: vincentzed <207368749+vincentzed@users.noreply.github.com >
2026-10-13 11:35:34 -07:00
6ade23696d
[Core] Keep request priority within the signed 64-bit range in radix caches ( #43920 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: qandrew <qandrew@users.noreply.github.com >
2026-10-13 11:33:26 -07:00
7f9f742535
[Rust] Refuse an unsupported multimodal model before the Rust listener binds ( #43925 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-13 11:30:43 -07:00
2a8e9db3b0
[lora] Support LoRA for the NemotronH_Nano_VL_V2 model family ( #39798 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Po-Han Huang <pohanh@nvidia.com >
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com >
2026-10-13 11:29:48 -07:00
ff3f8c348c
[Observability] Report per-request storage prefetch and staging waits in meta_info ( #43946 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-13 11:13:01 -07:00
Rain Jiang
e98c6d7b25
Rainj me/rust bench ( #43950 )
2026-10-13 11:07:42 -07:00
Kailash Ranganathan and Yanbin Jiang
c0584cf9af
[perf] [multi-lora] remove empty CTAs from fused LoRA expand and add split-K to shrink ( #41951 )
...
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com >
2026-10-13 10:16:14 -07:00
Zhaoyi Li
e9400c5a8f
[AMD][DI][CI] kimik26 2p1d-ep16: capture CUDA graph on decode (eager decode is slow) ( #43985 )
2026-10-13 08:46:53 -07:00
Zhaoyi Li
96f519e4fa
[AMD][DI][CI] glm52 2p1d-ep16: capture CUDA graph on decode (eager decode is too slow) ( #43977 )
2026-10-13 08:45:46 -07:00
11cd2701a8
fix(openai): preserve zero cached tokens when cache reporting is enabled ( #43005 )
...
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
2026-10-13 23:29:19 +08:00
3f1e185f6e
fix(openai): keep chat stream created timestamp stable ( #43026 )
...
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
2026-10-13 22:43:33 +08:00
jiaqiang-dot-liu and Cursor
4c5ed0597e
Skip fully-masked KV tiles in sliding-window extend attention ( #39769 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-13 22:40:53 +08:00
27b68303d0
fix(openai): omit absent tool_calls from chat responses ( #43027 )
...
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
2026-10-13 20:56:58 +08:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
8e37c6cf37
[CI] Run test_pi05_modelopt_fp8_cuda in the kernel-unit lane ( #44055 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 20:39:02 +08:00
Ho-Ren (Jack) Chuang and Xiaoyu Zhang
5100f8d11d
[DSA] Q8KV8 FP8 Sparse Prefill on GLM-5.3-Flash with BF16 KV Cache ( #41790 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-13 20:30:13 +08:00
amote-i
29974d543e
[NPU] [DOC] update kimi k3 cookbook and supported features on NPU ( #44052 )
2026-10-13 20:27:26 +08:00
jlqibm and Claude Opus 5
74e0a47863
Fix/compressed tensors nvfp4 hopper ( #40022 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-13 20:10:31 +08:00
LittleNyima and Mick Qian
5727f08932
[diffusion] feat: add configurable reference image resizing for MiniMax H3 Ref2VA ( #35839 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-13 19:48:18 +08:00
WenhaoZhang
85254b5a8f
[diffusion] comfyui: run Qwen-Image 2.1 in SGLD integrated mode ( #43581 )
2026-10-13 19:47:02 +08:00
Yunhao Zhou and 周云昊
991a29d0ad
fix(multimodal): convert grayscale tensors to RGB in PIL backend ( #42127 )
...
Co-authored-by: 周云昊 <zhouyunhao@zhouyunhaodeMacBook-Pro.local >
2026-10-13 19:03:48 +08:00
4b5b727145
Fix EVS for NemotronH_Omni_Reasoning_V3 ( #42933 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-13 10:59:00 +00:00
minke.yu
5df4f2c036
DEBUG-ONLY: use crc32 for rid fingerprint (builtin hash() is per-process randomized)
2026-10-13 18:12:56 +08:00
minke.yu
db1e8cdbe3
DEBUG-ONLY: ARDBG batch fingerprint + phase + hicache sync logs, sampled device AR (1/1024) + idle spin sampling (p2 wedge trap)
2026-10-13 18:01:36 +08:00
Michael and Claude Opus 5.5
aa15b06abe
[AMD][CI] Run test_tracing.py in the extra tier, matching CUDA ( #43928 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-13 02:45:58 -07:00
Xiaoyu Zhang
4e0fa52530
[Perf] Compact GDN causal-conv grids for skewed prefill batches ( #43570 )
2026-10-13 17:45:06 +08:00
Ziang Li
01f6840693
[FlashInfer] Add startup autotuning for MegaMoE ( #43789 )
2026-10-13 02:30:31 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
195cb7afb4
[CI] Use sgl-eval for GLM-5.3-Flash GSM8K tests ( #43975 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-13 02:26:50 -07:00
Shu Wang and Anish Shanbhag
e0c957e222
[HiCache][PD] Allow DSA index-K elision under HiCache and disaggregation ( #41769 )
...
Co-authored-by: Anish Shanbhag <ashanbhag@nvidia.com >
2026-10-13 02:07:54 -07:00
Shu Wang and Po-Han Huang
27be99ae73
[NVIDIA][MegaMoE] Fix the default spec decoding moe backend when the target model uses flashinfer megamoe ( #40772 )
...
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-13 02:00:24 -07:00
minke.yu
f3ac513d92
DEBUG-ONLY: limit ARDBG stack logging to ONE_SHOT_PUSH calls (volume control)
2026-10-13 16:01:18 +08:00
minke.yu
0054c5b8fe
DEBUG-ONLY: SGLANG_AR_SEQ_DEBUG seq logging for TP device AR + host CP gloo AR (p2 wedge repro)
2026-10-13 15:52:17 +08:00
Chunan Zeng and Claude Opus 5.5
7ff3a11284
[Perf] Enable fast-math for JIT activations on Blackwell ( #43236 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 15:18:05 +08:00
AMD-yanfeiwang
d8f8522348
[router] power_of_two: in-flight request load for --dp-aware decode ( #43483 )
2026-10-13 14:50:54 +08:00
Michal Shalev and Po-Han Huang
35a014c7e8
[PD][NIXL] Fix staging buffer startup by accepting slot_layer_ids ( #43723 )
...
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-12 23:48:13 -07:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
dbad07a06f
[Docs] Cookbook: refresh SemiAnalysis AgentX launches (DeepSeek-V4.1 GB200/GB300, GLM-5.2 FP8 B300) ( #43983 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-13 14:39:45 +08:00
50a144057c
[AMD][PD] enable mori transfer for PD decode DCP ( #37358 )
...
Signed-off-by: inkcherry <mingzhi.liu@amd.com >
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
Co-authored-by: HAI <hixiao@gmail.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-13 14:35:09 +08:00
Xiaoyu Zhang
46b7357779
[Diffusion] Speed up LTX-2 guidance and video output ( #43667 )
2026-10-13 14:23:33 +08:00
Jimmy Shong
3eebf987ed
[PD] Support heterogeneous TP for hybrid SWA KV transfer ( #40909 )
2026-10-12 23:21:54 -07:00
603a05e6c0
[ROCm][Spec] Route NGRAM tree reconstruction to the JIT kernel on HIP ( #43328 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
2026-10-12 23:17:49 -07:00
AMD-yanfeiwang
17c3b259f5
[router] Fix power_of_two with --dp-aware ( #42027 )
2026-10-13 14:15:36 +08:00
Zhaoyi Li
fc404bbbb3
[AMD][DI][CI] dsv4pro-fp8 2p1d-ep16-mtp: raise prefill mem fraction for MORI EP v2 memory ( #43944 )
2026-10-12 23:03:57 -07:00
Xinyi Song
a69e893b1a
[AMD] Fuse prefill index-Q RoPE fake-quant and fp4 quantize kernels in DSV4.1 ( #42911 )
2026-10-12 22:33:53 -07:00
jacky.cheng and Mick Qian
42e43cf33f
[AMD] Fix Flux2 kv-gather abort on non-contiguous K/V ( #43793 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-12 22:20:55 -07:00
kk and wunhuang
43356304b4
[AMD][Kimi-K3] Use public FlyDSL APIs in KDA decode kernels ( #43968 )
...
Co-authored-by: wunhuang <wunhuang@amd.com >
2026-10-12 22:09:55 -07:00
WenhaoZhang and Mick Qian
ea13359da9
[diffusion] perf: implement paint_use_remesh (decimate to 40k faces before UV unwrap) for Hunyuan3D ( #42770 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-13 12:28:32 +08:00
6a89181614
tokenizer perf optimization ( #39771 )
...
Co-authored-by: BBuf <1182563586@qq.com >
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
2026-10-13 12:27:21 +08:00
WenhaoZhang and Mick Qian
feba04fa49
[diffusion] fix: avoid redundant LoRA scaling while preserving exact numerics ( #43028 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-13 12:27:04 +08:00
WenhaoZhang
cda267ab0f
[diffusion] fix: keep MiniMax-H3 VAE fused norms off ROCm ( #43350 )
2026-10-13 12:26:16 +08:00
WenhaoZhang
b4822bae03
[diffusion] comfyui: run MiniMax H3, FastH3 and VDN in SGLD integrated mode ( #43504 )
2026-10-13 12:01:48 +08:00
4d5692b6cd
fix(openai): honor tool_choice=none in DeepSeek encoders ( #43006 )
...
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
2026-10-13 11:41:50 +08:00
Piotr Mazurek and Codex
40f5d9411f
Reuse Hugging Face LFM2-MoE configuration ( #43939 )
...
Co-authored-by: Codex <codex@openai.com >
2026-10-13 11:29:18 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
7bd41aec12
[CI] Remove Mistral Small 4 e2e test ( #43973 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-12 20:14:22 -07:00
Thomas Wang
c2dfd5d3d6
[AMD] Update v4 amd cookbook 1012 ( #43962 )
2026-10-12 19:57:55 -07:00
Xiaoyu Zhang
03416e8c0f
[Fix] Account for captured inputs in DeepGEMM masked workspace budget ( #43563 )
2026-10-13 10:53:10 +08:00
Liangsheng Yin
43e4e84731
[PD] Recompute retracted DeepSeek-V4 requests on prefill instead of crashing decode ( #43936 )
2026-10-12 19:32:03 -07:00
Yuwei An
07566cbf0d
[DSv4.1] Temporarily disable B300 PD/CP accuracy test ( #43948 )
2026-10-12 19:31:10 -07:00
75a0d91c1f
[Fix] Preserve HTTP errors for non-streaming /v1/messages ( #43796 )
...
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
2026-10-13 10:10:45 +08:00
45418d7dc6
[HiCache] Avoid full-store scans for uncached file prefix lookups ( #42077 )
...
Signed-off-by: apex-mochen <2756823972@qq.com >
Co-authored-by: apex-mochen <265981728+apex-mochen@users.noreply.github.com >
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu >
2026-10-12 18:52:50 -07:00
Yuwei An
23150f7ae0
[DSv4.1] Reduce sparse-prefill workspace padding ( #43921 )
2026-10-12 18:51:36 -07:00
Ziyi Xu and Claude Opus 5.5
107cff72d3
[DeepSeek-V4.1] mHC: drop seam_open, narrow precomputed stats in take_rows ( #43833 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 18:45:16 -07:00
e775fa7253
[Build][CI] Drop Python 3.10: require Python 3.11+ and run CUDA CI on Python 3.12 ( #43021 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com >
2026-10-12 18:40:16 -07:00
YAMY
59445b3da1
[DSV4][CP] Write low-ratio page indices when CP prefill runs paged attention ( #43935 )
2026-10-12 18:36:11 -07:00
Michael
7224fce1b5
[AMD] Recalibrate test_quark_mxfp4 GSM8K thresholds to observed MI355X noise ( #41980 )
2026-10-12 16:46:03 -07:00
Yizheng Jiao
6a7abcfc9b
[DeepSeek-V4.1] Admit the NIXL transfer backend for DSpark PD ( #43734 )
2026-10-12 15:32:17 -07:00
Khoa Pham
93809202a3
[Kernel] Expose bundled FlashMLA MegaAttention kernels and APIs ( #43918 )
2026-10-12 14:50:04 -07:00
Duyi-Wang
6781da8e5b
[AMD] [Docker] Update MoRI to v1.2.4 ( #43834 )
2026-10-12 14:45:42 -07:00
Thomas Wang
64cffed139
[AMD] Fix test_compact_attention_hip.py CI ( #43836 )
2026-10-12 14:43:19 -07:00
Thomas Wang
d6853f5066
[AMD] Add flydsl mxfp8 dense gemm kernel for dpsk v4.1 prefill ( #43803 )
2026-10-12 14:42:02 -07:00
Michael
6c2a682b59
[AMD] Keep the FP4 varctx schedule test off AITER's 2048-row compile ( #41793 )
2026-10-12 14:39:08 -07:00
Zhang, Jiejing and JohnQinAMD
6966c5d027
[ROCm] Eliminate HiCache migration stalls and use exact-size host allocations ( #42553 )
...
Signed-off-by: JohnQinAMD <yanyuan.qin@amd.com >
Co-authored-by: JohnQinAMD <yanyuan.qin@amd.com >
2026-10-12 14:05:51 -07:00
ix-48
b60444c621
Fix autograd retention in CUDA graph capture tail hooks ( #43854 )
2026-10-12 12:31:18 -07:00
Liangsheng Yin
0efbb3d2b9
[DFlash] Decide the Domino candidate pool in one place and apply simulated acceptance inside the accept ( #43791 )
2026-10-12 11:35:57 -07:00
Liangsheng Yin
392039d752
[DSpark] Share draft params, logits dtype, confidence and the accept tail between graph and eager ( #43790 )
2026-10-12 11:35:41 -07:00
Shuwen Wang
f92199b569
[Unified Cache] Abort requests whose external linker load fails instead of crashing the scheduler ( #40896 )
2026-10-13 00:03:21 +08:00
Sam Shleifer and Claude Fable 5
d88d5c325a
Prebuild fixup_zero_kv JIT module at TRTLLM-MLA backend init ( #37525 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-12 23:27:21 +08:00
Yuan Luo and luoyuan.luo
23a1f53fc3
[QSA] Bound the decode indexer's work by the context length ( #43463 )
...
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com >
2026-10-12 22:47:45 +08:00
Alexandr and Артем Савкин
1a553927b0
[CI][Diffusion] Add caching for npu GT images in CI ( #37426 )
...
Co-authored-by: Артем Савкин <58187114+OrangeRedeng@users.noreply.github.com >
2026-10-12 17:18:01 +03:00
279f1fad90
[diffusion] chore: bind worker CPUs to the GPU's NUMA node when memory policy is not permitted ( #43756 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 21:09:41 +08:00
Артем Савкин
d1357bf87b
[NPU] Diffusion CI fixes ( #43210 )
2026-10-12 14:53:51 +03:00
ming_wang
c6e822c29b
[Fix] Adapt NPU Qwen-VL processor patch to transformers 5.19 resize() signature change ( #43309 )
2026-10-12 19:24:05 +08:00
Sasha Sidorov and Shangming Cai
7953751bfa
[PD][Spec] Give a PD decode's first EAGLE draft step a proposal distribution ( #41783 )
...
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-12 18:49:52 +08:00
8afe107c24
[XPU] Upgrade XPU backend to PyTorch 2.14 ( #42003 )
...
Co-authored-by: Ashwini Rathi <arathi@habana.ai >
Co-authored-by: Y <ashwini.rathi@intel.com >
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-10-12 18:24:48 +08:00
C and Cheng Wan
28621739fc
[Fix] Make tokenizer request-state initialization atomic ( #41911 )
...
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com >
2026-10-12 03:10:57 -07:00
180933518e
[sgl-router] Serve the Anthropic Messages API on /v1/messages ( #42316 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
2026-10-12 17:56:08 +08:00
Jinyan Yi
f006fb49ec
[NPU] Stabilize HiCache and registered CI tests ( #43308 )
2026-10-12 17:10:26 +08:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
b0c84d4518
[Docs] Cookbook: refresh SemiAnalysis AgentX launches (GLM-5.2 B300, DeepSeek-V4.1 GB300) ( #43799 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-12 17:08:11 +08:00
Baizhou Zhang
450837e770
[Docs] Update GLM-5.3-Flash playground: B300 default, CP8, EP and PD ( #43802 )
2026-10-12 01:38:12 -07:00
BingjiaWang
62ff466ac0
[DSV4.1][PD][1/N] Support prefill node use cp parallel strategy ( #40323 )
2026-10-12 01:05:21 -07:00
BingjiaWang
54409c2fa0
[BCG][DSV4-Flash][CP] Support DeepSeek V4 prefill with interleave CP and breakable CUDA graphs ( #38603 )
2026-10-12 01:03:10 -07:00
Anupa Sajikumar and Siju Samuel
8d491fbc3e
[XPU] Support decode context parallelism (DCP) on Intel XPU ( #34355 )
...
Co-authored-by: Siju Samuel <siju.samuel@intel.com >
2026-10-12 15:50:33 +08:00
Ke Bao
a70a83bcd2
Keep NIXL Mamba checkpoint keys separate by TP rank ( #43594 )
2026-10-12 15:40:59 +08:00
Ke Bao
b6ba148384
Capture KV host transfer buffers ( #43711 )
2026-10-12 15:40:06 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
06887827c7
[CI] Move tests that do not need GB300 off the per-PR GB300 suite ( #43784 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-12 00:38:10 -07:00
5ca0695aa6
[HiCache] Preserve all MTP KV and recurrent state ( #40223 )
...
Co-authored-by: alphabetc1 <2508695655@qq.com >
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com >
2026-10-12 15:37:59 +08:00
Shuwen Wang
f5f5003323
[HiCache] Support Mamba in buffer-only host memory mode ( #36345 )
2026-10-12 00:35:16 -07:00
Shangming Cai and Claude Fable 5.1
4195980f5a
[PD] Poison the deferred abort ack when a Mooncake send fails ( #43719 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-12 15:33:09 +08:00
da4bf1ef8e
[Perf] Route BF16 mixed-batch decode tails through decode attention ( #43238 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-12 15:27:22 +08:00
Xinyuan Tong
a0e6effe50
[Fix] Link cuda_ipc JIT against CUDA driver stubs dir ( #43764 )
2026-10-12 15:26:32 +08:00
Baizhou Zhang
62c853b654
docs: clarify prefill CP terminology and simplify wording ( #43787 )
2026-10-12 00:24:57 -07:00
208df292e5
[diffusion][xpu] FLUX.2: enable the bit-exact Triton SwiGLU on XPU ( #41940 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-12 15:16:58 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
166b4ae7fc
[Fix] Adapt to transformers 5.18 / 5.19 changes ( #43688 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-11 23:24:57 -07:00
ashwini rathi
8ce76b88df
[XPU CI] Optimize PR CI per-file timeouts and fail hanging tests fast ( #42048 )
2026-10-12 14:24:26 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
130e11d50e
[Deps] Bump FlashInfer to 0.7.1 ( #43494 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-11 23:23:11 -07:00
sglang-bot and sglang-bot
f9bd83e7a9
docs: sync LMSYS SGLang blog cards ( #43462 )
...
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com >
2026-10-12 06:07:15 +00:00
Liangsheng Yin
9e0b001db4
[CI] Skip eval retries in the PD transfer-failure test ( #43769 )
2026-10-11 22:37:40 -07:00
Liangsheng Yin
2240049194
[RL] Return a failed weight update when the cache flush is refused ( #43758 )
2026-10-11 21:57:58 -07:00
a297cd3da1
[Fix] scheduler: stop latching batch_is_full when nothing can run ( #41872 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-10-11 21:56:19 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
bf703013f2
[Test] Re-anchor the VLM flashinfer throughput floor ( #43762 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-11 21:34:50 -07:00
d434ea444d
[DSV4.1] Reduce eager Engram extend preparation ( #42159 )
...
Co-authored-by: Shangming Cai <csmthu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 21:31:55 -07:00
Liangsheng Yin
a1001bcf3a
[PD] Retract transferred decode requests on a retract pause ( #43757 )
2026-10-11 21:29:39 -07:00
ChengYM
0d0be8fc98
[AMD][MORI] Add MORI EP V2 support ( #40666 )
2026-10-11 20:54:13 -07:00
Shunkangz
20c49f12c9
Support kv cache sharding for DSA indexer ( #40925 )
2026-10-12 11:50:23 +08:00
37d2e5c34f
[AMD][V4.1][*/N] Fix DP attention + TP MoE decode NaN and support bounded replay ( #43077 )
...
Co-authored-by: wunhuang <wunhuang@amd.com >
Co-authored-by: HAI <hixiao@gmail.com >
2026-10-11 20:47:13 -07:00
weireweire
296774da37
[Tokenizer] Require fastokens>=0.3.2 and include it in the all extra ( #43104 )
2026-10-11 20:41:53 -07:00
adenzhou1350
cd8ed1bb55
[Bugfix] Preserve available host memory for integrated MUSA devices ( #43398 )
...
Signed-off-by: Xucheng Zhou <aden1350@outlook.com >
2026-10-11 20:34:58 -07:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
6ac8602e8d
[CI] Fix TestQwenImageModulationCache after Qwen-Image modulation sharding ( #43747 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 20:31:29 -07:00
77a15cf4e2
[MemCache] Add shared component size and path APIs to the tree core interface ( #43722 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: Jialin <Jialin@users.noreply.github.com >
2026-10-11 20:14:07 -07:00
95ce9caaf2
[Rust] Lock hyper 1.12.0 and want 0.3.2 in the rust workspace ( #43724 )
...
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-11 20:12:55 -07:00
e4cfc0c42a
[MemCache] Split the SWA mapping move out of locked-full recovery as RecoverEvictedSWA ( #43721 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: Jialin <Jialin@users.noreply.github.com >
2026-10-11 20:11:52 -07:00
WangHemeng
0c52cb1519
[MoE] Skip block-padding rows in the TMA down activation and quant ( #42212 )
2026-10-12 11:10:30 +08:00
Alison Shao
1a7af3290d
Fix Mistral-native format detection when the Hub is unreachable ( #43446 )
2026-10-11 19:59:19 -07:00
Chunan Zeng
3c74f68a9c
[Fix] Initialize BF16 GEMM dispatch in the one-batch benchmark ( #43702 )
2026-10-12 02:53:02 +00:00
WenhaoZhang and Mick Qian
d0a098b2e0
[diffusion] perf: split and batch LTX-2.3 one-stage guidance passes under CFG parallel ( #43684 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-12 09:36:01 +08:00
WenhaoZhang
a954b395e6
[diffusion] perf: layerwise offload for the LTX-2.5 Gemma-4 text encoder ( #43683 )
2026-10-12 09:34:35 +08:00
WenhaoZhang
a67228aa43
[diffusion] fix: shard Qwen-Image adaLN modulation and text FFN under TP ( #43691 )
2026-10-12 09:32:52 +08:00
Liangsheng Yin
e6072935bf
[DSpark] Apply SGLANG_SIMULATE_ACC_LEN inside the accept instead of switching paths ( #43623 )
2026-10-11 15:33:24 -07:00
Liangsheng Yin
a65b776963
[DSpark] Add SGLANG_DSPARK_USE_GREEDY_DRAFT, an argmax draft accepted as one-hot ( #43670 )
2026-10-11 15:18:59 -07:00
Liangsheng Yin
3cc23b3b86
[DSpark] Merge the draft TP sync sites into dspark-draft ( #43669 )
2026-10-11 15:17:47 -07:00
Liangsheng Yin
00ba90fd1a
[DSpark] Run the same markov block in the draft graph tail; drop SGLANG_DSPARK_FOLDED_SAMPLING ( #43668 )
2026-10-11 15:15:59 -07:00
Liangsheng Yin
48abbdb845
[DSpark] Draft the eager markov block through MarkovSampleStep ( #43622 )
2026-10-11 15:13:02 -07:00
Liangsheng Yin
467209e838
[DSpark] Add MarkovSampleStep, one op for a markov draft step's add and token pick ( #43621 )
2026-10-11 15:10:58 -07:00
Liangsheng Yin
9943cf8eef
[DSpark] Run every markov head through one step interface ( #43620 )
2026-10-11 15:07:51 -07:00
Banruo Liu and Shangming Cai
d62b9c2480
[KV Events] Publish LoRA adapter names and namespaced block hashes ( #42455 )
...
Signed-off-by: Banruo Liu <lichangxinlivingshade@gmail.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-11 23:23:30 +08:00
97bfa5cc53
[diffusion] fix: select VSA-H3 for FastH3 when no transformer backend is given ( #43690 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 23:05:25 +08:00
Ke Bao
46a97e5540
Preserve serialized page dtype in NIXL copy fallback ( #43593 )
2026-10-11 22:39:14 +08:00
Ke Bao
b0a75c8196
Share MLA host capacity initialization ( #43595 )
2026-10-11 22:36:29 +08:00
Ke Bao
c938b6e454
Build Mamba host pools from device buffer infos ( #43596 )
2026-10-11 22:35:52 +08:00
Shuwen Wang
ef3e5c83c5
[HiCache] Keep storage workers alive on backend exceptions ( #43550 )
2026-10-11 22:26:56 +08:00
Mick and Mick Qian
11bfd868ac
[diffusion] perf: encode LTX video/audio through the shared bounded MP4 path ( #43656 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-11 21:15:49 +08:00
Shuwen Wang
0923bcd258
[Unified Radix] Invalidate lazy Mamba checkpoints before in-place tracking ( #43567 )
2026-10-11 21:12:51 +08:00
FeiHong Guo
cf88ed4352
[BugFix][DP] Prevent health checks from perturbing user request routing ( #35721 )
...
Signed-off-by: Phoenix3334 <Phoenix3334@users.noreply.github.com >
2026-10-11 21:08:55 +08:00
Shuwen Wang
d26b6a5a9c
fix: default Inkling to fa4 on Hopper ( #35292 )
2026-10-11 20:57:42 +08:00
trebladev and Mick Qian
586d98cc15
[Diffusion] Add ModelOpt FP8 checkpoint quantization for Pi0.5 ( #43062 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-11 20:55:59 +08:00
Mick and Mick Qian
04f740580c
[VLM] fix: repair multimodal VMM lease cleanup and source-stream ordering ( #43475 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-11 19:13:39 +08:00
efd14cfb66
[DeepSeek-V4.1] Use num_stages=1 for the hc_mix stats partial kernel ( #43070 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Yuan Luo <4908075+yuan-luo@users.noreply.github.com >
2026-10-11 19:03:19 +08:00
Sage
53295c6972
[kvevents] fix missing topics in replayed kv events ( #43386 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-10-11 17:25:43 +08:00
Liangsheng Yin
0a21a3b798
[Scheduler] Keep a DP-converted decode batch as the running batch ( #43283 )
2026-10-11 01:23:51 -07:00
jiaqiang-dot-liu
41652225d0
[Sampler] Sort only the max(top_k) leading columns in the torch sampler ( #43346 )
2026-10-11 01:00:59 -07:00
Liangsheng Yin
dab108b9f3
[Scheduler] Hold back decode rows on a mixed step when they do not fit the KV pool ( #43290 )
2026-10-11 00:18:27 -07:00
Liangsheng Yin
89e24e7a20
[KV canary] Expect sweep counters on the target runner only ( #43286 )
2026-10-10 23:52:17 -07:00
Liangsheng Yin
ad8b1e1f85
[Test] Do not hang scripted-runtime teardown when the server died ( #43285 )
2026-10-10 23:13:56 -07:00
Liangsheng Yin
448759b235
[Scheduler] Requeue preempted requests when admission admits nothing ( #43298 )
2026-10-10 22:46:55 -07:00
Qiaolin Yu
047405f534
[qwen 3.8 next] Fuse QSA prefill KV gathering and dtype conversion ( #41933 )
2026-10-10 22:40:23 -07:00
paulzhang-tm
5678b71b6c
[Spec] Allow Qwen3.5 backends to opt into packed target verification ( #43037 )
2026-10-10 22:39:17 -07:00
Nikolai
22d63f5e43
[AMD]Qwen add Sparse attention to capturing hip graphs ( #43383 )
2026-10-10 22:03:54 -07:00
Jzz1943
260551ebe9
[KDA] Support Helion MTP verification and harden prefill, decode ( #42857 )
2026-10-11 12:47:57 +08:00
SunskyXH and Xiaoyu Zhang
65e6bb6be7
[GDN] Fix CuTe DSL decode to read the [N, HV, V, K] state layout ( #43332 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-11 12:43:16 +08:00
a1500b10e3
[diffusion] Don't report video jobs completed after their outputs are deleted ( #43084 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-11 12:42:38 +08:00
Chunan Zeng and Claude Opus 5.5
aa5c969dc3
[Perf] Fuse BF16 writes to native HND KV cache storage ( #43237 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 12:41:18 +08:00
Shuwen Wang
58cb1a2be6
[Scheduler] Keep waiting-prefix LRU refresh consistent across TP ranks ( #43479 )
2026-10-10 21:18:41 -07:00
Avaya Aggarwal and niehen6174
3458c94fad
[diffusion] comfyui: consolidate executor conditioning-cache fixes ( #43492 )
...
Co-authored-by: niehen6174 <1639206518@qq.com >
2026-10-11 11:42:00 +08:00
Avaya Aggarwal and niehen6174
638330d975
[diffusion] comfyui: consolidate FluxAdapter correctness fixes ( #43490 )
...
Co-authored-by: niehen6174 <1639206518@qq.com >
2026-10-11 11:41:36 +08:00
Khoa Pham
6576e50cd5
[1/N][DSv41] Enable DP attention with MegaMoE ( #43228 )
2026-10-10 20:36:35 -07:00
b931b2c8bd
[diffusion] perf: run the attention section eager under torch.compile for MiniMax-H3 ( #43634 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 11:02:54 +08:00
b21826e609
[diffusion] fix: replay Qwen-Image-2.1 BCG graphs across prompt lengths ( #43633 )
...
Co-authored-by: Mick <mickqian@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 10:55:01 +08:00
b237afa547
[diffusion] fix: Qwen-Image BCG prompt padding misses every graph and attends the pad ( #43632 )
...
Co-authored-by: Mick <mickqian@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 10:54:18 +08:00
b3a65efe7e
[diffusion] perf: keep Cosmos3's fused kernels in the graph ( #43577 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 10:53:43 +08:00
d45ef0f2d2
[diffusion] perf: keep Qwen-Image-2.1 and Ulysses graphs whole ( #43586 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 10:52:43 +08:00
Shangming Cai and Kevin Flansburg
e1f43c9a61
[PD][HiCache] Fold the decode restore outcome into the MIN-reduced KV poll ( #43562 )
...
Co-authored-by: Kevin Flansburg <kflansburg@cloudflare.com >
2026-10-11 10:12:34 +08:00
Richard Wang
d60e0c8570
[diffusion] Support LLaDA-Image BF16 generation and editing ( #37907 )
2026-10-11 02:10:51 +00:00
e4b423256e
[diffusion] fix: serve whole graphs after offload-during-compile ( #43631 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 10:03:47 +08:00
Richard Wang
d7ed5b56f5
[diffusion] fix: keep the overlay probe from stalling cached startup ( #43590 )
2026-10-11 10:01:01 +08:00
156d54e56c
fix: wait for HiCache load-back before hybrid MLA KV reads ( #38217 )
...
Co-authored-by: zhouzechun <zhouzechun@kuaishou.com >
Co-authored-by: alphabetc1 <2508695655@qq.com >
2026-10-11 09:50:13 +08:00
Jzz1943 and Xiaoyu Zhang
9336681310
[DSA] Support TileLang FP8 KV cache on CUDA ( #42957 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-11 09:12:18 +08:00
Xiaoyu Zhang
33d2ef1756
[Fix] Reserve remaining tokens for ignore_eos requests during prefill admission ( #43564 )
2026-10-11 09:05:48 +08:00
WenhaoZhang and OnePunchMonk
5730345717
[diffusion] comfyui: fail early instead of silently dropping features in SGLD integrated mode ( #43495 )
...
Co-authored-by: OnePunchMonk <aggarwal.avaya@yahoo.com >
2026-10-11 08:58:23 +08:00
d1313dd5fe
[diffusion] fix: skip inductor's saved multi-kernel pick for torch.compile (Z-Image warm starts crash or hang) ( #43601 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 08:56:55 +08:00
3022989bad
[diffusion] CI: skip the LTX-2 RMSNorm-modulation guard test off the CUDA platform ( #43574 )
...
Co-authored-by: Mick <mickqian@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 08:54:37 +08:00
65386cb95a
[diffusion] perf: add the dynamic LoRA delta inside the second GEMM ( #43501 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 08:50:53 +08:00
e328c7ecbd
[diffusion] fix: skip QKV packing while layers are offloaded ( #43576 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 08:47:35 +08:00
7373665085
[diffusion] fix: keep exported buffers out of expandable segments for IPC all-to-all( #43584 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 08:46:30 +08:00
5e872557cb
[diffusion] feat: keep the TP all-reduce inside the graph ( #43602 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 08:45:54 +08:00
664e4c6499
[diffusion] feat: warn instead of disabling on unvalidated models for BCG( #43568 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 08:40:28 +08:00
23764b1242
[diffusion] feat: bind each GPU worker to its GPU's NUMA node ( #43565 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 08:38:58 +08:00
9933cf5437
[diffusion] feat: declare regional compile regions for flux ( #43575 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 08:33:29 +08:00
68e1a3dbb1
[diffusion] fix: accept request-level profiling on the image API ( #43573 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-11 08:30:20 +08:00
d32d8fa0d1
[Dsv4.1] Mega-mhc integration ( #42808 )
...
Co-authored-by: Yuwei An <yuwei.an@MacBook-Pro-yuweian.local >
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai >
2026-10-10 17:25:17 -07:00
Liangsheng Yin
7187bb593e
[Model] Keep gpt-oss router logits batch-invariant under deterministic inference ( #43624 )
2026-10-10 17:22:29 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
c640dc97ce
[CI] Fix Kimi-Linear DCP PD test startup ( #43598 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-10 16:53:54 -07:00
Cheng Wan
234ccba7f6
[Refactor] Declare every model's stages from a shared function and drop the meta-device neighbour build ( #43460 )
2026-10-10 12:43:09 -07:00
Cheng Wan
3b16eeedce
[Refactor] Declare pipeline neighbours from a shared declaration function ( #43459 )
2026-10-10 12:42:46 -07:00
Cheng Wan
3b717c0f13
[Refactor] Decide an exit's fixed facts when the stage is built ( #43458 )
2026-10-10 12:42:23 -07:00
Cheng Wan
a956aa192a
[Refactor] Bind a layer stack in one pass over each stage's data flow ( #43457 )
2026-10-10 12:42:00 -07:00
Cheng Wan
d2c597c8db
[Refactor] Decide input-scattered eligibility and stage rejections in one place ( #43456 )
2026-10-10 12:41:38 -07:00
Cheng Wan
b0dde87de2
[Refactor] Make residual read and update capabilities explicit and checked ( #43455 )
2026-10-10 12:41:16 -07:00
Cheng Wan
514feb1afc
[Fix] Keep residual rows and sums consistent across appends and pipeline handoffs ( #43454 )
2026-10-10 12:40:53 -07:00
Cheng Wan
8bd7f2d178
[Refactor] Record what a bound stage path does instead of reading it back ( #43453 )
2026-10-10 12:40:30 -07:00
0acd93c35c
[Scheduler] Use committed length in spec mamba track boundary check ( #43029 )
...
Co-authored-by: Ke Bao <ispobaoke@gmail.com >
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com >
2026-10-10 12:29:40 -07:00
Shuwen Wang and Ke Bao
f9cee8d2b2
[HiCache] Keep protected cgroup page cache charged in host budgets ( #42788 )
...
Co-authored-by: Ke Bao <ispobaoke@gmail.com >
2026-10-10 17:03:40 +00:00
Xiaoyu Zhang
f61163b4b0
[Docs] Let the Qwen3 B300 recipe use the default attention backend ( #43554 )
2026-10-10 10:00:04 -07:00
Xiaoyu Zhang
587b6d622d
[Docs] Document Qwen3-4B/8B H200 mixed-chunk latency tradeoffs ( #43551 )
2026-10-10 09:59:46 -07:00
Shunkangz
de487f8039
[kv-shard 3/4] Enable Control Plane C ( #40102 )
2026-10-10 22:53:58 +08:00
huangtingwei
406540ab5f
[HiCache] Add staged write-back for the page-unified KV cache layout ( #39606 )
2026-10-10 22:49:45 +08:00
Shuwen Wang
99d6f4e158
[MemCache] refactor: require an anchored lock receipt for non-root releases ( #41452 )
2026-10-10 22:43:49 +08:00
Shuwen Wang
44bc478863
[HiSparse] Avoid host synchronization in eager backup ( #41446 )
2026-10-10 21:49:55 +08:00
hhy-seven
616178a596
[Responses] Fix max_output_tokens overestimate on multimodal requests ( #35524 )
2026-10-10 13:42:53 +00:00
a510340aeb
[diffusion] perf: run head_dim-128 QK-norm + RoPE two rows per warp for every RoPE layout ( #43508 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 21:34:54 +08:00
72657f7ca5
[diffusion] perf: pipeline every USPAttention Ulysses path's exchange over the copy engine ( #43521 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 21:34:18 +08:00
Shuwen Wang
416ebe6090
[CI] Fix EAGLE3 CUDA parity prefill chunk alignment ( #43538 )
2026-10-10 21:11:17 +08:00
49e629d239
[Rust] Upgrade sglang-radix-tree to PyO3 0.29 ( #42807 )
...
Co-authored-by: colleen-tml <296620403+colleen-tml@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 20:56:50 +08:00
Bingxu Chen
a916d91cd3
[AMD][CI] Cap pyarrow below 26 for lmms-eval ( #43547 )
2026-10-10 18:59:49 +08:00
Sage
6fc8d9da32
[rust-server] preserve PD routing in chat and completions ( #42989 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-10-10 17:29:46 +08:00
Liangsheng Yin
c55790d38b
[CI] Drop CPU tests with no bug-catching signal and run Rust workspace tests in their own workflow ( #43503 )
2026-10-10 02:13:53 -07:00
siliangchen-amd
ff19159e16
[AMD] Add an opt-in AITER CK blockscale GEMM for gfx942 block-FP8 linears ( #41071 )
...
Signed-off-by: siliangchen-amd <SiLiang.Chen@amd.com
2026-10-10 01:31:20 -07:00
chuyeh
e35f60b9bc
[AMD][CI] Drop removed tp_rank/tp_size kwargs from small-M FP8 proj test ( #43318 )
2026-10-10 00:52:28 -07:00
d7d22e8a30
[AMD] Resolving bug in per batch size graph capture due to multiple capture runs per batch size ( #39840 )
...
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: HAI <hixiao@gmail.com >
2026-10-10 00:47:13 -07:00
d8c5eca3ac
[diffusion] fix: keep a LoRA unmerged under merge_mode auto when merging would round its update away ( #43385 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 15:02:03 +08:00
Richard Wang
441de659f4
[CI] Fix Python 3.10 failure in prefill complete integration test ( #43505 )
2026-10-10 00:00:03 -07:00
tao
6e8f3e09dd
[Model Loader] Add InstantTensor loader support ( #40453 )
2026-10-09 23:01:59 -07:00
cctry
33f6b00f32
Clamp default decode CUDA graph max_bs to max_running_requests ( #43172 )
2026-10-09 22:54:32 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
16be9df25f
[CI] Remove the GLM-5.3-Flash PTX KDA prefill nightly test ( #43470 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-09 22:52:17 -07:00
Yuwei An
8b2aaf7060
[Refactor][TCPCG] Retire backend selection and configuration (3/9) ( #42287 )
2026-10-09 22:42:35 -07:00
Yuwei An
0aa580f904
[Dsv4.1] Add DeepSeek V4.1 DSPARK TP and DP EP accuracy coverage ( #43478 )
2026-10-09 22:41:30 -07:00
YAMY
17852cdfe8
[DSv4.1] Initialize the full-to-SWA mapping for request-window pools ( #43165 )
2026-10-09 22:39:26 -07:00
Frederik Gossen
3831e7e091
[diffusion] Z-Image: autotune the shared transformer block at both token scales ( #35755 )
2026-10-10 13:04:43 +08:00
Raymond Huang and Claude Opus 5
aa79087017
[diffusion] fix: fail MiniMax-H3 quality="high" closed on substituted DiT weights ( #34978 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-10 13:03:44 +08:00
ccae3b3639
[diffusion] perf: pipeline MiniMax-H3's Ulysses exchange against cuDNN SDPA on Blackwell ( #43373 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 12:58:30 +08:00
Shangming Cai
1814ece57f
[PP+PD] Harden PPConsensusStore: msgspec wire format, drop malformed messages ( #43329 )
2026-10-10 12:38:38 +08:00
cctry
20948cdcc6
[PD] Reclaim parked optimistic prefills when request slots are exhausted ( #43442 )
2026-10-09 20:31:24 -07:00
Liangsheng Yin
7d44269907
[MemCache] Fix the strict busy check on prefill-aware SWA and tidy small leftovers ( #43474 )
2026-10-09 20:27:27 -07:00
f2e9a2bed1
[unified memory pool] Allocate without transfer waits, pin floating SWA reservations, drop the HiCache startup asserts ( #43206 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com >
2026-10-09 20:19:30 -07:00
Mick and Mick Qian
a65f4ffba5
[diffusion] docs: standardize diffusion cookbook command builders ( #43472 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-10 11:14:31 +08:00
436d73ddda
[diffusion] model: support HunyuanImage-3.0-Instruct ( #36311 )
...
Co-authored-by: Elizaveta Martirosian <you@example.com >
Co-authored-by: wes-lyu <294033513+wes-lyu@users.noreply.github.com >
Co-authored-by: Wes <its.wesley.lyu@gmail.com >
Co-authored-by: AuFlow <73925903+AuFlow@users.noreply.github.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-10 11:11:28 +08:00
506f140166
[sgl-router] Read finished /generate streams to the end; expose reorg admission caps as flags ( #43233 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 10:59:09 +08:00
Benjamin Truong and Xiaoyu Zhang
37678784c6
[Gemma4][Attention] Enable Gemma4 FA4 on SM90 with head-dim 512 support ( #42019 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-10 10:40:19 +08:00
55a8dffc87
Skip the eager activation reserve in KV sizing on a capped Full prefill role ( #43435 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: 842974287 <842974287@users.noreply.github.com >
2026-10-09 19:36:48 -07:00
a338b9858a
[PD] Receive decode KV into separate paged SWA host pools; let host-capable receivers own abort drain tracking ( #43410 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: cctry <cctry@users.noreply.github.com >
2026-10-09 19:33:06 -07:00
Vladislav Nosivskoy and BBuf
2ff3e6dce1
[SM90] Use native block32 DeepGEMM for DeepSeek-V4.1 ( #42604 )
...
Co-authored-by: BBuf <1182563586@qq.com >
2026-10-10 10:29:42 +08:00
Liangsheng Yin
465253ef52
[Scheduler] Fix the deterministic-inference hang when a prefill chunk is shorter than its alignment ( #43444 )
2026-10-09 19:06:18 -07:00
78218194ad
perf(kda): rebalance MTP verify precompute and state synchronization ( #42683 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-10 10:03:19 +08:00
Michael
37114afe20
[AMD][Fix] Saturate FP8 KV commits instead of reproducing e4m3fnuz NaN overflow ( #42982 )
2026-10-10 09:49:56 +08:00
Mick and Mick Qian
77ba54f7df
[diffusion] docs: add Wan-Animate-2 cookbook command picker ( #43380 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-10 09:45:16 +08:00
16423cb6d2
[sgl-router] Serve DeepSeek-V4.1 Flash through sglang-processor ( #43000 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 09:23:09 +08:00
5f94a45896
[srt] Allow mixed chunk prefill on the Full prefill CUDA graph backend ( #43461 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com >
2026-10-09 18:21:22 -07:00
Mick and Mick Qian
cde16aa64d
[diffusion] model: support Qwen-Image 2.1 Turbo and checkpoint sampling grids ( #43391 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-10 09:12:56 +08:00
e8fb52c526
[Fix] Keep request-pool row generations monotonic across clear() ( #43439 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: colinchan15 <colinchan15@users.noreply.github.com >
2026-10-09 18:08:54 -07:00
ashwini rathi
8cf4d73c84
[XPU] Fix XPU CI breaks from #41105 (KDA chunk_offsets) and #42588 (int4 test tp_group) ( #43334 )
2026-10-10 09:06:24 +08:00
Avaya Aggarwal
66e6b68d82
[diffusion] comfyui: report the real runtime import error ( #43178 )
2026-10-10 09:05:12 +08:00
ff3aa58f11
[diffusion] perf: run MiniMax-H3's QK-norm + RoPE two rows per war while bit-identical ( #43337 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 09:03:36 +08:00
bookinstock and Mick Qian
32f34e898b
[diffusion] fix: honor requested inference step count for MiniMax H3( #38671 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-10 09:02:57 +08:00
bd7c7971ba
[diffusion] perf: move MiniMax-H3's pipelined Ulysses copies off the SMs ( #43361 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 08:58:07 +08:00
Alison Shao
9b37e28b83
Disable DualGemm fusion for Inkling dense MLP ( #43447 )
2026-10-09 17:56:20 -07:00
6acaf3374a
[kernel] SwiGLU-OAI MXFP8 producer; move swizzled MXFP8 producers to ops/quantization ( #42810 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Denji-kk <51020465+Denji-kk@users.noreply.github.com >
2026-10-10 08:51:57 +08:00
Rain Jiang
961bcf481d
support DeepEP v2 on GLM-5.3 ( #43432 )
2026-10-09 16:57:12 -07:00
Yuwei An and Yuwei An
2d38934493
[DSv4.1] mxfp8 dispatch cache ( #42156 )
...
Co-authored-by: Yuwei An <yuwei.an@MacBook-Pro-yuweian.local >
2026-10-09 16:36:28 -07:00
Liangsheng Yin
46498bc8c5
[MLX] Sync decode KV to the pool once at release, clamped to the owned prefix ( #43292 )
2026-10-09 16:20:45 -07:00
Liangsheng Yin
114cb44920
[MLX] Read radix prefix slots from the model runner's req_to_token pool ( #43291 )
2026-10-09 16:18:58 -07:00
Liangsheng Yin
8020ea2ee9
[MLX] Drop a retracted request's runner state at its re-prefill ( #43284 )
2026-10-09 16:16:59 -07:00
Liangsheng Yin
f7e601b936
[MLX] Drain in-flight jobs before a retract-mode pause_generation ( #43282 )
2026-10-09 16:14:23 -07:00
f6c848c882
[PD] Use 1 to disable decode host receive and 0 for always-on ( #43407 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: cctry <cctry@users.noreply.github.com >
2026-10-09 15:50:52 -07:00
Alison Shao
f6e3444b2e
ci: diffusion-only option for the runner utilization report ( #41620 )
2026-10-09 15:47:47 -07:00
Cheng Wan
58f6976ce7
[Fix] Unblock CUDA graph startup for hybrid parallelism ( #43211 )
2026-10-09 15:39:59 -07:00
cb9bd1d4d4
[Perf] Reduce multi-detokenizer router IPC sends ( #42238 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-10-09 15:32:14 -07:00
66d840a8ef
[cuda_graph] Size dp-local decode graphs by the runner's own gather requirement ( #43303 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: charlotte12l <charlotte12l@users.noreply.github.com >
2026-10-09 15:21:23 -07:00
Kevin Mi and Claude Opus 5.5
6d8ddaf1d7
[AMD] MiniMax-M3 indexer CP: packed scoring for EAGLE verify rows ( #42614 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 15:05:50 -07:00
fbf8d730fd
[Fix] MiniMax-M3: default to the breakable prefill CUDA graph ( #41845 )
...
Co-authored-by: Hao Phan <htphan@nvidia.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
Co-authored-by: Po-Han Huang <pohanh@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 15:05:05 -07:00
Kevin Mi and Claude Opus 5.5
70e82f7b73
[AMD] Run the MiniMax indexer CP test on MI35X, not MI300 ( #43259 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 15:04:25 -07:00
weireweire and weireweire
46c2585faa
[DSpark] Fold draft sampling for DeepSeek-V4.1 heads under AUTO ( #42987 )
...
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com >
2026-10-09 14:40:28 -07:00
Huabin
2f1acef3d8
[Model] Add LLaDA2.2 Block Routing MoE support ( #31768 )
2026-10-09 12:14:44 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
72eab59bf6
[GLM-5.3-Flash] Enable breakable prefill CUDA graph by default ( #42845 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-10 02:41:02 +08:00
87845cda2b
[Multimodal] Chunk and cache audio items whose offsets span several placeholder runs ( #43301 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: YavorGIvanov <YavorGIvanov@users.noreply.github.com >
2026-10-09 10:57:01 -07:00
3bffe69b74
[moe] Fill num_token_non_padded for the customized A2A backend at EP 1 ( #43245 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: cctry <cctry@users.noreply.github.com >
2026-10-09 10:55:02 -07:00
3c612132fd
Overlap scheduler startup with the data parallel controller ( #43177 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: Yongji Wu <30348494+libertyeagle@users.noreply.github.com >
2026-10-09 10:54:22 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
5cbf949839
Fix torch.compile crash in fused gate-sigmoid-mul launcher ( #43384 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-09 23:51:30 +08:00
76ccf0850c
[diffusion] fix: fix TeaCache CFG state lifecycle and skip boundaries ( #42422 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-09 23:20:13 +08:00
121aa8dcc7
[diffusion] perf: fuse MiniMax-H3's RMSNorm and indexed AdaLN under quality lossless ( #43327 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 22:39:11 +08:00
Jimmy Shong
408f5d057a
docs: add Clef accuracy results to the deployment wizard ( #43349 )
2026-10-09 22:19:27 +08:00
1203eb9181
[diffusion] perf: pipeline MiniMax-H3's Ulysses exchange against attention over the copy engine ( #43164 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 22:04:29 +08:00
xiaofei-zheng
fba3ad4b1f
[Frontend] Parallelize long chat prompt encoding ( #41259 )
2026-10-09 21:30:22 +08:00
xiaofei-zheng
db32673b78
[AMD] Let the GLM DSA NextN draft declare its own shared-expert fusion architecture ( #41258 )
2026-10-09 21:28:45 +08:00
xiaofei-zheng
d5c3ae9752
[AMD] Eliminate remaining gfx95 FP8 scale relayout copies ( #41030 )
2026-10-09 21:27:38 +08:00
78996e3f02
[Performance] Batch Mooncake hybrid-pool metadata and puts ( #39696 )
...
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
Co-authored-by: Po-Han Huang <pohanh@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:12:42 +08:00
d511e6bd1d
[sgl-router] Parse tool calls when serving chat through /generate ( #42999 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:07:30 +08:00
67d063f63b
[diffusion] feat: allow per-role attention backend override ( #35310 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-09 20:57:29 +08:00
348689da40
[PP+PD] Fix #38206 Reduce bootstrap consensus latency ( #38959 )
...
Co-authored-by: inkcherry <mingzhi.liu@amd.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 20:53:48 +08:00
f1e0472fcc
[diffusion] model: support Wan-Animate-2 ( #42941 )
...
Co-authored-by: Ankit Dhall <8938083+ankitdhall@users.noreply.github.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-09 20:46:23 +08:00
Shuwen Wang
0955169ed2
[mem_cache] Bound FlexKV checkpoint stores and restore DSPARK cache checks ( #35694 )
2026-10-09 20:25:33 +08:00
59eb71a831
[diffusion] fix: align SGLD CFG combine formula with diffusers Z-Image ( #23772 )
...
Co-authored-by: SGLang CI <ci@sglang.ai >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-09 19:47:03 +08:00
Shuwen Wang
f48ed2127e
test: allow bounded checkpoint lag in Mamba HiCache tests ( #42523 )
2026-10-09 19:24:39 +08:00
3805d78796
[sgl-router] Serve /v1/chat/completions through /generate, with reasoning parsing ( #42998 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-09 19:17:52 +08:00
Xiaoyu Zhang
0eb24e7d5b
[Performance] Optimize Qwen3.8-Flash-Next BF16 decode on Blackwell ( #42175 )
2026-10-09 18:46:12 +08:00
516fd1c1b5
[diffusion] perf: tile MiniMax-H3's indexed modulation kernels by 2048 columns ( #43267 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 18:23:52 +08:00
Xinyuan Tong
91b3692a4c
docs: add thinking budget guide ( #41601 )
2026-10-09 17:50:34 +08:00
BingjiaWang
888de24cc8
Add BJWang-ant to CI_Permission ( #43321 )
2026-10-09 02:31:38 -07:00
黄孝君
e1c6ddca1d
[NPU] Bump SGLANG_KERNEL_NPU_TAG to 2026.9.0.post9 ( #43340 )
2026-10-09 17:30:38 +08:00
vOv and Xiaoyu Zhang
7cfecbef56
[diffusion][model] Add native SANA-Video 2.0 support ( #41492 )
...
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-09 17:28:03 +08:00
DevashishLal-CB
4debae7534
Add blackwell dual gemm fusion support for MLP ( #41887 )
...
Signed-off-by: Devashish Lal <devcode@meta.com >
2026-10-09 17:26:33 +08:00
Thomas Wang
5eca4d8caf
[AMD] Set maxseq length limit for DSV4.1 BCG metadata buffers ( #42570 )
2026-10-09 02:15:43 -07:00
Xinyi Song
23f890613e
[AMD] Patch aiter's group32 MXFP8 GEMM masked-scale fix ( #43319 )
2026-10-09 02:08:44 -07:00
Jimmy Shong
14e3414513
[Docs] Add Clef cookbook with selectable Clef Flash variant ( #43235 )
2026-10-09 01:56:45 -07:00
amote-i
ff146fa20c
[NPU] [DOC] remove --enable-torch-compile from Ascend support matrix ( #43336 )
2026-10-09 16:55:39 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
7578432c21
[LoRA][Test] Use the 0.9 ROUGE-L tolerance for the CUDA multi-LoRA batch test ( #43297 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-09 01:51:01 -07:00
Avaya Aggarwal and Mick Qian
d677848968
[diffusion] fix: fix CFG normalization to use per-sample norm instead of batch-flattened norm ( #41706 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-09 16:43:18 +08:00
Murthy L and Claude Opus 5
3ab142adc4
[diffusion] feat: enable breakable CUDA graphs for FLUX.2 Klein ( #36470 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-09 16:40:09 +08:00
b922f5ca20
[diffusion] fix: share the IPC all-to-all transport between AllToAll4D and USP ( #43269 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 16:31:55 +08:00
Mick and Mick Qian
17b2f35ae7
[diffusion] refactor: remove unused helpers and write-only state ( #43064 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-09 16:30:55 +08:00
James Xu and Mick Qian
ee4e2f7818
[diffusion] optimization: enable channels_last_3d for Cosmos3 VAE ( #28021 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-09 16:29:46 +08:00
Cheng Wan
dc0b394471
[Refactor] Remove Kimi K3's own layer communication and let the consumer set an FFN's exit ( #42936 )
2026-10-09 01:27:46 -07:00
Cheng Wan
305ad5503d
[Refactor] Run Kimi K3's fused collectives through stage boundaries ( #42898 )
2026-10-09 01:27:23 -07:00
Cheng Wan
2b469040d2
[Refactor] Support attention-TP dense FFNs and unpadded batches on stage boundaries ( #42897 )
2026-10-09 01:27:00 -07:00
Cheng Wan
b9ae3d1a65
[Fix] Run attention on a padded extend batch's real rows ( #42896 )
2026-10-09 01:26:38 -07:00
Cheng Wan
7203747cc8
[Fix] Shard Kimi K3's shared experts over the TP group they sum over ( #42895 )
2026-10-09 01:26:15 -07:00
Mick and Mick Qian
45ad463c87
[diffusion] refactor: deduplicate UniPC solvers and Gemma3 shard loading ( #42899 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-09 16:26:04 +08:00
Cheng Wan
125f88cb75
[Refactor] Run Kimi K3 through stage boundaries ( #42894 )
2026-10-09 01:25:51 -07:00
Cheng Wan
666da4f56b
[Fix] Kimi K3 KDA value head size and SiTU input alignment ( #42893 )
2026-10-09 01:25:28 -07:00
Cheng Wan
48a6c0a5d0
[Fix] Return an FFN's output to the attention rows at a pipeline handoff ( #42892 )
2026-10-09 01:25:05 -07:00
Cheng Wan
b2402ccad6
[Fix] Merge DeepSeek-V4 two-batch-overlap microbatches without a residual ( #42891 )
2026-10-09 01:24:42 -07:00
b130f8af02
[sgl-router] Serve /v1/completions through /generate, with the shared OpenAI layer ( #42997 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 16:04:04 +08:00
f620d733d2
[rust-processor] Share the reasoning splitter and tool constraint across hosts ( #42996 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-09 15:19:41 +08:00
37ae292e6f
[diffusion] fix: skip LTX-2 SP masks only for globally unpadded sequences ( #43134 )
...
Co-authored-by: yatholam <yatholam@amd.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-09 15:08:41 +08:00
Lianmin Zheng
c44a42e6d3
docs: reorder README workloads, hardware, and ecosystem ( #43307 )
2026-10-08 23:46:24 -07:00
e0608d890e
fix: use dp_size instead of attn_dp_size for msprobe rank id ( #43041 )
...
Co-authored-by: l30044004 <louyujing@huawei.com >
Co-authored-by: Claude Code <noreply@anthropic.com >
2026-10-09 09:40:43 +03:00
Gabriel Wu
217980566a
fix(fa4): isolate sheared-bias workspaces by CUDA stream ( #33000 )
2026-10-08 23:37:02 -07:00
areneau-cf and Jimmy Shong
bd2d73daa5
[model] support Clef and Clef-Flash decision models on /v1/systemone ( #42721 )
...
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com >
2026-10-09 14:25:52 +08:00
Michael
51f7ea84c7
[AMD] Keep MiniMax decode-score tile at 128 tokens off gfx950 ( #42772 )
2026-10-08 23:22:19 -07:00
08fc290f48
[AMD] gfx950 small-batch MoE: expert-count gate for the small sort and FP8 block-scale small-M kernel ( #41982 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-10-08 23:05:50 -07:00
cba1d07a94
[Memory] Share one full prefill graph output buffer across shapes for tuple and list outputs ( #43176 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: 842974287 <842974287@users.noreply.github.com >
2026-10-08 23:05:29 -07:00
e14ef4b658
[Scheduler] Skip the decode for a request whose last token is already in flight ( #42720 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-10-08 23:00:46 -07:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
1fc6b8962c
[Docs] Cookbook: AgentX launches for Qwen3.5 / Qwen3.8-Flash-Next; refresh GLM-5.2, DeepSeek-V4, DeepSeek-V4.1 ( #43083 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 13:55:35 +08:00
e46aaaa789
[diffusion] feat: accept strided row views in the packed QKV all-to-all ( #40588 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-09 13:55:31 +08:00
Colin Z
3c1703d2ff
[AMD] Support Quark DeepSeek-V4-MXFP4 checkpoints on the DSV4 stack, enabling shared-expert fusion and MTP ( #31054 )
2026-10-08 22:54:49 -07:00
Yuan Luo and luoyuan.luo
9a1aeacc3b
[GLM-5.3-Flash] Capture the KDA extend in breakable prefill CUDA graphs ( #41105 )
...
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com >
2026-10-09 13:37:20 +08:00
33e7b4cc4b
[diffusion] perf: decode plain base64 MiniMax-H3 materials without a per-character loop ( #43117 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 13:25:23 +08:00
Chetan Kumar Verma and Claude Fable 5.1
5d8e98b7fd
[XPU][Diffusion] Plain-torch scale-shift for inductor ( #41885 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-09 13:07:14 +08:00
saatwiknagpal
6e7beace14
[diffusion] fix: fix int32 offset overflow in the Ulysses packed-QKV pack kernel ( #43140 )
2026-10-09 11:51:46 +08:00
3ba90f1b52
[rust] Update Dynamo frontend crates to latest compatible releases ( #42666 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:46:01 +08:00
9172a73725
[sgl-router] Keep charset=utf-8 on streamed responses ( #43232 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:44:52 +08:00
Baizhou Zhang
33bffcd490
[CP] GLM-5.3-Flash Prefill CP implemented on layer boundaries ( #42044 )
2026-10-08 20:42:11 -07:00
metamergebot and joshuadeng
7859eff6ca
[srt] Inject "__req__" into custom_params only for custom logit processors ( #43181 )
...
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com >
2026-10-08 20:40:50 -07:00
4ee3f0f370
[Session] Skip the KV free when page alignment empties a streaming-session rewind ( #43183 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: Joshua Deng <91448271+joshuadeng@users.noreply.github.com >
2026-10-08 20:40:25 -07:00
4e3eaba619
[HiCache] Re-admit a split node's queued buffer-mode write instead of dropping it ( #43174 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-08 20:39:41 -07:00
HEJIAN SANG and Claude Opus 5.5
3e13f1b2c7
[Perf] Serialize OpenAI chat/completion responses in one pass ( #42889 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:17:25 +08:00
ormandj and kpham-sgl
438df9a2a4
[Spec] Fix wrong tokens from compact-verify CUDA graphs when a token tier is captured more than once ( #42709 )
...
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai >
2026-10-08 19:49:23 -07:00
Khoa Pham
8e19db19b0
[KDA] Default safe-gate Blackwell prefill to FlashInfer ( #43246 )
2026-10-08 19:46:28 -07:00
Chetan Kumar Verma and Claude Opus 5.5
fedab5537b
[XPU] Enable ring attention on XPUAttentionBackend ( #41406 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-10-09 10:41:11 +08:00
Mick and Mick Qian
df1e4f3f2b
[diffusion] refactor: share SANA-WM refiner block computation ( #42980 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-09 10:38:36 +08:00
Liangsheng Yin
f6569bce4d
[MemCache] Skip KV events and eviction config on a disabled radix cache ( #43253 )
2026-10-08 19:27:39 -07:00
Polisetty V R K Jyothendra Varma
e54f08c860
[XPU] Misc changes to support XPU ( #39817 )
...
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com >
2026-10-09 10:26:46 +08:00
ashwini rathi
51686cf8e7
[XPU] split Intel nightly into kernel-main and kernel-wheel jobs ( #41857 )
2026-10-09 09:58:39 +08:00
shodoco
86624f14eb
[Rust] Pass the media item fps hint to multimodal processors ( #43143 )
2026-10-08 17:51:06 -07:00
Liangsheng Yin
e0ab91e3ca
[MemCache] Rename SWAChunkCapPoolConfigurator to SWARequestCapPoolConfigurator ( #43202 )
2026-10-08 17:50:51 -07:00
b2786a670e
[MemCache] Count reclaimable tokens in the unified tree ledgers ( #43185 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-08 17:48:11 -07:00
Liangsheng Yin
b4fe282836
[MemCache] Delete ChunkCache ( #43201 )
2026-10-08 17:31:16 -07:00
Liangsheng Yin
508ba79688
[MemCache] Serve disabled hybrid-SWA models with UnifiedRadixCache ( #43200 )
2026-10-08 17:30:01 -07:00
Liangsheng Yin
265efe5e01
[MemCache] Serve disabled full-attention models with UnifiedRadixCache ( #43199 )
2026-10-08 17:27:24 -07:00
Liangsheng Yin
977e2719c7
[MemCache] Serve disabled pure-SWA models with PureSWARadixCache ( #43198 )
2026-10-08 17:25:54 -07:00
Liangsheng Yin
57ce47862d
[MemCache] Report no prefix sharing from a disabled RadixCache ( #43197 )
2026-10-08 17:23:11 -07:00
Khoa Pham
fc3385d530
feat(kda): integrate FlashInfer prefill checkpoints ( #41400 )
2026-10-08 17:16:33 -07:00
Khoa Pham
1e77db3215
[Perf] Avoid gate copies in gated RMSNorm ( #43227 )
2026-10-08 17:11:58 -07:00
20d1f3f56a
[Scheduler] Refresh waiting requests' cached prefixes under FCFS-like policies with LRU eviction ( #41593 )
...
Signed-off-by: Pei Li <peili@nvidia.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
Co-authored-by: Po-Han Huang <pohanh@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-08 17:05:04 -07:00
Ankur Singh
2b94cb4eb5
docs(cookbook): add Qwen3.8 GB300 long-context recipe ( #43159 )
...
Signed-off-by: Ankur-singh <ankusingh@nvidia.com >
2026-10-08 16:56:30 -07:00
bb43927404
[PD] Charge decode admission the page-rounded allocation when the radix cache is disabled ( #43173 )
...
Co-authored-by: hanming-lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-08 16:48:31 -07:00
Liangsheng Yin
2e64740442
[CI] Avoid Hugging Face Hub API calls when the local model cache is complete ( #43229 )
2026-10-08 16:27:31 -07:00
Liangsheng Yin
baee5f3b62
[CI] Drop the labeled trigger from the extra CI workflows ( #43234 )
2026-10-08 16:25:50 -07:00
Liangsheng Yin
ae4da8cd78
[MemCache] Skip SWA request-cap pool sizing when chunked prefill is off ( #43203 )
2026-10-08 16:19:35 -07:00
Liangsheng Yin
71584b62c0
[CI] Fix stale fakes in the NVFP4 MoE dispatch and CP strategy unit tests ( #43226 )
2026-10-08 15:51:39 -07:00
20bf93b6bf
[metrics] Call DPBalanceStats.create by keyword in the ratio ladder test ( #43218 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: hanming-lu <hanming-lu@users.noreply.github.com >
2026-10-08 15:24:38 -07:00
2abb761639
[metrics] Count per-rank DP attention pairs and their imbalance ( #43180 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: hanming-lu <hanming-lu@users.noreply.github.com >
2026-10-08 15:01:37 -07:00
metamergebot and Hanming Lu
0b658c12ff
[Bench][MoE] Deal the simulated round-robin experts evenly across EP ranks ( #43188 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
2026-10-08 15:01:16 -07:00
Siyuan Chen
b0e06359ac
[DSV4.1][PD][5/N] Decode node support DP attention ( #40177 )
2026-10-08 14:42:16 -07:00
Xuanteng Huang
d20bdc1443
Add per-token NVFP4 MoE support for ReLU2 activation ( #39199 )
...
Signed-off-by: Xuanteng Huang <xuanteng.huang@outlook.com >
2026-10-08 14:28:22 -07:00
Shunkangz and Claude Opus 5
3d8d44a4a9
[Qwen3.8 CP 3/4] Collocated prefill CP integration ( #39723 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-08 14:20:27 -07:00
Sage
5a5ea81036
[rust-server] warm up each rust listener in-process ( #43004 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-10-08 14:09:46 -07:00
e902207a4e
[Speculative] Support block verification for the DFlash family ( #43012 )
...
Co-authored-by: maocheng <mao.cheng@MacBook-Pro-MD663G60VV.local >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 13:59:52 -07:00
d1ba943016
[Model] Add support for EmbeddingGemma 2 ( #42782 )
...
Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 13:54:45 -07:00
Baizhou Zhang
b193ef382a
[CI] Clean some unnecessary DSV4 tests ( #43195 )
2026-10-08 13:53:41 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
d8448d2e62
[CI] Re-enable GB300 ( #43058 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-08 13:23:26 -07:00
metamergebot and hanming-lu
93f2acf614
[Metrics] Use widely supported bucket boundaries for the DP attention imbalance ratio ( #43187 )
...
Co-authored-by: hanming-lu <hanming-lu@users.noreply.github.com >
2026-10-08 12:58:58 -07:00
Ming Yang
3fef129ff8
[PD] Allow state-only KV checksum inputs ( #42904 )
2026-10-08 12:54:03 -07:00
075fd8fa0b
[Mamba] Describe the track snapshot dtype by its kernel instead of fp32 ( #43192 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com >
2026-10-08 12:47:46 -07:00
e3c5719713
[DFlash] Build the attention QKV projection through an overridable hook ( #43179 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: 842974287 <842974287@users.noreply.github.com >
2026-10-08 12:46:28 -07:00
kunchengit
00b3a48753
[dLLM] Add JointThresholdInDel algorithm for insertion & deletion decoding ( #31773 )
2026-10-08 12:44:14 -07:00
Aurick Qiao and Qiaolin-Yu
b2cb24995d
[Debug] Add opt-in NaN detection before sampling ( #42690 )
...
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com >
2026-10-08 12:20:57 -07:00
Shenxiu Liu and Qiaolin-Yu
393eaed509
[Fix] overlap scheduler: record_stream mix_running_indices for the forward-stream relay gather ( #34076 )
...
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com >
2026-10-08 12:19:35 -07:00
shodoco
1638c5d699
[http_server] Let custom /generate routes skip the second contract check ( #43114 )
2026-10-08 11:56:06 -07:00
214347891a
[Bugfix] Skip post-experts EP all-reduce on the FlashInfer cutlass FP4 all-gather MoE path ( #41072 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-08 18:01:27 +00:00
cbf49695c3
[CI] Add dedicated sglang-processor workflow ( #43022 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 01:57:05 +08:00
shodoco
e2fbc6a550
Stream a chunk once stream_interval tokens are unsent ( #42643 )
2026-10-08 10:12:37 -07:00
Eric Zhang
344b8a863b
[Perf] Fuse short-convolution checkpoint tracking metadata ( #43013 )
2026-10-09 00:26:15 +08:00
Ke Bao
642e81fbb8
Fix DSA index host buffer cleanup ( #43100 )
2026-10-09 00:21:38 +08:00
Ke Bao
46ad9eb439
Fix K-only host buffer cleanup ( #43101 )
2026-10-09 00:21:27 +08:00
billishyahao
4ab4d90d78
[AMD][DCP 3/N] add aiter asm for kimi k3 target verify ( #34537 )
2026-10-08 09:13:03 -07:00
Mick and Mick Qian
3030e31ff8
[diffusion] refactor: deduplicate request extraction and ComfyUI test helpers ( #42903 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 23:44:35 +08:00
黄孝君
9f5d871251
[NPU] Quote the arm64 Triton-ascend wheel URL in npu.Dockerfile ( #43035 )
2026-10-08 21:44:49 +08:00
943621c336
[sgl-router] Render DeepSeek-V4 through sglang-processor ( #42665 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-08 20:16:20 +08:00
11b1fb21de
[MiniMax-M3] Decode: paged K/V tile loads and a tiny GEMM for the router projection ( #38841 )
...
Co-authored-by: Hao Phan <htphan@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 19:39:31 +08:00
6e6ccb26f2
[MiniMax-M3] Instantiate the decode block top-k in register buckets ( #38615 )
...
Co-authored-by: Hao Phan <htphan@nvidia.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-08 19:33:17 +08:00
45abea0269
[rust-processor] DeepSeek-V4 parity with SGLang, plus a reusable parity harness and skill ( #42664 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 18:35:30 +08:00
Shangming Cai
9578cb1c63
[CI] Don't run the full GPU suite for renderer-only Cargo.lock changes ( #43025 )
2026-10-08 18:25:06 +08:00
Mick and Mick Qian
7dd512b0e8
[diffusion] refactor: deduplicate FLUX positions and RoPE application ( #43002 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 18:18:39 +08:00
Horace He
8ad24fb560
[Perf] Cache compiled ModelOpt exclusion patterns ( #42927 )
2026-10-08 17:10:25 +08:00
Liangsheng Yin
4b384df0c9
[CI] Keep the KL tests' LongBench cache outside the checkout and bound its download ( #43068 )
2026-10-08 01:17:31 -07:00
Liangsheng Yin
7b9c42cf49
[CI] Remove the unregistered chunked-prefill tests under test/manual ( #43081 )
2026-10-08 01:16:39 -07:00
Liangsheng Yin
89b5935793
[MemCache] Return only the loaded length from init_load_back; drop empty_device_indices ( #43023 )
2026-10-08 01:14:33 -07:00
Liangsheng Yin
5a4d01a3ae
[MemCache] Return only the matched length from match_prefix; read KV indices off the node path ( #42923 )
2026-10-08 01:10:14 -07:00
Spandan Tiwari
2d26823f94
[AMD] Drop static input_scale on aiter per-token FP8 path (use dynamic per-token) ( #42152 )
2026-10-08 01:06:34 -07:00
Cheng Wan
b3e6c87be3
[Refactor] Translate each KV loc once per iteration ( #42753 )
2026-10-08 00:58:28 -07:00
508ddc859a
[unified-memory] Admit DFLASH and DSPARK with fused draft KV ( #40492 )
...
Co-authored-by: Caihua Li <caihua.li@bytedance.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-08 00:56:21 -07:00
c2d491e38b
[unified-memory] Admit EAGLE/EAGLE3 onto fused draft KV, on mamba-MHA and MLA hosts ( #40491 )
...
Co-authored-by: Caihua Li <caihua.li@bytedance.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-08 00:55:54 -07:00
Michael
5afe4d6334
[AMD] Fix DSpark draft bucket tests for the raw in-graph metadata default ( #42230 )
2026-10-08 00:54:26 -07:00
8b80808366
[unified-memory] Fuse the draft model's KV into the target's page envelope (mechanism) ( #37628 )
...
Co-authored-by: Caihua Li <caihua.li@bytedance.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-08 00:54:19 -07:00
08d30f2300
[unified-memory] Speculative decoding on the unified memory pool: translate every read and write path ( #37627 )
...
Co-authored-by: Caihua Li <caihua.li@bytedance.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-08 00:52:21 -07:00
33883a5fa3
[AMD] Small-M W8A8 FP8 projection GEMM for Qwen3.5 AttnFP8 on gfx950 ( #41134 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-10-08 00:50:16 -07:00
b2cd65b8b0
[AMD] One-launch small-M MoE router for Qwen3.5 on gfx950 ( #41133 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-10-08 00:47:52 -07:00
f795fe352d
[Test] Add a docs drift check for server_arguments.mdx ( #38520 )
...
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com >
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-08 00:37:22 -07:00
Chunan Zeng
785005d21b
[DSpark] Mask uncommitted slots before commit-inject cache lookups ( #42624 )
2026-10-08 00:16:54 -07:00
Yuwei An
54b551df8e
[Refactor][TCPCG] Bind live forward batches in BCG eager calls (2/9) ( #42286 )
2026-10-07 23:55:41 -07:00
Mick and Mick Qian
5c5f593110
[diffusion] refactor: share KL VAE encode/decode and tiling operations ( #42841 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 14:43:19 +08:00
Mick and Mick Qian
d1b751d958
[diffusion] refactor: inherit existing defaults instead of repeating overrides ( #42963 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 14:42:21 +08:00
74308cf1d6
[diffusion] fix: wait for multimodal process teardown ( #40765 )
...
Signed-off-by: Xucheng Zhou <aden1350@outlook.com >
Co-authored-by: Mick <mickjagger19@icloud.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 14:36:17 +08:00
4f320fb85a
[diffusion] perf: FastH3 8-Step V2 native support, native Blackwell VSA and batched H3 VAE decode ( #37662 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: mickqian <mickqian@users.noreply.github.com >
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-08 14:35:27 +08:00
Mick and Mick Qian
57e9ff5403
[diffusion] refactor: share timestep retrieval across pipeline stages ( #42967 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 14:33:39 +08:00
Liangsheng Yin
0f5bc4af4b
[CI] Fix stale fakes in the NPU CUDA-graph backend unit test ( #43056 )
2026-10-07 23:04:48 -07:00
Mohammad Miadh Angkad
ac656ee79a
[CI] Re-enable B300 ( #42959 )
2026-10-07 22:03:16 -07:00
Jzz1943
b7b2975b57
fix(dsa): align FA3 metadata with unpadded queries ( #42793 )
2026-10-07 21:37:21 -07:00
hmalgewatta and Alex Nails
7327fb225f
[DSV4] Guard V4.1 indexer SM100 check on non-CUDA devices ( #42114 )
...
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
2026-10-08 12:06:28 +08:00
Thomas Wang
b72da1b817
[AMD] Update v4 amd cookbook 1008 ( #43033 )
2026-10-08 11:18:41 +08:00
AndyLi429 and AndyLi429
89bea053ac
[NPU] Add NPU cudagraph runner unit tests ( #40404 )
...
Co-authored-by: AndyLi429 <AndyLi429@noreply.gitcode.com >
2026-10-08 11:18:38 +08:00
chilltongx and chilltongx
e7530a3a26
[diffusion] fix: fix forward multimodal token type IDs in image encoding ( #34494 )
...
Co-authored-by: chilltongx <284668524+chilltongx@users.noreply.github.com >
2026-10-08 10:55:55 +08:00
WenhaoZhang
b0d7254fbc
[diffusion] CI: rebase LTX H100 timing references on recent CI runs ( #42986 )
2026-10-08 10:42:12 +08:00
Jiajun Li and Claude Opus 5.5
99463390f8
[Fix] MiMo-V2: accept checkpoints with the split attention layout ( #40978 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 19:18:32 -07:00
c892301ff7
[Platform] Thread vendor KV pool classes through the in-tree builders; add PlatformCapabilities ( #35594 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com >
2026-10-07 19:16:10 -07:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
7b6b6ee25e
[Docs] Cookbook: interactive SemiAnalysis InferenceX AgentX launches for GLM-5.2, DeepSeek-V4 and DeepSeek-V4.1 ( #42726 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 09:33:40 +08:00
Mick and Mick Qian
21fa2251a0
[diffusion] refactor: share sigma schedules and causal cross-attention ( #42848 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 09:29:33 +08:00
Mick and Mick Qian
28c08b02c3
[diffusion] refactor: deduplicate MOVA initialization ( #42983 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 09:28:06 +08:00
Mick and Mick Qian
243f3dc35f
[diffusion] refactor: deduplicate Qwen-VL inputs and component accuracy suites ( #42860 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 09:27:22 +08:00
Mick and Mick Qian
2489bd1d2e
[diffusion] refactor: share ModelOpt checkpoint conversion I/O ( #42856 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 09:26:39 +08:00
Mick and Mick Qian
35608e48d1
[diffusion] refactor: reuse the single-process parallel test fixture ( #42964 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 09:24:53 +08:00
ashwini rathi
b91919ffe2
[XPU][diffusion] Re-baseline Wan2.1-T2V-1.3B TextEncodingStage on B60 ( #42794 )
2026-10-08 09:22:41 +08:00
Mick and Mick Qian
287b8b793f
[diffusion] refactor: deduplicate VAE resampling and LTX tiling ( #42866 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 09:22:11 +08:00
Mick and Mick Qian
0c00bffc6f
[diffusion] refactor: deduplicate Qwen Image RoPE and SenseNova backbones ( #42843 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 09:20:55 +08:00
Mick and Mick Qian
1e2f524726
[diffusion] refactor: share stacked encoder weight loading ( #42988 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 09:16:03 +08:00
Liangsheng Yin
0427ea8032
[CI] Split the shared Cargo target dir per glibc ( #43024 )
2026-10-07 18:14:26 -07:00
Mick and Mick Qian
d52ea5ee90
[diffusion] refactor: deduplicate realtime transport and LingBot tests ( #42990 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-08 09:10:10 +08:00
Richard Wang
5cb7fb886c
[Feature] support pplx-decider-v1.1 and update the pplx-decider cookbooks ( #42645 )
2026-10-08 09:03:50 +08:00
Mick and Claude Opus 5.5
9101e895ea
[diffusion] feat: encode the MP4 while the VAE decodes (MiniMax-H3, Wan) ( #41819 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 09:01:19 +08:00
Cheng Wan
7760d5264a
[Refactor] Use destination layouts for model checkpoint mappings ( #42590 )
2026-10-07 16:24:22 -07:00
Cheng Wan
5e12831cec
[Refactor] Load auxiliary and expert checkpoints using owner partitions ( #42589 )
2026-10-07 16:24:03 -07:00
Cheng Wan
64c7f65eac
[Refactor] Retain process groups in parallel linear layers ( #42588 )
2026-10-07 16:23:40 -07:00
Cheng Wan
267e34516c
[Refactor] Select parallel groups for remaining model projections ( #42587 )
2026-10-07 16:22:57 -07:00
Cheng Wan
dec5d9d6fc
[Refactor] Let vision encoders select their parallel group ( #42586 )
2026-10-07 16:22:32 -07:00
Cheng Wan
b83827ea02
[Fix] Quantize reloaded FP8 weights after native sharding ( #42585 )
2026-10-07 16:20:47 -07:00
Cheng Wan
59740a728f
[Refactor] Let MLA projections select their parallel group ( #42484 )
2026-10-07 16:19:33 -07:00
Cheng Wan
961d7290fe
[Refactor] Let model MLPs select their parallel group ( #42468 )
2026-10-07 16:18:14 -07:00
Cheng Wan
4c5130ac5a
[Refactor] Let linear layers select their parallel group ( #42426 )
2026-10-07 16:15:26 -07:00
Liangsheng Yin
1066decce6
[Scheduler] Replace extend_range with extend_end derived from prefix_len ( #42822 )
2026-10-07 16:07:16 -07:00
Liangsheng Yin
5349d3e9d2
[Scheduler] Derive prefix KV indices from last_node at allocation; drop Req.prefix_indices ( #42825 )
2026-10-07 16:03:09 -07:00
Liangsheng Yin
e9240b3243
[Scheduler] Track prefill progress as prefix_len ( #42824 )
2026-10-07 15:59:06 -07:00
Kevin Mi and Claude Opus 5.5
79ef49cb6b
[Docs] GLM-5.3-Flash: measured MI355X MXFP4 agentic recipe ( #43015 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 15:49:49 -07:00
Liangsheng Yin
4bf8f28d15
[Scheduler] Fold match write-back into match_kv_cache ( #42823 )
2026-10-07 15:43:05 -07:00
Liangsheng Yin
b1ffe5c16a
[dLLM] Keep the request row across FDFO blocks ( #42846 )
2026-10-07 15:39:33 -07:00
Ping Qiu and Zhiqiang Xie
685d4d453d
[HiCache] Add SeaweedFS L3 storage backend ( #42399 )
...
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu >
2026-10-07 15:05:37 -07:00
WenhaoZhang
0b635266d4
[diffusion] CI: warm up CI cases at the request's quality level ( #42958 )
2026-10-07 23:26:36 +08:00
00b21d6415
[diffusion] fix: reset DiT cache state at the start of each request ( #42437 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-07 23:25:49 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
44d10eab5e
[GLM-V] Shard the vision tower over the attention-TP group under DP attention ( #42931 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-07 23:17:06 +08:00
d9a0f1cb34
[diffusion] fix: fix hybrid SP+TP for I2V models ( #41917 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-07 22:33:03 +08:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
1e86143bf0
[Bugfix] Return None for empty reasoning_content outside K2 Horizon ( #42962 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 21:02:40 +08:00
WenhaoZhang
492dbd0807
[CI] keep diffusion-only kernels and tests out of the srt stages ( #42706 )
2026-10-07 20:28:53 +08:00
Tianyang Gu and Claude Fable 5.1
aa5551d9b6
[diffusion] fix: do not interleave comfy-kitchen MXFP8 scales twice ( #38028 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-07 17:56:39 +08:00
YC Yen-Ching Tseng
c389cbaf99
[AMD][diffusion] Route bf16 fuse_scale_shift to the native fallback on gfx1250 ( #42859 )
2026-10-07 02:37:43 -07:00
YC Yen-Ching Tseng
49d7d60dfa
[AMD][diffusion] VAE attention to the math SDPA backend on gfx1250 ( #42858 )
2026-10-07 02:32:02 -07:00
Yihao Wang and Claude Opus 5.5
11972e5207
[deps] bump transformers to 5.19.0 ( #42784 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 17:04:08 +08:00
Rain Jiang
c52dcc5a01
The sgalng api specs ( #41585 )
2026-10-07 01:55:01 -07:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
ba77d30256
[FA4] Keep stream last in the SM100 MLA kernel so disk-cached objects load ( #42926 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-07 15:58:55 +08:00
Khoa Pham
e23093077e
[DSv4] Fix shared index cache accessor page size ( #42832 )
2026-10-07 00:53:17 -07:00
jiaryang
1c42ad3679
[ROCm] Skip the tilelang act_quant in the DSA indexer on gfx1250 ( #42747 )
2026-10-07 00:16:46 -07:00
Shuwen Wang and Claude Opus 5
2889158792
[MemCache] test: require every ComponentData field to be classified by each split hook ( #42518 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-07 14:53:34 +08:00
paulzhang-tm
c721602ad2
Cover SWA admission after declined HiCache restores ( #39839 )
2026-10-07 14:51:45 +08:00
Hyogeun Oh (오효근) and mickqian
1d019ce1c8
[diffusion] fix: honor the lazy transformer loader contract ( #15215 )
...
Signed-off-by: Zerohertz <ohg3417@gmail.com >
Co-authored-by: mickqian <mickqian@users.noreply.github.com >
2026-10-07 14:23:00 +08:00
Mark
b0dff4009f
[diffusion] cli: restore gated Diffusers model detection ( #34903 )
2026-10-07 14:21:37 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
f47d89564d
[KDA] Default GLM-5-Next spec verify to the CuTe DSL KDA kernel ( #42905 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-06 22:31:41 -07:00
AMD-yanfeiwang and Thomas Wang
9ddbba50dd
[AMD][DSV4] feat: enable PCP with fp8 unified_kv on gfx950 ( #39923 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
2026-10-06 22:09:18 -07:00
Seas0 and Mick Qian
43b4abd857
[diffusion] fix: validate SageAttention 2 head sizes without overriding component policy ( #41529 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-07 12:39:54 +08:00
Xinyi Song
168458fd90
[AMD][V4.1] Fix wo_b input flatten for packed fp8 activations on gfx950 ( #42796 )
2026-10-06 21:35:51 -07:00
Khoa Pham
9e26600f3a
[DSA] Clarify pooled-page naming and ownership ( #42671 )
2026-10-06 21:05:29 -07:00
CuzMi
bde01f8dc8
[Diffusion] Fuse Anima AdaLN modulation and cache the FFI callable ( #42608 )
2026-10-07 12:01:59 +08:00
Tuan Nguyen Gia and Cursor
c0df52be2d
[AMD] Fix Quark W4A4 MXFP4 MoE clamped-SwiGLU on the AITER path ( #34540 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-07 03:20:40 +00:00
Yi (Vincent) Zhong and Claude Opus 5.5
012cd3858b
[Fix] Make the fused KDA beta sigmoid bit-identical to torch.sigmoid ( #42611 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 20:04:59 -07:00
Thomas Wang
9a21d82958
[AMD] Update v4.1 amd cookbook 1006 ( #42847 )
2026-10-06 19:58:58 -07:00
Jay Zenith
35f18848db
[diffusion] fix: validate output_quality and fix return type annotation ( #19530 )
2026-10-07 10:53:29 +08:00
Yash Akhauri and BBuf
9c8fc6a8e4
[Bugfix] Preserve empty K2 Horizon reasoning in chat responses ( #42623 )
...
Co-authored-by: BBuf <1182563586@qq.com >
2026-10-07 10:44:55 +08:00
d7190b0122
[Perf] Accelerate DiffusionGemma with FA4 and CUDA graphs on Blackwell ( #42239 )
...
Co-authored-by: lianghougong <lianghouhong@xiaohongshu.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-07 10:31:00 +08:00
2f6e0ec45d
[DSv4.1] Trim prefill model-entry overhead ( #41704 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: BBuf <1182563586@qq.com >
2026-10-07 09:50:34 +08:00
7a4d6dfd2f
[diffusion] feat: name the quality tiers after what they guarantee ( #42370 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-07 09:49:30 +08:00
Lianmin Zheng
8f1790f595
Add Rust code owners and grant shodoco CI permissions ( #42826 )
2026-10-06 18:37:54 -07:00
660b17e8dc
[AMD] Let AITER unified verify run without host seq_lens ( #42635 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: zx3xyy <zx3xyy@users.noreply.github.com >
2026-10-06 18:27:27 -07:00
Eddie A. Tsai
e13933c4fd
[diffusion] docs: add gfx1151 Strix Halo serving guidance ( #41670 )
2026-10-07 09:14:07 +08:00
Michael
3e914e69d5
[AMD] ci: nightly-test the miles ROCm 10 MI30X image on MI300 ( #42771 )
2026-10-06 18:05:09 -07:00
WenhaoZhang
6da08fd7cf
[diffusion] CI: fix qwen cache-dit step 10/49 perf baselines ( #42495 )
2026-10-07 08:51:12 +08:00
Mick and Mick Qian
1351f32bb4
[diffusion] refactor: simplify Kandinsky 6 models, SR sampling and tests ( #42768 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-07 08:48:16 +08:00
f1065423d3
[PD] Stop reserving unallocated mamba spec-verify scratch in prefill KV sizing ( #42649 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: 842974287 <842974287@users.noreply.github.com >
2026-10-06 17:16:22 -07:00
Baizhou Zhang
f497b626b6
build: use one DeepEP ref for implementation and packaging ( #42827 )
2026-10-06 17:04:28 -07:00
Duyi-Wang
66dd4e15b9
[AMD][Docs] Add MI355X DeepSeek-V4 Pro PD recipes ( #42757 )
2026-10-06 16:58:40 -07:00
08aad0c196
[HiCache] Manage buffer-mode backups per storage pool ( #42633 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-06 16:45:05 -07:00
jain-ria and ishandhanani
fd62ee84f7
feat(grpc): expose follower metadata and node-local KV sources ( #39659 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-10-06 16:39:01 -07:00
Chunan Zeng and Cheng Wan
90aa1cb0fb
[Scheduler] Gather prefill-delayer queue timeout across ranks ( #42625 )
...
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com >
2026-10-06 16:15:34 -07:00
Jan Bernlöhr and Po-Han Huang
0a0fe001f1
fix: route Thor SM110 FP8 and ModelOpt NVFP4 auto backends ( #41765 )
...
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-06 16:02:45 -07:00
6b737fd4c6
Bind the MessageQueue remote socket atomically ( #42695 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: charlotte12l <charlotte12l@users.noreply.github.com >
2026-10-06 15:50:59 -07:00
Trevor Morris
44a2558d40
[NVIDIA] Fix nightly-cu134 build and update ( #42661 )
2026-10-06 14:52:42 -07:00
264d1c2015
Back the DFlash-family draft KV pool to the post-capture token count ( #42651 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: 842974287 <842974287@users.noreply.github.com >
2026-10-06 14:23:18 -07:00
4506927635
[Metrics] Add per-rank DP attention imbalance metrics ( #42755 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: hanming-lu <hanming-lu@users.noreply.github.com >
2026-10-06 14:20:41 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
38bdf25f8d
[Docker] Use uv-managed Python 3.12 and uv for all installs in the CUDA image ( #42612 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-06 14:10:09 -07:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
3eec178e48
[Spec][MegaMoE] Let the speculative draft choose its own W4A4 MXFP4 MegaMoE MMA type ( #42022 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 14:09:57 -07:00
Liangsheng Yin
12004043b8
[Metrics] Read scheduled prefill KV tokens from batch prefix_lens ( #42797 )
2026-10-06 13:11:29 -07:00
a34cba3c62
[HiCache] Resolve write_back to write_through under buffer_only host memory ( #42754 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-06 11:02:23 -07:00
0f57a39152
[rust-processor] Gate render, tokenizer and parser behind cargo features ( #42663 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 10:58:38 -07:00
Byron Hsu
abed685de1
[PD] Fix crash for allocation when mamba + pd + decode radix cache ( #41927 )
2026-10-06 09:36:31 -07:00
Michael and Claude
209e190ee5
[AMD] ci: build the miles ROCm 10 MI30X image nightly ( #41605 )
...
Co-authored-by: Claude <noreply@anthropic.com >
2026-10-07 00:00:11 +08:00
Emil Bogomolov and Emil Bogomolov
e17935820a
[diffusion] fix: fix width/height silently dropped on multipart /v1/videos requests ( #35108 )
...
Co-authored-by: Emil Bogomolov <zetyquickly@googlemail.com >
2026-10-06 23:31:31 +08:00
WenhaoZhang and Mick
2579875b5d
[diffusion] feat: support loading community ComfyUI-GGUF Qwen-Image-2.1 transformers ( #42556 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-06 23:30:48 +08:00
49bafb58cd
perf(mla): use native SATFINITE FP8 conversion in MLA KV/Q preparation ( #42659 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-06 22:47:47 +08:00
Xiaoshuai Zhang and jet
679d2851ba
fix(mimo-vl): keep multimodal features when capturing aux hidden states and correct vision preprocessing ( #40888 )
...
Co-authored-by: jet <dev@jetd.one >
2026-10-06 22:15:14 +08:00
16a23a672b
[diffusion] model: support Kandinsky 6 ( #42743 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Lev Novitskiy <57654885+leffff@users.noreply.github.com >
Co-authored-by: Kirill <kykozlov@gmail.com >
Co-authored-by: Artem Borisov <divotionchannel@gmail.com >
2026-10-06 22:13:29 +08:00
730f1f3e5b
[DeepSeek-V4.1] Unify mHC into one state machine and drop medium-batch fusion variants ( #42245 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: BBuf <1182563586@qq.com >
2026-10-06 21:01:42 +08:00
WenhaoZhang
2a5f45d3d4
[diffusion] fix: skip zero blocks when hashing large conditioning inputs ( #42748 )
2026-10-06 20:43:08 +08:00
WMC and jacky.cheng
ba94488b42
[diffusion] feat: default Qwen-Image 2.1 to VAE tiling on gfx1151 ( #41655 )
...
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-10-06 20:20:58 +08:00
Shangming Cai
1291f3175d
[rust-processor] Fix chat-template precedence wording in the processor README ( #42756 )
2026-10-06 19:10:26 +08:00
Shangming Cai and Claude Fable 5.1
28525b43c4
[sgl-router] Document and reject a zero request timeout ( #42751 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-06 19:07:37 +08:00
cf19ae6d04
[rust-processor] Split sglang-processor into tokenizer, render and parser ( #42662 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 18:59:02 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
45db430530
[DSA] Fix pooled page table refresh for target verify CUDA graph replay ( #42750 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-06 18:57:52 +08:00
2167cf90c3
[sgl-router] Bound streaming time-to-headers by --request-timeout-secs (retry 5/5) ( #42632 )
...
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 18:35:11 +08:00
YC Yen-Ching Tseng
f3b5a28f43
[AMD] M3-mxfp8 Nightly Test Update ( #42605 )
2026-10-06 02:22:53 -07:00
YC Yen-Ching Tseng
542addad32
[AMD] M3-MXFP4 Nightly Test ( #42602 )
2026-10-06 02:22:01 -07:00
662879e495
[sgl-router] Back off between retry attempts (retry 4/5) ( #42631 )
...
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-06 17:06:45 +08:00
Gawr Gura 0721
b563bbd597
[Fix] Don't let a failed deep_ep import-time check kill servers that never use DeepEP ( #40671 )
2026-10-06 01:58:02 -07:00
e49e971131
[mem_cache] Keep prefix_indices current for chunked requests kept out of the tree ( #42467 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-10-06 01:40:35 -07:00
bb4d0a13c1
[sgl-router] Retry failed dispatches on another worker (retry 3/5) ( #42630 )
...
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
2026-10-06 16:28:52 +08:00
Jzz1943
1be3013118
[diffusion] feat: support Ideogram TurboTime LoRA ( #30487 )
2026-10-06 16:15:21 +08:00
Max Kong
01345197af
[diffusion] chore: harden OpenAI upload filename handling ( #17270 )
...
=
2026-10-06 16:14:39 +08:00
Jinbao Chen and Mick Qian
d0494af08b
[diffusion] security: pin multimodal-gen ZMQ scheduler ingress to loopback on wildcard --host ( #36854 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-06 16:05:17 +08:00
Liangsheng Yin
c7f3ae1dd5
[mem_cache] Hold a request's tree lock as one TreeLock ( #42686 )
2026-10-06 00:59:38 -07:00
05340e9ecb
[sgl-router] Make one request dispatchable more than once (retry 2/5) ( #42629 )
...
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 15:24:02 +08:00
c683190b40
[PD] Wait for prefill completion before Mooncake early KV transfer ( #41395 )
...
Co-authored-by: Shangming Cai <csmthu@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-06 15:17:12 +08:00
8c273d6813
[sgl-router] Let selection skip workers a request already failed on (retry 1/5) ( #42628 )
...
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 15:10:51 +08:00
Rumit Desai
33ce0943e9
[Bugfix] Guard merge_state sgl_kernel import on non-CUDA platforms ( #42682 )
2026-10-06 14:19:20 +08:00
Joe
40fb5935c0
Allow platforms to declare Mamba extra-buffer support ( #36410 )
2026-10-06 14:18:49 +08:00
Liangsheng Yin
9a09b48da7
[Fix] Add maybe_hand_to_session to the one-batch benchmark TreeCacheNamespace ( #42723 )
2026-10-05 23:09:41 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
60878c3fba
[CI] Temporarily disable B300 ( #42704 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-05 22:27:54 -07:00
b524de2de6
[diffusion] optimization: release the allocator cache when the server idles not per request ( #41718 )
...
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-06 12:27:08 +08:00
e446698cdd
[SM120&90] Add CUDA fused Triton sparse-MLA prefill backend for DSA ( #32779 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
Co-authored-by: Po-Han Huang <pohanh@nvidia.com >
2026-10-05 21:08:05 -07:00
1e1bdeedbb
[Perf] Improve GLM-5.3-Flash DFlash decoding on B300 ( #42489 )
...
Co-authored-by: lianghougong <lianghouhong@xiaohongshu.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-06 12:05:37 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
fcbbefa82a
[GLM-5.3-Flash] Autotune MoE kernels at prefill sizes ( #42693 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-05 21:04:54 -07:00
Yuwei An
620c76f25d
[DSV4.1] Add opt-in TRT-LLM sparse attention support ( #41603 )
2026-10-06 12:02:10 +08:00
Zhang, Jiejing
c76faa1ff2
docs(cookbook): enable Top-K v2 for GLM-5.2 MI355X MTP ( #42613 )
2026-10-06 11:33:24 +08:00
Ke Bao
ed75fe67f1
Declare hybrid Mamba indexer host pool ( #40915 )
2026-10-06 10:18:36 +08:00
CuzMi
67959f80ab
[diffusion] optimization: fuse Anima FP32 split-half Q/K RoPE ( #42609 )
2026-10-06 10:18:09 +08:00
Thomas Wang
1abd356595
[AMD] Add engram host/device address mapping ( #42552 )
2026-10-05 19:12:12 -07:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
f2f0be731c
[Docs] GLM-5.2: move GB300 AgentX recipe to the agentic HiCache section and drop no-op settings ( #42167 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 09:40:30 +08:00
Mick and Mick Qian
21c9bbdf2c
[diffusion] fix: fix concurrent server perf dump writes across ranks ( #41541 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-06 09:29:51 +08:00
Yash Akhauri and Xiaoyu Zhang
36b216e2ee
[Docs] Refresh K2 Horizon checkpoint revisions ( #42130 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-06 09:19:04 +08:00
5d385007b3
[diffusion] kernels: take the Wan VAE decoder's remaining first-decode work off the request path ( #41830 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 09:05:29 +08:00
3120bb87a2
[diffusion] fix: fix the OOM hint's placement advice and add a 24 GB Wan2.1 recipe ( #41723 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 09:01:02 +08:00
Liangsheng Yin
ad7a9b53cc
[mem_cache] Make release_kv_cache's final checkpoint an explicit checkpoint flag ( #42270 )
2026-10-05 17:32:07 -07:00
Po-Han Huang (NVIDIA)
62af5f56a5
[CI] Drop the redundant and broken tc_piecewise tests ( #42571 )
2026-10-05 17:29:32 -07:00
Liangsheng Yin
c01d59eb90
[Session] Let a streaming session own its KV record and tree lock from the first turn ( #42537 )
2026-10-05 17:28:09 -07:00
Liangsheng Yin
f3fe753475
[misc] Rename SGLANG_OPT_SWA_RELEASE_LEAF_LOCK_AFTER_WINDOW to SGLANG_OPT_RELEASE_PREFILL_SWA ( #42657 )
2026-10-05 17:24:15 -07:00
Sage
4409d2fd22
[ci] add renderer automatic labeler ( #40923 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-10-05 17:12:51 -07:00
Yuwei An
18ec2ecce3
[Refactor][TCPCG] Relocate shared graph tensor and DSA head-gate helpers (1/9) ( #42285 )
2026-10-05 17:06:52 -07:00
ADiL
968726f3ce
[HiCache] Route --file-storage-path to the file storage backend ( #33883 )
...
Signed-off-by: adlashab <adlashab@amd.com >
2026-10-05 15:33:49 -07:00
61ba98443d
[DSA] k-pool: 256-token logical page so index page id = logical page id ( #42178 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Ke Bao <ispobaoke@gmail.com >
2026-10-05 15:02:44 -07:00
c765f8818a
[DFlash2] Keep the greedy selector walk in range for all-NaN score rows ( #41737 )
...
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com >
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 14:29:13 -07:00
maocheng23
57826a62ea
[Speculative] Add opt-in block verification ( #42297 )
2026-10-05 11:49:23 -07:00
70f0b7351e
[sgl-router] Improved policy balanced mode ( #42545 )
...
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
2026-10-05 09:48:37 -07:00
debanshd
efb62ce269
[diffusion] fix: resolve QKV layout inference for MiniMax-H3 checkpoints ( #40158 )
...
Signed-off-by: Debanshu Das <debanshu@google.com >
2026-10-05 23:01:37 +08:00
150f568900
[diffusion] kernels: stop specializing on per-request sequence lengths ( #41710 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 22:21:01 +08:00
a977e3b9d5
[diffusion] optimization: stream an oversized DiT in auto mode instead of OOMing ( #41721 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 21:58:27 +08:00
ad3e80d4de
[diffusion] chore: warm the image writer and the HTTP route models at startup ( #41833 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 21:58:00 +08:00
c83c84b422
[diffusion] chore: keep the DiT off component offload under explicit multi-GPU fsdp ( #41722 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 21:56:52 +08:00
wxy
23bf107469
[diffusion] optimization: cache LTX-2 RoPE coords to avoid per-step recompute ( #22441 )
2026-10-05 21:21:36 +08:00
Shiwen Cheng
91f9bf8d17
[diffusion] fix: fix scheduler host not working when ipv6 in multi modal gen ( #22813 )
2026-10-05 21:14:09 +08:00
liuke and Mick
f20f8e120d
[diffusion] model: restore folded H3 GGUF patch embedding ( #39523 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-05 21:10:06 +08:00
Mick and Mick Qian
83a6e1d39b
[diffusion] docs: keep model compatibility details in cookbook recipes ( #42227 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-05 20:02:43 +08:00
Akshat Anand and cipheraxat
860ebe720e
[diffusion] fix: skip warmup preferred preload when it would OOM ( #40761 )
...
Co-authored-by: cipheraxat <cipheraxat@users.noreply.github.com >
2026-10-05 19:59:35 +08:00
aiyueqi and Mick
8663e3b670
[diffusion] optimization: avoid full-video clone in VAE post-processing for MiniMax-H3 ( #40095 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-05 19:56:11 +08:00
d349e672bc
[diffusion] feat: recycle warmup-only entries of conditioning-cache at the first served store ( #41832 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 19:55:30 +08:00
7352073ec6
[diffusion] feat: support --async-output-save to overlap output saving with the next request ( #37549 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-05 19:53:38 +08:00
WenhaoZhang and Mick Qian
f048d5aa4b
[diffusion] feat: add community LoRA recipes and Kohya mapping ( #35857 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-05 18:22:56 +08:00
08fa37e48d
[sgl-router] Add reorg bucket config with per-group admission ( #42429 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
2026-10-05 18:17:32 +08:00
Shangming Cai and Claude Opus 5.5
ab4f8a44f4
[sgl-router] Point the design doc at the shared engine ranking ( #42572 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 18:16:42 +08:00
a96b146491
[sgl-router] Move pressure and prefix signals out of legacy policies ( #42428 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 18:10:29 +08:00
mrusanovsky and Po-Han Huang
26a1377c50
[Spec] LiLiCorr: named head MLPs and quantized head linears ( #42057 )
...
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-05 02:43:59 -07:00
WenhaoZhang and Mick Qian
284cda01fb
[diffusion] optimization: fuse the video VAE decoder's RMSNorm and QK RoPE for MiniMax-H3 ( #41906 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-05 17:30:39 +08:00
0263fdacb2
[diffusion] fix: keep generic warmup valid for step-floored models and stop reporting a failed warmup as warm ( #41641 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 17:23:06 +08:00
elvischenv and Claude Opus 5.5
55dcf3254f
Fix XQA draft extend with FP8 KV cache in trtllm_mha ( #41656 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-05 01:37:54 -07:00
Xiaoyu Zhang
abd6c374c7
[Perf] Optimize MiMo local audio attention on Hopper ( #42101 )
2026-10-05 16:14:56 +08:00
Michael
b3f967e809
Revert "Revert "[AMD][ROCm] Keep cos_sin_cache fp32 on HIP for fused QSA indexer kernel ( #41282 )"" ( #42558 )
2026-10-05 16:12:21 +08:00
Aleksi Vesanto and jacky.cheng
9f7d38ffd3
[diffusion] attention: Allow disabling sequence masking in SP ( #33855 )
...
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-10-05 15:22:53 +08:00
Aleksi Vesanto and jacky.cheng
c41b2d26e5
[diffusion][ROCm][Perf]: Set gfx942 AITER FMHA rounding mode to rtz instead of rtna ( #28650 )
...
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-10-05 15:21:04 +08:00
Mick and Mick Qian
dd2ffdb680
[diffusion] nightly: keep nightly regression baselines on a comparable methodology ( #42226 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-05 15:19:21 +08:00
438d9d2074
[Feature] DeepEP v2: expanded (do_expand=True) prefill dispatch ( #37261 )
...
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com >
2026-10-05 00:02:10 -07:00
cafb37a10d
[JIT] Fuse FP8 KV-cache quantization into the prefix-valid commit kernel ( #40785 )
...
Co-authored-by: Jayden Yu <jaydenyu@fb.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-05 14:47:54 +08:00
WenhaoZhang and Mick Qian
f218798006
[diffusion] CI: relax the load latency check and extend the 2-GPU retry deadline ( #42505 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-05 14:43:27 +08:00
520d6d0229
[DSv4.1] Prefill indexer: raw selection, row-pair schedule ids, no host syncs ( #41659 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: BBuf <1182563586@qq.com >
2026-10-05 14:33:37 +08:00
Liangsheng Yin
503b898540
[Session] Count streaming session KV by owner in pool accounting ( #42539 )
2026-10-04 22:15:57 -07:00
Michael
456d533e61
Revert "[AMD][ROCm] Keep cos_sin_cache fp32 on HIP for fused QSA indexer kernel ( #41282 )" ( #42541 )
2026-10-04 21:59:12 -07:00
Liangsheng Yin
1818db4e54
[CI] Drop the echoed command from /rerun-test result replies ( #42543 )
2026-10-04 21:50:53 -07:00
ashwini rathi
b1bbd74f28
[XPU][CI] Disable test_xpu_graph until tc_piecewise is removed ( #41634 ) ( #42524 )
2026-10-05 11:09:26 +08:00
Xiaoyu Zhang
f385390be5
[Refactor] Simplify GDN, vision RoPE and MLA kernel dispatch ( #42506 )
2026-10-05 09:37:53 +08:00
734cf3cf3b
[PD] Reload NIXL peer metadata after the peer is invalidated ( #41691 )
...
Co-authored-by: agent <agent@local>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com >
2026-10-05 00:49:26 +00:00
Yuwei An
f70e8c68fc
[Dsv4.1] Bounded replay with sparse mla path ( #42273 )
2026-10-04 17:19:14 -07:00
Eric.Chin.AMD and Bingxu Chen
b792228b35
[ROCm] topk v2: split one long row across blocks, the CDNA cluster-path equivalent ( #39931 )
...
Co-authored-by: Bingxu Chen <bingxche@amd.com >
2026-10-04 16:54:37 -07:00
Cheng Wan
1f299e9f2f
[Refactor] Build layer stacks in order with append_stages ( #42481 )
2026-10-04 16:40:24 -07:00
Cheng Wan
a99567ab74
[Fix] GigaChat 3.5: build the stage boundaries once ( #42480 )
2026-10-04 16:39:34 -07:00
Cheng Wan
5c8fb9e938
[Fix] Offer the fused MoE finalize all-reduce only to a block that owes the sum ( #42479 )
2026-10-04 16:39:06 -07:00
Cheng Wan
f07c1a398e
[Refactor] Build the Qwen4 experimental decoders from stage boundaries ( #42478 )
2026-10-04 16:38:36 -07:00
Cheng Wan
92d60351e2
[Refactor] Build the Hunyuan V4 decoder from stage boundaries ( #42477 )
2026-10-04 16:38:03 -07:00
Lianmin Zheng and jiayisuse
f5ccc4d7d9
[Kernel] Use explicit reciprocal scaling for recurrent Q/K normalization ( #42486 )
...
Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com >
2026-10-04 15:30:03 -07:00
Tarang Khanna
d475b5a25c
[weight_cache] Coordinate daemon readiness across ranks ( #38828 )
2026-10-04 15:20:13 -07:00
Aditya Sharma
caf96324fc
[Apple Silicon] Add an Apple Silicon (MPS) platform to the standard Torch model runner ( #36780 )
2026-10-04 15:03:40 -07:00
SuperSong and Cheng Wan
bab04cd791
fix(unified-memory): propagate unified-memory lazy checkpoint policy and handle unallocated session slots ( #42072 )
...
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-04 12:51:22 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
fea3c088a1
[CI] Temporarily disable GB300 ( #42463 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-04 12:24:04 -07:00
Zhang, Jiejing and kyle-256
aa0ce20201
[ROCm] GLM-5.2 decode path: decode-shaped MoE/MLA tiles, split speculative softmax, and bf16 GEMM routing ( #41725 )
...
Co-authored-by: kyle-256 <Kyle.Zhao@amd.com >
2026-10-04 11:19:37 -07:00
Xiaoyu Zhang
7a719e9a65
[AMD] Fix RoPE cache dtype and diffusion CI failures ( #42494 )
2026-10-05 00:36:26 +08:00
billishyahao
0343bc1483
[AMD][Docs] fix mori io and umbp playground ( #42512 )
2026-10-04 09:28:11 -07:00
Shuwen Wang
137c084000
[HiCache] refactor: retire in-flight storage prefetches through one helper ( #41453 )
2026-10-05 00:13:07 +08:00
WenhaoZhang
affa261e3d
[diffusion] feat: enable cuda graphs for observation encoding and denoising steps for action models ( #42171 )
2026-10-04 22:22:08 +08:00
WenhaoZhang
279d3f3888
[diffusion] feat: load ComfyUI/ai-toolkit fused gate_up LoRAs and diffusers metadata alpha for qwen-image-2.1 ( #42174 )
2026-10-04 22:21:13 +08:00
WenhaoZhang
44fac689f5
[diffusion] optimization: decode json pixel lists to numpy at the entrypoint ( #41895 )
2026-10-04 22:20:20 +08:00
Michael
ab9d8f029c
[AMD][DI][CI] mi355x spur: skip RDMA ports that are not PORT_ACTIVE; dump driver log on failure ( #41405 )
2026-10-04 21:23:54 +08:00
Shuwen Wang
35f3c96ff4
[CI] Fix bootstrap sender registration test on main ( #42487 )
2026-10-04 18:45:31 +08:00
Shangming Cai and Claude Opus 5
148da8e77d
[PD] Validate decode state layout once at registration ( #42051 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-10-04 18:32:30 +08:00
Xiaoyu Zhang
bc5a055dd7
[diffusion] move kernel validation into launchers and simplify dispatch ( #42391 )
2026-10-04 18:23:29 +08:00
Michael
29bcb74a89
[AMD][CI] Fix ROCm kernel wheel sources: drop eagle_utils, add DSV4 kernels ( #42461 )
2026-10-04 17:54:28 +08:00
SuperSong and Cheng Wan
6cec8f98ce
[Bugfix] Fix unified-memory compaction gates and pending page reuse ( #39982 )
...
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-04 02:48:08 -07:00
SuperSong and Cheng Wan
1d02a36bb7
fix(inkling): translate unified-memory checkpoint destinations to physical slots ( #38229 )
...
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-04 02:45:45 -07:00
WenhaoZhang and Claude Opus 5.5
1e490772e5
[diffusion] fix: pin the JoyAI-Echo overlay's source revision ( #42471 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-04 17:19:09 +08:00
ba68306256
[diffusion] feat: let requests choose the video libx264 preset ( #41797 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-04 16:22:36 +08:00
Tokha233 and WenhaoZhang
fbce0d9478
[diffusion] fix: load serialized h3 INT8 in ComfyUI integrated mode ( #42121 )
...
Signed-off-by: Tokha233 <61346912+Tokha233@users.noreply.github.com >
Co-authored-by: WenhaoZhang <42087078+niehen6174@users.noreply.github.com >
2026-10-04 16:08:56 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
0bdd0cc8dc
[Deps] Upgrade the CUDA PyTorch stack to 2.14 ( #38641 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-04 00:48:02 -07:00
Kotthagattu Meher Sai
9631f6f2d0
[XPU] Enable fused QK-norm + RoPE for Qwen3-MoE ( #40190 )
2026-10-04 15:45:18 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
7eb628612c
[Fix] Publish the parallel config the CLIP attention test reads ( #42460 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-04 00:44:51 -07:00
Baizhou Zhang
55c5415c17
[CP] Support scattered interleave CP inputs with expert parallelism ( #42041 )
2026-10-04 00:43:31 -07:00
937c0a6cfe
Fix unified HiCache physical transfers ( #39479 )
...
Co-authored-by: yhzhuang <yhzhuang@fb.com >
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com >
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
2026-10-04 00:38:56 -07:00
billishyahao
6cc661f223
[AMD][DCP 2/N] Enable aiter asm ps for kimi k3 prefill ( #34438 )
2026-10-03 23:34:45 -07:00
Liangsheng Yin
048b2a07f3
[Session] Release aborted streaming turns through the normal release path ( #42330 )
2026-10-03 22:30:25 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
1315d171a0
[Diffusion][CI] Add missing B200 Cirrascale runner E2E baselines ( #42329 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-03 21:10:32 -07:00
Brayden Zhong and Mohammad Angkad
a3275295c4
[MoE] Use FlashInfer A2A for prefill instead of AG+RS ( #39818 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-03 21:09:00 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
c12bc635ca
[CI] Install only userspace libgdrapi for GDRCopy ( #42233 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-03 21:05:35 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
0e6d7eba5f
[Fix] Count attention-DP ranks in one-batch bench running cap ( #42331 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-03 21:04:27 -07:00
Baizhou Zhang
fccc61c1c5
[CP] Support DP x TP x CP for attention with interleave CP strategy ( #42038 )
2026-10-03 20:57:36 -07:00
Aurick Qiao
f24db66729
[PD] Publish prefill DP rank whenever forced lookup is enabled ( #41786 )
2026-10-04 11:46:20 +08:00
metamergebot and metamergebot
169e6223ef
[PD] Drop the removed MooncakeKVSender args from test_prefill_complete_integration ( #42454 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
2026-10-03 20:29:06 -07:00
4611ea365e
[HiCache]: Fix host lock ownership across radix splits ( #38480 )
...
Signed-off-by: Lukas Humbel <lhumbel@nvidia.com >
Co-authored-by: Jialin Ouyang <jialino@meta.com >
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com >
2026-10-04 11:16:04 +08:00
ff219e159e
[AMD] MiniMax-M3: fuse sparse QK norm, RoPE and cache writes with AITER ( #35357 )
...
Co-authored-by: zijiec <300606707+zijiecode@users.noreply.github.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
2026-10-03 19:49:53 -07:00
sglang-bot and sglang-bot
8afef58af4
docs: sync LMSYS SGLang blog cards ( #41296 )
...
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com >
2026-10-04 02:26:05 +00:00
5606fb8592
[Fix] HiCache: carry registered Mamba slot side states (Qwen4-Exp PLE) through the host tier ( #39862 )
...
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com >
Co-authored-by: alphabetc1 <2508695655@qq.com >
Co-authored-by: ispobock <ispobaoke@gmail.com >
2026-10-04 10:15:59 +08:00
Michael
f2385314b2
[AMD] Add DeepSeek-V4.1-Flash MI35x nightly accuracy test ( #41476 )
2026-10-03 18:16:44 -07:00
Xinyi Song and kk
663984b5a1
[AMD] Keep aiter's tuned GEMM for the DeepSeek-V4 compressors ( #41931 )
...
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com >
2026-10-03 18:11:37 -07:00
trebladev
295a53c90e
[diffusion] optimization: fuse Flux3 rowwise FP8 quantization with triton ( #41671 )
2026-10-04 09:01:17 +08:00
71dc292b97
[AMD] Use AITER ASM prefill for MiniMax-M3 HD128 attention ( #41707 )
...
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
2026-10-03 17:54:49 -07:00
4ab720e655
[HiCache] Fix cgroup page-cache accounting and the sizing fallback when cgroup discovery fails ( #42420 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
Co-authored-by: nvjullin <nvjullin@users.noreply.github.com >
2026-10-03 16:01:32 -07:00
metamergebot and merrymercy
4301be7bdb
Pick the default nccl_port below the kernel ephemeral range ( #42405 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
2026-10-03 16:00:53 -07:00
Cheng Wan
10edb05e72
[Refactor] Let the remaining placement consumers read the parallel context ( #42348 )
2026-10-03 14:59:32 -07:00
07064fd4a4
[sgl-router] Credit routed prompts before their KV events arrive ( #42276 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
2026-10-03 14:57:49 -07:00
Cheng Wan
b2aa993cb8
[Fix] Skip a draft's decode recapture when it owns no decode graph ( #42347 )
2026-10-03 14:54:12 -07:00
Cheng Wan
1a18de4b29
[Fix] Load a draft's tensor weight update from its deployment TP rank ( #42346 )
2026-10-03 14:52:25 -07:00
sglang-bot and sglang-bot
061e712bab
chore: bump sgl-kernel version to 0.4.9 ( #42427 )
...
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com >
2026-10-03 14:29:04 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
3122b9c995
Update AOT kernels for Torch 2.14 ( #42366 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-03 14:26:34 -07:00
Liangsheng Yin
df53c978f1
[mem_cache] Run mamba models on UnifiedRadixCache when the radix cache is disabled ( #42354 )
2026-10-03 14:11:13 -07:00
Liangsheng Yin
3e9a120add
[Fix] Make Hf3fsMockClient reads and writes thread-safe with pread/pwrite ( #42425 )
2026-10-03 14:02:24 -07:00
8bf780dda5
feat(metrics): expose deferred decode KV release metrics ( #41128 )
...
Co-authored-by: Shangming Cai <csmthu@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-04 04:52:21 +08:00
Yanbin Jiang
749a89abd6
[LoRA] Reorganize kernels and add CODEOWNERS ( #42299 )
2026-10-03 13:43:40 -07:00
fd5e68f99e
[sgl-router] Repair KV-event sequence gaps from the engine's replay socket ( #42275 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
2026-10-04 04:43:17 +08:00
aaf7ebe077
Advertise the KV-event replay endpoint in /server_info ( #42274 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-04 04:42:33 +08:00
AMD-yanfeiwang
3dc7f4b816
fix(cuda-graph): remove unnecessary CP batch alignment ( #40543 )
2026-10-03 12:14:28 -07:00
AMD-yanfeiwang
e83c95b650
[AMD] Fix AITER weight slicing for CP decode TP ( #39950 )
2026-10-03 12:09:08 -07:00
AMD-yanfeiwang
f327f92424
[AMD] Port CP V2 to the DeepSeek-V4 HIP backend ( #34200 )
2026-10-03 12:08:17 -07:00
6acf2f4736
[PD] Keep ingesting requests while a prefill forward result is pending ( #42035 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: Jialin <Jialin@users.noreply.github.com >
2026-10-03 12:02:21 -07:00
b016ca406f
Support post-capture KV sizing for the unified hybrid-SWA pool ( #41961 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com >
2026-10-03 09:54:20 -07:00
2bae12b9b3
[HiCache] Fix write-back SWA insert backups tripping the write-through pending-ack assert ( #42264 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: 842974287 <842974287@users.noreply.github.com >
2026-10-03 09:45:56 -07:00
5b5d721239
[PD] Add opt-in prefill-complete decode KV allocation ( #40703 )
...
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
Co-authored-by: Po-Han Huang <pohanh@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-03 21:03:09 +08:00
5916999afb
[DSv4.1] Fused c1/c2 compress for eager extend, faster c2 decode ( #41660 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: BBuf <1182563586@qq.com >
2026-10-03 19:31:05 +08:00
d75d5b33f7
[DSv4.1] Faster fp4 index-K gather and combine_topk_swa_indices ( #41658 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: BBuf <1182563586@qq.com >
2026-10-03 18:25:22 +08:00
DarkSharpness and Claude Opus 5
379ec90fb2
[DSv4.1] Fold q_rope_store into fused_q_norm_rope ( #41657 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-03 18:24:41 +08:00
Cheng Wan
7be5e3473c
[Refactor] Drop the reduction-skip mechanisms stage boundaries no longer use ( #42312 )
2026-10-03 02:21:55 -07:00
Cheng Wan
71b04e02cf
[Refactor] Build the ZAYA1, IQuest-Q1 and Gemma 4 decoders from stage boundaries ( #42311 )
2026-10-03 02:21:34 -07:00
Cheng Wan
f85c2c4923
[Fix] GigaChat 3.5: apply the sandwich norms to the complete sums ( #42310 )
2026-10-03 02:21:10 -07:00
Cheng Wan
6faccf3e84
[Refactor] Build the GLM-4, GLM-Image and Granite MoE hybrid decoders from stage boundaries ( #42309 )
2026-10-03 02:20:40 -07:00
Cheng Wan
e4554fd5e5
[Refactor] Build the Llama and Nemotron-NAS decoders from stage boundaries ( #42308 )
2026-10-03 02:20:18 -07:00
Cheng Wan
740a4d5955
[Refactor] Build the Qwen2 decoders from stage boundaries ( #42307 )
2026-10-03 02:19:56 -07:00
Cheng Wan
2c739b225b
[Fix] Jet-Nemotron build and Granite MoE hybrid final norm ( #42306 )
2026-10-03 02:19:33 -07:00
Cheng Wan
b7fc516986
[Fix] EXAONE MoE under DP attention and DeepEP ( #42305 )
2026-10-03 02:19:19 -07:00
Cheng Wan
2fa17a6467
[Refactor] Build the ERNIE 4.5 VL MoE and EXAONE MoE decoders from stage boundaries ( #42304 )
2026-10-03 02:18:56 -07:00
Cheng Wan
f89ead192f
[Fix] EXAONE and ERNIE 4.5 VL MoE architecture, backend and PP issues ( #42303 )
2026-10-03 02:18:33 -07:00
Cheng Wan
be20e7453c
[Fix] Step-3.5 DeepEP routed scaling and Sarvam shared expert under dense TP1 ( #42302 )
2026-10-03 02:18:11 -07:00
Cheng Wan
b157d548bb
[Refactor] Let stage boundaries complete every stage-output sum ( #42301 )
2026-10-03 02:17:48 -07:00
Cheng Wan
d5a8d76bf5
[Refactor] Drop TBO op methods that no strategy ever schedules ( #42300 )
2026-10-03 02:17:26 -07:00
Liangsheng Yin
122b59152c
[mem_cache] Replace is_chunk_cache / is_tree_cache with supports_prefix_sharing ( #42362 )
2026-10-03 02:07:55 -07:00
Banruo Liu
8293f9af54
[PD] Centralize drain-aware abort acknowledgements ( #37077 )
2026-10-03 16:51:11 +08:00
saatwiknagpal
1093c501df
[diffusion] optimization: route subBlock sparse attention in head chunks ( #41985 )
2026-10-03 15:54:54 +08:00
Yanbin Jiang
254aa41f78
[Fix] Guard DeepSeek NVFP4 shared-expert fusion for LoRA and FP4 backends ( #42203 )
2026-10-03 00:37:57 -07:00
Tokha233
ce617d410b
[diffusion] fix: fix native FP8 format handling for FLUX 3 row-wise linears ( #42255 )
2026-10-03 15:23:41 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
00bcc25f6c
[HiCache] Fix DeepSeek-V4 storage backend crash from missing storage_format_tag ( #42215 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-02 22:27:57 -07:00
Jimmy Shong
0072e2cc08
[Fix] Validate per-layer rope_parameters on transformers 5.17 (Laguna-S/XS-2.1 nightly) ( #42214 )
2026-10-02 21:49:45 -07:00
Jacob0226 and Thomas Wang
af1bef3eaf
[AMD] GLM-5.3-Flash: fuse shared expert and KDA projections on Quark MXFP4 ( #41870 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
2026-10-02 20:47:47 -07:00
Richard Wang and Richard Wang
482b062d28
[Feature] serve pplx-decider decision checkpoints on /v1/systemone ( #42183 )
...
Co-authored-by: Richard Wang <11150595+rwang5203@users.noreply.github.com >
2026-10-03 10:18:53 +08:00
WenhaoZhang and Xiaoyu Zhang
41c6ffea89
[diffusion] fix: keep residual_gate_add on the JIT CUDA path for cont… ( #42240 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-10-03 10:12:11 +08:00
Xiaoyu Zhang
391e665dab
[Perf] Optimize DeepSeek V4.1 Flash Hopper paths and Blackwell prefill selection ( #41251 )
2026-10-03 10:04:48 +08:00
Mick and Mick Qian
65c7425a56
[diffusion] fix: respect explicit residency over pipeline preload hints ( #41622 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-03 09:54:21 +08:00
Liangsheng Yin
a8b54f138e
[Session] Run streaming sessions only on UnifiedRadixCache; reject unverified tree caches ( #42295 )
2026-10-02 18:15:59 -07:00
8ff02f5820
[QSA] Enable breakable prefill CUDA graphs for text-only Qwen3.8 Flash-Next (capture-safe metadata, MTP side-channel padding) ( #41729 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: Qiaolin Yu <liin1211@outlook.com >
2026-10-02 17:57:42 -07:00
Kevin Mi and Claude Opus 5
65f759144d
[HiCache] Let the MiniMax-M3 K-only host pool join a host pool group ( #42166 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-02 17:00:13 -07:00
Baizhou Zhang
3bde8eb2d8
[CP 1/5] Remove the CP adapter and separate interleave transport from boundary reduction ( #42036 )
2026-10-02 16:50:20 -07:00
Zhang, Jiejing
cbe070b6bf
[Docs][AMD] Update GLM-5.2 MI355X daily image to 20260930 ( #42278 )
2026-10-02 15:39:07 -07:00
b69a5f296a
[AMD] Add opt-in MiniMax-M3 TP4 indexer context partitioning ( #41488 )
...
Co-authored-by: ThomasNing <thomas.ning@amd.com >
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
2026-10-02 15:34:29 -07:00
fa7784d140
[unified-memory] Copy page envelopes in place during compaction ( #42252 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com >
2026-10-02 15:12:14 -07:00
e8fab85a02
[KDA] Add an opt-in decode-parity mode to the Triton multi-token recurrence ( #42251 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com >
2026-10-02 14:59:56 -07:00
19794f5743
[sgl-router] Add SGLang's /v1/classify ( #42258 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-03 05:56:16 +08:00
Liangsheng Yin
8a328e867b
[mem_cache] Rename the tree's request-level insert_req to checkpoint ( #42202 )
2026-10-02 14:42:35 -07:00
2ec9e3b11d
[eplb] Warm default-group NCCL P2P transports before KV-cache sizing ( #42026 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: luccafong <luccafong@users.noreply.github.com >
2026-10-02 14:34:55 -07:00
ebf161cea4
[sgl-router] Add SGLang's /v1/rerank ( #42205 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-03 05:33:40 +08:00
Kan Wu and Kan Wu
8c9030ce3f
[sgl-router] Unify reorg session and cache affinity modes ( #41990 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
2026-10-03 05:21:22 +08:00
Yuhao Wang
0098b9d29d
[Spec] Fix Qwen3.5 text model EAGLE3/DFLASH aux-layer capture ( #42002 )
2026-10-02 14:05:01 -07:00
raghotham
a64725d407
[HiCache] Fall back to host memory when no cgroup fs is mounted ( #42120 )
2026-10-02 13:59:40 -07:00
7dd93b01a5
[PD] Make the prebuilt last-token H2D copy non-blocking under overlap ( #42113 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: bbus <bbus@users.noreply.github.com >
2026-10-02 13:48:39 -07:00
Liangsheng Yin
07e1a4f082
[mem_cache] Skip the release-time insert on optimistic prefill requeue; make refresh_fill_ids public ( #42194 )
2026-10-02 13:46:32 -07:00
354ed2a7b6
Size the VMM graph-input exchange by the widest input across ranks ( #42125 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: oulgen <oulgen@users.noreply.github.com >
2026-10-02 13:45:32 -07:00
4a4dad1cf1
[Metrics] Label scheduler stage wall time by sampled forward overlap ( #42025 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: Jialin <Jialin@users.noreply.github.com >
2026-10-02 13:43:53 -07:00
ChangLiu0709 and Nikolai Protasov
f6fcda8827
[AMD][ROCm] Keep cos_sin_cache fp32 on HIP for fused QSA indexer kernel ( #41282 )
...
Co-authored-by: Nikolai Protasov <289648307+amd-nprotaso@users.noreply.github.com >
2026-10-02 11:57:03 -07:00
Shuwen Wang
89f21671bb
fix: preserve QSA indexer state through HiCache ( #39893 )
2026-10-02 23:49:38 +08:00
kk and wunhuang
a74f259f38
[AMD][V4.1][*/N] Fuse MXFP8 activation quant into producer kernels on gfx950 ( #42055 )
...
Co-authored-by: wunhuang <wunhuang@amd.com >
2026-10-02 08:34:44 -07:00
Alex O. P. and Po-Han Huang
e667ab10d5
[diffusion] quantization: ConvRot INT8 online W8A8 for DiTs (sgl-kernel + convrot_int8_customkernel) ( #38040 )
...
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-02 23:26:28 +08:00
Xiaoyu Zhang
1a2eb6c426
[Kernel] Use Cake softmax for qualified SM103 sampling workloads ( #42004 )
2026-10-02 23:06:43 +08:00
bb9a820f09
[diffusion] chore: make perf dump one writer per replica with no first-request NCCL setup ( #41711 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 21:44:10 +08:00
Michael
2cd92d10b9
[AMD] Skip the AITER #6042/#5967 patches on gfx1250 ( #42219 )
2026-10-02 21:10:01 +08:00
edee4308bc
[sgl-router] Add OpenAI /v1/embeddings ( #42191 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 20:18:24 +08:00
1330e437e1
[sgl-router] Add native /generate endpoint ( #42147 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 20:09:04 +08:00
Michael
e020b389b6
[AMD][CI] Partition stage-b-test-1-gpu-small-amd-mi35x to stop the 30-min timeout ( #41973 )
2026-10-02 19:29:20 +08:00
Mick and Mick Qian
0f71ec8656
[diffusion] UX: deduplicate SM120 FP8 fallback warnings across layers ( #41542 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-02 19:25:15 +08:00
Michael
544e46c080
[AMD][CI] Fix VLM MMMU nightly max_tokens for CoT prompt ( #41979 )
2026-10-02 18:59:05 +08:00
YAMY and Po-Han Huang
895525f062
[Mamba] Warm cache COW kernel before serving ( #36266 )
...
Signed-off-by: Yangmin Li <yangminl@nvidia.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-02 18:52:37 +08:00
Alex
1db33bc367
[PD][NIXL] Fix prepared dlists for KV entries of different lengths (DeepSeek-V4) ( #42005 )
2026-10-02 17:59:31 +08:00
Po-Han Huang (NVIDIA) and Claude Opus 5.5
41bd213c5f
[Test] Add get_swa_key_page_size to the Q8KV8 sparse-prefill fake KV pool ( #42211 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 16:00:15 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
d954c88a4f
[Fix] Update stale source-patch match in dumper comparator e2e test ( #42179 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-02 00:37:37 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
e7d9c0299b
[Docs] Keep the last CUDA 12 image tag fixed during install version bumps ( #42201 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-02 00:17:13 -07:00
sglang-bot and sglang-bot
d61a9c28ba
chore: bump docs install version to 0.5.21 ( #42013 )
...
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com >
2026-10-01 23:16:26 -07:00
Liangsheng Yin
50be533d09
[mem_cache] Rename cache_unfinished_req to checkpoint_req and count cache hits only when a request finishes ( #41520 )
2026-10-01 21:41:45 -07:00
Mingyi and Mick Qian
b906cd3f45
doc: refresh README ( #41967 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-02 12:11:33 +08:00
saatwiknagpal
bf8adf9602
[diffusion] fix: fix int32 offset overflow in SubBlock router kernels ( #41965 )
2026-10-02 11:22:53 +08:00
Khoa Pham
6be84e1b72
[Fix] Select the Marlin MoE runner for MiMo-V2 packed MXFP4 experts on SM90 ( #42172 )
2026-10-01 20:01:03 -07:00
Yuwei An
ef867fa40d
[Fix][DSV4.1] SWA page size with bounded replay ( #42128 )
2026-10-01 18:50:43 -07:00
jiaryang and Thomas Wang
58f0d250ec
[ROCm] Use the fused MLA absorb + RoPE + KV-write kernel for decode-sized forward modes only ( #41533 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
2026-10-01 18:45:29 -07:00
kk and wunhuang
29f6d408c0
[AMD][V4.1][*/N] Fix shared-expert fusion accuracy and speed up MoE routing on ROCm ( #42011 )
...
Co-authored-by: wunhuang <wunhuang@amd.com >
2026-10-01 18:19:06 -07:00
Kedar Potdar and Claude Opus 5.5
8acd36b56b
[Docs] GLM-5.2 GB300 NVFP4: add env vars from InferenceX AgentX recipe ( #42109 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-02 09:11:29 +08:00
Qiaolin Yu
67eab57057
[qwen 3.8 next] Fuse NEXTN verify and draft graph input preparation ( #41175 )
2026-10-01 17:17:34 -07:00
caihuali95 and Cheng Wan
2a3655fea1
[unified-memory] Name the fused KV translate for what it computes (7/7) ( #40331 )
...
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-01 17:14:03 -07:00
df530479c6
[unified-memory] Stride the KV translate kernel and route every translate through it (6/7) ( #40330 )
...
Co-authored-by: Caihua Li <caihua.li@bytedance.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-01 17:12:14 -07:00
9855e9c6f8
[unified-memory] Remove the kernel-page multiplier plumbing (5/7) ( #40329 )
...
Co-authored-by: Caihua Li <caihua.li@bytedance.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-01 17:11:25 -07:00
4c4e352a4b
[unified-memory] Mark the write loc physical and check it at every write door (4/7) ( #40328 )
...
Co-authored-by: Caihua Li <caihua.li@bytedance.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-01 17:10:37 -07:00
cdadb1c79a
[unified-memory] Token-major dense views for the unified memory pool (3/7) ( #38592 )
...
Co-authored-by: Caihua Li <caihua.li@bytedance.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-01 17:09:38 -07:00
caihuali95 and Cheng Wan
5df52e6382
[unified-memory] Build paged KV views through one helper (2/7) ( #40327 )
...
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-01 17:08:42 -07:00
caihuali95 and Cheng Wan
d3dbb71e5a
[unified-memory] Derive KV row addresses from strides, not shapes (1/7) ( #40326 )
...
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-10-01 17:07:47 -07:00
shyeh25 and Brayden Zhong
5423a4d885
[MegaMoE] Preserve W13 layout for ModelOpt NVFP4 experts ( #39388 )
...
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca >
2026-10-01 16:12:34 -07:00
f17f7705a5
[AMD] [GLM-5.3-Flash] Enable FP8 and MXFP4 serving on gfx950 ( #39273 )
...
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai >
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com >
Co-authored-by: long10024070 <long10024070@users.noreply.github.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: sunxxuns <126995791+sunxxuns@users.noreply.github.com >
Co-authored-by: Long Luong <long.luong2+personal@onemount.com >
2026-10-01 15:59:44 -07:00
Jan Bernlöhr and jbernloehr
80bb3fb651
[Fix] Select the MXFP4 MoE runner for MiMo-V2 packed experts on SM100 ( #41668 )
...
Co-authored-by: jbernloehr <janbernloehr@users.noreply.github.com >
2026-10-01 15:33:30 -07:00
Kan Wu and Kan Wu
a21dcbd320
[sgl-router] Support engines started with --api-key (--worker-api-key) ( #41996 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
2026-10-01 14:54:59 -07:00
Kan Wu and Kan Wu
d7565215ec
[sgl-router] Route to a specific DP rank inside multi-rank workers (--dp-aware) ( #41974 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
2026-10-01 14:50:59 -07:00
Martin Hua
52d5faed5b
[Bugfix] Llama4 local attention: read page ids from the graph tables in CUDA-graph capture/replay (page_size > 1) ( #38187 )
2026-10-01 14:42:22 -07:00
bf2d686a82
Add triton autotune on the Mamba2 SSD kernels ( #39130 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-10-01 14:25:37 -07:00
Jiajun Li and Claude Fable 5.1
dcf4e18283
fix(nccl): disable graph buffer registration for pausable graph pools ( #40648 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-01 12:34:23 -07:00
Shangming Cai and Claude Fable 5
a71c7d19e7
[Router] Document peer bootstrap: flags, RBAC, downward API, probe implications ( #42049 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-02 02:20:36 +08:00
3f4bfda13c
[Bugfix] Fix DeepSeek V4 Pro TP24 vocabulary-padding failure on Hopper ( #31801 )
...
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
Co-authored-by: Po-Han Huang <pohanh@nvidia.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-01 11:11:18 -07:00
a0fcebb3bb
NIXL: Use stride desc API ( #37827 )
...
Signed-off-by: Ilia Yastrebov <iyastrebov@nvidia.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-10-01 10:53:00 -07:00
jain-ria
fb92ba1127
[Rust] Wire the gRPC server into the frontend lifecycle ( #41964 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-10-01 10:44:56 -07:00
Joe and Alex Nails
f03a183719
Fix Kimi-K3 MLA output gate dispatch on non-CUDA devices ( #36406 )
...
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
2026-10-01 08:53:31 -07:00
Joe and Alex Nails
3031091c36
[Kimi-K3] Guard optimized paths by platform ( #40269 )
...
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
2026-10-01 08:53:05 -07:00
Артем Савкин
1bbce57d9e
[NPU] [Diffusion] Fix NPU multimodal-gen CI ( #41528 )
2026-10-01 18:02:37 +03:00
41cbe65de0
[sgl-router] Enforce compatible PD version groups ( #41613 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-01 20:24:20 +08:00
3b2ad1c6ae
[sgl-router] Fail fast and cancel decode when prefill fails ( #41612 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-01 19:35:56 +08:00
Aurick Qiao and Aurick Qiao
1a0250c5b6
[Disagg] Validate state strides before Mooncake transfers ( #41607 )
...
Co-authored-by: Aurick Qiao <6137920+aurickq@users.noreply.github.com >
2026-10-01 18:47:41 +08:00
d669efba84
[AMD][V4.1][*/N] Greedy dspark draft/accept under SGLANG_SIMULATE_ACC_LEN ( #41981 )
...
Co-authored-by: wunhuang <wunhuang@amd.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-10-01 03:35:28 -07:00
46637965ab
[sgl-router] Allow load-only routing without a tokenizer ( #41611 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-01 18:30:35 +08:00
3c4e218779
[AMD][V4.1][*/N] OPUS sparse prefill on gfx950 through layout conversion ( #42017 )
...
Co-authored-by: HAI <hixiao@gmail.com >
Co-authored-by: thomawan <thomawan@amd.com >
2026-10-01 03:21:25 -07:00
kk and wunhuang
f45c004e18
[AMD][V4.1][*/N] Build DSpark draft metadata inside the CUDA graph on ROCm ( #42014 )
...
Co-authored-by: wunhuang <wunhuang@amd.com >
2026-10-01 02:17:13 -07:00
黄孝君
7be0e85658
[CI][NPU][Diffusion] Bump ascend consistency GT to the CANN 9.1.0 baseline ( #41500 )
2026-10-01 17:13:28 +08:00
Xinyi Song
73ba6513f4
[AMD][V4.1][*/N] Switch the fp8 dense GEMMs on gfx950 to aiter's MXFP8 GEMM ( #41970 )
2026-10-01 02:04:39 -07:00
Xinyi Song
0224fcbd1c
[AMD][V4.1][*/N] Route low-ratio indexer and candidate-block top-k through top-k v2 ( #41947 )
2026-10-01 01:56:31 -07:00
Cheng Wan
5fab1f0f51
[Fix] Let the aiter DCP ASM decode test run without aiter ( #42018 )
2026-10-01 16:55:15 +08:00
Thomas Wang
69240ba7d6
[AMD][V4.1][*/N] Pick the gfx950 wo_a batched gemm tile by row count ( #41971 )
2026-10-01 01:52:39 -07:00
Thomas Wang
ffb134188b
[AMD] Update aiter version for v41 ( #42023 )
2026-10-01 01:49:45 -07:00
Shunkangz
266d9d1fa7
[Qwen3.8 CP 1/4] Context parallelism for QSA attention and the sparse indexer ( #39721 )
2026-10-01 01:28:55 -07:00
b51d4a04f0
[DFlash] Keep grouped-conv taps inside each request's block so NaN/Inf cannot leak across requests ( #41736 )
...
Co-authored-by: Rahul Chalamala <22563365+rchalamala@users.noreply.github.com >
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-01 00:18:37 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
42f2e4fcb7
[DSA] Stop forcing the per-step CPU seq_lens sync for the k-pool indexer ( #41987 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-01 00:16:28 -07:00
DarkSharpness
3ed6367d3f
[DSV4/DSA] Name the FlashMLA KV format and drop the V4.1 support probe ( #41337 )
2026-10-01 00:02:18 -07:00
d010e50ad6
[PPU][1/N] CI: Add backend registration and runner preflight ( #39788 )
...
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com >
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
2026-10-01 00:01:17 -07:00
amd-danli103
9c3262d855
[AMD][DSV4] feat: enable PD-disagg with fp8 unified_kv on gfx950 ( #39166 )
2026-09-30 23:42:31 -07:00
315080a0de
[sgl-router] Exclude portless prefills and retry bootstrap discovery ( #41610 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-01 14:32:14 +08:00
Shangming Cai and Claude Fable 5
c78fa09290
[Router] Drop the snapshot client's redirect-following fallback ( #42010 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-01 14:31:53 +08:00
Eric Zhang
947fee9384
[PD] Fix block scale registration for mixed full/SWA KV dtypes ( #41924 )
2026-10-01 14:25:04 +08:00
Xiaoyu Zhang and kernel-design-agents
77b64e353f
[KDA+Kimi K3] Speed up LTX-2.3 QK norm and split RoPE on H200 ( #41459 )
...
Co-authored-by: kernel-design-agents <334018530+kernel-design-agents@users.noreply.github.com >
2026-10-01 14:21:31 +08:00
Peng Wu
11500701f1
[DSV4] Reserve the FULL logical page in the c4 indexer pool like its KV pool ( #41163 )
2026-09-30 23:19:27 -07:00
Shangming Cai and Claude Fable 5
047be72a8c
[Router] Harden peer-bootstrap edges: refuse redirects and metric docs ( #41999 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-10-01 14:15:42 +08:00
chien-an-chen
99c9d65b32
[AMD] AITER MLA DCP decode ASM path (opt-in) ( #40750 )
2026-09-30 22:47:55 -07:00
billishyahao
dc7defd294
[Kimi-K3][DCP][DSpark] fix eager mode nonetype crash due to no extend_prefix_lens ( #38991 )
2026-09-30 22:35:25 -07:00
ashwini rathi
86807b87dd
[XPU] Fix test_qsa_indexer_xpu after defer_expansion change ( #41915 )
2026-09-30 22:34:10 -07:00
Ke Bao
5733bcaad3
Declare HiCache host pools for DSA indexer ( #40913 )
2026-10-01 13:08:26 +08:00
f02828ea39
fix(dsv4): ship the fp8 unified_kv rope pool through the direct external linker ( #40471 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: HaiShaw <hixiao@gmail.com >
Co-authored-by: billishyahao <bill.he@amd.com >
2026-09-30 21:46:56 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
40e1bb0f36
[Docs] Add NVIDIA NVFP4 checkpoint to GLM-5.3-Flash cookbook ( #41986 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-01 12:15:30 +08:00
3b6d37a56e
[Router] Gate readiness on peer bootstrap, and make it observable (13/13) ( #40699 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-01 11:55:46 +08:00
ebdeba2fee
[Fix] Emit Responses reasoning content-part events ( #41789 )
...
Co-authored-by: ai-jz <ai-jz@users.noreply.github.com >
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com >
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
2026-10-01 11:15:24 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
99c59a2ea6
Drop the GLM-5.3-Flash breakable prefill chunk-size pin ( #41837 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-30 19:26:58 -07:00
Shuwen Wang
bb62f125aa
[HiSparse] fix: return HiSparse slots on PD decode waiting abort and DeepSeek V4 free ( #41346 )
2026-10-01 10:24:51 +08:00
Shuwen Wang
c9d6c966d0
[HiSparse] fix: reject speculative decoding and mixed chunk at startup ( #41347 )
2026-10-01 10:24:15 +08:00
stanleys12 and Claude Opus 5
3fb721917d
[Fix] Require admin auth for /load_lora_adapter_from_tensors and /set_trace_level ( #41322 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-10-01 10:23:12 +08:00
Shuwen Wang
b62b21e7f4
[HiCache][PD] fix: keep decode load-back polling out of rank-local collectives ( #41393 )
2026-10-01 10:22:30 +08:00
zijie and Zijie Chen
0148d9a2fd
[AMD] Fix MiniMax-M3 EAGLE3 verification on ROCm ( #41497 )
...
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com >
2026-09-30 19:05:49 -07:00
Mick and Mick Qian
2ab1b537f8
[diffusion] docs: recommend docker for gpu deployments ( #41966 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-01 09:52:31 +08:00
36330b8c08
[Router] k8s e2e for cache-aware peer bootstrap (12/13) ( #40698 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-01 09:42:59 +08:00
Shafeeq Ibraheem and Yanbin Jiang
b4df214948
Fix LoRA usage-counter accounting across request lifecycle paths ( #31808 )
...
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com >
2026-09-30 18:37:53 -07:00
Sage and ishandhanani
baae0195d9
[ci] fix renderer image publish ( #41936 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-09-30 18:27:11 -07:00
jain-ria
ebccddce19
[Rust] Implement the Rust gRPC adapter for native generation ( #41766 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-09-30 18:17:42 -07:00
Yu-Yun Chang and kk
6af651ea0c
[AMD][Quark] Serve the Kimi-K3 MXFP4 checkpoint on ROCm ( #40811 )
...
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com >
2026-09-30 18:14:15 -07:00
leonardHONG and lianghougong
a702970ecb
[Perf] Use FA4 by default for MiMo on SM100 ( #41886 )
...
Co-authored-by: lianghougong <lianghouhong@xiaohongshu.com >
2026-10-01 09:05:12 +08:00
Sage
b37e6f79fc
[ci] grant ci permissions to sagearc ( #41935 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-30 17:21:41 -07:00
Liangsheng Yin
63321e3693
[Scheduler] Move internal-state readback and updates into a collaborator ( #41950 )
2026-09-30 16:40:57 -07:00
Liangsheng Yin
517cfa218c
[Test] Prune redundant scheduler unit tests ( #41952 )
2026-09-30 16:30:18 -07:00
Jan Bernlöhr and jbernloehr
488869c2d0
[Fix] Keep MiMo-V2 processor available without TorchCodec ( #41667 )
...
Co-authored-by: jbernloehr <janbernloehr@users.noreply.github.com >
2026-09-30 15:28:16 -07:00
826f15bf9c
[Fix] Preserve inline instructions for Responses and Kimi K3 Messages ( #40077 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-09-30 15:20:52 -07:00
Sage
8e4bcc5ca9
[rust-renderer] sglang-processor lib ( #41896 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-30 15:09:35 -07:00
f050ab194a
fix(spec): enable Qwen3.5 EAGLE3 capture and streaming overlap ( #41154 )
...
Co-authored-by: Yuhao Wang <59914901+jin-chuan@users.noreply.github.com >
Co-authored-by: Liangsheng Yin <95566987+hnyls2002@users.noreply.github.com >
2026-09-30 14:41:46 -07:00
Cheng Wan
a9c97c9f69
[Feature] Add --attn-dp-size and deprecate --enable-dp-attention ( #41818 )
2026-09-30 13:34:18 -07:00
Jason Mancuso and hnyls2002
201c4b9b2a
[RL, Spec] Introduce top-p mask capture for spec and add DFlash/DSpark impl ( #34201 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-09-30 13:32:26 -07:00
Cheng Wan
986cb8268b
[Refactor] Enter draft TP scopes by attention ownership and drop ModelRunner.tp_group ( #41817 )
2026-09-30 13:28:50 -07:00
Cheng Wan
e7f6a99313
[Refactor] Add make_pp_layers so models stop handing their PP position down ( #41816 )
2026-09-30 13:28:34 -07:00
Cheng Wan
00659ac099
[Refactor] Stop passing models' layers the placement they already read ( #41815 )
2026-09-30 13:28:17 -07:00
Cheng Wan
c7fa37a5ca
[Refactor] Let FusedMoE's weight-loading helpers read the layer's MoE-TP rank ( #41814 )
2026-09-30 13:27:58 -07:00
Cheng Wan
499e86db98
[Refactor] Keep only the TP and PP groups on the model runner ( #41813 )
2026-09-30 13:27:31 -07:00
Cheng Wan
a09aedf1ca
[Refactor] Drop model placement attributes and parameters nothing reads ( #41812 )
2026-09-30 13:27:11 -07:00
Cheng Wan
b0cc246ff8
[Refactor] Drop placement values nothing reads ( #41811 )
2026-09-30 13:26:51 -07:00
Cheng Wan
4f3ee94d25
[Fix] Make the Solar model constructible and runnable ( #41810 )
2026-09-30 13:26:32 -07:00
Cheng Wan
a659e6461f
[Fix] Size the prefill delayer's gather buffer to its TP group ( #41809 )
2026-09-30 13:26:12 -07:00
Cheng Wan
ecc0f0dec2
[Fix] Pass the draft's attention ownership to DFLASH's eager LiLiCorr scope ( #41808 )
2026-09-30 13:25:54 -07:00
Cheng Wan
e289989933
[Fix] Shard MoE WNA16 and Quark INT4-FP8 weights by the MoE placement ( #41807 )
2026-09-30 13:25:38 -07:00
Cheng Wan
3f20738846
[Fix] State the configured MoE-DP width in the weight-cache fingerprint ( #41806 )
2026-09-30 13:25:19 -07:00
Cheng Wan
e27b6e0971
[Fix] Read TensorCast's WORLD placement under its current names ( #41805 )
2026-09-30 13:25:03 -07:00
YAMY
4407a21f3c
[Spec][PP] Fix hybrid recurrent-state commit and micro-batch pairing under PP x speculative decoding ( #40001 )
2026-09-30 13:05:39 -07:00
547286bb5a
[Deps] Bump transformers to 5.17.0 ( #39012 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
2026-09-30 12:52:02 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
036a3b3e14
[Test] Lower the SM120 NVFP4 KV GSM8K threshold to 0.60 ( #41799 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-30 12:50:42 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
eae808903f
[Deps] Bump FlashInfer to 0.7.0.post1 ( #40709 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-30 12:42:19 -07:00
4dfc0ac131
[BugFix] Pass token-major Q/K tensors from Gemma-3 to RadixAttention ( #37984 )
...
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
Co-authored-by: Po-Han Huang <pohanh@nvidia.com >
2026-09-30 12:05:41 -07:00
Kevin Mi and Kevin Mi
3a398442bf
dsv4.1-amd: serve DeepSeek-V4.1 on gfx950 ( #41308 )
...
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai >
2026-09-30 11:47:14 -07:00
9f9f9e702a
[HiCache] Attribute buffer-mode storage hits against the joint device match ( #41758 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-09-30 11:46:30 -07:00
zijiexia
d26ec4690d
docs: remove unreliable DeepWiki badge ( #41851 )
2026-09-30 10:51:36 -07:00
Shuwen Wang and Shangming Cai
aec4b799e6
[PD][Mamba] fix: free the COW mamba slot of decode requests dropped before preallocation ( #41451 )
...
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-09-30 17:50:44 +00:00
jain-ria
21ed9fbf44
[Rust] Extract shared runtime.v1 protobuf bindings ( #41726 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-09-30 10:13:08 -07:00
Sohom chakraborty and Shuwen Wang
28c5e7f5cb
[Fix] Stop PD-decode queue_time from counting decode time before a retraction ( #41380 )
...
Signed-off-by: Sohom Chakraborty <sohomchakraborty.iitkgp@gmail.com >
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com >
2026-09-30 23:17:03 +08:00
bd66ce343e
[diffusion] nightly: measure every framework with one client end-to-end methodology ( #41689 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 19:52:20 +08:00
4b3f63ec70
[Router] Prove a bootstrapped replica answers like the one it copied (11/13) ( #40697 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-30 19:19:26 +08:00
4c930125c7
[Router] Ask the fleet when a graft's splice goes unwitnessed (10/13) ( #40696 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-30 18:59:21 +08:00
Cheng Wan
53225fadf4
[Fix] Stop the namespace census at the leaf a read names ( #41838 )
2026-09-30 03:27:51 -07:00
f8500474c9
[Router] Share one fleet-wide fetch across a discovery burst (9/13) ( #40695 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-30 18:07:51 +08:00
jacky.cheng
47dcde70f4
[Docs] Add MI355X FP8 agentic recipe to the Qwen3.5 cookbook ( #41849 )
2026-09-30 17:44:19 +08:00
Yuan Luo and luoyuan.luo
57c81258d4
[KDA] Enable the ptx_kda prefill backend on SM100 (B200 / GB200) ( #41445 )
...
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com >
2026-09-30 17:24:51 +08:00
Yuan Luo and luoyuan.luo
c0296186b7
[KDA] Fix ptx_kda prefill NaN without a gate lower bound and workspace growth ( #41572 )
...
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com >
2026-09-30 17:23:54 +08:00
paulzhang-tm and Ke Bao
8055ccd2cd
[Linear Attention] Expose GDN/KDA prefill hooks and auxiliary cache accounting ( #40227 )
...
Co-authored-by: Ke Bao <ispobaoke@gmail.com >
2026-09-30 17:19:02 +08:00
EdwardXuy and EdwardXuy
9b7464c8dd
ci: temporarily disable A5 (950) nightly job during power maintenance ( #41856 )
...
Co-authored-by: EdwardXuy <EdwardXuy@users.noreply.github.com >
2026-09-30 17:17:39 +08:00
jiaryang
e9b0d0c0a3
[AMD] Gate flashinfer and TRT-LLM DSA paths on CUDA ( #40911 )
2026-09-30 02:10:02 -07:00
Michael
d7cce54885
[AMD] Resolve QSA packed-varlen decode to aiter on HIP ( #41513 )
2026-09-30 02:01:15 -07:00
amote-i
964c45cf31
[NPU] [DOC]: add Kimi-K3 NPU PD disaggregation recipes ( #41717 )
2026-09-30 16:42:46 +08:00
4d4d3f2e5c
[Router] Sweep the fleet for a peer snapshot and hand it to the pump (8/13) ( #40694 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-30 16:34:12 +08:00
45c8ddddd3
[diffusion] feat: check the final MP4 in-process instead of spawning ffprobe ( #41825 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-30 16:26:39 +08:00
WenhaoZhang
0931a72eb2
[diffusion] feat: support MiniMax H3 RL ( #34365 )
2026-09-30 16:23:56 +08:00
Michael
b87a241a6f
[AMD] Add Qwen3.8-Flash-Next-FP8 nightly validation ( #36901 )
2026-09-30 00:54:36 -07:00
682f4ef32f
[ROCm] Select DSA indexer top-k wave size by arch (wave32 on gfx1250) ( #41687 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
2026-09-30 00:47:04 -07:00
Shuwen Wang
51cae5f303
[HiCache][PD] fix: clamp the decode restore to the prefix promised to prefill ( #41450 )
2026-09-30 15:26:15 +08:00
heziiop and sglang-npu-bot
ec0006673a
[NPU] Add decode context parallel support for dsa models ( #37787 )
...
Co-authored-by: sglang-npu-bot <sglangnpu@163.com >
2026-09-30 15:22:16 +08:00
Shangming Cai
8c75ad9e62
chore: bump mooncake version to 0.3.13.post1 ( #41678 )
2026-09-30 15:16:54 +08:00
Zhaoyi Li
3b537e96e2
[AMD][DI][CI] Leave the GLM-5.2 MTP decode room to load its Triton kernels ( #41135 )
2026-09-30 00:02:42 -07:00
Zhaoyi Li
25de118ae3
[AMD][DI][CI] Give the dsv4pro-fp4 MTP decode more memory headroom ( #41238 )
2026-09-29 23:57:25 -07:00
Zhaoyi Li
4207f1ae23
[AMD][DI][CI] Record the commit the MI355X nightly tested ( #41099 )
2026-09-29 23:55:38 -07:00
Zhaoyi Li
f85ca901e2
[AMD][DI][CI] Fix four failures in the MI355X disaggregation nightly ( #41025 )
2026-09-29 23:54:40 -07:00
24ee67d2fa
Readme refresh ( #41787 )
...
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com >
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai >
2026-09-29 23:48:38 -07:00
eb9c9ee99d
[Qwen4-Exp] Optional fp8 (e4m3) storage for the compressed QSA indexer cache ( #39614 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
Co-authored-by: YAMY1234 <74099316+YAMY1234@users.noreply.github.com >
2026-09-29 22:11:46 -07:00
Qiaolin Yu
88f95f4b87
[qwen next] Fuse QSA KV preparation and sparse block expansion ( #40972 )
2026-09-29 21:54:34 -07:00
Qiaolin Yu
b660100c00
[qwen 3.8 next] Fuse QSA graph replay metadata across draft steps ( #41173 )
2026-09-29 21:52:28 -07:00
JohnQinAMD and Claude Opus 5.5
964fd6af92
[ROCm] GLM-5.2: gfx950 four-kernel fused DSA indexer decode path ( #38583 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-29 21:52:00 -07:00
hhhh1252023
be66c1b460
[NPU][CI] Fix stale multimodal-gen job name in fast-fail health check ( #41824 )
2026-09-30 12:39:23 +08:00
Cheng Wan
d41904f05e
[Refactor] Build seven more decoders from stage boundaries ( #41779 )
2026-09-29 21:16:45 -07:00
Cheng Wan
113e6a1496
[Refactor] Simplify layer boundary internals ( #41778 )
2026-09-29 21:15:58 -07:00
Wenfei (Charles) Qi
7fb85a9d88
[MLX] Fix startup after logical token capacity change ( #41314 )
2026-09-29 21:15:56 -07:00
Cheng Wan
9ee065e49a
[Fix] Step-3.5: build the shared expert without TP under all-to-all MoE backends ( #41754 )
2026-09-29 21:11:37 -07:00
Cheng Wan
faa5064f6e
[Refactor] Send producer-written residual streams through to_pp ( #41753 )
2026-09-29 21:11:02 -07:00
Cheng Wan
843f50f5c1
[Fix] Reduce the Sarvam dense MLP through its row-parallel projection ( #41752 )
2026-09-29 21:10:17 -07:00
Cheng Wan
ea855f0daf
[Refactor] Leave decoder stacks through final_norm(skip_empty=True) ( #41751 )
2026-09-29 21:08:19 -07:00
Cheng Wan
01b5f572b7
[Refactor] Rename layer boundary protocols, contracts and helpers ( #41750 )
2026-09-29 21:05:48 -07:00
cb3ba2f2b7
[XPU] Add dynamic_expert_bias and track_state support ( #39618 )
...
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Singh <rohitsi2@iil-login.iind.intel.com >
Co-authored-by: Singh <rohitsi2@iil-gnrap01.iind.intel.com >
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com >
2026-09-30 12:05:35 +08:00
Alison Shao
8854857a99
[CI] Bound maintenance API requests to 30 seconds ( #41800 )
2026-09-29 20:02:02 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
7b30ff26ee
[CI] Evaluate the maintenance gate with github-script instead of jq ( #41625 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-29 18:47:51 -07:00
fc9bdc8a3e
[MoE] Keep the moe_align_block_size pad fill inside sorted_token_ids ( #41759 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: oulgen <oulgen@users.noreply.github.com >
2026-09-29 17:43:17 -07:00
Baizhou Zhang
e5cec303ae
[Fix] Restore deferred layer dumps and pin SentencePiece for InternVL ( #41777 )
2026-09-29 17:28:30 -07:00
sglang-bot
cae69be584
chore: bump sgl-kernel version to 0.4.8 ( #41784 )
2026-09-29 17:19:21 -07:00
DarkSharpness
57d4adeb83
[Kernel] Expose FlashMLA kv_format in sgl-kernel sparse decode ( #41336 )
2026-09-29 17:16:52 -07:00
Jason Mancuso
ba3e04c063
[CI] Grant CI permissions to four contributors ( #41775 )
2026-09-29 16:42:01 -07:00
Cheng Wan
8d2d874758
[Fix] Release consumed residual contributions ( #41749 )
2026-09-29 16:38:05 -07:00
Jiajun Li and Shi Dong
3d4953839c
[Fix] MoE: require TopK layer_id to ensure routed expert captures ( #40980 )
...
Co-authored-by: Shi Dong <hello@dongshi.me >
2026-09-29 16:29:27 -07:00
Jiajun Li
f0e4001930
[Fix] Allocate SWA KV pools inside the memory-saver region ( #40979 )
2026-09-29 16:29:02 -07:00
Yuwei An
d3f8a8f4f5
[Spec/Prefill Coordination] Dspark support ( #41623 )
2026-09-29 16:08:01 -07:00
Liangsheng Yin
211b1d9784
[Refactor] Share PD routing fields across request models ( #41756 )
2026-09-29 15:11:18 -07:00
37a47737c8
[Perf] Tune SM90 GDN recurrent verify launch for small batches ( #41486 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-09-29 15:03:33 -07:00
Mohammad Miadh Angkad and hnyls2002
4885b563c3
[Test] Demote PD test RDMA openability check to a warning ( #41681 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-09-29 14:52:31 -07:00
84523d6785
[Spec] Reuse K3 auxiliary outputs across decode CUDA graph sizes ( #40159 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-09-29 14:24:51 -07:00
jain-ria
46beacf13d
[grpc rust] Define semantic frontend contract ( #41709 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-09-29 13:40:04 -07:00
James Liu and Claude Opus 5.5
77a81996ce
[PD] Preserve bootstrap metadata in native Messages requests ( #39711 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 13:32:54 -07:00
3c4653b6a3
[Metrics] Fix PD latency histogram accounting ( #39706 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-09-29 13:12:09 -07:00
zijiexia and Claude Sonnet 5.5
f884231f5a
[Docs] Add IQuestLab card logo to cookbook landing page ( #41746 )
...
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com >
2026-09-29 12:47:21 -07:00
51ace48387
[Metrics] Add request-level TPOT histogram (sglang:request_time_per_output_token_seconds) ( #40275 )
...
Co-authored-by: Gilford Ting <gilford@modal.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-29 12:20:22 -07:00
ishandhanani
9a16e47437
chore: grant jain-ria CI permissions ( #41728 )
2026-09-29 11:56:51 -07:00
ashwini rathi
bf80732b92
Fix pip install: exclude multimodal_gen/.claude symlink from package data ( #41703 )
2026-09-29 11:35:11 -07:00
8f2a9638f7
[metrics] Fix non-streaming TTFT by flushing the first output through detokenization ( #38600 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-09-29 11:24:52 -07:00
Colin Z
875dd41e6f
[AMD] Fix Load and Inference of MLA models with Quark PTPC FP8 attention on ROCm ( #28734 )
2026-09-29 11:20:10 -07:00
Chunan Zeng
affc602f51
[CI] Extend DeepGEMM GB300 validation timeout to three hours ( #41720 )
2026-09-29 10:57:20 -07:00
ced5e9fe49
[Kimi] Enable GB300 TP4 and GB200/GB300 TP16 SP collectives ( #35330 )
...
Co-authored-by: Ajit Mistry <ajmistry@amistry.nvidia.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
2026-09-29 10:18:11 -07:00
CYJiang
fa090f7755
[MUSA] Fix fused MoE GEMV registration and torchada pin ( #41444 )
2026-09-29 09:51:08 -07:00
ashwini rathi
98fce73d5b
[XPU] publish nightly docker image with sgl-kernel-xpu built from main ( #41650 )
2026-09-29 20:09:53 +08:00
4a4e1ce014
[Router] Hold a booting rank's batches and graft a snapshot on the pump (7/13) ( #40693 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-29 20:07:37 +08:00
8d1643b21e
[Router] Vet a peer's snapshot before it may touch the tree (6/13) ( #40692 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-29 19:58:11 +08:00
8b3a4cad8b
[Router] Track a booting rank's bootstrap and fetch a peer's snapshot (5/13) ( #40691 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-29 19:06:13 +08:00
7bcfcf5aa0
[Router] Watch EndpointSlices for sibling router replicas (4/13) ( #40690 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-29 18:56:32 +08:00
c5a38d3998
[sgl-router] Add --tokenizer-backend fast and --tokenizer-l1-cache-mb ( #41268 )
...
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-29 18:55:58 +08:00
Michael and Claude
5828cd4927
[AMD] Add GLM-5.3 MI30x and MI35x nightly accuracy tests ( #41602 )
...
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-29 02:49:01 -07:00
Rumit Desai and Xiaoyu Zhang
f731e82f09
Fix MUSA detection under torch.compile fullgraph ( #41065 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-09-29 17:26:27 +08:00
Carrie Chen and Brayden Zhong
0e586fd12d
fuse shared experts with routed experts in MegaMoE's DeepGEMM ( #39313 )
...
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca >
2026-09-29 17:25:10 +08:00
c9cb32e310
[AMD] fix kda decode flydsl import ( #40546 )
...
Co-authored-by: Bingxu Chen <bingxche@amd.com >
Co-authored-by: Michael <13900043+michaelzhang-ai@users.noreply.github.com >
2026-09-29 02:20:42 -07:00
Liang Gong
34a1234d21
chore: expose agent skills via .agents directories ( #40515 )
2026-09-29 17:12:28 +08:00
Michael
05817a40c9
[AMD] Fix GLM-5.3 quark MoE MI35x test runner config ( #41464 )
2026-09-29 16:46:52 +08:00
Jiacong Fang
bd78095030
[Fix] Add name mapping in load_weights of nvidia/LocateAnything-3B ( #41059 )
2026-09-29 00:38:58 -07:00
Mick and Mick Qian
dc1bd46802
[diffusion] feat: support bounded exact conditioning cache across native models ( #40470 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-09-29 15:27:22 +08:00
eeecho
c7be3e935b
[SM120] Add optional FlashInfer PCIe-IPC all-reduce for switch-free hosts ( #34528 )
2026-09-28 23:08:44 -07:00
Anupa Sajikumar and Ma Mingfei
2ff52e3ceb
[Intel][XPU]Enable chunked prefill scnearios for XPU with UT ( #33804 )
...
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com >
2026-09-29 13:28:25 +08:00
9ae43b0eee
[XPU] Support compressed-tensors W4A16 by reusing the torch int4pack path ( #40828 )
...
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Singh <rohitsi2@iil-gnrap01.iind.intel.com >
Co-authored-by: Singh <rohitsi2@iil-login.iind.intel.com >
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com >
2026-09-29 13:16:03 +08:00
Cheng Wan
5781206833
[PD] Give FakeKVReceiver ensure_abort_notified ( #41618 )
2026-09-28 21:59:17 -07:00
jain-ria and ishandhanani
069523f999
[Rust] Extract a transport-neutral frontend core ( #39385 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-09-28 21:18:50 -07:00
Cheng Wan
6dad152490
[Refactor] Document layer boundary contracts and integration ( #41557 )
2026-09-28 20:18:27 -07:00
Cheng Wan
49e7bd179d
[Refactor] Group layer boundary unit tests ( #41556 )
2026-09-28 20:18:08 -07:00
Cheng Wan
de9f850b36
[Refactor] Rename the module to layer_boundary ( #41555 )
2026-09-28 20:17:50 -07:00
Cheng Wan
701213c867
[Refactor] Retire the layer facade and simplify boundary internals ( #41554 )
2026-09-28 20:17:33 -07:00
Cheng Wan
9b89b96230
[Refactor] Migrate specialized decoder and overlap boundaries ( #41553 )
2026-09-28 20:17:17 -07:00
Cheng Wan
49b5e76e45
[Refactor] Construct independent decoder stage boundaries ( #41552 )
2026-09-28 20:17:00 -07:00
Cheng Wan
acffb0d20d
[Refactor] Select reduction fusion at the consumer ( #41551 )
2026-09-28 20:16:44 -07:00
Cheng Wan
cb1fa3c399
[Refactor] Capture auxiliary states at residual reads ( #41550 )
2026-09-28 20:16:26 -07:00
Cheng Wan
42f58b4da7
[Refactor] Carry residual state across stage boundaries ( #41549 )
2026-09-28 20:15:03 -07:00
Cheng Wan
6f72cc3b64
[Refactor] Centralize decoder output access ( #41548 )
2026-09-28 20:14:04 -07:00
Cheng Wan
0a8ca85565
[Refactor] Group communicator fusion and CP adapters ( #41547 )
2026-09-28 20:10:50 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
e49d7fb170
Remove GLM-4.1V-9B-Thinking from encoder DP MMMU test ( #41608 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-28 19:45:21 -07:00
Liangsheng Yin
d9074f7943
[Test] Fail fast when PD test RDMA devices are not openable by ibverbs ( #41600 )
2026-09-28 19:34:58 -07:00
James
6e1df69623
[npu]support NPU 910C L2 memcache offload ( #41527 )
2026-09-29 10:30:01 +08:00
77091cea68
[Radix Cache] Sync Rust TreeCore and make it the default ( #39627 )
...
Co-authored-by: Ke Bao <ispobaoke@gmail.com >
Co-authored-by: Zhangheng <hzh0425@apache.org >
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com >
2026-09-29 10:18:57 +08:00
WenhaoZhang
49d26ff0c2
[diffusion] model: support flux 3 action robot policies ( #41066 )
2026-09-29 10:17:19 +08:00
79cafec013
[Model] Add IQuest Q1 support and MTP draft ( #41590 )
...
Co-authored-by: zelong huang <yzhu@ubiquant.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: zelong518 <zelonghuang05@gmail.com >
Co-authored-by: chenwenxiaolive <cocoshirolive@gmail.com >
2026-09-28 17:21:31 -07:00
Stanislav Petrov and Stanislav Petrov
b95329a74d
[Docs] Add GigaChat 3.5 and GigaChat 3.5 Reasoning cookbook pages ( #41118 )
...
Co-authored-by: Stanislav Petrov <stapetrov@sberbank.ru >
2026-09-29 08:16:22 +08:00
7889fbab9a
[PD] Keep EAGLE DP graph and token metadata consistent ( #32196 )
...
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com >
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai >
2026-09-28 17:15:51 -07:00
Shuwen Wang
89e1316eae
[mem cache] refactor: remove the index-K continuous getters orphaned by the CP v1 removal ( #41469 )
2026-09-28 16:41:07 -07:00
Hrithvik Alex and Hrithvik Alex
f124c5ceec
[Model Loader] Stop checkpoint prefetch after iterator completion ( #41588 )
...
Co-authored-by: Hrithvik Alex <hrithvik@baseten.co >
2026-09-28 16:27:09 -07:00
9654e5c4bc
dsv4.1-amd: fused mHC boundary and all-reduce + mHC post kernels ( #41021 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai >
2026-09-28 16:15:08 -07:00
Kevin Mi and Claude Opus 5.5
096b066fb4
dsv4.1-amd: gfx950 sparse decode attention and sorted top-k ( #41020 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-28 16:14:28 -07:00
Zhang, Jiejing
26d7539799
[Docs][AMD] Update GLM-5.2 MI355X daily image ( #41597 )
2026-09-28 15:42:54 -07:00
Trevor Morris
b07cb9e9e7
[NVIDIA] Update deepgemm, deep-ep, sgl-kernel in CUDA 13.4 image, use cuda base image ( #40987 )
2026-09-28 14:12:46 -07:00
Nan Jiang
6fa3fe69e2
[PD] Keep the sampling mask of a replayed rebootstrap token ( #41235 )
2026-09-28 12:02:27 -07:00
Shangming Cai and Claude Fable 5
2a1c477648
[PD] Defer decode KV release on every transfer failure, not only decode-initiated aborts ( #41404 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-29 02:19:51 +08:00
YAMY
f209e67767
[Spec] Model-agnostic last-stage draft embedding under pipeline parallelism ( #39643 )
2026-09-28 10:22:40 -07:00
Kurkur
6624999385
[Fix][NPU] fix dp-attn hang when pin_mem is True on NPU ( #40446 )
2026-09-29 00:41:43 +08:00
triple-mu
2e7e0802f4
[diffusion] Qwen-Image 2.1: fuse Q/K RMSNorm + RoPE + KV packing into one CUDA kernel and project Q/K/V with one packed GEMM ( #41339 )
2026-09-28 21:45:39 +08:00
DarkSharpness and Claude Opus 5.5
8151378c1a
[DeepSelect] Add page-table transform to top-k and tighten the layout contract ( #41364 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-28 21:43:57 +08:00
DarkSharpness and Claude Opus 5.5
0318a8d0af
[Doc] Add kernel benchmark rule on L2 cache reuse ( #41545 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-28 20:25:58 +08:00
Shuwen Wang
da3eb6db32
[Unified Memory] Fix Inkling conv-checkpoint track ids written to virtual slot numbers ( #41144 )
2026-09-28 05:19:19 -07:00
Shuwen Wang
bddeb61d94
[Unified Memory] fix: preserve FP8 dtype in unified MHA pool ( #38133 )
2026-09-28 05:19:03 -07:00
Mick and Mick Qian
090439bf8f
[diffusion] docs: consolidate Qwen-Image 2.1 guidance in its cookbook ( #41540 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-09-28 20:04:32 +08:00
6aacca2d91
[Router] Name a replica's siblings with --kv-peer-selector (3/13) ( #40689 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-28 18:49:09 +08:00
Raiden Makoto and Raiden-Makoto
fa443412c5
[AMD] [GLM5] Fuse shared expert into AITER MoE on gfx950 ( #41161 )
...
Signed-off-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com >
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com >
2026-09-28 03:40:09 -07:00
2f88c65289
[Router] Serve the cache-aware tree at /internal/kv_snapshot (2/13) ( #40688 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-28 18:38:52 +08:00
47ad1928a2
[sgl-router] Scope input_ids forwarding by renderer: all text chats for DeepSeek-V4 ( #41226 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-09-28 18:37:40 +08:00
karverma-amd
85be04978d
[AMD][DSV4] moe: enable shared-expert fusion on the grouped-topk path (megamoe) ( #40943 )
2026-09-28 03:21:57 -07:00
Denji-kk
6caf0ff4ae
perf(multimodal): offload CPU feature hashing with bounded admission ( #39539 )
2026-09-28 17:51:05 +08:00
Mick and Mick Qian
c02eaadeb2
[diffusion] optimization: populate cpu weight stores before host registration to speedup layerwise-offload initialization ( #40439 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-09-28 17:45:58 +08:00
Peng Wu
6582425854
Let predicate-registered linear-attention models carry the mamba radix-cache leaves ( #41165 )
2026-09-28 17:07:35 +08:00
df378bca42
[Router] Give the cache-aware tree a snapshot surface (1/13) ( #40687 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-28 16:59:59 +08:00
Peng Wu
546bfa7221
[Kimi-K3] Merge fused_qkvg_proj into the loader-seeded packed_modules_mapping ( #41164 )
2026-09-28 16:46:11 +08:00
ashwini rathi and Pramod Kumar
e8eeff2956
[XPU] Bump sglang-kernel-xpu wheel to v0.3.0 ( #41220 )
...
Co-authored-by: Pramod Kumar <144990617+pramodkumar-habanalabs@users.noreply.github.com >
2026-09-28 16:15:12 +08:00
Baizhou Zhang
7ee7bef79d
docs: add prefill context parallelism guide and design draft ( #39354 )
2026-09-28 00:26:02 -07:00
Ke Bao
6405cf2881
Fix chat template cache key order ( #41517 )
2026-09-28 15:17:09 +08:00
flb_
f4de6abee6
feat(npu): Support returning indexer top-k results ( #39060 )
2026-09-28 14:46:45 +08:00
Rain Jiang
6c4f36e47a
Rust server unify datapath for mm and generate requests ( #39679 )
2026-09-27 23:36:07 -07:00
e75e3b8a98
[Bugfix] fix(hicache): wait for decode offload before retraction ( #30899 )
...
Co-authored-by: agent <agent@local>
Co-authored-by: Kevin Flansburg <6134007+kflansburg@users.noreply.github.com >
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-27 23:24:19 -07:00
6be9c78cf4
[Intel GPU] Xpu/weekly simple model enablement 2026 09 21 ( #40664 )
...
Co-authored-by: Amrutha M <amrutha.m@intel.com >
Co-authored-by: charlesxu91 <charlesxu.mi@gmail.com >
Co-authored-by: KMS07 <meher.sai.kotthagattu@intel.com >
Co-authored-by: Avi Fenesh <aviarchi1994@gmail.com >
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com >
2026-09-28 13:49:18 +08:00
34f090053e
[AMD] [Docker] Remove unused LLVM 18 setup from ROCm TileLang build ( #41387 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
Co-authored-by: quitenode <quitenode@users.noreply.github.com >
2026-09-27 22:41:08 -07:00
25de80780a
[AMD] Add GLM-5.3-Flash MI35x nightly test ( #36903 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
Co-authored-by: quitenode <quitenode@users.noreply.github.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-09-27 22:40:03 -07:00
Michael
3d2827236d
[AMD] Register Triton data movement tests in PR CI ( #41137 )
2026-09-27 22:38:28 -07:00
Khoa Pham
55cc90b533
[CI] Real-model Kimi-Linear PD parity at page, DCP virtual-page, chunk and cached-prefix boundaries ( #41378 )
2026-09-27 20:23:07 -07:00
Xinguo Zhu
cc012abd21
[CI] fix CI regression on xeon ( #41002 )
2026-09-28 10:28:01 +08:00
Yihao Wang and Mick Qian
8b2ca8ecc2
[diffusion] feat: support multiple task types for pipelines ( #38762 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-09-28 10:00:45 +08:00
Kurkur
6cbdfec453
[Fix][NPU] Fix performance degradation caused by serial execution of two-stage ACL op calls on graph ( #40814 )
2026-09-28 09:32:16 +08:00
ashwini rathi
0823651b0b
[XPU] Disable test_ngram_corpus on XPU and detect XPU tests dynamically in CI filter ( #41075 )
2026-09-28 09:09:40 +08:00
Cheng Wan
81f27fb3a7
[Refactor] Replace LayerScatterModes with LayerFacts and remove ScatterMode ( #41443 )
2026-09-27 16:56:11 -07:00
Cheng Wan
06d012eaca
[Refactor] Give a layer its CuTe DSL kernels at construction instead of a subclass ( #41442 )
2026-09-27 16:55:47 -07:00
Cheng Wan
341d5273bd
[Refactor] Build the boundary into any stage with one construction ( #41441 )
2026-09-27 16:55:25 -07:00
Cheng Wan
644014bc98
[Refactor] Declare each stage's residual read and update, and give each stage its own entry ( #41440 )
2026-09-27 16:55:04 -07:00
Cheng Wan
0e907b755d
[Refactor] Split the layer communicator into a package (move only) ( #41439 )
2026-09-27 16:54:41 -07:00
Cheng Wan
427c9e05c1
[Refactor] Choose every layer's boundaries from declarations and remove the scatter-mode selection ( #41438 )
2026-09-27 16:54:18 -07:00
Cheng Wan
55d2ee5546
[Refactor] Step-3.5: complete the dense MLP's sum through ffn_exit ( #41437 )
2026-09-27 16:53:57 -07:00
Cheng Wan
0971450c16
[Fix] LongCat-Flash under attention DP: branch and merge the dense FFNs through the communicators ( #41436 )
2026-09-27 16:53:36 -07:00
Cheng Wan
4fac02fe5d
[Refactor] Choose MoE layers' boundaries from declarations when moe_dp_size equals attn_cp_size ( #41435 )
2026-09-27 16:53:15 -07:00
Cheng Wan
ecb8e9fc8f
[Refactor] Falcon-H1: complete the FFN's sum through ffn_exit ( #41434 )
2026-09-27 16:52:53 -07:00
Cheng Wan
19eb56fa57
[Fix] Falcon-H1: count the Mamba mixer's output once under tensor parallelism ( #41433 )
2026-09-27 16:52:31 -07:00
Cheng Wan
31329c222c
[Fix] Run MoE layers under attention DP and GQA prefill CP on the declared DP × CP gather ( #41432 )
2026-09-27 16:52:10 -07:00
Cheng Wan
f0ecf15c84
[Refactor] Move CuTe DSL-fused layers onto the declared boundaries and FFN-exit kernel entries ( #41431 )
2026-09-27 16:51:48 -07:00
Cheng Wan
8c43c667cb
[Refactor] Nemotron-H: build each layer's boundaries from its stage and the previous one ( #41430 )
2026-09-27 16:51:27 -07:00
Cheng Wan
6f370c3bcd
[Refactor] Build each decoder boundary from the declarations of its two sides ( #41429 )
2026-09-27 16:51:05 -07:00
Cheng Wan
2e7f0bb28c
[Refactor] Publish the LoRA token layout from the batch's FFN input rows ( #41428 )
2026-09-27 16:50:43 -07:00
Cheng Wan
c289656434
[Refactor] Let the MoE declare whether its skipped reduction is one TP all-reduce ( #41427 )
2026-09-27 16:50:23 -07:00
Cheng Wan
deaa7f1463
[Refactor] Run two-batch-overlap layers on the declared boundaries ( #41426 )
2026-09-27 16:50:03 -07:00
Cheng Wan
6f2d2437b6
[Refactor] Run MHC layers on the shared boundary steps with MHC's residual operations ( #41425 )
2026-09-27 16:49:42 -07:00
Cheng Wan
f5ae989a0e
[Refactor] Choose fully-DP dense and DSA / MLA prefill CP layers' steps from declarations ( #41424 )
2026-09-27 16:49:21 -07:00
Cheng Wan
cad9eaf27d
[Fix] Gather a dense FFN's input across attention DP and CP in one DP sum ( #41423 )
2026-09-27 16:48:59 -07:00
Cheng Wan
1c919e401d
[Fix] Keep one copy of CP-replicated rows in the DP gather ( #41422 )
2026-09-27 16:48:39 -07:00
Cheng Wan
7a833ad0a2
[Refactor] Choose a GQA prefill CP extend's steps from declarations ( #41421 )
2026-09-27 16:48:17 -07:00
Cheng Wan
380a4d0332
[Refactor] Run every batch of a layer from one BoundarySteps ( #41420 )
2026-09-27 16:47:55 -07:00
Cheng Wan
9f87d28db6
[Refactor] Choose the LayerNorm SP region's and input-scattered batches' steps from declarations ( #41419 )
2026-09-27 16:47:32 -07:00
Cheng Wan
846181f1c4
[Refactor] Give the fused prepare_mlp kernels an explicit contract ( #41418 )
2026-09-27 16:47:11 -07:00
Cheng Wan
446d181e23
[Refactor] Choose the boundaries of plain-TP dense layers and MoE layers from declarations ( #41417 )
2026-09-27 16:46:50 -07:00
Yoni Gozlan and Yoni Gozlan
583fada954
Port chat_parsing core ( #40477 )
...
Co-authored-by: Yoni Gozlan <yonigozlan@users.noreply.github.com >
2026-09-27 16:14:24 -07:00
78eee88113
[Spec] Add LiLiCorr: a candidate-lattice reranker for DFlash drafts ( #37462 )
...
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com >
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-27 15:36:48 -07:00
Shangming Cai and Claude Fable 5
14ab74ef95
[PD] Fan drain abort ACKs out to every decode peer of the room ( #41402 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-28 05:16:23 +08:00
Thomas Wang
b252aceffe
[AMD] Update v4 cookbook for megamoe, fp8 kv attn, BCG ( #41458 )
2026-09-27 10:53:21 -07:00
e581520c67
[HiCache] Add the page-unified KV load-back JIT kernel ( #39726 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: huangtingwei <141888744+huangtingwei9988@users.noreply.github.com >
Co-authored-by: Zhangheng <hzh0425@apache.org >
2026-09-28 01:45:02 +08:00
Xinyuan Tong
d10a09ecf3
Add MiniMax arch fallback to auto parser resolution ( #40930 )
2026-09-28 01:33:27 +08:00
CuzMi
b2800461e5
[Diffusion] Reuse bit-exact packed SwiGLU for Ming-Image ( #41266 )
2026-09-27 23:57:43 +08:00
1ba84a4b59
[bench] Take each request's prompt length from the server ( #39889 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-09-27 23:57:18 +08:00
Yuan Luo and luoyuan.luo
c311bc9611
[VLM] Introduce FA4 into ViT for SM100/SM103 ( #41344 )
...
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com >
2026-09-27 23:04:21 +08:00
588049fe2e
[diffusion] fix: run an all-valid attention mask on the backend's unmasked kernel ( #41309 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-27 22:54:46 +08:00
Kangrui Du
84622ce9d5
[diffusion] fix: lora-wrapped linears crash qwen-image and minimax-h3 inference ( #41272 )
...
Signed-off-by: rockdu <kangrdu@gmail.com >
2026-09-27 22:54:10 +08:00
a0ba1968b1
[Diffusion] Apply latent-ids and packing to caller-provided initial latents ( #34418 )
...
Co-authored-by: aimicahchen <aimicahchen@tencent.com >
Co-authored-by: WenhaoZhang <42087078+niehen6174@users.noreply.github.com >
2026-09-27 18:32:20 +08:00
WenhaoZhang
f1e62e3a2e
[diffusion] feat: add MiniMax-H3 to ComfyUI integrated mode ( #35990 )
2026-09-27 17:58:54 +08:00
WenhaoZhang
ae47bcd4da
[diffusion] feat: minimax-h3 spectrum skip-step + fused RMSNorm/AdaLN ( #35684 )
2026-09-27 17:56:13 +08:00
WenhaoZhang
2a14f65fd7
[diffusion] feat: support in-place lora merge/unmerge under layerwise offload ( #36192 )
2026-09-27 17:54:40 +08:00
Liangsheng Yin
37556c1b91
[mem_cache] Replace cache_finished_req with insert_req; release_kv_cache frees and unpins ( #41281 )
2026-09-27 02:53:16 -07:00
Xiaoyu Zhang and kernel-design-agents
0bc074db83
[KDA+Kimi K3] Speed up SANA-Video residual gate add on H200 ( #41305 )
...
Co-authored-by: kernel-design-agents <334018530+kernel-design-agents@users.noreply.github.com >
2026-09-27 16:57:12 +08:00
Qiaolin Yu
aa7a976807
[qwen 3.8 next] Fuse small CUDA graph input buffer copies ( #41166 )
2026-09-27 01:27:56 -07:00
Shunkangz and Zhangheng
df1fc6ab64
[kv-shard 3/4] Enable Control Plane B ( #39964 )
...
Co-authored-by: Zhangheng <hzh0425@apache.org >
2026-09-27 14:19:08 +08:00
Jianghai and aimicahchen
acc15c193b
[Diffusion] Preserve per-sample rollout trajectories across multi-output merge ( #34416 )
...
Co-authored-by: aimicahchen <aimicahchen@tencent.com >
2026-09-27 13:08:45 +08:00
Shuwen Wang
38d865489a
[DSV4.1][HiCache] fix: wait for the layer transfer before reading low-ratio index-K ( #41345 )
2026-09-26 22:04:03 -07:00
Liangsheng Yin
d27efca353
[mem_cache] Never free the protected prefix on request release ( #41312 )
2026-09-26 22:00:02 -07:00
b4584ab838
[sgl-router] Take SGLang's render defaults: --default-chat-template-kwargs and thinking/effort envs ( #41221 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-27 11:01:17 +08:00
Kevin Mi and x
effb752188
dsv4.1-amd: KV cache layouts, FP4 indexer, compressor and router kernels ( #41019 )
...
Co-authored-by: x <x>
2026-09-26 19:44:47 -07:00
Ankith Averineni
425a1f8f24
[AMD] Honor an explicit triton moe_runner_backend for mxfp8 on ROCm ( #41377 )
2026-09-26 17:18:08 -07:00
Lianmin Zheng and Lu Fang
fc9e1c8d29
[MemCache] Fix LMCache component cursors and per-cache backend selection ( #41328 )
...
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com >
2026-09-26 11:24:39 -07:00
c10fa03fc4
Make sliding-window caching and speculative batch padding extensible ( #41325 )
...
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com >
Co-authored-by: jasonjk-park <jasonjk@fb.com >
2026-09-26 11:24:02 -07:00
Thomas Wang
c73f7077eb
[AMD] Fix jit broken on rocm env ( #41356 )
2026-09-26 09:59:59 -07:00
Khoa Pham
7bdd8fe6ec
[CI] Merge the Kimi-Linear PD DCP4 nightly tests and drop exact-token parity ( #41321 )
2026-09-26 09:48:00 -07:00
Ariel and Claude Fable 5.1
e1e97e8b49
Pin triton_kernels num_warps for MXFP4 MoE below Hopper (6x gpt-oss decode on RTX 4090) ( #41292 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-27 00:43:25 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
cdbea5dccf
[CI] Move DeepSelect into the attention kernel group to fix the namespace test ( #41349 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-26 21:39:00 +08:00
Shangming Cai and Claude Fable 5
710a41ba57
[sgl-router] Book the input_ids forwarding outcome only for built bodies ( #41342 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-26 19:39:01 +08:00
DarkSharpness and Claude Opus 5.5
a947b56bc7
[CI] Add __main__ entry to test_deep_select.py ( #41341 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-26 18:12:41 +08:00
Cheng Wan
bd37e7f513
[Refactor] Choose a dense layer's boundaries under attention DP from both sides' declarations ( #41257 )
2026-09-26 02:41:43 -07:00
Cheng Wan
9cd6616c24
[Refactor] Pick the two-batch-overlap split's layout moves once and remove execute ( #41256 )
2026-09-26 02:41:09 -07:00
Cheng Wan
d3fcf8997c
[Refactor] Run the LayerNorm SP region's boundary steps in the communicator itself ( #41255 )
2026-09-26 02:40:34 -07:00
Cheng Wan
35b369ebba
[Refactor] Declare at construction the layers whose FFN completes its own reduction ( #41254 )
2026-09-26 02:39:58 -07:00
Cheng Wan
c081d4aacc
[Refactor] Drive an FFN exit's flags and its completion from one selection ( #41253 )
2026-09-26 02:39:25 -07:00
Cheng Wan
8fc3ce48da
[Refactor] Choose prepare_attn / prepare_mlp steps and fused kernels at construction ( #41252 )
2026-09-26 02:38:03 -07:00
7bc988446d
[DeepSeek V4.1] Add DeepSelect JIT kernel. ( #40556 )
...
Co-authored-by: DarkSharpness <2040703891@qq.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-26 17:36:48 +08:00
Hexu Zhao and hexu.zhao
a5c8afab48
[Perf] Mamba2 selective_state_update up to 2x faster on B200 via 8x1 launch config for dstate 128 (+7.5% Nemotron-3-Super serving) ( #41223 )
...
Co-authored-by: hexu.zhao <zhaohexu2001@gmail.com >
2026-09-25 23:27:01 -10:00
Sundara Raman Ramachandran
3ed56a341b
[Score API] Setwise scoring: CausalLM support (batched + --enable-mis) ( #41188 )
2026-09-25 23:02:44 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
4bb645b8e2
[DSA] Fix the pooled-indexer breakable prefill bridge under DP attention ( #41311 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-25 22:52:22 -07:00
8772916e06
[DSv4.1] Move the low-ratio index top-k into dsv4/low_ratio_indexer ( #41125 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-09-25 22:22:57 -07:00
7152aabdcb
[sgl-router] Track input_ids forwarding outcomes per chat request ( #41185 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-26 13:09:46 +08:00
Xiaoyu Zhang
38ec649048
[Refactor] Restore logical kernel groups and test organization ( #41243 )
2026-09-26 12:30:44 +08:00
Mick and Mick Qian
e538489e0e
[diffusion] fix: copy small files into overlay materialized trees instead of linking them ( #41298 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-09-26 12:22:47 +08:00
WMC and jacky.cheng
dfdbff931d
[diffusion] fix: fix Qwen-Image 2.1 default RGBA output ( #41150 )
...
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-09-26 12:21:24 +08:00
Lianmin Zheng and Lu Fang
3ecb6cd729
[MemCache] Unify component eviction cursors and lock receipts ( #41276 )
...
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com >
2026-09-25 20:53:35 -07:00
Olga Miroshnichenko
0e2aac500b
[ROCm][Perf] aiter: page-level KV view for gfx950 fp8 page-64 asm prefill ( #36505 )
2026-09-25 19:23:46 -07:00
Liangsheng Yin
f35ec18d8b
[Test] Remove more unit tests that mirror implementation or never run in CI ( #41297 )
2026-09-25 19:22:14 -07:00
Shuwen Wang
4273e7de77
[PD] fix: cache resumed decode-radix requests from root instead of an unlocked re-match ( #41261 )
2026-09-26 10:21:41 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
63d320c723
[DSA] Chunk the kpool indexer MQA logits by query rows under a free-memory budget ( #40854 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-25 19:13:13 -07:00
Mick and Mick Qian
2ee82fe158
[diffusion] model: support Anima Base v1.0 ( #41011 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-09-26 09:53:51 +08:00
Liangsheng Yin and DarkSharpness
0967a013c2
[DSv4.1] Move the ratio-1/2 index top-k ops into kernels/ops/attention/dsv4 ( #41291 )
...
Co-authored-by: DarkSharpness <2040703891@qq.com >
2026-09-25 18:28:04 -07:00
Xiaoyu Zhang
85374532f2
[diffusion] Fuse lossless Klein packed QK RMSNorm and RoPE on Hopper ( #40490 )
2026-09-26 09:24:21 +08:00
Liangsheng Yin
c842d32b0a
[CI] Harden /rerun-test dispatch and partitioning ( #41285 )
2026-09-25 17:39:50 -07:00
50e50e13ab
[DCP] Use logical token capacity for PD admission and load reporting ( #39731 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai >
2026-09-25 17:39:47 -07:00
Liangsheng Yin
1f6ce4b068
[Test] Remove unit tests that only mirror implementation or never run in CI ( #41286 )
2026-09-25 16:15:02 -07:00
Liangsheng Yin
2afd343934
[misc] Call all_gather_single / reduce_scatter_single to drop torch deprecation warnings ( #41284 )
2026-09-25 16:14:13 -07:00
Liangsheng Yin and BBuf
efd9a40bb8
[PD] Share one head-slice helper across mooncake, mori, and nixl ( #39660 )
...
Co-authored-by: BBuf <1182563586@qq.com >
2026-09-25 16:13:18 -07:00
Yuwei An and Claude Opus 5.5
95047d3464
[Fix] Fix cpu CI fail introduced by pr #38652 ( #41287 )
...
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-25 15:55:21 -07:00
Liangsheng Yin
516ab77619
[Test] Route all sgl-eval benchmarks through run_sgl_eval and deprecate run_eval ( #41280 )
2026-09-25 15:48:38 -07:00
Kevin Mi and Claude Opus 5.5
27e883a20d
dsv4.1-amd: gfx950 MXFP8 matmul kernels and fp8-grid producers ( #41018 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-25 15:45:40 -07:00
Erik Wijmans and Ethan Su
c421d16563
[sglang][lora] Support DP attention in LoRA backends ( #36389 )
...
Co-authored-by: Ethan (Yusheng) Su <yushengsu@radixark.ai >
2026-09-25 12:03:47 -10:00
cf5df82680
[KVCache] Support lmcache unified radix cache ( #38652 )
...
Signed-off-by: chunxiaozheng <1179548172@qq.com >
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com >
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com >
2026-09-25 14:08:45 -07:00
Ethan (Yusheng) Su
d3e3945e41
[LoRA] Size dense row/column-parallel LoRA buffers from the base linear's real shard ( #39379 )
2026-09-25 14:01:08 -07:00
Liangsheng Yin
f88572a57c
[Test] Add in-process sgl-eval adapter and move validated GSM8K tests to it ( #41216 )
2026-09-25 13:59:17 -07:00
Liangsheng Yin
f337f01b80
[Test] Remove dead eval modules and point GSM8K/MMLU docs to sgl-eval ( #41215 )
2026-09-25 12:50:31 -07:00
ac62ddf164
[Rust frontend] Decode input_ids without untagged buffering ( #41246 )
...
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com >
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com >
2026-09-25 12:47:38 -07:00
Nan Jiang
24b6930076
[Sampling] Stream sampling masks as per-request arrays ( #40986 )
2026-09-25 12:20:25 -07:00
6d35cee7fc
Fix multimodal feature offload races ( #40621 )
...
Co-authored-by: cctry <cctry@fb.com >
Co-authored-by: Yongji Wu <yongji@meta.com >
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu >
2026-09-25 12:18:08 -07:00
metamergebot and Yonghao Zhuang
671000aee6
[unified-memory] Honor move gates in float relocation and size auto HiCache from host capacity ( #41248 )
...
Co-authored-by: Yonghao Zhuang <yhzhuang@meta.com >
2026-09-25 12:17:28 -07:00
31b931edc3
fix(multimodal): return 400 for corrupt image inputs ( #28131 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: Justin Tong <justintong0323@users.noreply.github.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
2026-09-25 19:51:33 +02:00
0154f72b48
[ROCm][Bugfix] Keep quantization for mixed Quark Qwen3.5 MTP checkpoints ( #39064 )
...
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-09-25 10:31:29 -07:00
Even Zhou
67bb6a58d0
Revert "[NPU] Fuse FIA KV-cache K/V writes into one npu_scatter_pa_kv_cache call" ( #41132 )
2026-09-25 23:56:22 +08:00
Xiaoyu Zhang
2f5c9ac43d
[diffusion] Enable lossless Cosmos3 Super T2I QK fusion on Hopper TP2 ( #40486 )
2026-09-25 18:01:06 +08:00
Jianghai and mickqian
a749d84303
[Diffusion] Fix AttributeError in grouped forward_batch by installing the residency manager ( #34417 )
...
Co-authored-by: mickqian <mickqian@users.noreply.github.com >
2026-09-25 17:27:38 +08:00
ashwini rathi
62122c838a
[XPU] Disable test_ngram_corpus on XPU and extend XPU CI path filter ( #41224 )
2026-09-25 01:54:05 -07:00
Cheng Wan
3450d68d4e
[Refactor] Decide an FFN exit's completion once and declare the group it owes ( #41200 )
2026-09-25 01:50:35 -07:00
Cheng Wan
26a3214cda
[Refactor] Take a layer's last-layer fact from its scatter-mode plan ( #41199 )
2026-09-25 01:49:33 -07:00
Cheng Wan
8f5a636f8d
[Refactor] Build prepare_mlp and the layout moves from named steps ( #41191 )
2026-09-25 01:48:30 -07:00
Cheng Wan
b7f6d04a9a
[Refactor] Move Step-3.5, GLM5-Next, Dots3, MiniMax-M3 and Qwen3.5 onto ffn_exit ( #41198 )
2026-09-25 01:47:43 -07:00
Cheng Wan
963e9fb42d
[Refactor] Leave the FFN reduction to the next layer under attention DP ( #41197 )
2026-09-25 01:47:03 -07:00
Cheng Wan
9d7f44bdbb
[Refactor] Carry a deferred FFN all-reduce as UnreducedOutput and complete it in the next layer without the fused kernel ( #41196 )
2026-09-25 01:46:03 -07:00
Cheng Wan
402df23188
[Fix] Stop counting a deferred FFN sum more than once: replicated TP1 shared expert, dense reduce_scatterv ( #41195 )
2026-09-25 01:45:00 -07:00
Cheng Wan
1409f469be
[Fix] Plan NextN / MTP draft layers as one-layer models and fix the Bailing V2 NextN draft ( #41194 )
2026-09-25 01:43:55 -07:00
Cheng Wan
51c92e8df1
[Fix] Complete the all-reduce when the flashinfer fused norm declines a batch ( #41193 )
2026-09-25 01:42:16 -07:00
kk
515f5be77e
[AMD] Integrate Aiter MegaMoEv2 for DeepSeek-V4 ( #35619 )
2026-09-25 00:47:08 -07:00
Shuwen Wang
434c2e3adc
[HiCache] fix: Drain pending backups before internal Mamba write-back ( #41092 )
2026-09-25 15:06:24 +08:00
Shuwen Wang
cbe137727e
[HiCache] Give trailing sidecar storage transfers a contiguous prefix_keys chain ( #40456 )
2026-09-25 15:06:05 +08:00
Mick and Mick Qian
0fb699ac03
[diffusion] model: support Ming-Image Design and Design-Layer ( #41067 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-09-25 14:44:37 +08:00
metamergebot and Xingyu Liu
9c8340a2e5
[DeepEP v2] Let a model package supply its per-rank prefill dispatch bound ( #41201 )
...
Co-authored-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com >
2026-09-24 23:39:03 -07:00
Xinyi Song
ec070ec8c8
[AMD] Fix int32 offset overflow in Triton DSv4 KV store kernels ( #41159 )
2026-09-24 23:11:59 -07:00
Ting SUN and Xinyuan Tong
3e7e652900
fix(openai): reject request-supplied chat_template by default ( #28135 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com >
2026-09-25 07:48:34 +02:00
Richard Wang
16d1c93b3d
[feat] add a system one compatible /v1/systemone route ( #41208 )
2026-09-25 12:59:11 +08:00
cec70a4708
[Fix] Keep the target's DP sync slot in draft scopes ( #41062 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com >
2026-09-24 21:48:39 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
3fc7a669bf
[CI] Move GLM-5.2 layer-split test to extra-b-test-8-gpu-b300 ( #41207 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-24 21:25:48 -07:00
e0b4d66733
[Perf] Lazy-load built-in model definitions and nixl_ep at startup ( #41061 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: Yongji Wu <30348494+libertyeagle@users.noreply.github.com >
2026-09-24 20:29:33 -07:00
7325b38412
[PD] Honor gracefully_exit in disaggregation event loops and keep non-zero-rank launchers alive on SIGTERM ( #40793 )
...
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com >
Co-authored-by: Shiyan Deng <dsy842974287@meta.com >
2026-09-24 20:29:07 -07:00
akhilg-nv
37ebcac3c4
Support XQA backend for SpecDec verify ( #32269 )
2026-09-24 20:14:46 -07:00
Mick and Mick Qian
6103a8c743
[diffusion] feat: add opt-in SRT prompt enhancement to image and video APIs ( #41095 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-09-25 10:00:04 +08:00
jacky.cheng
9dc4c5d891
[AMD] Restore non-DCP Mamba checkpoint donation to fix agent-mode cache hit at high conc with HiCache ( #40907 )
2026-09-24 18:08:10 -07:00
Liangsheng Yin
8ca82118e0
[mem_cache] Drop is_insert from cache_finished_req; release rows from release_kv_cache ( #40988 )
2026-09-24 17:23:26 -07:00
Shuwen Wang and hnyls2002
a2025b8c4d
[DSV4] Account for FlashMLA physical KV page padding in memory budgets ( #41091 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-09-24 17:14:08 -07:00
Sundara Raman Ramachandran
2f2f9d12f8
[Score API] Setwise Scoring Support ( #38965 )
2026-09-24 16:50:58 -07:00
7c5bdb940c
[PD] Add a none decode retraction backup and subclass seams in the PD queues ( #41103 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: Zhiqiang Xie <zqx@meta.com >
2026-09-24 16:13:54 -07:00
466e985f4b
[HiCache] Batch buffer-only KV backups within each flush ( #40960 )
...
Co-authored-by: oss-sync bot (pranjalssh) <pranjalssh@users.noreply.github.com >
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com >
2026-09-24 16:13:02 -07:00
Caio Rocha and Caio Rocha
e047e50d4e
Add 8-node AllReduce/AllGather and MNVLS algorithm support to MSCCL++ ( #37442 )
...
Co-authored-by: Caio Rocha <caiorocha@microsft.com >
2026-09-24 15:51:29 -07:00
Yuwei An
3ec86307f2
Fix mixed chunk prefill with DP speculative coordination ( #41179 )
2026-09-24 15:41:49 -07:00
6bbd689ab4
[Bugfix] Align DeepSeek-V4.1 reasoning effort budgets ( #39929 )
...
Co-authored-by: fengtinglei <fengtinglei@bytedance.com >
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-09-24 15:21:06 -07:00
Liangsheng Yin
0c578d97fe
[DSV4] Size compressed pools from one per-ratio table in DSV4PoolConfigurator ( #41049 )
2026-09-24 15:19:02 -07:00
Qiaolin Yu
a1eb691ab5
[Docs] Enable Qwen3.8 Flash Next NVIDIA NVFP4 on B200/B300/GB300 ( #41046 )
2026-09-24 14:19:00 -07:00
Liangsheng Yin
182f62d2de
[Fix] Patch set_dp_buffer_len_from_batch in DP spec prefill coordination test ( #41162 )
2026-09-24 13:52:11 -07:00
Shuwen Wang
961404b010
[DSV4] Fix TRTLLM uniform FP8 KV memory budgeting ( #41090 )
2026-09-24 13:25:00 -07:00
36f59982fa
MoE: small-batch sorting path with fused mxfp8 quantisation ( #36559 )
...
Co-authored-by: mikevin920 <mikevin920@yahoo.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Kevin Mi <mikevin920@gmail.com >
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-09-24 13:18:18 -07:00
0b53305f48
MiniMax-M3: MXFP8 dense-only block convert + aiter MXFP8 MoE on gfx950 ( #36574 )
...
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: mikevin920 <mikevin920@yahoo.com >
Co-authored-by: Kevin Mi <mikevin920@gmail.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com >
2026-09-24 13:16:59 -07:00
Qiaolin Yu
b129504f0e
[qwen 3.8 next] Fuse Qwen PLE gate and convolution preparation for target verify ( #40041 )
2026-09-24 13:13:34 -07:00
Thomas Wang
1446e24d13
[AMD] Add .co for deepseek v4 fp8 decode kernel and add group decode opt ( #41120 )
2026-09-24 12:21:34 -07:00
Cheng Wan
5bb850d536
[Refactor] Split prepare_attn into a reduction step and per-quant-format residual steps ( #41084 )
2026-09-24 12:16:54 -07:00
Cheng Wan
a80055d3c6
[Fix] Broadcast requests along attention CP before attention TP ( #41083 )
2026-09-24 12:16:27 -07:00
Cheng Wan
7c9f74c6e1
[Fix] Step-3.5: stop dense layers from summing their output twice under DP attention ( #41082 )
2026-09-24 12:15:58 -07:00
Cheng Wan
62ae032c46
[Refactor] Share the MoE output all-reduce between models ( #41097 )
2026-09-24 12:15:33 -07:00
Cheng Wan
5df1667077
[Refactor] Pass each layer stack's output through a communicator exit ( #41081 )
2026-09-24 12:15:07 -07:00
Cheng Wan and Mick Qian
07a3735e94
[Fix] Complete the deferred FFN all-reduce before deepstack addition and aux hidden-state capture ( #41080 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-09-24 12:14:39 -07:00
Cheng Wan
928e683f94
[Fix] Complete the deferred FFN all-reduce before a pipeline-parallel send ( #41079 )
2026-09-24 12:13:57 -07:00
ChangLiu0709
6a14b80141
[AMD] GLM-5.2 MI355X MXFP4: bump image to 20260923 daily ( #41109 )
2026-09-24 12:02:01 -07:00
Khoa Pham
1b03d31b31
[Test] Run the Qwen3.5 Triton DCP nightly with the radix cache enabled ( #41155 )
2026-09-24 12:00:09 -07:00
f404db94d4
[DP attention] Publish DP buffer sizes from a ForwardBatch ( #40858 )
...
Co-authored-by: hanminglu <hanminglu@fb.com >
Co-authored-by: oss-sync bot (pranjalssh) <pranjalssh@users.noreply.github.com >
2026-09-24 11:34:19 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
86b3558b41
[Fix] Skip the DCP target-verify MLA kernel during FlashInfer autotune ( #41138 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-24 11:19:28 -07:00
78b382b1ad
Support unified memory decode host pools ( #39478 )
...
Co-authored-by: yhzhuang <yhzhuang@fb.com >
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com >
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
2026-09-24 11:02:15 -07:00
Trevor Morris
984994e3aa
fix: Triton 3.8 compatbility to support DSV4.1-Flash in CUDA 13.4 image (Rubin) ( #40805 )
2026-09-24 10:25:35 -07:00
Jackey Hua and Claude Fable 5.1
5bb24e399d
[Quant] ModelOpt mixed precision: dispatch block-FP8 MoE experts and derive the block size ( #38726 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-24 12:30:01 -04:00
Xinyu Jiang and Zhiyao Jiang
8ec65e89c1
[AMD] ci: move the Miles ROCm 7.2 nightly build to 7.2.4 ( #40387 )
...
Co-authored-by: Zhiyao Jiang <jessicajiang324@gmail.com >
2026-09-25 00:09:19 +08:00
Shangming Cai and Claude Fable 5
77173ffb08
[HiCache] Demote SWA KV to host on write_back eviction instead of dropping it ( #40712 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-09-24 23:56:17 +08:00
Ting SUN
752801e4d1
fix(sampling): validate sampling_seed is an int within int64 range ( #28960 )
...
Signed-off-by: Ting Sun <suntcrick@gmail.com >
2026-09-24 17:25:13 +02:00
17a7484c06
[chore] surface the cookbook to users who pip install sglang ( #40866 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-24 22:56:41 +08:00
Shangming Cai and Claude Opus 5
cd11037009
[PD] Enable deferred decode-side KV release by default ( #41023 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-24 22:35:11 +08:00
黄孝君
4142235c2b
[NPU] Update CANN version to 9.1.0 ( #40524 )
2026-09-24 21:35:31 +08:00
WenhaoZhang
ec7eb6bd79
[diffusion] update code owner ( #41130 )
2026-09-24 21:31:50 +08:00
Xiaoyu Zhang
ea5baf4022
[Refactor] Retire the model-specific Kimi K3 kernel namespace ( #40922 )
2026-09-24 21:25:00 +08:00
paulzhang-tm
28ec6700da
[HiCache] Make host reclamation independent of transfer order ( #40512 )
2026-09-24 21:03:25 +08:00
3177d10ca6
Refactor the Cute-DSL AR fusion to support DeepseekV2 archs (GLM-5.3, etc.) ( #39816 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com >
2026-09-24 06:00:58 -07:00
ronnie_zheng
ce06a14444
[Diffusion] migrate the whole _register_configs from registry.py to the model own config file ( #40612 )
2026-09-24 19:29:49 +08:00
Xiaoyu Zhang
c6565c1a6d
[Diffusion] Enable lossless SANA-Video eager conv fusions for 12.6% lower latency ( #40388 )
2026-09-24 19:21:32 +08:00
Alan Kao
a69583f70f
[AMD] Add tuned dsv4 shape ( #40996 )
2026-09-24 04:06:40 -07:00
AMD-yanfeiwang
0a59830114
[AMD][DSV4] fp8 unified_kv decode: wave-aware split count past 40 tokens ( #40878 )
...
Signed-off-by: Yanfei Wang <yanfwang@crsuse2-m2m-v2-017.us-east2-a.compute.internal >
2026-09-24 03:41:23 -07:00
Xiaoyu Zhang
26c63335e2
[Diffusion] Fuse lossless LingBot World FP32 normalization ( #40425 )
2026-09-24 18:41:10 +08:00
Zhang, Jiejing
d94d784441
[AMD] Tune Triton sparse MLA on gfx950 and make split-K workspaces graph-safe ( #39059 )
2026-09-24 03:21:53 -07:00
Mohammad Miadh Angkad
bb1c98baa2
Fix TBO child batch missing dp_spec_prefill_coordination_applied ( #41096 )
2026-09-24 02:44:55 -07:00
Xiaoyu Zhang
c85df5becc
[Diffusion] Fuse lossless Wan VAE post-ops for LongLive 2 I2V ( #40405 )
2026-09-24 17:40:07 +08:00
Xiaoyu Zhang
261cb826e6
[Diffusion] Fuse LongCat GELU+cat and support Edit-Turbo BCG ( #40384 )
2026-09-24 17:27:56 +08:00
Mick and Mick Qian
174a5f37b9
[feature] add per-item candidate token scoring and calibration ( #40826 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-09-24 17:00:13 +08:00
Cheng Xin
41812afe43
[NPU] Fix DSV4 hard-coding kv dtype ( #40510 )
2026-09-24 16:53:44 +08:00
Yihao Wang
81f2b43ab5
[ci] pr-gate: add generic require-label input and support pull_request_target ( #40999 )
2026-09-24 01:04:29 -07:00
pllimax
d65503fece
[NPU] Remove the LLaDA2.0-mini basic-function test case ( #40905 )
2026-09-24 15:58:48 +08:00
Yuwei An
1417345f5f
[Experimental] Preserve speculative decoding during prefill across DP ranks ( #40118 )
2026-09-24 00:01:12 -07:00
Liangsheng Yin
cae4927fbf
[DSV4] Budget the ratio-2 pair state pool in DSV4PoolConfigurator ( #41048 )
2026-09-23 23:53:46 -07:00
Michael
a164c6d9f1
[AMD] Register mem-cache unit tests in PR CI ( #40812 )
2026-09-23 23:35:51 -07:00
YC Yen-Ching Tseng and Bingxu Chen
43af9fcbb6
[AMD][DI][CI] Move MI355X disagg nightly to ROCm 10 ( #41053 )
...
Co-authored-by: Bingxu Chen <Bingxu.Chen@amd.com >
2026-09-24 14:24:39 +08:00
YC Yen-Ching Tseng
9f6fc55332
[AMD][Diffusion] FlyDSL fused norm kernels on wave32 targets (gfx1250) ( #37751 )
2026-09-23 23:06:55 -07:00
32290dda2c
[AMD] Small-M MXFP4 fused-MoE kernel for gfx950 (Qwen) ( #40204 )
...
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-09-23 22:45:33 -07:00
Shuwen Wang
90663ccb41
[DSV4] fix: size the C4 state ring by the page it is addressed by ( #40337 )
2026-09-23 22:05:32 -07:00
cctry and cctry
efff836911
[PD] Add decode host receive for custom transfer backends ( #40238 )
...
Co-authored-by: cctry <cctry@fb.com >
2026-09-23 21:57:33 -07:00
Theresa Shan
7a9feacf91
[Docs] DeepSeek-V4 MI355X Pro Official PD pairs with DSpark and UMBP ( #41024 )
2026-09-23 21:48:27 -07:00
fanxingran and Thomas Wang
0dc8b29e15
[ROCm][DSA] Enable AITER fused FP8 indexer writer ( #40710 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
2026-09-23 21:46:35 -07:00
Nan Jiang
c3685dff42
[Sampling] Add selected/support sampling logprob modes ( #40932 )
2026-09-23 21:38:19 -07:00
Zhaoyi Li
8eedf616f5
[AMD][DI][CI] Say which image the MI355X nightly ran on ( #41026 )
2026-09-24 12:18:03 +08:00
Jimmy Shong
be495a6174
[Fix] Use cached prefix lengths for FlashInfer full-attention ragged prefill ( #40796 )
2026-09-23 17:23:25 -10:00
jacky.cheng
e2f4fedf04
[AMD] Critical fix enabling Qwen3.8 FP8: restore dropped fused shared-expert weights ( #40754 )
2026-09-23 20:21:33 -07:00
Shijin Zhang and Brayden Zhong
77983865d8
[Moe] Honor swiglu_limit clamped activation in flashinfer_cutlass runner ( #39939 )
...
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca >
2026-09-23 20:10:03 -07:00
Xia Weiwen and Copilot
5c44214d6a
[XPU] Qwen3.8-flash-next enablement ( #37213 )
...
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com >
2026-09-24 11:06:30 +08:00
f4d9d2e670
[RL] Add RL weight-update sessions and support updating spec draft runners ( #40777 )
...
Co-authored-by: Jiajun Li <guapisolo@gmail.com >
Co-authored-by: Nan Jiang <59716405+nanjiangwill@users.noreply.github.com >
Co-authored-by: maocheng23 <maocheng@berkeley.edu >
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai >
Co-authored-by: JD-ETH <jaedon.guo@gmail.com >
Co-authored-by: Letian Ruan <ruanlt420@gmail.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-09-23 20:04:57 -07:00
Yueming Yuan and zyzshishui
f4b90382a2
[RL] Keep DSA cuda-graph state and the graph pool intact across TMS pause/resume ( #40804 )
...
Co-authored-by: zyzshishui <zyzshishui@gmail.com >
2026-09-23 19:55:26 -07:00
Liangsheng Yin
cc6e8b0f25
[Fix] Derive per-runner hybrid SWA layer ids on ModelLayerInfo instead of mutating ModelConfig ( #40983 )
2026-09-23 19:52:55 -07:00
jacky.cheng
e98b2f7538
[AMD] Speed up Wan2.2 DiT FP8 attention per-tensor quantization ( #34695 )
2026-09-23 19:49:26 -07:00
jacky.cheng
8b5d77c268
[AMD] Add a Triton packed sparse decode path for QSA on ROCm ( #38876 )
2026-09-23 18:59:17 -07:00
amote-i
82cdd72588
[NPU] [DOC] Remove --enforce-shared-experts-fusion from npu docs ( #41000 )
2026-09-24 09:54:29 +08:00
hanwlax
03fcbe147f
[NPU] Support batch invariant FIA graphs for deterministic inference ( #39607 )
2026-09-24 09:53:17 +08:00
Wang, FangYuan
81b81664a6
Revert "[AMD] Fix DeepSeek-V4 accuracy by not passing num_token_non_padded to MoE topk" ( #40946 )
2026-09-23 18:38:19 -07:00
Liangsheng Yin
621136e094
[mem_cache] Remove unused helpers in mem_cache, storage backends, and metrics ( #40963 )
2026-09-23 18:16:44 -07:00
Liangsheng Yin
81cb895b85
[Fix] Recover from stale torch extension locks in every cpp_extension loader ( #40989 )
2026-09-23 17:53:57 -07:00
ec75d3d30f
MiniMax-M3: allocate the lightning-indexer K cache in fp8 on gfx95 ( #36549 )
...
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Kevin Mi <mikevin920@gmail.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai >
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com >
2026-09-23 17:52:02 -07:00
zijiexia and Claude Opus 5.5
ffac53d779
[Doc] Add H200 recipes to MiMo-V2.6 cookbook ( #40969 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-23 16:49:35 -07:00
Yueming Yuan
542a043f1b
[RL] Release the weight-checker snapshot once compare passes ( #37284 )
2026-09-23 16:06:02 -07:00
James Liu and Claude Opus 5.5
208f6f7501
[Spec] Support DFLASH for Kimi K3 ( #40794 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-09-23 15:58:39 -07:00
Alison Shao
3f697899a3
ci: reinstall torch/triton left incomplete by a cancelled job ( #40819 )
2026-09-23 15:55:02 -07:00
Cheng Wan
0010f56a0d
[Test] Remove obsolete configuration migration guards ( #40976 )
2026-09-23 15:33:54 -07:00
6fe4b66f5c
[RL] Keep pause_generation and weight updates from deadlocking each other ( #40779 )
...
Co-authored-by: maocheng23 <maocheng@berkeley.edu >
Co-authored-by: Zhichenzzz <zczeng@uw.edu >
2026-09-23 15:13:42 -07:00
Cheng Wan
79fec592a8
[Refactor] Read parallel placement in consumers ( #40638 )
2026-09-23 14:57:25 -07:00
f2a1366584
MiniMax-M3: wave64 histogram-select decode top-k, and raise kMaxNumBlocks for CUDA graphs ( #36560 )
...
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Kevin Mi <mikevin920@gmail.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com >
2026-09-23 14:50:13 -07:00
4fa2c9c1de
MiniMax-M3: run the sparse prefill main attention through AITER Gluon paged attention ( #36546 )
...
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Alex Sun <alex.s@amd.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
Co-authored-by: Kevin Mi <mikevin920@gmail.com >
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com >
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai >
2026-09-23 14:49:10 -07:00
Cheng Wan
f766397f2e
[Refactor] Trim server configuration and runtime context comments ( #40971 )
2026-09-23 14:47:13 -07:00
Liangsheng Yin
c39a1c0663
[mem_cache] Free the rows below the SWA evict floor on all-SWA request release ( #40798 )
2026-09-23 14:46:28 -07:00
Cheng Wan
ab0c31bd94
[Refactor] Move eleven more MoE models to LayerCommunicator.ffn_exit ( #40871 )
2026-09-23 14:43:41 -07:00
Cheng Wan
e1048a215c
[Refactor] Let LayerCommunicator own the FFN exit in Qwen3-MoE, DeepSeek-V2 and GLM4-MoE ( #40870 )
2026-09-23 14:42:58 -07:00
Cheng Wan
7dd9640752
[Refactor] Compare token layouts instead of group sizes when selecting communicator paths ( #40869 )
2026-09-23 14:41:55 -07:00
Cheng Wan
e31cbc01a2
[Fix] Stop deferring the last layer's FFN all-reduce in five models ( #40868 )
2026-09-23 14:41:14 -07:00
Cheng Wan
701cf7e47e
[Refactor] Run Nemotron-H DP attention through the standard layer communicator ( #40867 )
2026-09-23 14:40:33 -07:00
Cheng Wan
379e8f916b
[Fix] Capture complete Nemotron auxiliary hidden states ( #40801 )
2026-09-23 14:39:50 -07:00
Cheng Wan
9ebe42238b
[Fix] Reduce Nemotron MTP attention outputs once ( #40800 )
2026-09-23 14:39:04 -07:00
Cheng Wan
0e80c73c92
[Fix] Avoid duplicate residual in LongCat MoE shortcut ( #40799 )
2026-09-23 14:37:39 -07:00
Liangsheng Yin
954458567e
[mem_cache] Remove unreachable RadixCache paths in KV canary and HiCache accessors ( #40807 )
2026-09-23 13:22:58 -07:00
2909f84e3b
[JIT] Add an occupancy-preserving L1 carveout preference ( #40767 )
...
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com >
Co-authored-by: Oguz Ulgen <oulgen7@gmail.com >
2026-09-23 11:49:34 -07:00
890a9605ce
[Metrics] Log forward and forward+idle occupancy over total wall time ( #40802 )
...
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com >
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com >
2026-09-23 11:48:31 -07:00
4bb5611dba
[Fix] Give the full prefill CUDA graph replay view the captured bucket's input_ids ( #40851 )
...
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com >
Co-authored-by: fei-xx <135589532+fei-xx@users.noreply.github.com >
2026-09-23 11:48:03 -07:00
f2f223e697
Allow attention layers to opt out of the prefill wrapper ( #40683 )
...
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com >
Co-authored-by: metamergebot <324680979+metamergebot@users.noreply.github.com >
2026-09-23 11:39:22 -07:00
Byron Hsu
f2eebd5533
[RL] Fix Kimi K3 expert-count lookup for routed-expert capture ( #40700 )
2026-09-23 11:18:04 -07:00
7fef014d51
feat(kv-hints): add kv hint envelope to request transport ( #38891 )
...
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com >
2026-09-23 10:31:13 -07:00
Even Zhou
5c154c214d
Revert " [NPU] Enable piecewise CUDA graph support on NPU" ( #40895 )
2026-09-23 23:38:14 +08:00
Xiaoyu Zhang
c3e985203a
[Diffusion] Remove unused standalone benchmarks and deduplicate kernel tests ( #40924 )
2026-09-23 22:42:03 +08:00
Xiaoyu Zhang
8993f790be
[Diffusion] Fuse lossless SenseNova RoPE for 5% faster H200 inference ( #40374 )
2026-09-23 22:37:22 +08:00
Xiaoyu Zhang
4e60d70ba4
[Diffusion] Fuse Joy Image Edit QKV concatenation and avoid QK copies ( #40494 )
2026-09-23 22:20:14 +08:00
Xiaoyu Zhang
172b1b4825
[Diffusion] Accelerate Cosmos3 Edge on Hopper with lossless fusions ( #40386 )
2026-09-23 21:16:58 +08:00
iridiumine
16e353d93e
[NPU] Skip fused gmm1+swiglu for swiglu_limit (SiLU-with-clamp) checkpoints ( #40438 )
2026-09-23 21:02:54 +08:00
iridiumine
3fdd63a562
[NPU] Fuse FIA KV-cache K/V writes into one npu_scatter_pa_kv_cache call ( #40445 )
2026-09-23 21:02:39 +08:00
Xiaoyu Zhang
abef3efb64
[Diffusion] Fuse rounded SwiGLU for quantized MiniMax-H3 MLPs ( #40378 )
2026-09-23 19:55:48 +08:00
hanwlax
2d25767759
[NPU] Enable piecewise CUDA graph support on NPU ( #28417 )
2026-09-23 17:30:30 +08:00
inkcherry and Duyi-Wang
58f622ebc4
Reduce decode bootstrap latency with request-owned speculative KV ( #38978 )
...
Signed-off-by: inkcherry <mingzhi.liu@amd.com >
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-09-23 02:24:53 -07:00
TianDi101 and Duyi-Wang
4cd63da996
[Unified Cache] Dedup replicated MLA/DSA KV in the UMBP direct linker ( #38778 )
...
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-09-23 02:15:01 -07:00
a89f849158
[ROCm] Fuse the MLA q absorb into the RoPE + KV-write kernel on gfx950 ( #38340 )
...
Co-authored-by: JohnQinAMD <zhengsheng1997@gmail.com >
Co-authored-by: RuibinCheung <ruibzhan@amd.com >
2026-09-23 02:07:54 -07:00
siliangchen-amd
401d5aedf3
[AMD] Drop the unreachable vLLM fallback from ROCm FP8 activation quant ( #40879 )
2026-09-23 01:59:15 -07:00
Yoray Zack
de123f38bb
[3/N] elastic-ep: Recapture decode CUDA graphs after scale-up ( #33723 )
2026-09-23 01:51:30 -07:00
aa0b65c7c7
[AMD] Fix DeepSeek-V4 accuracy by not passing num_token_non_padded to MoE topk ( #39804 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: HAI <hixiao@gmail.com >
2026-09-23 01:35:56 -07:00
RuibinCheung and xiaobochen-amd
48c3854620
[ROCm] feat: enable aiter allreduce fusion for GLM models ( #39790 )
...
Co-authored-by: xiaobochen-amd <xiaobo.chen@amd.com >
2026-09-23 01:24:08 -07:00
MatsueYu
d4dcce12d4
[npu] decoding procedure optimization on qwen3.5/3.6 ( #35958 )
2026-09-23 15:23:51 +08:00
Ziru and Niu Ziru
0b0f947265
[CPU] Add fused_sigmod_mul_cpu operators to the Meta Muse Glimmer model. ( #39538 )
...
Co-authored-by: Niu Ziru <niuziru@a4bf018d3341.jf.intel.com >
2026-09-23 15:12:43 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
86cb4a0f97
[CI] Update GLM-5.3-Flash H200/B200 test args ( #40862 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-22 23:21:04 -07:00
Polisetty V R K Jyothendra Varma
abca3b2e52
[Intel GPU] Add DeepSeek-V2-Lite-Chat-FP8 gsm8k e2e accuracy nightly test on XPU ( #39781 )
...
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com >
2026-09-23 14:11:30 +08:00
ANSHUMAN TRIPATHY
eb5e8c84be
Speculative Decoding with NGRAM support for XPU ( #31362 )
2026-09-23 14:04:20 +08:00
Zhaoyi Li
aa0feca536
[AMD][DI][CI] Use a node-local model cache on the SPUR cluster ( #40813 )
2026-09-22 23:01:55 -07:00
Shuwen Wang
66ce8c55cc
[HiCache] ci: add HiCache and unified radix rerun group ( #40831 )
2026-09-23 13:56:24 +08:00
a3d11f9d95
[diffusion] feat: support permanent lifetime for layerwise resident layers ( #40599 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-23 13:49:09 +08:00
28be39f72e
[deepep_v2] support GLM-5.3-Flash (Glm5NextForConditionalGeneration) ( #40466 )
...
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-22 22:32:34 -07:00
jacky.cheng and Eddie A. Tsai
58988bee2f
[AMD] Add diffusion (Wan2.2) extras to gfx1151 Docker image ( #40844 )
...
Co-authored-by: Eddie A. Tsai <56776714+vvvdwbvvv@users.noreply.github.com >
2026-09-22 22:18:55 -07:00
Jeff Hanson and Jeff Hanson
5b8d8b2f88
[Fix] Keep Inkling automatic tool grammar active across the response ( #40468 )
...
Co-authored-by: Jeff Hanson <2469458+jshanson7@users.noreply.github.com >
2026-09-23 13:14:36 +08:00
Shuwen Wang
ced0a2c98c
[Hisparse] fix: account for MiniMax HiSparse full-pool memory ( #40743 )
2026-09-23 13:13:48 +08:00
Eric Zhang
9d267655e3
Fix NIXL transfer of MXFP8 KV block scales ( #40792 )
2026-09-23 13:12:04 +08:00
973fb44471
[Diffusion] Support MiniMax-H3 PDD(Parallel Decoding Distillation) inference ( #40568 )
...
Co-authored-by: Forrest Lin <forrestl@nvidia.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-23 12:57:26 +08:00
Shuwen Wang
1d59ce7c90
fix: partition selective CI reruns into matrix jobs ( #40760 )
2026-09-22 20:37:02 -07:00
zhaozx-cn
525f14040d
[NPU]fix ci hicache oom ( #40824 )
2026-09-23 11:16:50 +08:00
Shuwen Wang
224a247d5b
[HiSparse] ci: add cross-directory rerun test group ( #40749 )
2026-09-23 11:07:03 +08:00
c2bc456b64
[chore] point agents at the cookbook before test configs ( #40818 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-23 10:54:15 +08:00
Shuwen Wang
2bc435d32e
[HiCache] fix: bound the controller reset join so a stalled storage thread cannot hang the scheduler ( #40312 )
2026-09-23 10:52:03 +08:00
Kan Wu and Claude Fable 5.1
2e3568567b
[sgl-router] Launch reorg routing with existing policy options ( #40766 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-23 10:18:11 +08:00
Richard Wang
40048f6e51
[dLLM] feat: support DiffusionGemma serving ( #34061 )
2026-09-23 09:59:31 +08:00
Liangsheng Yin
78980a3b0b
[misc] Remove deprecated endpoints, env vars and aliases past two releases ( #40795 )
2026-09-22 18:14:45 -07:00
Alison Shao
0cd8be351d
ci: stop Runner Utilization Report from draining the shared API quota ( #40791 )
2026-09-22 18:01:40 -07:00
Qi Sun and Xiaoyu Zhang
06008c170d
[dsv4.1]Optimize FP4 indexer by skipping invisible tiles ( #40431 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-09-23 07:48:43 +08:00
Liangsheng Yin
4ce23542bf
[HiCache] Remove the unused HiRadixCache ( #40787 )
2026-09-22 16:17:12 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
7b977ce5dc
[Fix] Decide the MoE padded-row bound from the layer scatter mode ( #40672 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-09-22 16:02:17 -07:00
Liangsheng Yin
d34f7b2326
[mem_cache] Clean up SWA/Mamba radix cache leftovers and drop SGLANG_ENABLE_UNIFIED_RADIX_TREE ( #40780 )
2026-09-22 15:40:57 -07:00
19ee4b566b
fix(function_call): buffer complete DeepSeek DSML invokes ( #39632 )
...
Co-authored-by: Xinyuan Tong <justintong0323@gmail.com >
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
2026-09-22 15:38:44 -07:00
Jialin Ouyang
b77833c504
[MM] Keep scheduler padding in packed token arrays ( #40357 )
2026-09-22 15:12:07 -07:00
c19dc43cc7
[AMD] [GLM-5.3-Flash Day 0] Load the MXFP4 MTP draft layer ( #39779 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: CHANG <jacchang@crsuse2-slog-003.us-east2-a.compute.internal >
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
2026-09-22 15:07:51 -07:00
jain-ria
01275aab6b
fix(grpc): expose native response timeout as a server argument ( #40644 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-09-22 14:57:38 -07:00
Liangsheng Yin
9a53f75124
[mem_cache] Remove the experimental C++ radix tree ( #40775 )
2026-09-22 14:19:45 -07:00
Faradawn Yang
d72629e4ba
Add GB200/GB300 hardware to Qwen3.5 ( #40770 )
...
Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com >
2026-09-22 14:16:43 -07:00
sglang-bot and sglang-bot
c08ffef709
docs: sync LMSYS SGLang blog cards ( #40655 )
...
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com >
2026-09-22 21:14:07 +00:00
Xinyuan Tong and Zijie Xia
542c817bfa
[Docs] Fix benchmark table column overflow in cookbook deployment panel ( #40114 )
...
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai >
2026-09-22 14:07:47 -07:00
3afdde5f05
[AMD] [GLM-5.3-Flash Day 0] Enable the k-pool DSA indexer on gfx950 ( #39341 )
...
Co-authored-by: Yikai Zhang <ykzhang12@gmail.com >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
2026-09-22 13:14:30 -07:00
077c319914
[AMD] [GLM-5.3-Flash Day 0] Enable speculative decoding (MTP) on ROCm ( #39778 )
...
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com >
2026-09-22 13:10:39 -07:00
d6cc283da7
[AMD] [GLM-5.3-Flash Day 0] Enable zero-RoPE TileLang DSA on gfx950 ( #38547 )
...
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com >
Co-authored-by: Thomas Wang <thomawan@amd.com >
Co-authored-by: Kevin Mi <mikevin920@yahoo.com >
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai >
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-22 13:08:25 -07:00
Yuwei An
4cbf290fb9
[Fix] Handle chunked paged MQA metadata in DSV4.1 eager forwards ( #40637 )
2026-09-22 12:59:04 -07:00
720617bb5b
[AMD] Reuse KV gather indices across ASM context prefill layers ( #39901 )
...
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-09-22 12:08:05 -07:00
5c18780c31
[Router] Keep e2e workers inside the job's CUDA_VISIBLE_DEVICES allotment ( #40686 )
...
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai >
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-22 11:32:12 -07:00
Cheng Wan
db73f35f4a
Fix the Inkling per-expert sync test and collect it in the weekly CPU run ( #40725 )
2026-09-22 11:22:07 -07:00
Shangming Cai
6f4c2b9b91
[HiCache] Demote internal-node mamba states on write_back eviction ( #40680 )
2026-09-23 02:18:58 +08:00
Cheng Wan
91c329cc86
Take the model config out of the parallel group build, and finish retiring the parallel getters ( #40707 )
2026-09-22 10:55:21 -07:00
Pei Li
66454d704b
[Feature] Support --tokenizer-worker-num > 1 in the offline Engine API ( #40260 )
2026-09-22 10:50:01 -07:00
Sage
b4701d672d
[rust-renderer] decouple renderer sampling from protocols ( #40747 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-22 10:48:28 -07:00
Sage
8ab21c8a94
[ci] publish renderer image ( #40639 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-22 09:48:12 -07:00
Sage
a7b96fdad7
[ci] run cpu ci for renderer-only changes ( #40636 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-22 09:47:48 -07:00
YAMY and zhangxiaolei
6fd98c98b9
[Qwen3.8-Next] Pipeline-parallel serving and PD-prefill MTP for Qwen4-Exp ( #40501 )
...
Co-authored-by: zhangxiaolei <zhangxiaolei.666@bytedance.com >
2026-09-22 08:46:44 -07:00
c79510cc2a
[DSv4.1] Score prefill consumer index layers on candidate blocks with DeepGEMM ( #40352 )
...
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com >
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com >
2026-09-22 22:06:06 +08:00
4a1b69abc8
[diffusion] docs: correct the resident-layer help text to match its scope ( #40593 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-22 20:32:59 +08:00
pllimax
244db08d60
[NPU][CI] Constrain evalscope dependency versions ( #40133 )
2026-09-22 20:15:11 +08:00
Wookje Han
bc3a63ebdb
[Fix] Missing SWA eviction during decode preallocation ( #40309 )
2026-09-22 20:11:40 +08:00
Shunkangz
ddebc52f23
[kv-shard 3/4] Enable Control plane ( #38468 )
2026-09-22 19:57:44 +08:00
2524b61f9b
[diffusion] feat: allow a component use retain its layerwise resident set ( #40592 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-22 19:56:21 +08:00
Shangming Cai
861b11f087
[PD] Simplify late-abort quiescent ack branch to else ( #40711 )
2026-09-22 19:08:47 +08:00
amote-i
cb8dab06be
[NPU] [DOC] Remove duplicated features in npu docs ( #40714 )
2026-09-22 19:04:17 +08:00
cctry
8ef6d31cb9
[PD] Preserve abort ACKs until in-flight KV transfers drain ( #40645 )
2026-09-22 18:46:07 +08:00
Emre Albayrak and Xiaoyu Zhang
9d58189c12
[diffusion] attention: add fp8_fa_sm120 FP8 backend for SM120 GPUs ( #40175 )
...
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-09-22 17:44:37 +08:00
debbb5cde9
[AMD] Drop the redundant scale zero-fill before AITER per-tensor FP8 quant ( #40557 )
...
Co-authored-by: siliangchen-amd <{"message":"Although you appear to have the correct authorization credentials, the `amd-eng-emu` enterprise has an IP allow list enabled, and your IP address is not permitted to access this resource.","documentation_url":"https://docs.github.com/rest/users/users#get-the-authenticated-user ","status":"403"}siliangchen-amd@users.noreply.github.com >
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com >
2026-09-22 17:43:31 +08:00
Michael
3f00fb7e2e
[AMD] Register unified KV page-zeroing test in PR CI ( #40123 )
2026-09-22 02:16:31 -07:00
Zhaoyi Li
70a2bb2a30
[AMD][DI] Keep loopback in UCX_NET_DEVICES ( #40641 )
2026-09-22 02:15:18 -07:00
jacky.cheng and chuyeh
790551c382
[AMD] Skip full-vocab softmax in EAGLE topk==1 draft on ROCm ( #35872 )
...
Co-authored-by: chuyeh <chuyeh@users.noreply.github.com >
2026-09-22 01:57:37 -07:00
chuyeh and jacky.cheng
c2f14bfbc4
[AMD] Use exact CU share for gfx950 segment-plan headroom ( #39503 )
...
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-09-22 01:49:59 -07:00
inkcherry
367e3700cf
avoid host sync in DSpark prefill slot expansion ( #40111 )
2026-09-22 01:20:49 -07:00
d00adfd19c
[AMD] Fix deferred Kimi-K3 forget gate in fused in-projection ( #39525 )
...
Co-authored-by: Tom Jen <tomjen12@localhost.localdomain >
Co-authored-by: HAI <hixiao@gmail.com >
2026-09-22 01:07:25 -07:00
6412ad8c64
[AMD] Pack Qwen3.5 GDN input projections on ROCm ( #39902 )
...
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
2026-09-22 00:52:40 -07:00
Po-Han Huang (NVIDIA)
771c9d782d
[PD] Validate Mooncake EFA allocator compatibility ( #39973 )
2026-09-22 15:12:29 +08:00
Suraj
1815e490de
[observability] Fix negative queue_time for retracted requests ( #39312 )
2026-09-22 15:08:01 +08:00
jshn9515
c948114d58
[DOC] Update quickstart guide to use sglang serve for launching the server ( #40647 )
2026-09-22 15:00:35 +08:00