This website requires JavaScript.
6e755e4114
[Logging] Downgrade missing TokenizerManager request state log to warning (#36625 )
faceless void
2026-09-15 07:18:56 +08:00
0163f8ff74
[AMD] Fix registered HiCache host pointer aliases (#35233 )
AMD-yanfeiwang
2026-09-15 06:49:05 +08:00
2fca6d69aa
bumping sgl-deep-gemm to 0.2.0 (#39371 )
Baizhou Zhang
2026-09-14 15:30:31 -07:00
710a044caf
[dLLM] Add rwang5203 as code owner and grant CI permissions (#39483 )
Richard Wang and Richard Wang
2026-09-14 15:27:15 -07:00
dad8c074e7
Scope prefetch cache state to the request attempt (#39318 )
cctry
2026-09-14 15:03:03 -07:00
d72e59508b
Reland fix(qsa): clamp the compress gather to the rows (#38346 ) (#39446 )
Qiaolin Yu and ehuaa
2026-09-14 14:05:41 -07:00
9128d57966
[Spec] Allow speculative workers to stage prefill shared reads (#38554 )
paulzhang-tm
2026-09-14 17:03:27 -04:00
5c2de3f355
[PD] Preserve the prefill rank during rebootstrap (#39357 )
Byron Hsu and Byron Hsu
2026-09-14 09:20:37 -07:00
7465e42b7a
[NPU][CI] Add CANN 9.1.0 and Ascend a5 nightly suites (#38833 )
pllimax
2026-09-14 22:39:42 +08:00
433c999dd0
[NPU][CI] Fix sglang.test.ascend import failure in multi-node e2e pods (#39403 )
pllimax
2026-09-14 22:23:47 +08:00
242d8a70c0
[AMD] GLM-5.2 MI355X MXFP4: bump image to 20260913, enable TOPK_V2 (#39406 )
ChangLiu0709
2026-09-14 14:50:52 +01:00
d5f1c593c1
[NPU] Remove temperature/top_p from Qwen3.5-397B-A17B perf test (#39047 )
pllimax
2026-09-14 21:48:50 +08:00
f81fbc749a
[Fix] Merge adjacent KV-row frees so a mid-page split under DCP cannot double-free (#38941 )
Kurt Shuster
2026-09-14 09:44:32 -04:00
2123aca87e
[Fix] Wait for PDL before reading DeepSeek V4 K cache locations (#38409 )
Aurick Qiao and Aurick Qiao
2026-09-14 06:04:28 -07:00
87db743021
[NPU] Fix device mismatch in SWA mask for DSpark verify graph capture (#39353 )
iridiumine
2026-09-14 19:36:33 +08:00
66c7bc838e
[misc] Revert #38346 , #33426 , #39061 and #39219 (#39405 )
Liangsheng Yin
2026-09-14 03:04:07 -07:00
5aa9b8fb3e
[AMD][DSV4] feat: enable fp8 two-pool unified_kv on gfx950 (#37413 )
amd-danli103
2026-09-14 17:49:11 +08:00
95140a7b0c
[docs] DeepSeek-V4: MI355X PD disaggregation recipes for all three strategies (#39396 )
Theresa Shan
2026-09-14 17:11:51 +08:00
5200508b0f
[AMD][gfx95] Fill the chunked-prefill compute budget exactly (#32888 )
Jacob0226 and Thomas Wang
2026-09-14 15:35:15 +08:00
3eeb7d37f9
[AMD] gfx950 assembly attention: length-aware split-KV for dynamic workload (#39172 )
2026-09-13 23:48:03 -07:00
a4781c9fe5
[NPU] Fix error due to missing parameter quant_linear passing (#37384 )
ZeyuanChen2000
2026-09-14 14:37:44 +08:00
ce66ba2844
[AMD] Fix AITER FP8-Q unified-attention Test (#39360 )
YC Yen-Ching Tseng
2026-09-14 08:35:24 +02:00
edf9584be9
[AMD][CI] Disable Wave attention test file on ROCm 10 (#39356 )
Bingxu Chen
2026-09-14 14:34:18 +08:00
0d95a9c1ff
[HiCache] Keep the file backend temp file name within NAME_MAX (#38925 )
Shuwen Wang
2026-09-14 13:12:28 +08:00
2f5cc8e33e
[AMD] Align Qwen3.5 MI355X cookbook with AttnFP8-V2 and HiCache direct / page_first_direct (#39358 )
jacky.cheng
2026-09-14 12:46:17 +08:00
2fd835b9c1
[Fix] Don't write conv state from the fused KDA verify kernel (#39219 )
Mohammad Miadh Angkad and mmangkad
2026-09-14 12:26:57 +08:00
60f6f03409
Fix MUSA detection in compiled prefill path (#39061 )
Jan Bernlöhr
2026-09-14 06:23:51 +02:00
f2111715cd
[fa] Make the FlashAttention backend extensible by subclasses (#33426 )
Kurt Shuster and Baizhou Zhang
2026-09-14 00:21:25 -04:00
39e147443b
[Session] Fix image append positions and parent metadata (#39145 )
2026-09-13 21:06:47 -07:00
5a132c061b
[Perf] trtllm_mla: reuse the fused fp8 KV/Q prepare on target verify (#39232 )
Jackey Hua
2026-09-13 23:59:00 -04:00
42b5af8c62
[diffusion] docs: refresh the VDN-H3 on b200 numbers in cookbook (#39244 )
2026-09-13 20:50:40 -07:00
96a95171c7
chore: bump sglang-kernel version to 0.4.7 (#39346 )
sglang-bot and sglang-bot
2026-09-13 20:49:37 -07:00
3480112a9c
[diffusion] fix: serve requests that turn cfg off on a cfg-parallel server (#39303 )
Mick and Mick Qian
2026-09-14 11:26:37 +08:00
4c71d14fba
[HiCache][LoRA] Simplify decode offload hash inputs (#39162 )
2026-09-13 20:23:31 -07:00
f539c1fc65
[sgl-router] Stream outcome observability for 2xx SSE streams (#38737 )
Kan Wu
2026-09-13 19:46:44 -07:00
bf9773e1da
HiCache: Add @rank_consensus to various functions (#37425 )
Chao Shi and Zhangheng
2026-09-14 10:44:21 +08:00
ca8ecc6a6f
[NPU] Support DSV4 host memory cache management (#37382 )
Cheng Xin
2026-09-14 10:38:50 +08:00
7b4972a9cb
chore: add hzh0425 and xiezhq-hermann as Rust tree and HiSparse code owners (#39337 )
Shuwen Wang
2026-09-14 10:34:55 +08:00
6410800af9
[unified-memory] Enable prefill cuda-graph capture (#37418 )
Cheng Wan
2026-09-13 19:17:19 -07:00
2ec4bbcbd4
[unified-memory] PD disaggregation for every unified pool shape (#37506 )
Cheng Wan
2026-09-13 19:16:43 -07:00
6388b6cfb1
[feat] Add base NpuSRTPlatform implementation (#36472 )
Kurkur
2026-09-14 09:32:24 +08:00
4358a1617c
chore: bump sgl-kernel version to 0.4.7 (#39324 )
sglang-bot and sglang-bot
2026-09-13 15:48:50 -07:00
5ebb16005d
[DeepSeek-V4.1] Bump FlashMLA to the fork's rebase head (v4.1 kernels) (#39171 )
Baizhou Zhang and Chunan Zeng
2026-09-13 15:43:19 -07:00
ff228d11fe
[HiCache] Release buffer prefetch anchor locks during storage cleanup (#38483 )
Shuwen Wang
2026-09-14 04:47:21 +08:00
ddc1df1203
fix(router): simplify bucket context limit check (#39259 )
ishandhanani
2026-09-13 12:27:59 -07:00
6220f45d8e
[PD] Add /v1/responses support to the HTTP PD router (#36141 )
2026-09-13 23:48:13 +09:00
7f1f8c706a
[mem_cache][10/N] refactor: drop the redundant _component suffix in unified_cache/components (#35644 )
Shuwen Wang and Claude Opus 5
2026-09-13 21:38:20 +08:00
14a131ad5b
[PD][OpenAI] Gate /v1/responses persistence behind --enable-response-store, default off (#39122 )
Shangming Cai and Xinyuan Tong
2026-09-13 21:20:47 +08:00
f9fca05803
[diffusion] fix: make the sp sequence gather pass contiguous shards (#39291 )
2026-09-13 20:30:52 +08:00
9ffe548738
[diffusion] fix: don't route an unreadable checkpoint into the native fallback (#39292 )
2026-09-13 20:30:24 +08:00
14b647cf27
chore: add HiSparse coordinator and allocator code owners (#38682 )
Shuwen Wang
2026-09-13 18:42:33 +08:00
d24eacaecd
[Test] Add unit test for muse_glimmer_format (#37015 )
Hritik Raj
2026-09-13 16:10:02 +05:30
7078e5ffbc
[HiCache] Publish a host store event for storage-prefetch refills (#38486 )
Jundong Liu and Shuwen Wang
2026-09-13 18:29:14 +08:00
a7cf4a6fbc
[Unified Cache][7/N] Support MTP, EAGLE, and DSpark draft KV caches in the external linker (#37914 )
huangtingwei and hzh0425
2026-09-13 18:22:06 +08:00
cebca698e2
[Qwen3.8] Enable NVIDIA NVFP4 on DGX Spark with file-backed PLE and PDL router fix (#39126 )
2026-09-13 01:23:41 -07:00
d6fabb74b4
[AMD][Fix] Fix aiter bpreshuffle GEMM for output sizes it cannot dispatch for qwen3.5 mxfp-attn-fp8-v2 TP4 (#37564 )
jacky.cheng and HAI
2026-09-13 14:11:44 +08:00
d2f054d916
[diffusion] CI: make diffusion GT generation usable without the publish token, and cover the 5090 lane (#39255 )
2026-09-13 13:59:43 +08:00
492f346d88
[diffusion] CI: set an explicit x264 preset for video output (#38657 )
2026-09-13 13:47:41 +08:00
6d6d42d1c0
[Fix] Preserve GLM tool argument types across JSON Schema unions (#39136 )
William Hu and Xinyuan Tong
2026-09-13 01:47:13 -04:00
34b2904741
[Session] Work with PD and Fix empty continuations (#39038 )
2026-09-12 22:24:32 -07:00
24b6c1c7f5
Fix multimodal embedding cache retaining full batches through views (#39120 )
cctry
2026-09-12 21:37:14 -07:00
7e3d18bbcc
Add a provider hook for prefill-buffer ceilings (#39182 )
cctry and cctry
2026-09-12 21:37:10 -07:00
7763f666f3
Fix DeepGEMM release MegaMoE validation without RDMA (#39245 )
Baizhou Zhang
2026-09-12 20:27:29 -07:00
ec5fba5777
[AMD] Add dspark config and agentic workload section for deepseek-v4 model (#39252 )
Thomas Wang
2026-09-13 10:32:01 +08:00
206034e520
Keep graph-pool borrows on their allocation stream (#39180 )
cctry and cctry
2026-09-12 19:26:09 -07:00
18cc55dc0b
Expose a capacity check for graph-pool borrows (#39178 )
cctry and cctry
2026-09-12 19:24:57 -07:00
23bc4c6ed9
[Diffusion] Return Qwen-Image-Layered outputs and preserve CFG2 rounding (#38549 )
Xiaoyu Zhang and Mick Qian
2026-09-13 09:15:47 +08:00
3e035a3513
[Diffusion] Optimize Qwen-Image-Edit attention on Hopper (#38584 )
Xiaoyu Zhang and Mick Qian
2026-09-13 09:15:01 +08:00
fa663e7297
config: delete the redundant full stamp in initialize_model_parallel (#39202 )
Cheng Wan
2026-09-12 17:27:21 -07:00
fa260f26da
config: delete dead ensure_model_parallel_initialized (#39137 )
Cheng Wan
2026-09-12 17:27:12 -07:00
6804eeaabe
config: an out-of-tree replacement point for every resolution-pipeline step (#39134 )
Cheng Wan
2026-09-12 17:26:54 -07:00
a8b5616303
Fix DeepGEMM release dependencies and bound GPU validation (#39241 )
Baizhou Zhang
2026-09-12 17:00:58 -07:00
6657f7d844
[MoE][ROCm] Admit the unified Triton router on ROCm, including single-group routing (#38328 )
2026-09-13 07:21:28 +08:00
a66451c058
[GLM-5.3 Flash] Restore and enable KPool metadata fusion (#38845 )
2026-09-12 16:01:11 -07:00
288627e400
[AMD] Document GLM-5.2 MXFP4 recipe update on MI355X (#39230 )
Zhang, Jiejing
2026-09-12 15:57:15 -07:00
21289cfd50
[AMD] Fix Dspark accept length and reduce host bubble on DSV4 (#39116 )
Xinyi Song
2026-09-12 15:21:43 -07:00
b5a2aebc7e
[Docs] GLM-5.3-Flash cookbook: fixed MTP 5/1/6, EP1 + flashinfer_trtllm on Blackwell (#39213 )
Xinyuan Tong
2026-09-13 03:56:57 +08:00
7ae4af8187
Scope graph-pool borrowing to the runtime and reduce fragmentation (#39177 )
cctry and cctry
2026-09-12 11:45:17 -07:00
2784a86062
Reuse live CUDA graph executables during dedup registration (#39176 )
cctry and cctry
2026-09-12 11:44:17 -07:00
25a5641cf2
[Session + MM] Fix text positions in session continuations (#39144 )
2026-09-12 10:31:55 -07:00
ae1acf822d
[GraniteMoE] Load split per-expert quantized MoE weights (#37679 )
2026-09-12 11:10:20 -04:00
7b89b95168
[diffusion] feat: pick the attention backend by measuring it (#38689 )
2026-09-12 21:52:16 +08:00
dc3171c322
[diffusion] CI: remove mova ulysses two-gpu CI case (#39097 )
Mick and Mick Qian
2026-09-12 21:48:04 +08:00
925e684a88
[Feat][Responses API] Support custom tools, encrypted reasoning replay, developer tier and model validation (#38690 )
2026-09-12 21:29:12 +08:00
6dc7b3421b
Inference Support Mamba 2 and 1 (#34556 )
desmond-intel
2026-09-12 18:21:18 +05:30
fd32226706
Fix gpt-oss RunAI streamer weight ownership (#38908 )
Kunal and Claude Fable 5.1
2026-09-12 08:21:59 -04:00
bd45cd50ca
[Unified Tree] Port SWA Branching-Point Caching to the Rust TreeCore (#37584 )
Shuwen Wang
2026-09-12 18:37:57 +08:00
0b415fa573
[HiCache][LoRA] Isolate storage pages by extra key (#38577 )
Yanbin Jiang and Shuwen Wang
2026-09-12 03:18:14 -07:00
b9cb96496d
[Unified Tree] Preserve aux LRU recency when splitting nodes (#38482 )
Shuwen Wang
2026-09-12 18:17:38 +08:00
6953dae005
[Qwen 3.8 Next] Remove unused tokenwise QSA implementation and tests (#38960 )
Qiaolin Yu
2026-09-12 01:32:01 -07:00
7bc4eb3740
[AMD] Update MI355X MXFP4 HiCache defaults and quick-reduce quantization for Qwen3.5 cookbook (#39104 )
ChangLiu0709
2026-09-12 08:59:32 +01:00
7c195b9151
[AMD] Fix Quark load of MiniMax-M3 MXFP4 index_qkv_proj (#37254 )
Tuan Nguyen Gia
2026-09-12 10:50:41 +03:00
bf3305b65e
[diffusion] feat: read mapped layers directly when the host cannot cache them (#39022 )
Mick and Claude Opus 5
2026-09-12 15:48:03 +08:00
0a57403468
[Perf] Optimize Qwen3-VL unique-image serving on H100 (#36411 )
Xiaoyu Zhang and Cursor
2026-09-12 15:16:59 +08:00
6ba96d329f
[DCP] Resolve --dcp-comm-backend to fi_a2a/a2a by default for every model (#39165 )
Khoa Pham and Claude Fable 5.1
2026-09-11 23:48:42 -07:00
981b947568
[Fix] Seed raw tokenizer_path for smg-grpc-servicer in gRPC mode (#39105 )
Shangming Cai and Claude Fable 5
2026-09-12 14:41:15 +08:00
56a4f47ca6
[CI] Fix base-a wait for skipped CPU matrix (#39195 )
Baizhou Zhang
2026-09-11 23:39:53 -07:00
3bb2a5231a
[CI] Extend DeepGEMM SM90 test timeout to four hours (#39194 )
Baizhou Zhang
2026-09-11 23:36:54 -07:00
1cdc5bca5e
fix(qsa): clamp the compress gather to the rows this forward has (#38346 )
ehuaa
2026-09-12 14:24:17 +08:00
0d08668821
[Cookbook] Kimi-K3: keep DCP under HiCache L1+L2 with DSPARK (#39190 )
Khoa Pham and Claude Fable 5.1
2026-09-11 23:14:24 -07:00