Commit Graph
18221 Commits
Author SHA1 Message Date
Aurick QiaoandAurick Qiao 2123aca87e [Fix] Wait for PDL before reading DeepSeek V4 K cache locations (#38409)
Co-authored-by: Aurick Qiao <6137920+aurickq@users.noreply.github.com>
2026-09-14 21:04:28 +08:00
iridiumine 87db743021 [NPU] Fix device mismatch in SWA mask for DSpark verify graph capture (#39353) 2026-09-14 19:36:33 +08:00
Liangsheng Yin 66c7bc838e [misc] Revert #38346, #33426, #39061 and #39219 (#39405) 2026-09-14 03:04:07 -07:00
amd-danli103 5aa9b8fb3e [AMD][DSV4] feat: enable fp8 two-pool unified_kv on gfx950 (#37413) 2026-09-14 02:49:11 -07:00
Theresa Shan 95140a7b0c [docs] DeepSeek-V4: MI355X PD disaggregation recipes for all three strategies (#39396) 2026-09-14 02:11:51 -07:00
Jacob0226andThomas Wang 5200508b0f [AMD][gfx95] Fill the chunked-prefill compute budget exactly (#32888)
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-09-14 00:35:15 -07:00
3eeb7d37f9 [AMD] gfx950 assembly attention: length-aware split-KV for dynamic workload (#39172)
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-09-13 23:48:03 -07:00
ZeyuanChen2000 a4781c9fe5 [NPU] Fix error due to missing parameter quant_linear passing (#37384) 2026-09-14 14:37:44 +08:00
YC Yen-Ching Tseng ce66ba2844 [AMD] Fix AITER FP8-Q unified-attention Test (#39360) 2026-09-13 23:35:24 -07:00
Bingxu Chen edf9584be9 [AMD][CI] Disable Wave attention test file on ROCm 10 (#39356) 2026-09-13 23:34:18 -07:00
Shuwen Wang 0d95a9c1ff [HiCache] Keep the file backend temp file name within NAME_MAX (#38925) 2026-09-14 05:12:28 +00:00
jacky.cheng 2f5cc8e33e [AMD] Align Qwen3.5 MI355X cookbook with AttnFP8-V2 and HiCache direct / page_first_direct (#39358) 2026-09-14 12:46:17 +08:00
Mohammad Miadh Angkadandmmangkad 2fd835b9c1 [Fix] Don't write conv state from the fused KDA verify kernel (#39219)
Co-authored-by: mmangkad <mohammad.angkad@radixark.ai>
2026-09-13 21:26:57 -07:00
Jan Bernlöhr 60f6f03409 Fix MUSA detection in compiled prefill path (#39061) 2026-09-13 21:23:51 -07:00
Kurt ShusterandBaizhou Zhang f2111715cd [fa] Make the FlashAttention backend extensible by subclasses (#33426)
Signed-off-by: Kurt Shuster <kurt@thinkingmachines.ai>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2026-09-13 21:21:25 -07:00
39e147443b [Session] Fix image append positions and parent metadata (#39145)
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: Manik Singhal <manikvsinghal.pub@gmail.com>
2026-09-13 21:06:47 -07:00
Jackey Hua 5a132c061b [Perf] trtllm_mla: reuse the fused fp8 KV/Q prepare on target verify (#39232) 2026-09-13 20:59:00 -07:00
42b5af8c62 [diffusion] docs: refresh the VDN-H3 on b200 numbers in cookbook (#39244)
Co-authored-by: haochengxi <xihc@berkeley.edu>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
2026-09-14 11:50:40 +08:00
sglang-botandsglang-bot 96a95171c7 chore: bump sglang-kernel version to 0.4.7 (#39346)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-09-13 20:49:37 -07:00
MickandMick Qian 3480112a9c [diffusion] fix: serve requests that turn cfg off on a cfg-parallel server (#39303)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-14 11:26:37 +08:00
4c71d14fba [HiCache][LoRA] Simplify decode offload hash inputs (#39162)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
Co-authored-by: alphabetc1 <2508695655@qq.com>
2026-09-14 11:23:31 +08:00
Kan Wu f539c1fc65 [sgl-router] Stream outcome observability for 2xx SSE streams (#38737) 2026-09-13 19:46:44 -07:00
Chao ShiandZhangheng bf9773e1da HiCache: Add @rank_consensus to various functions (#37425)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-09-14 10:44:21 +08:00
Cheng Xin ca8ecc6a6f [NPU] Support DSV4 host memory cache management (#37382) 2026-09-14 10:38:50 +08:00
Shuwen Wang 7b4972a9cb chore: add hzh0425 and xiezhq-hermann as Rust tree and HiSparse code owners (#39337) 2026-09-14 10:34:55 +08:00
Cheng Wan 6410800af9 [unified-memory] Enable prefill cuda-graph capture (#37418) 2026-09-13 19:17:19 -07:00
Cheng Wan 2ec4bbcbd4 [unified-memory] PD disaggregation for every unified pool shape (#37506) 2026-09-13 19:16:43 -07:00
Kurkur 6388b6cfb1 [feat] Add base NpuSRTPlatform implementation (#36472) 2026-09-14 09:32:24 +08:00
sglang-botandsglang-bot 4358a1617c chore: bump sgl-kernel version to 0.4.7 (#39324)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-09-13 15:48:50 -07:00
Baizhou ZhangandChunan Zeng 5ebb16005d [DeepSeek-V4.1] Bump FlashMLA to the fork's rebase head (v4.1 kernels) (#39171)
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
2026-09-13 15:43:19 -07:00
Shuwen Wang ff228d11fe [HiCache] Release buffer prefetch anchor locks during storage cleanup (#38483) 2026-09-13 20:47:21 +00:00
ishandhanani ddc1df1203 fix(router): simplify bucket context limit check (#39259)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
2026-09-13 12:27:59 -07:00
6220f45d8e [PD] Add /v1/responses support to the HTTP PD router (#36141)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-13 22:48:13 +08:00
Shuwen WangandClaude Opus 5 7f1f8c706a [mem_cache][10/N] refactor: drop the redundant _component suffix in unified_cache/components (#35644)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 21:38:20 +08:00
Shangming CaiandXinyuan Tong 14a131ad5b [PD][OpenAI] Gate /v1/responses persistence behind --enable-response-store, default off (#39122)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-13 21:20:47 +08:00
f9fca05803 [diffusion] fix: make the sp sequence gather pass contiguous shards (#39291)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 20:30:52 +08:00
9ffe548738 [diffusion] fix: don't route an unreadable checkpoint into the native fallback (#39292)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 20:30:24 +08:00
Shuwen Wang 14b647cf27 chore: add HiSparse coordinator and allocator code owners (#38682) 2026-09-13 18:42:33 +08:00
Hritik Raj d24eacaecd [Test] Add unit test for muse_glimmer_format (#37015)
Signed-off-by: Hritik003 <hritik.raj@nutanix.com>
2026-09-13 18:40:02 +08:00
Jundong LiuandShuwen Wang 7078e5ffbc [HiCache] Publish a host store event for storage-prefetch refills (#38486)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-09-13 10:29:14 +00:00
huangtingweiandhzh0425 a7cf4a6fbc [Unified Cache][7/N] Support MTP, EAGLE, and DSpark draft KV caches in the external linker (#37914)
Co-authored-by: hzh0425 <hzh0425@apache.org>
2026-09-13 18:22:06 +08:00
cebca698e2 [Qwen3.8] Enable NVIDIA NVFP4 on DGX Spark with file-backed PLE and PDL router fix (#39126)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: rdxa <rdxa@rdxa-int-spark-01.yvb.moe>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Manrique <nanomlm@gmail.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
2026-09-13 16:23:41 +08:00
jacky.chengandHAI d6fabb74b4 [AMD][Fix] Fix aiter bpreshuffle GEMM for output sizes it cannot dispatch for qwen3.5 mxfp-attn-fp8-v2 TP4 (#37564)
Co-authored-by: HAI <hixiao@gmail.com>
2026-09-12 23:11:44 -07:00
d2f054d916 [diffusion] CI: make diffusion GT generation usable without the publish token, and cover the 5090 lane (#39255)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 13:59:43 +08:00
492f346d88 [diffusion] CI: set an explicit x264 preset for video output (#38657)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 13:47:41 +08:00
William HuandXinyuan Tong 6d6d42d1c0 [Fix] Preserve GLM tool argument types across JSON Schema unions (#39136)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-13 13:47:13 +08:00
34b2904741 [Session] Work with PD and Fix empty continuations (#39038)
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: Manik Singhal <3400497+Manikvsin@users.noreply.github.com>
2026-09-12 22:24:32 -07:00
cctry 24b6c1c7f5 Fix multimodal embedding cache retaining full batches through views (#39120) 2026-09-12 21:37:14 -07:00
cctryandcctry 7e3d18bbcc Add a provider hook for prefill-buffer ceilings (#39182)
Co-authored-by: cctry <17473714+cctry@users.noreply.github.com>
2026-09-12 21:37:10 -07:00
Baizhou Zhang 7763f666f3 Fix DeepGEMM release MegaMoE validation without RDMA (#39245) 2026-09-12 20:27:29 -07:00