This website requires JavaScript.
175973d834
[Diffusion] Fuse Qwen-Image residual norm and NVFP4 quantization (#37129 )
Xiaoyu Zhang and Cursor
2026-09-01 08:54:10 +08:00
079afaffb1
[Diffusion] Fuse FLUX.2 gated residual normalization on Blackwell (#37112 )
Xiaoyu Zhang and Cursor
2026-09-01 08:38:33 +08:00
6715debb2a
[Diffusion] Fuse Qwen-Image FP8 QKV projection and Blackwell epilogue (#37123 )
Xiaoyu Zhang and Cursor
2026-09-01 08:34:41 +08:00
07c8f7294d
Bump sgl-deep-gemm to 0.1.7 (#37279 )
Baizhou Zhang
2026-08-31 17:16:34 -07:00
455232de6e
[Cookbook] Enable DSpark on the DeepSeek-V4 Flash Vision low-latency recipes (#37301 )
zijiexia
2026-08-31 16:43:54 -07:00
ef9e58fd6d
feat(unified-memory): three sub-pools for mamba + hybrid-SWA models (#35177 )
2026-08-31 15:10:12 -07:00
98cb3535b7
feat(unified-memory): byte-budget sizing, feasibility floor, and a conservation verifier (#35158 )
2026-08-31 15:09:28 -07:00
961beee9e5
fix(unified-memory): four boot/correctness fixes on the hybrid model paths (#35154 )
2026-08-31 15:08:43 -07:00
88cf5c9541
[Cookbook] Add DeepSeek-V4-Flash-Vision-Exp to the DeepSeek-V4 page (#37293 )
zijiexia
2026-08-31 14:52:28 -07:00
2530204502
[mem_cache] Make release, row-reuse asserts, and presence checks read the KV record (#37167 )
Liangsheng Yin
2026-08-31 12:46:15 -07:00
95f0f41021
[CI] Move tests onto the right CI stages (#34074 )
Liangsheng Yin and Baizhou Zhang
2026-08-31 12:39:59 -07:00
579270d459
[Rust] Simplify request defaults and document batch header ABI (#37226 )
Lianmin Zheng
2026-08-31 12:31:35 -07:00
2138494272
[Rust] Keep sampling and scheduler wire schemas in sync (#37222 )
Lianmin Zheng
2026-08-31 12:31:07 -07:00
48098b5f23
[Rust] Derive server address and accept signed env values (#37221 )
Lianmin Zheng
2026-08-31 12:30:40 -07:00
1da86b9801
[Rust] Split and rename embedded server components (#37220 )
Lianmin Zheng
2026-08-31 12:28:43 -07:00
cf51650335
fix(config): retain pre-engine resolution declarations (#37195 )
ishandhanani
2026-08-31 14:24:29 -05:00
9e9d26a4af
Fix Mooncake serving benchmark trace rows (#37201 )
Shuwen Wang
2026-09-01 02:43:59 +08:00
549166b819
fix(gateway): bump wfaas to 1.0.2 so ContinueNextStep unblocks dependents (#37249 )
Juhyun-Kim-Memphis
2026-09-01 03:30:34 +09:00
07d84ebd6d
[2/N][Mixed] Mixed chunk prefill with spec enabled (#36933 )
Yuwei An and Claude Fable 5
2026-08-31 10:48:07 -07:00
9cf157c252
[Radix Cache] Add Rust TreeCore backend with shared parity tests (#32710 )
2026-08-31 09:26:20 -07:00
52e1c24744
[Diffusion] Fuse FLUX.2 token concatenation and NVFP4 quantization (#37141 )
Xiaoyu Zhang
2026-08-31 21:33:02 +08:00
d60d658f5f
[Kernel] Add GB300 Triton MoE configs for GLM-4.5 FP8 (#37159 )
Xiaoyu Zhang and Song Bian
2026-08-31 21:31:04 +08:00
771e613d96
[Diffusion] Fuse Qwen-Image final adaptive LayerNorm (#37144 )
Xiaoyu Zhang
2026-08-31 18:22:20 +08:00
1ed9bfac2c
[Diffusion] Fuse Qwen-Image FP8 norm and activation quantization (#37156 )
Xiaoyu Zhang and Cursor
2026-08-31 18:19:42 +08:00
a874b83c2c
test: re-enable DSV4-Flash W8A8 8p nightly perf cases (#37214 )
pllimax
2026-08-31 17:20:46 +08:00
63b2adbeac
[NPU] Fix evalscope accuracy parsing and add glm5_1 aime26 request timeout (#36459 )
pllimax
2026-08-31 17:19:03 +08:00
a53718e88c
fix(ci): update attention backend test fixtures (#37219 )
YAMY
2026-08-31 02:15:17 -07:00
0674be736c
[AMD] build gfx1250 release image from main (#37225 )
YC Yen-Ching Tseng
2026-08-31 11:04:04 +02:00
6580d5cd9a
weight cache: key daemon paths by GPU UUID (#36101 )
2026-08-31 15:44:45 +07:00
3865efc9f7
[AMD] support gfx1250 on ROCM 10 (#36871 )
+1
2026-08-31 10:19:11 +02:00
712a720c8a
[KDA] Fused-accept state advance for FlashInfer KDA MTP verify (#33722 )
Yuan Luo and luoyuan.luo
2026-08-31 16:11:49 +08:00
f61bb7b40a
[unified-memory] Drop the vacated 'dense' qualifier and the restating comments (#37170 )
Cheng Wan
2026-08-31 00:54:13 -07:00
8bb776dc48
feat(unified-memory): read unified pool from attention backends fa3/flashinfer/trtllm_mha/flashmla (#34613 )
2026-08-30 23:58:24 -07:00
29578d5578
refactor(unified-memory): translate the KV write location once, at ForwardBatch construction (#35245 )
2026-08-30 23:52:14 -07:00
4f997a432a
test: re-enable FlashInfer per-token NVFP4 coverage (#36985 )
Ziang Li
2026-08-30 23:44:09 -07:00
3ed3326631
Decouple speculative draft capacity from runtime state (#36897 )
Lianmin Zheng and Ming Yang
2026-08-30 23:31:25 -07:00
2cb3f32b03
[diffusion] chore: preserve exact component identity during loading (#36875 )
Mick
2026-08-31 14:17:57 +08:00
5e679b0cad
[CI] Speed up lint: cache pre-commit envs + mint, drop redundant work (#37203 )
Alex Nails and Claude Fable 5
2026-08-30 23:16:31 -07:00
b77cac06a9
[PP] Support prefill CUDA graph proxy tensors (#36248 )
YAMY and github-actions[bot]
2026-08-30 23:11:45 -07:00
5d92e60783
[Unified Cache Linker][3/N]: Add backend-independent linker core (#37151 )
Zhangheng and 晟海
2026-08-31 14:07:39 +08:00
10b67aa7a1
xpu: move prefill-only model tests to the nightly-xpu-1-gpu grid (#36814 )
ashwini rathi
2026-08-31 11:33:48 +05:30
62c470697e
[diffusion] chore: enforce component attention backend application (#36907 )
Mick
2026-08-31 14:00:02 +08:00
28690f5aa5
[diffusion] chore: detect quantized transformer replacements (#36916 )
Mick
2026-08-31 13:55:50 +08:00
3a6ed55999
[Fix] Shut hicache test servers down gracefully before SIGKILL (#37194 )
Liangsheng Yin
2026-08-30 22:45:40 -07:00
afbca97f74
add suffix for xpu kernel upload space (#36422 )
Zaili Wang and Alex Nails
2026-08-31 13:41:22 +08:00
5d12ad4fd7
[mem_cache] Move mamba state and retraction_backup into ReqKvInfo (#37164 )
Liangsheng Yin
2026-08-30 22:03:01 -07:00
9a9e167179
[Bugfix] Fix full prefill CUDA graph padding and EAGLE capture (#35588 )
Aurick Qiao
2026-08-30 21:30:24 -07:00
2ea6d17eab
fix(staging): make empty staging rings reusable (#37166 )
YAMY
2026-08-30 20:59:57 -07:00
5972211977
[AMD] Fix the QuickReduce bf16 cast failing to build for CDNA (#37132 )
YC Yen-Ching Tseng and Alex Nails
2026-08-31 05:57:29 +02:00
62f86ce470
[CI] Fix unreachable FakeReq field initialization (#37182 )
Shuwen Wang
2026-08-31 11:51:40 +08:00
881cbfe54c
[diffusion] feat: add exact component precision overrides (#36991 )
Mick
2026-08-31 11:12:07 +08:00
4dc7dc8518
[Fix] Transformers-fallback (GPT-NeoX) + KV pool config (DeepSeek-VL2) (#35244 )
Pavan Sivaram Girijala and YanbingJiang
2026-08-31 08:35:50 +05:30
046454404a
[Spec] Add LFM2 and LFM2-MoE DSpark speculative decoding support (#31041 )
Piotr Mazurek and Khoa Pham
2026-08-31 05:03:59 +02:00
7700602278
[PD] Align defensive protocol behavior across Mooncake, NIXL, and Mori (#35281 )
jambow0320 and Shangming Cai
2026-08-31 10:57:56 +08:00
df75ec5f77
[Intel GPU] Add rust support to XPU docker images (#35877 )
Polisetty V R K Jyothendra Varma
2026-08-31 08:22:57 +05:30
3139ceaeec
[XPU] Use SYCL kernels for topk_transform on XPU (#33318 )
Cui Lily and Ma Mingfei
2026-08-31 10:33:19 +08:00
a9d5ca723a
[CPU] Fix weight missing issue in fused_input_proj_cpu for GPTQ INT4 for Qwen 3.5 (#35805 )
YanbingJiang
2026-08-31 10:14:41 +08:00
4f761e8649
[Deps] Bump FlashInfer to 0.6.18 (#36954 )
Mohammad Miadh Angkad
2026-08-31 10:02:39 +08:00
0da6a66856
[MLX] Fix startup crash when reporting preloaded weights (#37035 )
Ali Ihsan Nergiz
2026-08-31 01:36:13 +01:00
bb5e619860
[diffusion] perf: absorb Qwen-Image output projection biases (#37116 )
Xiaoyu Zhang
2026-08-31 08:25:37 +08:00
4bb8de34cc
[mem_cache] Share one ReqKvInfo between a streaming session slot and its request (#37108 )
Liangsheng Yin
2026-08-30 16:36:48 -07:00
4bea51d885
feat(unified-memory): dense KV views for uniform-row MHA/SWA models (#34602 )
2026-08-30 15:10:12 -07:00
007ef5e23a
[mem_cache] Move req_pool_idx into ReqKvInfo (#37094 )
Liangsheng Yin
2026-08-30 14:46:21 -07:00
8a87079dbb
Fix stale GLM MoE routing after runtime weight updates (#35883 )
Yuzhen Zhou and Jiajun Li
2026-08-30 14:13:34 -07:00
5ab97c4f44
[Diffusion] Cache Qwen-Image modulation across serial CFG branches (#37090 )
Xiaoyu Zhang
2026-08-31 01:40:29 +08:00
8c28cdd116
[Diffusion][Kernel] Fuse Wan2.2 NVFP4 bias + GELU on Blackwell (#37075 )
Xiaoyu Zhang
2026-08-31 01:37:35 +08:00
9a03bc2dc3
[CI] Fix stale GPU capability test patches (#37148 )
YAMY
2026-08-30 10:12:32 -07:00
6a9366f036
[Unified Cache Linker][2/N]: Add device pool assembly for external linkers (#37098 )
Zhangheng and 晟海
2026-08-30 23:54:00 +08:00
84e56982b6
[Fix] Fix transformer loader fallback test fixture (#37142 )
YAMY
2026-08-30 08:28:11 -07:00
c9eb475a88
[Unified Cache][1/N]: Support cache contract for external linker (#37091 )
Zhangheng and 晟海
2026-08-30 22:24:13 +08:00
fe694986a2
[diffusion] chore: make malformed component execution options fail-fast (#37049 )
Mick
2026-08-30 21:06:19 +08:00
e6a6492057
[vlm] fix: preserve per-request vit graph metadata for qwen-vl (#37043 )
Mick
2026-08-30 21:02:14 +08:00
e9a7157615
[diffusion] feat: allow cache-dit with dit layerwise offload (#35858 )
WenhaoZhang
2026-08-30 20:55:41 +08:00
26c754e06e
[vlm] fix: recover multimodal decode and processor failures (#36983 )
Mick
2026-08-30 20:50:19 +08:00
aa483ab782
[diffusion] feat: support streaming native vae weights directly to gpu (#37004 )
Mick
2026-08-30 20:48:09 +08:00
e51a3ae65e
[Config] Round 5.2: the per-model declarations get their own modules (#37087 )
Cheng Wan and Claude Opus 5
2026-08-30 02:24:35 -07:00
7e751153eb
[Config] Round 5.1: the published-side readers ask the bags, and a platform fact gets one address (#37086 )
Cheng Wan
2026-08-30 02:18:33 -07:00
a6e4021368
[diffusion] chore: reject incompatible transformer fallback (#36917 )
Mick
2026-08-30 16:33:14 +08:00
e635577431
[rotary] Fix the fused Qwen3.5 RoPE kernel discarding mrope height and width (#34446 )
Jason Wiemels
2026-08-30 00:37:40 -07:00
d249672ad3
Scatter mm embeddings with row index_copy_ instead of masked_scatter_ to cut transient GPU memory (#37070 )
Oguz Ulgen
2026-08-30 00:15:39 -07:00
7399c2b558
[AMD] Update v4 amd cookbook 0830 (#37092 )
Thomas Wang
2026-08-30 14:55:28 +08:00
512df615de
[Bugfix] Hold references to fire-and-forget tasks in disaggregation (#33048 )
Lin Junrong
2026-08-30 14:26:34 +08:00
5ec959965b
[mem_cache] Settle extend kv_committed_len inside alloc_for_extend (#37085 )
Liangsheng Yin
2026-08-29 23:14:31 -07:00
0438b16154
[mem_cache] Move kv_committed_len into ReqKvInfo (#37078 )
Liangsheng Yin
2026-08-29 22:42:25 -07:00
5b7c62d5d6
fix(frontend): bound stop strings and regex patterns (#37029 )
EchO
2026-08-30 13:25:26 +08:00
78fa921189
[ROCm] Fix QuickReduce fp16 saturation corrupting bf16 all-reduces (106M non-finite -> 0, +0.3%) (#34484 )
Alex Nails
2026-08-29 22:08:17 -07:00
67bd163a48
[AMD] Cherry-pick dsv4 fp4 kv-cache fix aiter commit (#37083 )
Thomas Wang
2026-08-30 12:17:36 +08:00
7f540274a7
Use Flashinfer 0.6.18 release for CUDA 13.4 package (#37073 )
Trevor Morris
2026-08-29 21:06:00 -07:00
a7ee399904
Bump sgl-deep-gemm to v0.1.6 (#36998 )
Baizhou Zhang
2026-08-29 19:51:02 -07:00
6be767c2d2
[mem_cache] Move cache_protected_len and swa_evict_floor into ReqKvInfo (#36982 )
Liangsheng Yin
2026-08-29 19:37:53 -07:00
ed39568e79
[MoE] Add extension points for custom runner backends (#32665 )
2026-08-29 22:03:41 -04:00
ca8ff035c3
fix(hicache): isolate decode offload state per request (#37026 )
EchO and Guanyi Chen
2026-08-30 09:15:05 +08:00
fbecd75c83
[AMD] fix: do not emit a shared-expert marker twice on the per-rank slot path (#36515 )
karverma-amd
2026-08-29 20:09:33 -05:00
9a489f8d2f
[Test] Move gpqa and aime25 onto sgl-eval, drop unused eval paths (#36979 )
Liangsheng Yin
2026-08-29 17:36:13 -07:00
032fe91bf1
Handle unlimited tokenizer context lengths (#37054 )
Neeraj Kashyap and Baizhou Zhang
2026-08-29 17:29:11 -07:00
f60bc73c58
[Spec] Fix Dspark and Dflash state divergence across TP rank (#33614 )
2026-08-29 17:27:43 -07:00
a1fe4e30a9
[Kernel] Fix SM90 FP8 decode regression with benchmarked M/K/N routing (#37018 )
2026-08-30 03:59:30 +04:00
00fbb6e8ac
[Perf] Tune the W4AFP8 DeepEP low-latency requant launch geometry (#35760 )
Alex Nails and Claude Opus 5
2026-08-29 16:03:01 -07:00
3a0f1a1344
[NPU] fix: reach torch>=2.8 CUDA memory-pool APIs lazily via torch._C (#29100 )
2026-08-29 15:55:14 -07:00
09ecb9aaaa
📝 [NPU] Use vendor-neutral wording in quantization comments (#36768 )
Junlin Wu
2026-08-30 04:05:35 +08:00