Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main

Mono Color

• 8bb776dc48 feat(unified-memory): read unified pool from attention backends fa3/flashinfer/trtllm_mha/flashmla (#34613) 4 people 2026-08-30 23:58:24 -07:00
• 29578d5578 refactor(unified-memory): translate the KV write location once, at ForwardBatch construction (#35245) 3 people 2026-08-30 23:52:14 -07:00
• 4f997a432a test: re-enable FlashInfer per-token NVFP4 coverage (#36985) Ziang Li 2026-08-30 23:44:09 -07:00
• 3ed3326631 Decouple speculative draft capacity from runtime state (#36897) Lianmin ZhengandMing Yang 2026-08-30 23:31:25 -07:00
• 2cb3f32b03 [diffusion] chore: preserve exact component identity during loading (#36875) Mick 2026-08-31 14:17:57 +08:00
• 5e679b0cad [CI] Speed up lint: cache pre-commit envs + mint, drop redundant work (#37203) Alex NailsandClaude Fable 5 2026-08-30 23:16:31 -07:00
• b77cac06a9 [PP] Support prefill CUDA graph proxy tensors (#36248) YAMYandgithub-actions[bot] 2026-08-30 23:11:45 -07:00
• 5d92e60783 [Unified Cache Linker][3/N]: Add backend-independent linker core (#37151) Zhanghengand晟海 2026-08-31 14:07:39 +08:00
• 10b67aa7a1 xpu: move prefill-only model tests to the nightly-xpu-1-gpu grid (#36814) ashwini rathi 2026-08-31 11:33:48 +05:30
• 62c470697e [diffusion] chore: enforce component attention backend application (#36907) Mick 2026-08-31 14:00:02 +08:00
• 28690f5aa5 [diffusion] chore: detect quantized transformer replacements (#36916) Mick 2026-08-31 13:55:50 +08:00
• 3a6ed55999 [Fix] Shut hicache test servers down gracefully before SIGKILL (#37194) Liangsheng Yin 2026-08-30 22:45:40 -07:00
• afbca97f74 add suffix for xpu kernel upload space (#36422) Zaili WangandAlex Nails 2026-08-31 13:41:22 +08:00
• 5d12ad4fd7 [mem_cache] Move mamba state and retraction_backup into ReqKvInfo (#37164) Liangsheng Yin 2026-08-30 22:03:01 -07:00
• 9a9e167179 [Bugfix] Fix full prefill CUDA graph padding and EAGLE capture (#35588) Aurick Qiao 2026-08-30 21:30:24 -07:00
• 2ea6d17eab fix(staging): make empty staging rings reusable (#37166) YAMY 2026-08-30 20:59:57 -07:00
• 5972211977 [AMD] Fix the QuickReduce bf16 cast failing to build for CDNA (#37132) YC Yen-Ching TsengandAlex Nails 2026-08-31 05:57:29 +02:00
• 62f86ce470 [CI] Fix unreachable FakeReq field initialization (#37182) Shuwen Wang 2026-08-31 11:51:40 +08:00
• 881cbfe54c [diffusion] feat: add exact component precision overrides (#36991) Mick 2026-08-31 11:12:07 +08:00
• 4dc7dc8518 [Fix] Transformers-fallback (GPT-NeoX) + KV pool config (DeepSeek-VL2) (#35244) Pavan Sivaram GirijalaandYanbingJiang 2026-08-31 08:35:50 +05:30
• 046454404a [Spec] Add LFM2 and LFM2-MoE DSpark speculative decoding support (#31041) Piotr MazurekandKhoa Pham 2026-08-31 05:03:59 +02:00
• 7700602278 [PD] Align defensive protocol behavior across Mooncake, NIXL, and Mori (#35281) jambow0320andShangming Cai 2026-08-31 10:57:56 +08:00
• df75ec5f77 [Intel GPU] Add rust support to XPU docker images (#35877) Polisetty V R K Jyothendra Varma 2026-08-31 08:22:57 +05:30
• 3139ceaeec [XPU] Use SYCL kernels for topk_transform on XPU (#33318) Cui LilyandMa Mingfei 2026-08-31 10:33:19 +08:00
• a9d5ca723a [CPU] Fix weight missing issue in fused_input_proj_cpu for GPTQ INT4 for Qwen 3.5 (#35805) YanbingJiang 2026-08-31 10:14:41 +08:00
• 4f761e8649 [Deps] Bump FlashInfer to 0.6.18 (#36954) Mohammad Miadh Angkad 2026-08-31 10:02:39 +08:00
• 0da6a66856 [MLX] Fix startup crash when reporting preloaded weights (#37035) Ali Ihsan Nergiz 2026-08-31 01:36:13 +01:00
• bb5e619860 [diffusion] perf: absorb Qwen-Image output projection biases (#37116) Xiaoyu Zhang 2026-08-31 08:25:37 +08:00
• 4bb8de34cc [mem_cache] Share one ReqKvInfo between a streaming session slot and its request (#37108) Liangsheng Yin 2026-08-30 16:36:48 -07:00
• 4bea51d885 feat(unified-memory): dense KV views for uniform-row MHA/SWA models (#34602) 3 people 2026-08-30 15:10:12 -07:00
• 007ef5e23a [mem_cache] Move req_pool_idx into ReqKvInfo (#37094) Liangsheng Yin 2026-08-30 14:46:21 -07:00
• 8a87079dbb Fix stale GLM MoE routing after runtime weight updates (#35883) Yuzhen ZhouandJiajun Li 2026-08-30 14:13:34 -07:00
• 5ab97c4f44 [Diffusion] Cache Qwen-Image modulation across serial CFG branches (#37090) Xiaoyu Zhang 2026-08-31 01:40:29 +08:00
• 8c28cdd116 [Diffusion][Kernel] Fuse Wan2.2 NVFP4 bias + GELU on Blackwell (#37075) Xiaoyu Zhang 2026-08-31 01:37:35 +08:00
• 9a03bc2dc3 [CI] Fix stale GPU capability test patches (#37148) YAMY 2026-08-30 10:12:32 -07:00
• 6a9366f036 [Unified Cache Linker][2/N]: Add device pool assembly for external linkers (#37098) Zhanghengand晟海 2026-08-30 23:54:00 +08:00
• 84e56982b6 [Fix] Fix transformer loader fallback test fixture (#37142) YAMY 2026-08-30 08:28:11 -07:00
• c9eb475a88 [Unified Cache][1/N]: Support cache contract for external linker (#37091) Zhanghengand晟海 2026-08-30 22:24:13 +08:00
• fe694986a2 [diffusion] chore: make malformed component execution options fail-fast (#37049) Mick 2026-08-30 21:06:19 +08:00
• e6a6492057 [vlm] fix: preserve per-request vit graph metadata for qwen-vl (#37043) Mick 2026-08-30 21:02:14 +08:00
• e9a7157615 [diffusion] feat: allow cache-dit with dit layerwise offload (#35858) WenhaoZhang 2026-08-30 20:55:41 +08:00
• 26c754e06e [vlm] fix: recover multimodal decode and processor failures (#36983) Mick 2026-08-30 20:50:19 +08:00
• aa483ab782 [diffusion] feat: support streaming native vae weights directly to gpu (#37004) Mick 2026-08-30 20:48:09 +08:00
• e51a3ae65e [Config] Round 5.2: the per-model declarations get their own modules (#37087) Cheng WanandClaude Opus 5 2026-08-30 02:24:35 -07:00
• 7e751153eb [Config] Round 5.1: the published-side readers ask the bags, and a platform fact gets one address (#37086) Cheng Wan 2026-08-30 02:18:33 -07:00
• a6e4021368 [diffusion] chore: reject incompatible transformer fallback (#36917) Mick 2026-08-30 16:33:14 +08:00
• e635577431 [rotary] Fix the fused Qwen3.5 RoPE kernel discarding mrope height and width (#34446) Jason Wiemels 2026-08-30 00:37:40 -07:00
• d249672ad3 Scatter mm embeddings with row index_copy_ instead of masked_scatter_ to cut transient GPU memory (#37070) Oguz Ulgen 2026-08-30 00:15:39 -07:00
• 7399c2b558 [AMD] Update v4 amd cookbook 0830 (#37092) Thomas Wang 2026-08-30 14:55:28 +08:00
• 512df615de [Bugfix] Hold references to fire-and-forget tasks in disaggregation (#33048) Lin Junrong 2026-08-30 14:26:34 +08:00
• 5ec959965b [mem_cache] Settle extend kv_committed_len inside alloc_for_extend (#37085) Liangsheng Yin 2026-08-29 23:14:31 -07:00
• 0438b16154 [mem_cache] Move kv_committed_len into ReqKvInfo (#37078) Liangsheng Yin 2026-08-29 22:42:25 -07:00
• 5b7c62d5d6 fix(frontend): bound stop strings and regex patterns (#37029) EchO 2026-08-30 13:25:26 +08:00
• 78fa921189 [ROCm] Fix QuickReduce fp16 saturation corrupting bf16 all-reduces (106M non-finite -> 0, +0.3%) (#34484) Alex Nails 2026-08-29 22:08:17 -07:00
• 67bd163a48 [AMD] Cherry-pick dsv4 fp4 kv-cache fix aiter commit (#37083) Thomas Wang 2026-08-30 12:17:36 +08:00
• 7f540274a7 Use Flashinfer 0.6.18 release for CUDA 13.4 package (#37073) Trevor Morris 2026-08-29 21:06:00 -07:00
• a7ee399904 Bump sgl-deep-gemm to v0.1.6 (#36998) Baizhou Zhang 2026-08-29 19:51:02 -07:00
• 6be767c2d2 [mem_cache] Move cache_protected_len and swa_evict_floor into ReqKvInfo (#36982) Liangsheng Yin 2026-08-29 19:37:53 -07:00
• ed39568e79 [MoE] Add extension points for custom runner backends (#32665) 3 people 2026-08-29 22:03:41 -04:00
• ca8ff035c3 fix(hicache): isolate decode offload state per request (#37026) EchOandGuanyi Chen 2026-08-30 09:15:05 +08:00
• fbecd75c83 [AMD] fix: do not emit a shared-expert marker twice on the per-rank slot path (#36515) karverma-amd 2026-08-29 20:09:33 -05:00
• 9a489f8d2f [Test] Move gpqa and aime25 onto sgl-eval, drop unused eval paths (#36979) Liangsheng Yin 2026-08-29 17:36:13 -07:00
• 032fe91bf1 Handle unlimited tokenizer context lengths (#37054) Neeraj KashyapandBaizhou Zhang 2026-08-29 17:29:11 -07:00
• f60bc73c58 [Spec] Fix Dspark and Dflash state divergence across TP rank (#33614) 3 people 2026-08-29 17:27:43 -07:00
• a1fe4e30a9 [Kernel] Fix SM90 FP8 decode regression with benchmarked M/K/N routing (#37018) 3 people 2026-08-30 03:59:30 +04:00
• 00fbb6e8ac [Perf] Tune the W4AFP8 DeepEP low-latency requant launch geometry (#35760) Alex NailsandClaude Opus 5 2026-08-29 16:03:01 -07:00
• 3a0f1a1344 [NPU] fix: reach torch>=2.8 CUDA memory-pool APIs lazily via torch._C (#29100) 3 people 2026-08-29 15:55:14 -07:00
• 09ecb9aaaa 📝 [NPU] Use vendor-neutral wording in quantization comments (#36768) Junlin Wu 2026-08-30 04:05:35 +08:00
• 6afb5e1771 Gate the idle-loop tree-cache sanity check behind a default-off env (#36205) 3 people 2026-08-29 12:10:28 -04:00
• 000c636342 docs: state that HiCache L2 is instance-private and only L3 is shared (#37050) Shuwen WangandClaude Opus 5 2026-08-29 23:47:50 +08:00
• 97781eb7f3 [diffusion] fix: honor explicit offload in resident requirements (#36905) Mick 2026-08-29 22:48:31 +08:00
• b24bd44556 [diffusion] feat: avoid direct GPU parameter copies (#36832) Mick 2026-08-29 22:43:25 +08:00
• cdbfe90b4a [HiCache] Align chunked CUDA host registrations (#36798) HZYandZhangheng 2026-08-29 14:36:12 +02:00
• 4d53767b09 config: the lazy imports that buy nothing become eager (#36975) Cheng WanandClaude Opus 5 2026-08-29 04:21:54 -07:00
• f0d621cfa6 config: the dead record parameters go (#36974) Cheng WanandClaude Opus 5 2026-08-29 04:20:55 -07:00
• 1a3e152f03 config: six more runtime readers ask the bags (#36973) Cheng WanandClaude Opus 5 2026-08-29 04:19:14 -07:00
• b65e677e48 config: the resolution callbacks into the record go to zero (#36972) Cheng WanandClaude Opus 5 2026-08-29 04:18:05 -07:00
• 48b88e1256 config: the resolution pipeline's dispatcher leaves the record (#36896) Cheng WanandClaude Opus 5 2026-08-29 04:16:25 -07:00
• 46ccd7ce3e Add Laguna-XS-2.1 / S-2.1 NVFP4 nightly gsm8k tests (#35547) Joe RowellandJimmy Shong 2026-08-29 12:34:46 +02:00
• a328c19c81 [HiCache] buffer mode: decide staged-fetch fate against the live tree (#36834) Zhiqiang Xie 2026-08-29 01:51:47 -07:00
• 0a585d5bb1 [BugFix] Allow model_loader_extra_config with remote_instance + modelexpress backend (#34639) Joel Gustafsonandmmangkad 2026-08-29 02:43:10 -05:00
• 78eef34356 [NPU] [BugFix] Fix discontinuous input for FIA operator in GLM4.7‑Flash (#36170) silencejade 2026-08-29 15:03:05 +08:00
• 46334460a6 [CI] Temporarily disable GB300 jobs (#36981) Baizhou Zhang 2026-08-28 23:56:54 -07:00
• 24a3a63e6b [misc] Keep req.kv non-optional and key KV ownership on req_pool_idx (#36958) Liangsheng Yin 2026-08-28 23:47:42 -07:00
• 0e1146d04f [diffusion] optimization: optimize Pi0.5 inference and bounded graph serving (#34599) Xiaoyu Zhang 2026-08-29 14:47:32 +08:00
• f8f501f2e8 [diffusion] fix: fix nvfp4 diffusion models on sm_120 (RTX PRO 6000 / RTX 50xx) (#35739) 王鹤男andMick 2026-08-29 14:42:49 +08:00
• fa474b0441 [diffusion] fix: fix image encoder parallel folding proposal (#36863) Mick 2026-08-29 14:34:36 +08:00
• f5319af1c9 [diffusion] chore: honor explicit component offload (#36931) Mick 2026-08-29 14:29:57 +08:00
• a25df83fe3 [Cookbook] Run accuracy benchmarks through sgl-eval (#36977) Liangsheng Yin 2026-08-28 23:29:38 -07:00
• 4d7d2ebb44 [Docker] Install AI Dynamo nightly (#36946) ishandhanani 2026-08-29 01:27:52 -05:00
• d1ce017665 [diffusion] feat: delegate recognized quantized components to transformers (#36902) Mick 2026-08-29 14:26:50 +08:00
• 3c1d77be21 [diffusion] feat: respect component weight overrides for upsamplers (#36874) Mick 2026-08-29 14:23:11 +08:00
• f93b48c627 [diffusion] refactor: resolve indexed component weight sets (#36883) Mick 2026-08-29 14:21:25 +08:00
• 7f2ee22b70 [AMD] Fix eager metadata for AITER EAGLE draft extend (#36915) zijiecandjacky.cheng 2026-08-28 22:56:38 -07:00
• 24c9251ac5 [AMD][Spec][PD] Enable the PD DSA fused-TopK seed remap on ROCm (#36714) 3 people 2026-08-28 22:34:23 -07:00
• ec962fd12d [Fix] Fall back to the process-group broadcast for DSA topk when PyNCCL is absent (#36963) Liangsheng Yin 2026-08-28 22:20:35 -07:00
• 3904b309db Add configurable HTTP/2 connection window (#36920) Eric Zhang 2026-08-29 01:18:42 -04:00
• 3760296be8 [PD] Pack DCP1→DCP-N PD KV transfers into dest-contiguous RDMA blocks (#35762) 3 people 2026-08-28 21:41:30 -07:00
• 5f216fc33f qwen 3.8 rebase (#35758) +818 people 2026-08-28 20:41:34 -07:00
• ca8cc101b8 [NPU] add causal conv1d for ascend kda backend (#35021) zhaozx-cn 2026-08-29 11:39:45 +08:00