This website requires JavaScript.
acfde25d34
Carry deferred attention operands and reuse multimodal shared memory (#39870 )
2026-09-17 01:17:15 -07:00
882577451e
Restrict SafeUnpickler standard-library globals (#39858 )
Lianmin Zheng and jiayisuse
2026-09-17 01:09:55 -07:00
71ef869ece
[AMD][Bugfix] Fix vattn_asm HIP error 709 under CUDA graph capture on ROCm 10 (#39513 )
chuyeh and jacky.cheng
2026-09-17 16:06:04 +08:00
3ce7e2a29f
[sgl-router] Render chat prompts with dynamo-render (#38983 )
Kan Wu and Claude Fable 5.1
2026-09-17 00:57:20 -07:00
15b256bdb0
[HiCache] fix: resolve Mooncake local_hostname per node for runtime attach (#29668 )
Teng Ma and Teng Ma
2026-09-17 15:51:21 +08:00
aebae58b8c
[Moe] Fix flashinfer_trtllm silently dropping swiglu_limit clamped SwiGLU activation (#39920 )
Shijin Zhang
2026-09-17 15:25:17 +08:00
f3c0256771
[diffusion] chore: remove retired auto-residency workload helpers (#39884 )
Mick and Mick Qian
2026-09-17 15:16:35 +08:00
44bd359082
[NPU][Diffusion] Optimize SenseNova-U1 batched generation (#39382 )
faceless void and ronnie_zheng
2026-09-17 15:08:20 +08:00
3401b75240
dsv4.1: Engram module and request history support (#39666 )
Liangsheng Yin and BBuf
2026-09-16 23:49:47 -07:00
8ae4a39b50
Gate mamba extra-buffer predicates on uses_mamba_radix_cache (#37474 )
2026-09-16 23:39:12 -07:00
fd6f96bf96
Re-land dp-attention local control broadcast test (#39835 )
Ke Bao and Sam Shleifer
2026-09-17 14:33:25 +08:00
6460082c05
[Mamba] Fix checkpoint depth for prefixes that end off the radix page (#39115 )
Kurt Shuster and Ke Bao
2026-09-17 02:30:57 -04:00
fa8d22e665
[AMD][DSV4] Allow moe_a2a_backend='mori' with DSpark + dp attention (#39910 )
karverma-amd
2026-09-17 01:09:41 -05:00
9d0a8d7536
[CPU] Upgrade torch cpu dependencies to version 2.14. (#37939 )
Ziru and Niu Ziru
2026-09-17 14:01:43 +08:00
7782a2a1c8
[Benchmark] Limit warmup concurrency in serving benchmark (#39398 )
BingjiaWang
2026-09-17 13:58:10 +08:00
c525ed8f02
[diffusion] fix: preserve explicit attention backends during autotune (#39882 )
Mick and Mick Qian
2026-09-17 13:28:55 +08:00
f6f69334ba
[XPU] Allow the nightly XPU docker build to target a branch or tag (#39796 )
ashwini rathi
2026-09-17 10:53:09 +05:30
1c4b130bf7
[CI] Unify basic and speculative sanity accuracy checks with MMLU (#39906 )
Liangsheng Yin
2026-09-16 22:19:05 -07:00
923e4a56d4
[CI] Fix sanity evaluation and diffusion test suite blockers (#39892 )
Liangsheng Yin
2026-09-16 21:47:41 -07:00
25ce8063f7
Pipeline parallelism x speculative decoding (EAGLE/MTP) compatibility (#30775 )
2026-09-17 12:41:54 +08:00
2733afe54e
[diffusion] Fix the XPU capability gates that broke the Wan2.2 A14B DiT path (#36825 )
Chetan Kumar Verma and Claude Opus 5
2026-09-17 09:45:07 +05:30
464fffbec8
dsv4.1: chat encoding and tool parsing (#39665 )
Liangsheng Yin and BBuf
2026-09-16 20:49:42 -07:00
33d46376a6
[HiCache] Yield idle scheduler so storage workers can drain (#38504 )
Shuwen Wang
2026-09-17 10:56:11 +08:00
241a5b9823
MiniMax-M3: allow shared-experts fusion on ROCm gfx942 and newer (#36576 )
Chunan Zeng and Kevin Mi
2026-09-16 19:55:47 -07:00
329ffc89b9
[Mooncake] Fix silent SSD offload corruption when TP/PP ranks share ssd_offload_path (#31926 )
Michele Palazzi and Teng Ma
2026-09-17 04:51:03 +02:00
84d7604b7e
[XPU] weekly simple model enablement 2026/09/14 (#39439 )
2026-09-16 19:43:04 -07:00
4793f56835
[HiCache] Keep hybrid transfer layer maps stage-local under PP (#39699 )
2026-09-16 19:18:44 -07:00
0daa040e48
Support LongBench v2 in one-batch server benchmarks (#39874 )
2026-09-16 19:16:03 -07:00
4fb9b5b5ba
feat(hicache): support NPU Mamba states with FIA and async IO (#32500 )
qyb233
2026-09-17 10:12:26 +08:00
4c85172f3a
[HiCache] Back up MXFP8 KV scales in the host pool (#39089 )
Shuwen Wang
2026-09-17 09:40:26 +08:00
0443e3179f
fix(kda_prefill): fence shared writes before async proxy reads (#39124 )
William Hu
2026-09-16 21:37:19 -04:00
7780882f18
[AMD][bugfix] Fix dsv4 server launch (#39875 )
Xinyi Song
2026-09-16 18:28:02 -07:00
c89c63fa38
dsv4.1: vision tower and image preprocessing (#39668 )
2026-09-16 18:11:16 -07:00
408d2334c3
dsv4.1: mHC computation and compensated projections (#39664 )
Liangsheng Yin
2026-09-16 17:11:55 -07:00
46ae84df15
dsv4.1: Hopper FP8 matmul kernels and tuning (#39657 )
2026-09-16 16:54:19 -07:00
dc067c7d8c
[Fix] Allow closed object schemas in Outlines prevalidation (#39869 )
Liangsheng Yin
2026-09-16 16:42:17 -07:00
d78b35076a
[CI] Fix missing elfutils headers in CUDA 13.4 DeepGEMM build (#39855 )
Chunan Zeng
2026-09-16 16:41:29 -07:00
c2443458e1
dsv4.1: RoPE and FP4 packing kernels (#39656 )
2026-09-16 16:17:12 -07:00
f45aad44bd
[gRPC] Expose native pause status (#37488 )
William Arnold and ishandhanani
2026-09-17 07:16:52 +09:00
35b7589e1a
dsv4.1: candidate indexer library (#39671 )
2026-09-16 15:15:18 -07:00
13d593b6cf
dsv4.1: compression, KV I/O, and metadata kernels (#39652 )
2026-09-16 13:54:07 -07:00
869674b3a7
[Fix] Prevalidate JSON Schema support per grammar backend (#37839 )
Justin Perlman
2026-09-16 16:10:15 -04:00
43390f63f5
[qwen 3.8 next] change the testing model in test_qwen4_exp_models.py (#39662 )
Qiaolin Yu
2026-09-16 12:47:45 -07:00
b02e16a895
[Router] Fleet-wide sampling contract 3/3: splice injection without re-serializing (#39002 )
2026-09-16 09:59:48 -07:00
3e03879f68
[Router] Drain readiness before SIGTERM shutdown so k8s deregisters the pod first (#39016 )
2026-09-16 09:59:26 -07:00
279339f113
[sgl-router] Share model-file discovery for chat formatters (#39485 )
Kan Wu
2026-09-16 09:36:11 -07:00
f0bf652534
Add Ling-3.0-flash-VL model support (#38526 )
Xinyuan Tong
2026-09-17 00:19:26 +08:00
1b78083b42
[PD] Add optional KV transfer checksums (#39500 )
cctry
2026-09-16 08:53:30 -07:00
a813224e78
[ROCm][DSV4] Enable breakable CUDA graph prefill (#37810 )
AMD-yanfeiwang and Duyi-Wang
2026-09-16 23:52:11 +08:00
ad94978adf
[sgl-router] Rename chat encoder to chat formatter (#39459 )
Kan Wu and Claude Fable 5.1
2026-09-16 08:46:11 -07:00
5aaa18207c
Revert "[CI] Add e2e test for dp-attention local control broadcast" (#39828 )
Ke Bao
2026-09-16 23:41:50 +08:00
e41026f434
[sgl-router] Forward input_ids only for string content; count tokenize errors only when forwardable (#39458 )
Kan Wu and Claude Fable 5.1
2026-09-16 07:49:41 -07:00
cc171fbad0
feat: support custom OTLP trace service name (#35802 )
ymren
2026-09-16 22:37:42 +08:00
00a9a81b67
[NPU][CI] Fail fast and speed up long-running qwen3.6 accuracy cases (#39813 )
pllimax
2026-09-16 20:20:37 +08:00
e7f7447333
[kv-shard 2/4] Sharded pools (#37615 )
Shunkangz and Zhangheng
2026-09-16 19:22:23 +08:00
76e06febab
[CI] Add e2e test for dp-attention local control broadcast (#39437 )
2026-09-16 03:11:40 -07:00
7b7620774c
Fix device context during NIXL backend initialization (#38774 )
Aurick Qiao and Aurick Qiao
2026-09-16 02:59:24 -07:00
a3bf25dc62
[AMD] Clamp MORI intranode grid GPUs (#39763 )
inkcherry and Duyi-Wang
2026-09-16 17:13:59 +08:00
7d5696b3a1
dsv4.1: communication kernels and wrappers (#39653 )
+4
2026-09-16 02:09:40 -07:00
5a0c1e21e9
[AMD] Preserve deterministic inference when Lean Attention is enabled (#37740 )
Bingxu Chen
2026-09-16 17:09:08 +08:00
2cbfaefbf9
[AMD] Avoid the FP8 wo_a path when the weight is BF16 (#38453 )
Bingxu Chen
2026-09-16 16:57:15 +08:00
5212797880
[NPU] fix npu docker workspace directory (#39732 )
Jensen and ronnie_zheng
2026-09-16 16:51:38 +08:00
fc5a979f21
[misc] Merge FlashInfer autotune caches across spec workers, pad MXFP4 TP shards, drop dead ngram attrs (#39678 )
Liangsheng Yin and BBuf
2026-09-16 01:50:29 -07:00
0e528dc9ff
bugfix:fix unifiedcache c128 radix cache management (#39426 )
vstone-w
2026-09-16 16:47:42 +08:00
8baeded6f3
[Router] Pin to the prefix owner when the whole fleet is queueing (--saturation-queue-floor) (#39169 )
2026-09-16 01:16:24 -07:00
ad28b91fae
ci: fix always-failing coverage job, add by-GPU-count view (#39697 )
Alison Shao
2026-09-16 01:13:08 -07:00
9c8d4641ff
[Fix] Fix GLM5 mHC PP forward (#39720 )
Void
2026-09-16 15:32:53 +08:00
54f0d72adf
[Router] Fleet-wide sampling contract 2/3: enforce and inject per request (#39001 )
2026-09-16 00:29:33 -07:00
7835f1de9a
[Router] Add the shutdown-drain configuration surface (#39015 )
2026-09-16 00:26:50 -07:00
5beb2fd552
[NPU] Avoid device synchronization in Ascend sampling (#39404 )
Jensen
2026-09-16 15:15:16 +08:00
e2d56bbbfc
[Router] Bound the e2e worker memory budget so prefill graph capture stops OOMing (#39713 )
2026-09-16 00:13:09 -07:00
d634320e48
[Router] Count open HTTP exchanges until their response body finishes (#39014 )
2026-09-16 00:04:38 -07:00
faaff1eca8
dsv4.1: Top-k kernels and candidate selection (#39648 )
+4
2026-09-15 23:54:35 -07:00
a64be2e430
[Kernel] Add H20 block-FP8 MoE configs for GLM-5.3-Flash EP4/EP8 (#38913 )
Hank Han
2026-09-16 14:42:18 +08:00
6b33338242
[HiCache] Forward prefix metadata to v2 storage calls (#39567 )
Shuwen Wang and Zhangheng
2026-09-16 14:05:42 +08:00
91f691c490
dsv4.1: standalone kernels and Python wrappers (#39646 )
Liangsheng Yin
2026-09-15 22:31:18 -07:00
afde31a2f5
[router] Speak cleartext h2c on both edges: serve it inbound, forward it outbound (#39006 )
2026-09-15 22:30:31 -07:00
c1f5b4736a
[Router] Add --worker-queue-limit: stop sending cache-affinity traffic to a queueing worker (#39168 )
2026-09-15 22:24:48 -07:00
3f8eb35ead
[PD] Do not admit intake-rejected requests to a PD handoff (#38935 )
Shangming Cai and Claude Opus 5
2026-09-16 12:44:49 +08:00
100e1cd0d9
[Fix] Spawn, don't fork, the benchmark server process (#34712 )
Dayananda V and Claude Opus 5
2026-09-16 10:14:09 +05:30
5f17e3a75f
[Router] Fleet-wide sampling contract 1/3: the config surface (#39000 )
2026-09-15 21:39:06 -07:00
4678536df6
[CPU] Add fp8_per_tensor_scaled_mm_cpu kernel (#32618 )
Xinguo Zhu and AKatydid
2026-09-16 12:31:05 +08:00
f60652a43e
[AMD] Update deepseek-v4 PDI and cache policy setting for agentic workload (#39702 )
Thomas Wang
2026-09-16 12:16:30 +08:00
444b29c932
[Diffusion] Clean up obsolete worker plumbing, dead helpers, and tests (#39293 )
Xiaoyu Zhang
2026-09-16 12:11:45 +08:00
954bb6804a
[AMD][bugfix] Fix DSV4 MTP crash (#39547 )
Xinyi Song
2026-09-15 21:10:23 -07:00
537ac52c5b
[SRT] Clean up no-op compiler pass, dead helpers, and migration tests (#39295 )
Xiaoyu Zhang
2026-09-16 12:04:21 +08:00
6331e43081
[AMD] Prefer HIP Top-K for GLM-5.x on ROCm (#39631 )
Zhang, Jiejing
2026-09-15 20:55:57 -07:00
a9bb4d7d45
[AMD] Align Qwen3.5 MI355X HiCache cookbook with kernel / page_first (#39572 )
jacky.cheng and ChangLiu0709
2026-09-16 11:22:28 +08:00
5f6dd44edc
[NPU] [bugfix] Fix undefined require_attn_tp_allgather (#39561 )
chx96642264
2026-09-16 11:02:40 +08:00
935cbf24ec
Accept MXFP8 dispatch in FlashInfer A2A TRT-LLM MoE (#39613 )
Mohammad Miadh Angkad and Mohammad Angkad
2026-09-16 10:23:15 +08:00
d4ad368ed9
[Intel XPU] Enable fused_moe_triton tuning on XPU and add tuned DeepSeek-OCR-2 configs (#28723 )
Rahul Vijayaraghavan
2026-09-16 07:50:22 +05:30
f11cd8ab0e
[XPU] Use torch scaled_mm for XPU block FP8 linear (#35605 )
Cao E
2026-09-16 10:10:31 +08:00
2cb51f5d22
[CPU] [Diffusion] Add fused scale-shift and norm kernels for CPU (#33452 )
Yuxingwang-intel and Ma Mingfei
2026-09-16 10:01:29 +08:00
f920be4b09
[AMD] GLM-5.2 NextN: cast draft fused MoE to per-channel FP8 (#39155 )
Zhang, Jiejing and fanxingran
2026-09-15 18:56:48 -07:00
7eedd57ab0
[ROCm] Fix EAGLE spec-decode verify silently sampling greedy on HIP (#37134 )
2026-09-16 09:49:49 +08:00
2f7d04da5d
dsv4.1: Rust extension modules for image preprocessing, KV pool names, and PD bootstrap (#39677 )
Liangsheng Yin
2026-09-15 18:11:48 -07:00
dbd7281b06
fix(npu): fix hybrid KV transfer with PP prefill in PD disaggregation (#38402 )
Jinyan Yi
2026-09-16 09:10:17 +08:00
4e9e407d37
[HiCache] Label radix-cache metrics per rank and split the "shrunk" prefetch reason (#39280 )
Zhiqiang Xie
2026-09-15 17:54:04 -07:00
c9a8fba991
[DSV4][BCG] Optimize the heavy memory use of C4 Indexer when BCG is enabled (#36534 )
Siyuan Chen and Yuwei An
2026-09-16 08:23:11 +08:00
bbef93e22a
[Router] Add a KV storage-tier coverage row to the Grafana dashboard (4/4) (#39111 )
2026-09-15 17:09:11 -07:00
