-
45194794d0
Apply gemma's position offset out-of-place instead of in-place (#26799)
fzyzcjy
2026-05-31 09:47:50 +08:00
-
13ca55afa3
Make qwen3's set_embed_and_head idempotent (#26798)
fzyzcjy
2026-05-31 09:47:07 +08:00
-
9b4be9c574
[core] Compute dimensions/return_pooled_hidden_states in ForwardBatch.init_new (#26779)
Liangsheng Yin
2026-05-30 17:38:01 -07:00
-
7ed53d15f3
[config] Recognize custom hybrid SWA models via hf_text_config.is_hybrid_swa (#23988)
Bruce Changlong Xu
2026-05-30 08:21:10 -07:00
-
e279b0bf72
Optimize large add_constant tensors (#24755)


2026-05-30 22:25:19 +08:00
-
b421e60eed
【NPU】【bugfix】fix server error when mtp unquant (#26389)


2026-05-30 20:01:19 +08:00
-
282c46133f
[Scheduler] Defer prefill input_ids H2D to forward stream, unify resolve via future_map (#25945)
Liangsheng Yin
2026-05-30 02:58:13 -07:00
-
acd689b407
feat: add SGLANG_RAY_BUNDLE_INDICES for fine-grained Ray bundle index control (#24667)
Haichuan Hu
2026-05-30 17:19:50 +08:00
-
90eb894564
[NPU] fix model llava-onevision-qwen2-7b-ov torch compiles error in npu case (#26573)
ZeyuanChen2000
2026-05-30 17:10:58 +08:00
-
714bcd84e2
[parallel] Support moe_dense_tp_size == attn_tp_size to share the attention TP group (#23996)
Bruce Changlong Xu
2026-05-30 02:10:07 -07:00
-
a952e9174f
[MLX] Support Qwen3.5 (dense) Model (#25754)


2026-05-30 17:05:02 +08:00
-
7c5708cba7
[DeepSeek-V4] Add mhc_fused_post_pre kernel (#25976)

Qichao LiandQichao Li
2026-05-30 17:04:51 +08:00
-
0d9a2a9de3
[MoE Refactor] Migrate SM90 Cutlass W4A16 to MoeRunner (#26489)

Yuan Luoandluoyuan.luo
2026-05-30 17:02:56 +08:00
-
c93a559e5a
Fix MoE LoRA wrapper exposing moe_runner_config (#26710)
huangtingwei
2026-05-30 16:59:15 +08:00
-
3b61a1f935
[Bugfix] Optimize metadata allocation and transfer for mooncake intraNode NVLink (#26707)


2026-05-30 16:52:01 +08:00
-
edfe8d34e8
[CI] ci-coverage-overview: schedule + manual only, include XPU/MUSA/multimodal_gen (#26619)
Alison Shao
2026-05-30 01:39:18 -07:00
-
b4bf489dea
ci: allow /rerun-test to dispatch nightly/weekly CUDA tests (#26624)
Alison Shao
2026-05-30 01:35:01 -07:00
-
716e670d3d
[bugfix]: size CuteDSL MoE allgather buffers for the worst-case forward (#26696)
Jimmy Shong
2026-05-30 00:27:20 -07:00
-
7662210406
[UnifiedTree]: Support eviction priority (#26549)
Zhangheng
2026-05-30 15:19:43 +08:00
-
c048ebd10d
[Hicache]: skip flaky test (#26764)
Zhangheng
2026-05-30 15:16:12 +08:00
-
02aeed5387
[NPU] DFlash Speculative Decoding Support NPU (#23122)
gjsheu
2026-05-30 15:13:59 +08:00
-
fe4b29d391
[Bugfix] Fix Ascend NPU CP attention for batch size > 1 (#26705)
AndyLi429
2026-05-30 15:07:39 +08:00
-
23a825c694
[DOC] [NPU] add qwen3.5-397b best practice to doc_new (#26709)
silencejade
2026-05-30 14:20:24 +08:00
-
1c79015434
Drop dead ScheduleBatch return_routed_experts/return_indexer_topk fields (#26760)
Liangsheng Yin
2026-05-29 23:16:09 -07:00
-
6f1c9fc77b
[RL] Fix crash when the reqs in a batch have a mix of
return_routed_experts = True and False. (#26423)



2026-05-29 20:46:10 -07:00
-
804f01a823
Allow PR test and lint workflows to trigger on non-main bases (#26721)
fzyzcjy
2026-05-30 10:22:59 +08:00
-
1f850e67f2
[core] Fix crashes on the
gpu_only spec_v2 path (#26738)
Liangsheng Yin
2026-05-29 19:12:03 -07:00
-
c2ac37dbcc
[Bug] ngram verify: keep
batch.seq_lens_sum in sync after accept (#26753)
Liangsheng Yin
2026-05-29 18:00:11 -07:00
-
95cd2fd29f
[lora] More efficient pinned memory (#20876)
Erik Wijmans
2026-05-29 17:04:59 -07:00
-
a5e6a8887a
[attention] Fallback to Triton merge_state when FlashInfer hits CUDA thread limit (#23993)
Bruce Changlong Xu
2026-05-29 16:30:49 -07:00
-
6ea69efb7f
[RL] Forward Kimi K2.5 weight hooks to language model (#26744)

Byron HsuandByron Hsu
2026-05-29 15:08:58 -07:00
-
6ce49e5f4c
[Utils] Support configure log level at runtime (#26583)
Chao Shi
2026-05-30 05:49:06 +08:00
-
cf66693b35
[Model] Add Qwen3-MoE MTP (#26468)



2026-05-29 14:31:05 -07:00
-
4ff1296f5e
Optimize get load calls (/v1/loads) using shared-memory load snapshots (#26348)

Lianmin Zhengandcctry
2026-05-29 13:40:26 -07:00
-
3cecc77ccb
[perf] Fuse NVFP4 gate_up_gemm + swiglu + output FP4 quant (#26626)
Qiaolin Yu
2026-05-29 13:16:24 -07:00
-
6b5f0d0ccb
[core] Make spec_v2
seq_lens_cpu optional via backend needs_cpu_seq_lens; Triton opts out (#26128)

Liangsheng YinandQiaolin-Yu
2026-05-29 13:00:32 -07:00
-
ff8ed7a302
[refactor] unify cuda-graph capture/replay across attention backends (#26665)

Cheng WanandClaude Sonnet 4.6
2026-05-29 12:46:42 -07:00
-
7fb7b41a3e
[docs] Qwen3.5 cookbook: multi-node, MTP TP overrides, dense mamba flag (#26695)
![gemini-code-assist[bot]](/assets/img/avatar_default.png)

2026-05-29 12:10:14 -07:00
-
f113ece5cc
Revert "improve: combine vit calls for images from different reqs from one batch (#25910)" (#26442)

Bingxu ChenandCursor Agent
2026-05-30 02:13:34 +08:00
-
ec075d8bc5
Fix DRAFT_EXTEND_V2 CG metadata: align test fixture and Triton with production seq_lens convention (#26651)

Cheng WanandClaude Sonnet 4.6
2026-05-29 02:46:45 -07:00
-
eb5d4827e8
[UnifiedTree]: Split unified tree kl ci into multiple files to reduce GPU usage. (#26666)
Zhangheng
2026-05-29 17:46:34 +08:00
-
4585f8eb95
[refactor] remove unused op_mlp (#26673)
akhoroshev
2026-05-29 16:38:56 +07:00
-
8652001b6a
fix: use req.req_pool_idx instead of loop variable for req_to_token i… (#26534)

chenxb002andkjuuii
2026-05-29 17:34:42 +08:00
-
ed85bcf8c3
pin kernels<0.15 (#26704)
Liangsheng Yin
2026-05-29 01:46:57 -07:00
-
544f3039d5
[PD] Fix IB device validation for JSON mappings (#26114)
Teng Ma
2026-05-29 16:44:49 +08:00
-
9062f583db
[ROCm] Eliminate redundant contiguous copy in MLA attention on ROCm MXFP4 (#25463)
Rita Brugarolas
2026-05-29 01:28:32 -07:00
-
05bef58297
[Misc] Tiny change default building branch of sgl-deep-gemm (#26699)
Baizhou Zhang
2026-05-29 01:18:36 -07:00
-
3e8d8736c8
fix stage-b-test-2-npu-a2 image (#26698)
monkeyLoveding
2026-05-29 16:09:52 +08:00
-
a722b1a437
[CPU] fix incorrect index of b_ptr in fused_sigmoid_gating_delta_rule… (#26634)
blzheng
2026-05-29 16:08:12 +08:00
-
3ecf2c76ad
[CPU] Add GPT-OSS model optimization for CPU (#16775)


2026-05-29 16:05:26 +08:00
-
5601b7139d
[core] Make overlap-schedule WAR barrier CUDA-only (#26646)
Bingxu Chen
2026-05-29 16:02:31 +08:00
-
4d1163e6a9
[PD][MoRI] Align hybrid state transfer with per-component schema (#26539)
Niko Ma
2026-05-29 15:54:46 +08:00
-
a42a7654a2
Update MooncakeStore batch tests to use v1 APIs (#25880)
Chizheng Fang
2026-05-29 15:18:05 +08:00
-
ace730db48
[AMD] Work around HIP TPOT regression from Event.wait() in MTP seq lens resolution (#26672)
Arik
2026-05-29 15:18:02 +08:00
-
6e9bd82714
[AMD][CI] Update v4 CI setting and move the task to main branch (#26662)
Thomas Wang
2026-05-29 15:12:40 +08:00
-
b2eed9e16d
[Apple Silicon] Add custom Metal RoPE kernel with fused KV cache store (#22868)

Aditya SharmaandXiaodong Ye
2026-05-29 12:39:33 +05:30
-
7dff4118b9
Ignore
.humanize folder (#26680)

Jiajun LiandJiajun Li
2026-05-29 00:09:21 -07:00
-
08ec19872c
[HotFix][Ling 2.6] Fix HybridLinearAttn dispatcher for Ling-2.6 (#26474)

Yuan Luoandluoyuan.luo
2026-05-29 14:50:15 +08:00
-
0fb0ea7aac
test: add trtllm_mha EAGLE-draft CG runner coverage (chain) (#26669)

Cheng WanandClaude Sonnet 4.6
2026-05-28 23:46:06 -07:00
-
69362cbc2c
[Doc] Update benchmark instruction for dsv4 (#26668)
Baizhou Zhang
2026-05-28 23:37:35 -07:00
-
5850aa14c3
fix(mooncake): honour MOONCAKE_PROTOCOL so EFA hardware can select efa transport (#25083)

王鹤男andwhn09
2026-05-29 14:21:15 +08:00
-
73c99e3361
Ensure multi-node MM embedding cache consistency in insert_batch (#25959)



2026-05-29 13:59:46 +08:00
-
2dfbc3d781
test: strengthen CG-replay coverage with prod-fill padding, metadata invariants, and pad-ratio sweep (#26658)

Cheng WanandClaude Sonnet 4.6
2026-05-28 22:43:29 -07:00
-
226649e3b7
[Fix] Fix FP8 Online Quantization (#26415)

Colin Zandyichiche@amd.com
2026-05-28 22:00:09 -07:00
-
f16816f043
fix: copy seq_lens in TRTLLM MHA draft decode cuda graph capture (#26521)
Yongji Wu
2026-05-28 21:55:33 -07:00
-
b47366fbf9
[NPU]: Optimize xgrammar token bitmask on NPU with AscendC (#24133)
yuefeng Wu
2026-05-29 12:38:06 +08:00
-
fba083c80f
[CI] Split PP tests into base and extra suites (#26648)
Shangming Cai
2026-05-29 12:15:27 +08:00
-
40f91e6697
[Bugfix] [DSA] [Hisparse] Broadcast TP Rank 0 Topk Indexes to other TPs (#24654)


2026-05-29 12:14:46 +08:00
-
84698b21e7
rename unittest as unittests (#26654)
Cheng Wan
2026-05-28 21:14:06 -07:00
-
3ea9607d1c
[diffusion] model: update to new model format (#26492)
Dawid Majchrowski
2026-05-29 06:08:45 +02:00
-
621a79728c
test: stabilize Gemma4 26B-A4B MTP GSM8K test with deterministic inference + tuned threshold (#26653)

Khoa PhamandCursor
2026-05-28 20:58:48 -07:00
-
dc4e7bc479
Fix TRTLLM MHA draft decode cache seqlens replay (#26655)
Lianmin Zheng
2026-05-28 20:58:16 -07:00
-
79c844527c
Upgrade xgrammar to 0.2.1 (#25676)
Xinyuan Tong
2026-05-29 04:40:07 +01:00
-
272066566f
[AMD] Pin compressed-tensors<0.16.0 for srt_hip (fixes ROCm 7.2 nightly build) (#26591)
YC Yen-Ching Tseng
2026-05-29 11:34:45 +08:00
-
b1173c8c14
[NPU] Enhance accuracy for model Step3_5 from 0 to 88% (#24582)
McZyWu
2026-05-29 11:29:30 +08:00
-
36d0a6e08e
[EPD] Optimize the Mooncake backend (#22587)

LucQueenandZhengWG
2026-05-29 10:42:24 +08:00
-
569ee93357
[AMD] ci: switch CACHE_HOST to a fresh path to fix "No space left on device" (#26642)
Michael
2026-05-29 10:40:54 +08:00
-
7619e77b7b
[NPU] chore: basic software upgrade (#26466)
monkeyLoveding
2026-05-29 10:04:15 +08:00
-
54b06f199c
[lora] Share MoE LoRA Info (#24160)
Erik Wijmans
2026-05-28 19:01:47 -07:00
-
e381312664
Revert "Fix FA DRAFT_EXTEND_V2 cache extent" (#26628)

Cheng WanandClaude Sonnet 4.6
2026-05-28 18:45:33 -07:00
-
0f8104ef15
[XPU] Fix Device Assignment (#26257)
Rohit Kumar Singh
2026-05-29 07:08:11 +05:30
-
6258947039
NPU Nightly Pipeline Skip Test Case Adaptation and Recovery Testing (#26353)
liuxianglong17
2026-05-29 09:29:25 +08:00
-
1c2857b064
bugfix: --decrypted-draft-config-file not applied (#25960)
McZyWu
2026-05-29 09:15:05 +08:00
-
fffdfb6fcb
[Fix][NPU] Preserve existing packed_modules_mapping when merging model-level fused module mappings (#25755)
Kurkur
2026-05-29 09:11:36 +08:00
-
a8cfae0b30
doc: update step-3.7-flash docker image tag (#26625)
Yuhao Yang
2026-05-29 08:40:17 +08:00
-
f66f56c6bd
Add attention-backend unit-test suite under test/registered/attention/unittest (#26517)

Cheng WanandClaude Opus 4.7
2026-05-28 17:30:31 -07:00
-
3bdea78ad1
model: support Step-3.7-Flash (#26565)


2026-05-29 08:00:54 +08:00
-
0597242797
[CI] /rerun-test: descend into directories that a glob matched (#26614)
Alison Shao
2026-05-28 16:52:42 -07:00
-
407106300e
test/disaggregation: dp-attention keeps total_tokens e2e, simpler LB algos -> unit tests (#26617)
Liangsheng Yin
2026-05-28 16:50:46 -07:00
-
3e255fd493
fix: Graceful fallback to CustomAllReduce when full_nvlink is not True (#25650)
Yujun Dong
2026-05-29 07:28:33 +08:00
-
4f92e63c99
Let unittest._ShouldStop propagate through retry() so subTest+failfast works (#26616)

Cheng WanandClaude Opus 4.7
2026-05-28 16:26:17 -07:00
-
ec78fa6518
test/registered: cleanup pure model e2e tests (moves, splits, dedup, kit) (#26610)
Liangsheng Yin
2026-05-28 15:41:46 -07:00
-
435c4ffb30
[CI] Clean DeepSeek V4 tests and installation scripts (#26609)
Baizhou Zhang
2026-05-28 15:22:59 -07:00
-
f838adb7d4
bench_serving: add Zipfian shared-prefix sampling to generated-shared-prefix (#26378)
Jimmy Shong
2026-05-28 14:39:46 -07:00
-
97d129f8c6
# feat(bench): add SPEED-Bench dataset support to bench_serving (#24149)


2026-05-29 00:37:00 +03:00
-
cd65be98df
[CP] Add back qwen 30b test (#26604)
Baizhou Zhang
2026-05-28 14:34:41 -07:00
-
93445e6359
[spec decoding] support kimi-k2.6-eagle3.1-mla draft (#26506)

Khoa PhamandCursor
2026-05-28 14:26:39 -07:00
-
68706e615a
[SPEC] fix: use effective max draft tokens for adaptive spec initiali… (#26354)

shuwennandmaodoudou168
2026-05-29 04:33:11 +08:00
-
3ca8cb470f
[BugFix] preserve cached token details in multi-tokenizer output (#26590)
Cameron Quilici
2026-05-28 15:31:55 -05:00
-
690d4cdd94
Revert "[CI] FA3: ascending cuda-graph capture to avoid varlen workspace IMA (#26532) (#26550)" (#26600)
Mohammad Miadh Angkad
2026-05-29 04:17:37 +08:00