This website requires JavaScript.
da1793f63a
update ascend feature docs (#20506 )
amote-i
2026-03-16 11:09:20 +08:00
f4393bf3f6
Fix correctness test issue for bench_one_batch (#20650 )
gaopengff
2026-03-16 11:05:36 +08:00
e1eb25880f
[Diffusion] Add a benchmark for rmsnorm/fuse_add_rmsnorm (#20632 )
Xiaoyu Zhang
2026-03-16 09:50:33 +08:00
e9fae69e5f
docs: align environment variable reference with environ defaults (#20419 )
AnonTokyo
2026-03-16 09:07:29 +08:00
3879c466b4
[CI] Add Nemotron 3 Super 120B nightly 8-GPU tests (#20616 )
Mohammad Miadh Angkad
2026-03-16 09:03:20 +08:00
35c249b4de
[OpenAI] Log raw request payload for --log-requests (#20605 )
Zhirui
2026-03-16 08:45:00 +08:00
d852f26cb6
Fix dual-stack socket handling: IPV6_V6ONLY, IPv4-first, is_port_available all-family check (#20643 )
Liangsheng Yin
2026-03-15 17:17:23 -07:00
7c498a6538
[DOC] add documents for encoder global mm cache (#20636 )
Teng Ma
2026-03-16 07:44:21 +08:00
c2245d9b55
ci: add runner dropdown to rerun-ut workflow dispatch (#20645 )
Liangsheng Yin and Claude Opus 4.6
2026-03-15 16:40:14 -07:00
53f831691a
fix: propagate grammar errors and improve llguidance backend (#20467 )
jellysnack
2026-03-16 02:11:18 +03:00
116aef8504
[Test] Move embedding tests into test/registered/embedding/ and unit/ (#20642 )
Liangsheng Yin
2026-03-15 14:48:43 -07:00
1145805e7d
Fix socket utilities and reserve_port for IPv6 dual-stack support (#20491 )
psaab and hnyls2002
2026-03-15 14:29:10 -07:00
8397a8a301
[CI] Extract wait-for-jobs composite action and gate on stage-a-cpu-only (#20641 )
Liangsheng Yin
2026-03-15 13:52:48 -07:00
23c191afb6
fix(docs): correct quantization documentation (#20301 ) (#20619 )
Mook
2026-03-15 09:33:12 -07:00
c3483e8e97
[CI] Move existing unit tests into unit directory (#20631 )
Ke Bao
2026-03-15 23:25:18 +08:00
e2be31824f
[CI] Add ut coverage tool (#20628 )
Ke Bao
2026-03-15 21:13:45 +08:00
45dd06f4e0
Remove insecure auto-format label workflow (#20629 )
Xiaoyu Zhang
2026-03-15 21:04:29 +08:00
0cff070828
[CI] Add READMEs for unit test directory structure (#20626 )
Ke Bao
2026-03-15 20:27:05 +08:00
7142a594f9
[CI]: Add CI For HiMambaRadixTree and qwen3.5 (#20540 )
hzh0425 and ispobock
2026-03-15 20:16:12 +08:00
1c456a0af5
VLM: add Conv2dLayer/Conv3dLayer to fix PyTorch 2.9.1 CuDNN Conv3d (#20282 )
Yuhao Yang and wili-65535
2026-03-15 19:17:44 +08:00
f07529b947
[diffusion] CI: use dedicated HF token for accessing restricted models (#20620 )
Mick
2026-03-15 19:15:28 +08:00
4f91b3069e
Skip flaky test in CI for disaggregation decode offload (#20622 )
Shangming Cai
2026-03-15 16:30:34 +08:00
7c773ddb0a
[Fix] Slice input_embeds to extend_input_len in prepare_for_extend (#20376 )
Kit Fraser-Taliente
2026-03-15 00:07:05 -07:00
7458407437
Fix InternVL and vision attention for non-CUDA backends (e.g. XPU) (#19997 )
Juan Muneton and Yang Wang
2026-03-14 23:24:41 -07:00
1ac6a26464
fix: Nemotron chunk size alias (#20458 )
shuwenn
2026-03-15 14:23:39 +08:00
fc7f9c1de7
Rename --stream-output to --incremental-streaming-output (#20614 )
Liangsheng Yin
2026-03-14 23:22:33 -07:00
538acb4c46
fix: Add .text property to HttpResponse to prevent AttributeError (#20518 )
shuwenn
2026-03-15 13:59:32 +08:00
a6ecf050be
diffusion: fix helios accuracy issue (#20036 )
Yuhao Yang
2026-03-15 13:55:51 +08:00
6c5bf53a36
[Doc] Clarify that --chat-template is required for Qwen3-Reranker (#20596 )
2026-03-14 16:43:48 -07:00
93afe15b43
chore: bump flashinfer version to 0.6.6 (#20480 )
sglang-bot and sglang-bot
2026-03-14 13:05:10 -07:00
574dbe23b2
Add piecewise cuda graph for Qwen3-Next FP8 flashinfer_trtllm moe backend (#18184 )
Xiaowei Wang
2026-03-15 04:03:31 +08:00
3e643967e6
[CI] Add Nemotron 3 Super 120B CI tests for BF16 and NVFP4 (#20575 )
Mohammad Miadh Angkad
2026-03-15 03:30:27 +08:00
39008955ff
Revert "[AMD][MORI] Fix MTP crash with FP4/FP8 dispatch and add NEXTN dispatch env vars." (#20602 )
Baizhou Zhang
2026-03-14 12:12:42 -07:00
5ab2cfe9a8
[Diffusion] Clean upstream fa3 in hopper (#20576 )
Xiaoyu Zhang
2026-03-14 23:41:23 +08:00
22e67876d6
[Omni] Optimize AudioEncoder for Qwen3_Omni_Thinker (#18185 )
Yuan Luo and luoyuan.luo
2026-03-14 23:00:17 +08:00
574aa2d723
[diffusion]: remove stale offload-manager cleanup in denoising stage (#20587 )
Ratish P
2026-03-14 20:26:57 +05:30
25e38216b6
[kernel slimming] Clean many useless sgl-kernel deprecated kernels (#20277 )
Xiaoyu Zhang
2026-03-14 16:45:54 +08:00
75a7879fd4
[Model] Support Nemotron 3 Super NVFP4 (#20407 )
Mohammad Miadh Angkad
2026-03-14 15:56:26 +08:00
c95dc88f86
[CI] migrate ascend-gptq from test/srt to test/registered (#19628 )
SoluMilken
2026-03-14 15:28:57 +08:00
f9e4221b71
[Diffusion] add mova and hunyuanvideo to perf skills (#20563 )
Xiaoyu Zhang
2026-03-14 13:49:50 +08:00
99a3b25c9b
[PP] Fix recv tensor dict potential race condition (#20341 )
Shangming Cai
2026-03-14 13:35:01 +08:00
c330b687a1
[Bugfix] Fix GLM-4.6V vision regression in glm4v_moe and glm_ocr (#20463 )
Xinyuan Tong
2026-03-14 04:48:28 +00:00
dfd0a77a9a
[bugfix] Add prev_prefix_len parameter to HiMambaRadixCache's _insert_helper() (#20539 )
ziruiliu
2026-03-14 09:54:14 +08:00
4659b08fcf
update CI_PERMISSIONS.json (#20551 )
Qiaolin Yu
2026-03-13 14:52:06 -07:00
0eea80bc00
[AMD][MORI] Fix MTP crash with FP4/FP8 dispatch and add NEXTN dispatch env vars. (#20453 )
Duyi-Wang
2026-03-14 05:03:17 +08:00
c37ef7f18b
[AMD] diffusion refactor: move ROCM VAE optimization to Platform abstraction (#20496 )
YC Tseng
2026-03-14 04:10:05 +08:00
d764f414a1
[AMD] fix mori unittest (#20524 )
billishyahao
2026-03-14 00:37:45 +08:00
a16f03c1e3
Revert "feat: add banner to sgl-model-gateway (#20471 )" (#20536 )
Simo Lin
2026-03-13 09:20:32 -07:00
654fc02cf1
[gRPC] Extract gRPC servicer into standalone package (#20478 )
Simo Lin
2026-03-13 09:13:29 -07:00
be7a0311a0
[Diffusion] Fix and validate diffusion skills benchmarking/profiling workflow (#20528 )
Xiaoyu Zhang
2026-03-13 21:11:37 +08:00
b1246c50f8
Fix chunked prefill and KV cache leaks for streaming sessions (#20476 )
2026-03-13 02:36:55 -07:00
287dc12b05
Fix hicache log metrics (#20504 )
Ke Bao
2026-03-13 16:29:58 +08:00
f8668d9e78
[Fix] Add fallback for flashinfer allreduce fusion (#20384 )
Baizhou Zhang
2026-03-13 01:24:55 -07:00
b638b25b22
[diffusion] UX: suppress excessive logging from httpx and httpcore (#20452 )
Mick
2026-03-13 14:43:09 +08:00
9c8777c80f
[AMD][Qwen3.5] aiter a8w8 gemm configuration (#19826 )
seungrokj and HaiShaw
2026-03-13 15:23:58 +09:00
70d4aabe42
Add CLI args to conveniently support tuning more models (#12922 )
Chongchong Tian and gemini-code-assist[bot]
2026-03-13 14:10:55 +08:00
63ecdcbb18
Expose async LoRA interface to Offline Engine (#18636 )
Antonin Vidon
2026-03-13 02:09:47 -04:00
d4e68ead1d
[quant] Ignore FP8 quantization layers (#20340 )
2026-03-13 13:59:39 +08:00
e00328d1e5
[Diffusion] Opt qwen-image-edit with fuse_residual_layernorm_scale_shift_gate_select01_kernel (#20395 )
Xiaoyu Zhang and Yihan Chen
2026-03-13 13:15:22 +08:00
197f807134
[RadixTree][7/N Refactor]: Refactor mamba radix tree, release dup kvcache in insert func (#19429 )
hzh0425
2026-03-13 12:28:32 +08:00
f605612b87
[HTTP] Fix /GET HTTP route when ollama endpoint is not set. (#20494 )
Liangsheng Yin
2026-03-12 20:54:32 -07:00
154af9e46c
update CI_PERMISSIONS.json (#20405 )
Mick
2026-03-13 10:50:35 +08:00
3deaf61345
misc: update CODEOWNERS (#20488 )
Yineng Zhang
2026-03-12 19:36:17 -07:00
7ecf07b8f4
[jit_kernel] Temporarily Skip Flaky JIT Kernel GDN Test and Add PR Label (#20436 )
Xiaoyu Zhang
2026-03-13 09:34:22 +08:00
65dd08153d
Fix Test* mixin classes being collected as standalone pytest tests (#20417 )
Pai Liu
2026-03-13 12:18:45 +11:00
9865f11421
[bugfix] fix bug when enable prefill delay and DP (#20134 )
LinyuanLi
2026-03-13 08:51:09 +08:00
c21ddbc785
[Minor] fix type annotations and invalid method calls in constrained … (#20132 )
zzhpro
2026-03-13 07:42:46 +08:00
78a467c74a
[AMD] [diffusion] feat: enable AITer GroupNorm for VAE decode on ROCm (#20170 )
YC Tseng and HaiShaw
2026-03-13 02:38:19 +03:00
f5a4a5429f
Revert early HTTP port reservation (#17754 , #19805 ) (#20468 )
Kangyan-Zhou and Claude Opus 4.6
2026-03-12 16:17:33 -07:00
9c6f166600
tiny fix the usage of test_logprobs (#20473 )
Qiaolin Yu
2026-03-12 15:37:45 -07:00
b227e53ebf
feat: add banner to sgl-model-gateway (#20471 )
Simo Lin
2026-03-12 14:25:27 -07:00
af2807e146
[LoRA][I] Add MOE LoRA JIT alignment kernel and tests (#19710 )
2026-03-12 12:23:46 -07:00
e29305c120
[GDN] Add benchmark for sglang gdn prefill (#20428 )
2026-03-12 22:25:02 +08:00
a57a44739f
[diffusion] deps: upgrade diffusers from 0.36.0 to 0.37.0 (#20318 )
Yuhao Yang
2026-03-12 19:17:28 +08:00
318a40fdfb
[Bug-fix] Fix gpu fault when run the test with dp-attention-enabled and max-concurrency is over 256 (#20399 )
kk and wunhuang
2026-03-12 17:32:03 +08:00
4e5ca92249
[diffusion]: clear file-path-only outputs on all ranks to prevent TP GPU memory skew (#20353 )
Ratish P
2026-03-12 14:59:09 +05:30
5f1bfb0d28
[Security] Fix /rerun-ut bypassing run-ci gate for fork PRs (#20424 )
Liangsheng Yin and Claude Opus 4.6
2026-03-12 02:24:29 -07:00
1e2983c98e
[AMD] Fix FP8 assertion failure in aiter MLA decode by falling back to self.k_scale (#19935 )
jacky.cheng
2026-03-12 16:48:51 +08:00
067353f67b
[Test] Refactor KL divergence and prefix cache branching to kits (#19715 )
roikoren755
2026-03-12 10:11:59 +02:00
46b558445d
Fix default_max_tokens compute error in responses api when mtp is opened (#18932 )
0xNullPath
2026-03-12 16:00:48 +08:00
ac1310b300
Fix wrong BaseKVCacheMethod import in test_modelopt_fp8kvcache.py (#20403 )
Pai Liu
2026-03-12 18:23:46 +11:00
2c03a5c6c7
Fix global server args not set error in test_triton_moe_wna16.py (#20412 )
Pai Liu
2026-03-12 18:22:59 +11:00
91f701e5f1
[AMD][Mori] Add unittest for mori ep low latency and tbo (#20374 )
billishyahao
2026-03-12 13:38:26 +08:00
dd82678b2d
[NPU] Support mamba cache transfer for NPU (#20364 )
Hexq0210
2026-03-12 12:49:21 +08:00
abc672e717
[Benchmark] use flashinfer bench_gpu_time instead of triton do_bench (#20305 )
Mook
2026-03-11 21:04:30 -07:00
ae7c2397b9
Fix FA3 swa spec pg_size > 1 (#20369 )
Ke Bao
2026-03-12 11:42:01 +08:00
649d6f2bc8
[GDN] Change Attention State Layout from [N, HV, K, V] to [N, HV, V, K] (#20283 )
Yuan Luo
2026-03-12 10:53:12 +08:00
8787cf4566
Fix the scope of io_backend in NSATokenToKVPoolHost (#20327 )
huangtingwei
2026-03-12 10:33:11 +08:00
7771fcd4b3
Update CodeOwners (#20329 )
Baizhou Zhang
2026-03-11 19:11:09 -07:00
9b55a98a67
perf(qwen3_5): replace einops rearrange with torch.flatten in GatedDe… (#20386 )
Vedant V Jhaveri
2026-03-11 18:51:27 -07:00
25bd83033d
Enable Piecewise CUDA Graph for NemotronH Hybrid (Mamba+Attention) Models (#19903 )
Vedant V Jhaveri
2026-03-11 18:16:38 -07:00
677e446e51
[NPU] Convert cu_window_seqlens to CPU for npu_flush_attention_unpad operator (#20328 )
fy
2026-03-12 09:08:43 +08:00
67f02681c9
[AMD] Support speculative decoding v2 for aiter backend on ROCm/HIP (#17450 )
2026-03-11 17:01:01 -07:00
acab24a76a
fix: gracefully abort last request in retract_decode on OOM (#19881 )
shuwenn
2026-03-12 06:13:03 +08:00
88d2fc19b1
feature: support X-Data-Parallel-Rank header to specific dp-rank. (#19832 )
doujiang24
2026-03-12 05:53:33 +08:00
af4c28904d
[PD] Fix the infinite loop in deocde resolve_pending_reqs (#20371 )
Shangming Cai
2026-03-12 05:11:19 +08:00
ab4b863546
fix ci by removing nvidia-cutlass-dsl-libs-base and force reinstall n… (#20380 )
Rain Jiang
2026-03-11 13:37:33 -07:00
2720ea2667
[Typo] Fix H200 doc links pointing to H20 section in deepseek_v3.md (#20383 )
Mook
2026-03-11 13:35:20 -07:00
252ef90fc2
[Generative Score API] Fix on prefill-only scheduler running batch loss track problem (#14320 )
2026-03-11 13:15:50 -07:00
a54d71e967
[Benchmark] Add sglang-embedding backend to bench_serving (#20017 )
satyamk7054 and Satyam Kumar
2026-03-11 13:13:16 -07:00
amote-i