This website requires JavaScript.
7c90840bad
[CI] Key scheduled CUDA suites by runner_config instead of hand-written jobs (#34186 )
Liangsheng Yin
2026-08-09 16:44:53 -07:00
4a5d7d3c67
[DSV4] Fix silent KV corruption when speculative draft tokens > 4 (#34189 )
Liangsheng Yin and Deleter-D
2026-08-09 15:54:41 -07:00
57f2105118
docs(skill): record where config is read now that the seed is off limits (#34097 )
Cheng Wan
2026-08-09 14:46:14 -07:00
b4284f3eb7
config: the KV-cache configurator reads the bags (#34096 )
Cheng Wan
2026-08-09 14:45:42 -07:00
e216c2bc59
config: the runner and scheduler read resolved config from the bags (#34095 )
Cheng Wan
2026-08-09 14:45:11 -07:00
a2199c1dee
config: pin that resolution is reproducible from the raw input (#34094 )
Cheng Wan and Claude Fable 5
2026-08-09 14:44:39 -07:00
63833f8034
config: business code no longer reads the published ServerArgs (#34081 )
Cheng Wan
2026-08-09 14:44:08 -07:00
110bf7e6a8
config: retire the hidden global fallbacks and the mamba-extra-buffer instance reads (#34080 )
Cheng Wan
2026-08-09 14:43:28 -07:00
967cac801c
[AMD] [CI] Register the DeepSeek-V4-Pro-DSpark MI35x nightly job so its suite actually runs (#34147 )
Michael
2026-08-09 14:10:28 -07:00
bfeb9a8af2
[feat] Add language_model_only parameter support for Qwen35 (#22867 )
2026-08-10 02:47:36 +08:00
c2fbe2f6d8
Add skill for the logprob consistency tests (#34169 )
Ke Bao
2026-08-10 01:33:00 +08:00
168eba3257
[Fix] Speculative decoding crashes with DP-Attention (#33892 )
Eric Zhang
2026-08-09 12:56:36 -04:00
2ab96d0fe0
Fix mlx unit test batch mock (#34181 )
Ke Bao
2026-08-10 00:55:55 +08:00
fb72a37fde
Add deterministic logprob-consistency test for inkling-small nvfp4 (#34168 )
Ke Bao
2026-08-09 22:22:27 +08:00
3fe65e0654
Deterministic gumbel sampling: clamp u=1 so masked tokens can't be sampled (#33423 )
Kurt Shuster and Claude Fable 5
2026-08-09 09:43:27 -04:00
fc40684b32
[srt] Fix sconv state memory corruption on specdec (#34043 )
Eric Zhang
2026-08-09 09:42:54 -04:00
11d03eaeef
runtime: Add flashinfer rmsnorm + quant fusion support SM90, SM100, SM120- #32994 (#33471 )
2026-08-09 05:15:03 -07:00
7120f3ee13
fix: support FA4 backend for GLM4.7-flash (#33436 )
Yuzhen Zhou
2026-08-09 05:05:38 -07:00
71043b9dbb
Revert parallel request lifecycle tracking from #32588 (#34160 )
Lianmin Zheng
2026-08-09 03:10:02 -07:00
fcc5468cce
Fix deterministic inference all-reduce for tp>1 (#34159 )
Ke Bao
2026-08-09 18:08:14 +08:00
c500674124
Switch inkling per-commit test to nvfp4 (#32402 )
Ke Bao
2026-08-09 16:49:49 +08:00
78cd60b4e3
[srt] Reuse batched Mamba boundary mask (#33477 )
Leon Gao
2026-08-09 01:47:05 -07:00
51470b376f
[diffusion] feat: support sol-attn sparse attention backend for h3 (#33702 )
WenhaoZhang
2026-08-09 16:26:06 +08:00
ce1b9f88b6
config: derive the runner's DCP topology from its ParallelState (#34133 )
Khoa Pham and Claude Opus 5
2026-08-09 01:18:24 -07:00
22e003580b
[Kimi K3] optimize: preprocess cpu-transport images on the vision owner (#33921 )
Mick
2026-08-09 16:13:20 +08:00
d0aa37b49b
[diffusion] fix: update weight from tensor detects device by uuid (#32685 )
Kangrui Du
2026-08-09 01:11:28 -07:00
bc285b2064
refactor: clean up logits processor helpers (#34158 )
Lianmin Zheng
2026-08-09 00:22:57 -07:00
1ebd6fab6c
[CI] Gate Kimi-K3 acceptance length on the GSM8K average (#34145 )
Liangsheng Yin
2026-08-08 20:28:42 -07:00
4792ab1e90
[CI] Pin the rust frontend parity test to eager prefill (#34146 )
Liangsheng Yin
2026-08-08 20:07:19 -07:00
f6cbdc1dd1
docs(diffusion): refresh skills for latest runtime (#34143 )
Xiaoyu Zhang
2026-08-09 10:30:46 +08:00
38c007dfe5
[diffusion] FLUX.1: route the adaLN LN+modulate sites through the bit-exact fused LayerNorm+modulate kernel (H200 1024^2 lossless denoise -1.2%, e2e wall -2.9%) (#34126 )
Xiaoyu Zhang
2026-08-09 09:52:37 +08:00
6424fec326
[diffusion] Bit-exact data-movement elimination for the Wan causal VAE decoder (H200 LongLive2 704x1280x61f: decode 2.80->2.32 s lossless / 2.12->1.67 s quality=high, e2e -10.7%) (#34125 )
Xiaoyu Zhang and Claude Fable 5
2026-08-09 09:50:56 +08:00
33ed5d4413
[diffusion] perf_logger: SYNC_STAGE_PROFILING must drain the GPU queue for stage records too (fixes 2-3x inflated DecodingStage readings) (#34124 )
Xiaoyu Zhang and Claude Fable 5
2026-08-09 09:49:52 +08:00
cea16bf229
Fix Qwen3-MoE producing garbage with the mori a2a backend (#34006 )
Yuankai Chen
2026-08-09 06:44:01 +08:00
ba7abd4f92
[AMD] Support DeepSeek V4 DSpark on AMD HIP platform (#30964 )
Wang, FangYuan
2026-08-09 06:22:14 +08:00
a59bb931c6
Fix DCP KV head mapping for GQA models (#32858 )
2026-08-08 14:25:09 -07:00
c2d90db1e3
[CI] Add Kimi-K3 low-latency performance check (#34089 )
Mohammad Miadh Angkad
2026-08-09 04:42:59 +08:00
c4f018ba1d
[Refactor] Separate ROCm-specific DeepSeek MHA and MLA forward paths (#31531 )
Diya Peng
2026-08-08 15:39:36 -05:00
3fbb5330c7
Fix the router GEMM inaccuracy when using _front_w in Kimi-K3 (#33764 )
Brayden Zhong
2026-08-08 12:23:40 -07:00
dc9624deb2
[diffusion] Clean up kernels and shared fast paths (#34085 )
Xiaoyu Zhang
2026-08-09 00:37:00 +08:00
ec5199b906
[jit_kernel] Fix missing JIT kernel namespaces (#34106 )
Mohammad Miadh Angkad and BBuf
2026-08-08 22:15:05 +08:00
f3ed82b3a8
[inkling] Let Anthropic thinking=disabled map to reasoning effort "none" (#33913 )
Eric Zhang and Claude Opus 5
2026-08-08 09:59:32 -04:00
548ff545c5
[diffusion] fix: guard sage attention sm90 bindings (#34107 )
2026-08-08 16:32:40 +03:00
db75dfe10f
fix: always capture default prefill CUDA graph (#33352 )
Mick
2026-08-08 19:24:49 +08:00
cf2d4fd679
docs: clarify K3 VLM feature transport (#34099 )
Mick
2026-08-08 19:23:00 +08:00
cfb354bcfc
[Bugfix] Fix batched KV free aliasing (#34067 )
Leon Gao
2026-08-08 03:34:44 -07:00
a1ca76b24b
[Scheduler] Unify WAR read-done gating behind shared-read boundary declarations (#34052 )
Liangsheng Yin
2026-08-08 03:26:36 -07:00
2c0188cc78
[Fix] Give the piecewise CUDA graph test stub an hf_config (#34100 )
Liangsheng Yin
2026-08-08 02:04:16 -07:00
f6a6f5bf1e
[CI] Trim redundant nightly test registrations (#34070 )
Liangsheng Yin and Baizhou Zhang
2026-08-08 01:42:46 -07:00
dd5d82bead
[NPU] Improve the execution efficiency and maintainability of pr‑test‑npu (#33724 )
2026-08-08 16:42:44 +08:00
e732c0a9dc
[UT][NPU] add unit tests for ascend_torch_native_backend and mla_preprocess (#32505 )
xdtbynd
2026-08-08 16:15:12 +08:00
4ad5bb5d9a
[jit_kernel] Move JIT kernels into namespace sglang (#33400 )
DarkSharpness and Claude Fable 5
2026-08-08 16:10:15 +08:00
5fdf6cd18f
[MoE] Single-launch moe_align for tiny batches with many experts (#32395 )
Yuan Luo
2026-08-08 16:08:25 +08:00
891445676c
[diffusion] Sana: bit-exact fused aten LayerNorm+modulate under BCG (H200 denoise -4.8%) (#34015 )
Xiaoyu Zhang and Claude Fable 5
2026-08-08 16:05:25 +08:00
d747bd052e
feat(vlm): auto-select cuda vmm on multi-node mnnvl (#33936 )
Mick
2026-08-08 16:00:58 +08:00
db3898fec1
fix: avoid piecewise prefill graph for trtllm_mla (#32785 )
Mick
2026-08-08 16:00:10 +08:00
afb4f37ca5
[Inkling] silu_and_mul: replace helion kernels with plain Triton (#33903 )
Sam Shleifer and Claude Fable 5
2026-08-08 03:00:23 -04:00
d238e36b24
[Fix] Restore data_parallel_rank alias on native /generate (dp-aware gateway routing is silently dropped) (#33565 )
Sam Shleifer and Claude Fable 5
2026-08-08 02:59:16 -04:00
c9444deef4
Docker: install DeepEP from release wheels (#34041 )
Baizhou Zhang
2026-08-07 23:51:09 -07:00
c69d59395b
[inkling] Render tool-result media instead of coercing content to str (#33898 )
Eric Zhang and Claude Opus 5
2026-08-08 02:39:19 -04:00
6185ed8011
Add registered short-conv tests and backend extensions (#34045 )
Aurick Qiao
2026-08-07 23:38:03 -07:00
6679d9b60c
[AMD] Add K3 verified mla kernel for DSpark on triton backend (#33981 )
Thomas Wang
2026-08-08 13:59:26 +08:00
b61a06921e
moe: the shared-experts-fusion decision is a per-runner value the loader installs (#33889 )
Cheng Wan
2026-08-07 22:42:58 -07:00
eda0ddc260
config: delete the dead get_server_args() bindings across the repo (#33888 )
Cheng Wan
2026-08-07 22:41:53 -07:00
a5af27f49e
config: retire ServerArgs.derive; per-runner values are constructor arguments (#33887 )
Cheng Wan
2026-08-07 22:41:09 -07:00
5dffa06fe1
[diffusion] GLM-Image bit-exact fused aten LayerNorm+modulate / qk-LN (H200 30-step denoise -8.1%) (#34008 )
Xiaoyu Zhang and Claude Fable 5
2026-08-08 13:26:11 +08:00
148f15b0af
[diffusion] FLUX.1 fused adaLN modulate (bit-exact) + RoPE cache hoist, LN-affine folding behind quality=high (H200 e2e -3.5% lossless / -6.9% high) (#34004 )
Xiaoyu Zhang
2026-08-08 13:07:42 +08:00
f64328c7f6
[diffusion] feat: support quant-videogen prq kv-cache quantization (memory-saving) for causal-dit (#32581 )
2026-08-07 21:56:54 -07:00
24c84dfa68
[diffusion] doc: add parallelism overview (#33704 )
Mick and Claude Fable 5
2026-08-08 11:36:06 +08:00
55f02e6887
Support Intern-S2-Mobius (#33691 )
RunningLeon and Xinyuan Tong
2026-08-08 11:06:57 +08:00
a25c330eb1
[diffusion] feat: cross-node sequence parallelism (Ulysses x Ring) (#33327 )
Mick and Claude Fable 5
2026-08-08 10:56:28 +08:00
52afe87a08
[diffusion] fix: stop runai-model-streamer's rank-discovery collective from firing on independent per-rank loads (#33969 )
Mick and Claude Fable 5
2026-08-08 08:54:08 +08:00
209857334e
[Scheduler] Gate SWA eviction on accumulated tokens (#33404 )
paulzhang-tm
2026-08-07 20:28:26 -04:00
633838b0ec
Add the 8-gpu Inkling consistency test (#34009 )
Ke Bao
2026-08-08 08:24:08 +08:00
ce84df0fa1
Fix deterministic inference for Inkling (#33417 )
Ke Bao
2026-08-08 07:48:15 +08:00
1d812865dc
[Laguna] fix YaRN mscale double-application in rope config (#33558 )
Joe Rowell and Jimmy Shong
2026-08-08 01:46:08 +02:00
c59d2b4329
[Fix] Judge the phase-checker device-assert test by its FAIL line, not the exit code (#34017 )
Liangsheng Yin
2026-08-07 16:19:05 -07:00
86f373daff
docs(cookbook): DeepSeek-V4-Flash-0731 — drop chunked-prefill/autotune flags on B300 low-latency (#34044 )
Douglas Yang and Claude Opus 5
2026-08-07 16:12:56 -07:00
3dc91366ac
[HiCache] write_back: reclaim duplicated host copy first under host pressure (#33777 )
Zhiqiang Xie
2026-08-07 15:59:59 -07:00
eb3cc879e0
Install DeepEP from release wheels (#33932 )
Baizhou Zhang
2026-08-07 15:38:44 -07:00
115cd7bde1
docs: update checkpoint to Qwen3.5 NVFP4 V2 for InfX (#32945 )
Faradawn Yang
2026-08-07 15:26:07 -07:00
b2f9603f93
[bugfix] Stop/EOS inside a spec accept run beats the max_new_tokens finish (#33758 )
2026-08-07 15:18:14 -07:00
07297049e9
config: route DCP topology reads through get_parallel() (#33925 )
Khoa Pham and Claude Opus 5
2026-08-07 14:53:54 -07:00
df3aa20d89
Reland serving-time Triton load diagnostics (#33908 )
cctry and kangwangamd
2026-08-07 14:51:43 -07:00
8fc66d1a62
docker: pin Kimi images to SGLang commit (#34030 )
Baizhou Zhang
2026-08-07 14:49:26 -07:00
0da25ee6f7
Docs: Ling-3.0-flash cookbook — serve native 256K, drop YaRN override (#33882 )
Xinyuan Tong
2026-08-08 05:46:51 +08:00
8600457731
[Spec] Propagate state capture outputs in DFlash (#34026 )
Nan Jiang
2026-08-07 14:42:30 -07:00
9e3f6b746b
fix(mamba): widen causal_conv1d token offsets to int64 (#33665 )
YAMY
2026-08-07 14:31:48 -07:00
b3ee679467
[MXFP8] Use FlashInfer CUTLASS for dense GEMM on SM120, delete Triton path (#33208 )
Brayden Zhong and Brayden Zhong
2026-08-07 14:30:43 -07:00
699fcdc936
Fix _pa_swa_prefill_lens off-by-one in FlashAttentionBackend (#33379 )
Dmitrii Sergeev
2026-08-08 00:06:48 +03:00
62a28197c0
Autotune flashinfer extend buckets at warmup (#32556 )
Sam Shleifer and Claude Fable 5
2026-08-07 17:06:05 -04:00
1480687cff
[CP]: Support CP V2 Strategy for dsv4 (#33532 )
Zhangheng
2026-08-08 05:03:27 +08:00
8e7d361def
[perf] Compute input logprobs without materializing the full-vocab log-softmax (#31958 )
Sam Shleifer
2026-08-07 17:01:08 -04:00
8a22b8305d
docker: add Kimi K3 artifacts and build hpc-ops with C++20 (#33956 )
Baizhou Zhang
2026-08-07 13:50:44 -07:00
b53a39c5e4
Limit prefill delayer debug logs to rank zero (#34020 )
Hanming Lu and Lianmin Zheng
2026-08-07 13:41:19 -07:00
7f6b4cb94b
Add CUDA VMM multimodal feature transport (#33899 )
Oguz Ulgen and Yinghai Lu
2026-08-07 13:39:54 -07:00
3c51e29deb
Responses support (#32689 )
2026-08-07 13:21:46 -07:00
6c7498113f
[diffusion] Enable breakable CUDA graph for SANA (H200 1024px e2e -26%, bit-exact) (#33989 )
Xiaoyu Zhang and Claude Fable 5
2026-08-07 23:54:39 +08:00
12de7fb1f6
Remove the HiMambaRadixTree that is no longer in use (#33468 )
Zhangheng
2026-08-07 23:01:40 +08:00
d4be483efb
[diffusion] Enable breakable CUDA graph for LTX-2 (H200 two-stage e2e 10.75 s -> 6.90 s, 1.56x) (#33885 )
Xiaoyu Zhang
2026-08-07 22:29:14 +08:00