Commit Graph
6055 Commits
Author SHA1 Message Date
Feng Su fb3d1419fd [tracing] sglang tracing v2: support exporting tracing data asynchronously (#30023) 2026-08-10 15:23:05 +08:00
Yuwei AnandClaude Opus 5 06f32bab6b [BCG][5/N] MLA Fully Support (#33661)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 23:52:14 -07:00
Liangsheng YinandBrayden Zhong d6a066131c [Feature] Support NVFP4 token embedding in ModelOpt mixed-precision checkpoints (#34222)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
2026-08-09 23:37:54 -07:00
2969ab3d41 [MLX] Window-bounded SWA KV storage and in-graph sampling (#34166)
Co-authored-by: Siming Deng <siming_deng_stat@163.com>
Co-authored-by: R0CKSTAR <yeahdongcn@gmail.com>
Co-authored-by: Jiminator <Jiminator@users.noreply.github.com>
Co-authored-by: damahua <damahua@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 21:28:51 -07:00
Jan BernlöhrandMohammad Miadh Angkad 5d85f25f75 fix(minimax): use routed TRT-LLM for NVFP4 MoE auto on SM100 (#32229)
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
2026-08-10 11:45:14 +08:00
Khoa Pham ee3ee8393c fix: preserve GQA head mapping in Triton DCP prefill (#34161) 2026-08-09 18:52:18 -07:00
Xiaoyu ZhangandClaude Opus 4.8 441910f926 [diffusion] LTX-2 quality=high fused RMSNorm+modulate + FFN GELU epilogue (H200 ltx23-one-stage denoise 45.85->43.24 s, ~matches torch.compile) (#34172)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-10 09:46:17 +08:00
Mick c20e99bd22 fix(vlm): preserve Kimi-K3 GPU JPEG accuracy (#34163) 2026-08-10 09:42:52 +08:00
553dc0f936 [MLX] Support gpt-oss: sliding-window attention, attention sinks, sm_scale (#30050)
Co-authored-by: R0CKSTAR <yeahdongcn@gmail.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
2026-08-09 18:32:36 -07:00
Ke Bao 68b961e9fb Add kda replayssm tests (#33630) 2026-08-10 09:11:20 +08:00
Ke Bao 25b7015064 Fix stale track rows corrupting conv checkpoints under the prefill graph (#34184) 2026-08-10 08:57:23 +08:00
Liangsheng Yin 7c90840bad [CI] Key scheduled CUDA suites by runner_config instead of hand-written jobs (#34186) 2026-08-09 16:44:53 -07:00
Liangsheng YinandDeleter-D 4a5d7d3c67 [DSV4] Fix silent KV corruption when speculative draft tokens > 4 (#34189)
Co-authored-by: Deleter-D <867909454@qq.com>
2026-08-09 15:54:41 -07:00
Cheng Wan b4284f3eb7 config: the KV-cache configurator reads the bags (#34096) 2026-08-09 14:45:42 -07:00
Cheng Wan e216c2bc59 config: the runner and scheduler read resolved config from the bags (#34095) 2026-08-09 14:45:11 -07:00
Cheng WanandClaude Fable 5 a2199c1dee config: pin that resolution is reproducible from the raw input (#34094)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 14:44:39 -07:00
Cheng Wan 63833f8034 config: business code no longer reads the published ServerArgs (#34081) 2026-08-09 14:44:08 -07:00
Cheng Wan 110bf7e6a8 config: retire the hidden global fallbacks and the mamba-extra-buffer instance reads (#34080) 2026-08-09 14:43:28 -07:00
Ke Bao 2ab96d0fe0 Fix mlx unit test batch mock (#34181) 2026-08-10 00:55:55 +08:00
Ke Bao fb72a37fde Add deterministic logprob-consistency test for inkling-small nvfp4 (#34168) 2026-08-09 22:22:27 +08:00
Kurt ShusterandClaude Fable 5 3fe65e0654 Deterministic gumbel sampling: clamp u=1 so masked tokens can't be sampled (#33423)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 21:43:27 +08:00
11d03eaeef runtime: Add flashinfer rmsnorm + quant fusion support SM90, SM100, SM120- #32994 (#33471)
Signed-off-by: Devashish Lal <devcode@fb.com>
Co-authored-by: Devashish Lal <devcode@fb.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-08-09 20:15:03 +08:00
Yuzhen Zhou 7120f3ee13 fix: support FA4 backend for GLM4.7-flash (#33436) 2026-08-09 20:05:38 +08:00
Lianmin Zheng 71043b9dbb Revert parallel request lifecycle tracking from #32588 (#34160)
Signed-off-by: Lianmin Zheng <lianminzheng@gmail.com>
2026-08-09 03:10:02 -07:00
Ke Bao c500674124 Switch inkling per-commit test to nvfp4 (#32402) 2026-08-09 16:49:49 +08:00
Leon Gao 78cd60b4e3 [srt] Reuse batched Mamba boundary mask (#33477) 2026-08-09 16:47:05 +08:00
Mick 22e003580b [Kimi K3] optimize: preprocess cpu-transport images on the vision owner (#33921) 2026-08-09 16:13:20 +08:00
Liangsheng Yin 1ebd6fab6c [CI] Gate Kimi-K3 acceptance length on the GSM8K average (#34145) 2026-08-08 20:28:42 -07:00
Liangsheng Yin 4792ab1e90 [CI] Pin the rust frontend parity test to eager prefill (#34146) 2026-08-08 20:07:19 -07:00
Xiaoyu Zhang 38c007dfe5 [diffusion] FLUX.1: route the adaLN LN+modulate sites through the bit-exact fused LayerNorm+modulate kernel (H200 1024^2 lossless denoise -1.2%, e2e wall -2.9%) (#34126) 2026-08-09 09:52:37 +08:00
Xiaoyu ZhangandClaude Fable 5 6424fec326 [diffusion] Bit-exact data-movement elimination for the Wan causal VAE decoder (H200 LongLive2 704x1280x61f: decode 2.80->2.32 s lossless / 2.12->1.67 s quality=high, e2e -10.7%) (#34125)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 09:50:56 +08:00
Xiaoyu ZhangandClaude Fable 5 33ed5d4413 [diffusion] perf_logger: SYNC_STAGE_PROFILING must drain the GPU queue for stage records too (fixes 2-3x inflated DecodingStage readings) (#34124)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 09:49:52 +08:00
Wang, FangYuan ba7abd4f92 [AMD] Support DeepSeek V4 DSpark on AMD HIP platform (#30964) 2026-08-08 15:22:14 -07:00
a59bb931c6 Fix DCP KV head mapping for GQA models (#32858)
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 14:25:09 -07:00
Mohammad Miadh Angkad c2d90db1e3 [CI] Add Kimi-K3 low-latency performance check (#34089) 2026-08-08 13:42:59 -07:00
Diya Peng c4f018ba1d [Refactor] Separate ROCm-specific DeepSeek MHA and MLA forward paths (#31531) 2026-08-08 13:39:36 -07:00
Xiaoyu Zhang dc9624deb2 [diffusion] Clean up kernels and shared fast paths (#34085) 2026-08-09 00:37:00 +08:00
Eric ZhangandClaude Opus 5 f3ed82b3a8 [inkling] Let Anthropic thinking=disabled map to reasoning effort "none" (#33913)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 21:59:32 +08:00
Mick db75dfe10f fix: always capture default prefill CUDA graph (#33352) 2026-08-08 19:24:49 +08:00
Leon Gao cfb354bcfc [Bugfix] Fix batched KV free aliasing (#34067) 2026-08-08 03:34:44 -07:00
Liangsheng Yin a1ca76b24b [Scheduler] Unify WAR read-done gating behind shared-read boundary declarations (#34052) 2026-08-08 03:26:36 -07:00
Liangsheng Yin 2c0188cc78 [Fix] Give the piecewise CUDA graph test stub an hf_config (#34100) 2026-08-08 02:04:16 -07:00
Liangsheng YinandBaizhou Zhang f6a6f5bf1e [CI] Trim redundant nightly test registrations (#34070)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2026-08-08 01:42:46 -07:00
dd5d82bead [NPU] Improve the execution efficiency and maintainability of pr‑test‑npu (#33724)
Co-authored-by: Even Zhou <even.y.zhou@outlook.com>
Co-authored-by: sglang-npu-bot <sglangnpu@163.com>
2026-08-08 16:42:44 +08:00
xdtbynd e732c0a9dc [UT][NPU] add unit tests for ascend_torch_native_backend and mla_preprocess (#32505) 2026-08-08 16:15:12 +08:00
DarkSharpnessandClaude Fable 5 4ad5bb5d9a [jit_kernel] Move JIT kernels into namespace sglang (#33400)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 16:10:15 +08:00
Yuan Luo 5fdf6cd18f [MoE] Single-launch moe_align for tiny batches with many experts (#32395) 2026-08-08 16:08:25 +08:00
Xiaoyu ZhangandClaude Fable 5 891445676c [diffusion] Sana: bit-exact fused aten LayerNorm+modulate under BCG (H200 denoise -4.8%) (#34015)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-08 16:05:25 +08:00
Mick d747bd052e feat(vlm): auto-select cuda vmm on multi-node mnnvl (#33936) 2026-08-08 16:00:58 +08:00
Mick db3898fec1 fix: avoid piecewise prefill graph for trtllm_mla (#32785) 2026-08-08 16:00:10 +08:00