Commit Graph
14516 Commits
Author SHA1 Message Date
a3d00eff58 [Perf] srt: Record on the host whether an extend batch completes a prompt (#45487)
Co-authored-by: Lu Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-21 11:58:25 -07:00
7e5256d1ba [Feat] srt: Balance uvicorn tokenizer workers with per-worker SO_REUSEPORT listeners (#45482)
Co-authored-by: luccafong <116399278+luccafong@users.noreply.github.com>
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com>
2026-10-21 11:57:26 -07:00
Baizhou Zhang f06d1bb1f4 [Perf] cuda-graph: Capture FlashInfer KDA prefill (#45473) 2026-10-21 11:57:23 -07:00
740d5f9f9e [Perf] scheduler: Plan hybrid Mamba extend batches without host-device syncs (#45472)
Co-authored-by: Lu Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Shiyan Deng <20773414+842974287@users.noreply.github.com>
2026-10-21 11:56:50 -07:00
5e79333c95 [Refactor] radix-tree: Reuse the insert walk, component hooks and bindings for custom tree cores (#45439)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Jialin <Jialin@users.noreply.github.com>
2026-10-21 10:53:26 -07:00
paulzhang-tm b57b5e2286 [Fix] disaggregation: Synchronize deferred decode KV release across TP ranks (#45453) 2026-10-22 01:37:28 +08:00
Khoa Pham ed268dd320 [Fix] moe: Apply SGLANG_SIMULATE_* routing on DeepSeek-V4.1 vision-biased gates (#45628) 2026-10-21 10:00:15 -07:00
vorapolsiloai 6e98c84c60 [Perf] [AMD] quark: Add opt-in AITER ASM FP4 GEMM for Quark dense linear (#39598) 2026-10-21 09:37:47 -07:00
Michael 4b1a44bc77 [Test] [AMD] diffusion: Avoid Python 3.11-only enterContext in test_usp_ring_replicated (#45409) 2026-10-22 00:11:08 +08:00
Huang LongandShangming Cai af49ada16d [Fix] pd: Allow runtime role switching with DP and DP attention (#44215)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-21 22:58:29 +08:00
8e3e2980db [Feat] [Intel] disaggregation: Support XPU for staging-buffer KV transfer (#40861)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-21 22:09:04 +08:00
amote-i 2815eed49c [Fix] [NPU] pre_warm_nccl: init npu backend before the parallel runtime touches a device (#43039) 2026-10-21 19:44:30 +08:00
Avaya Aggarwalandniehen6174 62b4127668 [Feat] comfyui: Apply ControlNet residuals in Flux integrated mode (#43698)
Co-authored-by: niehen6174 <1639206518@qq.com>
2026-10-21 19:29:43 +08:00
ca72dad953 [Fix] rust-server: Answer health probes a busy scheduler skipped on the response ring (#45475)
Co-authored-by: Lu Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-21 03:41:36 -07:00
Jimmy Shong 5bf10d4c79 [Fix] nemotron: Correct weight-export failures and GEMV dispatch (#45082) 2026-10-21 18:38:35 +08:00
Jzz1943 2e7c9d851c [Feat] NIXL: Support per-GPU UCX device selection (#43482) 2026-10-21 17:54:11 +08:00
Shangming Cai be65391abd [Feat] mooncake: Enable the failed-session probe by default with backoff and idle pruning (#45338) 2026-10-21 17:16:19 +08:00
Thomas Wang 1278087336 [Fix] [AMD] mono-kernel: Update sgl-amd-mono version (#45546) 2026-10-21 02:02:03 -07:00
2c9d919071 [Fix] srt: Pin the source address of TCP ZMQ connects for the peer allowlist (#45437)
Co-authored-by: Lu Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
2026-10-21 01:59:03 -07:00
21f775cd97 [Feat] rust-server: Let the launcher set the default max_new_tokens for /generate (#45431)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-21 01:58:34 -07:00
Aurick QiaoandXiaoyu Zhang 7a632cf998 [Fix] quantization: Preserve BF16 NaNs in per-token FP8 quantization (#42689)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
2026-10-21 16:52:28 +08:00
8bd6aa9a6c [Feat] parser: Enrich chat_parsing streaming events (#40478)
Co-authored-by: Yoni Gozlan <yonigozlan@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
2026-10-21 01:46:26 -07:00
AlexandJimmy Shong dbe0a9a7c9 [Fix] nemotron_vl: Preserve tokenized prompts during media expansion (#44184)
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
2026-10-21 01:40:51 -07:00
Liangsheng Yin 6efdd33d07 [Refactor] spec: Move platform dispatch and Triton kernels into kernels/ops (#45504) 2026-10-21 01:22:16 -07:00
6cac4dc3fe [Refactor] diffusion: Rollout controller as a scheduler-step plugin instead of mixins (#44603)
Signed-off-by: rockdu <kangrdu@gmail.com>
Co-authored-by: WenhaoZhang <42087078+niehen6174@users.noreply.github.com>
Co-authored-by: JingwenGu0829 <75733630+JingwenGu0829@users.noreply.github.com>
2026-10-21 16:04:33 +08:00
Avaya AggarwalandWenhaoZhang fee32a5c1f [Feat] ComfyUI: Add online INT8 quantization for BF16 Qwen-Image-2.1 single files (#44865)
Co-authored-by: WenhaoZhang <42087078+niehen6174@users.noreply.github.com>
2026-10-21 16:01:18 +08:00
Jzz1943 a361f0c410 [Fix] kda: Align graph tracking branch dtype with the recurrent state pool (#44886) 2026-10-21 15:58:34 +08:00
f7b8c6fe40 [Fix] glm-5.2: Record side-stream work right before its join to keep CUDA-graph replay on one stream (#45349)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-21 15:36:54 +08:00
Hongkuan Zhouandishandhanani 8acfabbd67 [Fix] observability: Correct speculative FPM timing and decode lengths (#38711)
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-10-21 15:31:18 +08:00
Junlin Wu 844d2b5dec [Fix] [NPU] qwen3.5: Fix dense serving on Ascend 950 (#32745) 2026-10-21 09:50:38 +03:00
Jimmy Shong 8599b7251f [Chore] deps: Bump the OpenAI SDK to 2.8.0 for Responses shell tools (2/8) (#45291) 2026-10-20 23:49:30 -07:00
wutao fb6e6a09ce [Fix] [NPU] disaggregation: Fix host-pool backup for decode retraction (#43814) 2026-10-21 14:48:01 +08:00
Khoa PhamandAMD-yanfeiwang da7e3f1c12 [Fix] cache: Accept packed draft layer mappings in the UMBP linker (#44929)
Co-authored-by: AMD-yanfeiwang <yanfei.wang@amd.com>
2026-10-20 23:27:22 -07:00
Thomas Wang bd5a445bf7 [Fix] [AMD] dsv41: Set moe act to fp8 by default (#45480) 2026-10-20 23:09:27 -07:00
956b9a24ee [Perf] [AMD] qwen3.5: Fuse activation quant into small-M FP8 block-scale GEMM on gfx950 (#42229)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
2026-10-20 22:32:30 -07:00
Aarne Talman 4b5c9c4b5e [Perf] [AMD] attention: Restore the gfx950 extend tile for head_dim <= 128 (#45099) 2026-10-20 22:02:50 -07:00
32c26788f3 [Fix] mamba: Keep prefill rows' checkpoint tracking through mixed chunked prefill (#45435)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: YavorGIvanov <YavorGIvanov@users.noreply.github.com>
2026-10-20 21:56:50 -07:00
Liangsheng Yin 573b5716f5 [Fix] spec: Take post-verify KV move targets from the verify window (#45491) 2026-10-20 21:12:41 -07:00
1c2f42470c [Fix] disaggregation: Keep a requeued optimistic prefill request's multimodal inputs (#45440)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: libertyeagle <30348494+libertyeagle@users.noreply.github.com>
Co-authored-by: fei-xx <135589532+fei-xx@users.noreply.github.com>
2026-10-20 20:27:34 -07:00
metamergebotandshodoco 6bd1c76735 [Fix] rust-server: Drop the fixed 64-item multimodal cap in favor of --limit-mm-data-per-request (#45432)
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-20 20:25:22 -07:00
eb91a41f38 [Feat] unified_cache: Support FULL + MAMBA MLA models (Kimi-K3) in the direct external linker (#44977)
Co-authored-by: Yutong Wu <yutong.wu@amd.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-10-21 10:47:17 +08:00
Chunan Zeng 93088901fb [Chore] attention: Remove the tokenspeed_mla backend and dependency (#44803) 2026-10-20 18:50:05 -07:00
unclezhou c0d18c0254 [Fix] [NPU] dspark: Fix attn_tp > 1 draft wo_b all_reduce group (#40399) 2026-10-21 09:20:15 +08:00
b326e8349d [Perf] mamba: Copy tracked decode states at HBM bandwidth in one persistent launch (#45213)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: YavorGIvanov <YavorGIvanov@users.noreply.github.com>
2026-10-20 16:52:15 -07:00
Baizhou Zhang 5b1dbc2c80 [Fix] [NVIDIA] modelopt-nvfp4: Keep MoE global scales in gate/up order (#45289) 2026-10-20 16:49:13 -07:00
kkandwunhuang 9686edcf80 [Perf] [AMD] dsv41: faster mHC boundary Triton tile for 96+ rows, bitwise unchanged (#45124)
Co-authored-by: wunhuang <wunhuang@amd.com>
2026-10-20 16:39:34 -07:00
Cheng Wan 027208ae02 [Doc] layer-boundary/moe: Correct guides and comments that drifted from the code (#44896) 2026-10-20 16:26:40 -07:00
Cheng WanandAditya Suresh 299a75ac5d [Refactor] moe: Derive routing inputs from the layer and active forward (#44994)
Co-authored-by: Aditya Suresh <152752311+adi-suresh01@users.noreply.github.com>
2026-10-20 16:26:15 -07:00
kkandwunhuang 533020b500 [Perf] [AMD] dsv41: run gfx950 MXFP8 dense GEMMs on aiter's preshuffled FlyDSL kernel (#45123)
Co-authored-by: wunhuang <wunhuang@amd.com>
2026-10-20 16:19:32 -07:00
Cheng WanandAditya Suresh 4f8f06faf3 [Refactor] moe: Derive layer contracts from the selected quant method (#44993)
Co-authored-by: Aditya Suresh <152752311+adi-suresh01@users.noreply.github.com>
2026-10-20 16:18:45 -07:00