Commit Graph
13790 Commits
Author SHA1 Message Date
MickandMick Qian 6103a8c743 [diffusion] feat: add opt-in SRT prompt enhancement to image and video APIs (#41095)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-25 10:00:04 +08:00
jacky.cheng 9dc4c5d891 [AMD] Restore non-DCP Mamba checkpoint donation to fix agent-mode cache hit at high conc with HiCache (#40907) 2026-09-24 18:08:10 -07:00
Liangsheng Yin 8ca82118e0 [mem_cache] Drop is_insert from cache_finished_req; release rows from release_kv_cache (#40988) 2026-09-24 17:23:26 -07:00
Shuwen Wangandhnyls2002 a2025b8c4d [DSV4] Account for FlashMLA physical KV page padding in memory budgets (#41091)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-24 17:14:08 -07:00
Sundara Raman Ramachandran 2f2f9d12f8 [Score API] Setwise Scoring Support (#38965) 2026-09-24 16:50:58 -07:00
7c5bdb940c [PD] Add a none decode retraction backup and subclass seams in the PD queues (#41103)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <zqx@meta.com>
2026-09-24 16:13:54 -07:00
466e985f4b [HiCache] Batch buffer-only KV backups within each flush (#40960)
Co-authored-by: oss-sync bot (pranjalssh) <pranjalssh@users.noreply.github.com>
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
2026-09-24 16:13:02 -07:00
Caio RochaandCaio Rocha e047e50d4e Add 8-node AllReduce/AllGather and MNVLS algorithm support to MSCCL++ (#37442)
Co-authored-by: Caio Rocha <caiorocha@microsft.com>
2026-09-24 15:51:29 -07:00
Yuwei An 3ec86307f2 Fix mixed chunk prefill with DP speculative coordination (#41179) 2026-09-24 15:41:49 -07:00
6bbd689ab4 [Bugfix] Align DeepSeek-V4.1 reasoning effort budgets (#39929)
Co-authored-by: fengtinglei <fengtinglei@bytedance.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-09-24 15:21:06 -07:00
Liangsheng Yin 0c578d97fe [DSV4] Size compressed pools from one per-ratio table in DSV4PoolConfigurator (#41049) 2026-09-24 15:19:02 -07:00
Shuwen Wang 961404b010 [DSV4] Fix TRTLLM uniform FP8 KV memory budgeting (#41090) 2026-09-24 13:25:00 -07:00
36f59982fa MoE: small-batch sorting path with fused mxfp8 quantisation (#36559)
Co-authored-by: mikevin920 <mikevin920@yahoo.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Kevin Mi <mikevin920@gmail.com>
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-09-24 13:18:18 -07:00
0b53305f48 MiniMax-M3: MXFP8 dense-only block convert + aiter MXFP8 MoE on gfx950 (#36574)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mikevin920 <mikevin920@yahoo.com>
Co-authored-by: Kevin Mi <mikevin920@gmail.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
2026-09-24 13:16:59 -07:00
Qiaolin Yu b129504f0e [qwen 3.8 next] Fuse Qwen PLE gate and convolution preparation for target verify (#40041) 2026-09-24 13:13:34 -07:00
Thomas Wang 1446e24d13 [AMD] Add .co for deepseek v4 fp8 decode kernel and add group decode opt (#41120) 2026-09-24 12:21:34 -07:00
Cheng Wan 5bb850d536 [Refactor] Split prepare_attn into a reduction step and per-quant-format residual steps (#41084) 2026-09-24 12:16:54 -07:00
Cheng Wan a80055d3c6 [Fix] Broadcast requests along attention CP before attention TP (#41083) 2026-09-24 12:16:27 -07:00
Cheng Wan 7c9f74c6e1 [Fix] Step-3.5: stop dense layers from summing their output twice under DP attention (#41082) 2026-09-24 12:15:58 -07:00
Cheng Wan 62ae032c46 [Refactor] Share the MoE output all-reduce between models (#41097) 2026-09-24 12:15:33 -07:00
Cheng Wan 5df1667077 [Refactor] Pass each layer stack's output through a communicator exit (#41081) 2026-09-24 12:15:07 -07:00
Cheng WanandMick Qian 07a3735e94 [Fix] Complete the deferred FFN all-reduce before deepstack addition and aux hidden-state capture (#41080)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-24 12:14:39 -07:00
Cheng Wan 928e683f94 [Fix] Complete the deferred FFN all-reduce before a pipeline-parallel send (#41079) 2026-09-24 12:13:57 -07:00
f404db94d4 [DP attention] Publish DP buffer sizes from a ForwardBatch (#40858)
Co-authored-by: hanminglu <hanminglu@fb.com>
Co-authored-by: oss-sync bot (pranjalssh) <pranjalssh@users.noreply.github.com>
2026-09-24 11:34:19 -07:00
Mohammad Miadh AngkadandMohammad Angkad 86b3558b41 [Fix] Skip the DCP target-verify MLA kernel during FlashInfer autotune (#41138)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-09-24 11:19:28 -07:00
78b382b1ad Support unified memory decode host pools (#39478)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-09-24 11:02:15 -07:00
Trevor Morris 984994e3aa fix: Triton 3.8 compatbility to support DSV4.1-Flash in CUDA 13.4 image (Rubin) (#40805) 2026-09-24 10:25:35 -07:00
Jackey HuaandClaude Fable 5.1 5bb24e399d [Quant] ModelOpt mixed precision: dispatch block-FP8 MoE experts and derive the block size (#38726)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-24 12:30:01 -04:00
Shangming CaiandClaude Fable 5 77173ffb08 [HiCache] Demote SWA KV to host on write_back eviction instead of dropping it (#40712)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-24 23:56:17 +08:00
Ting SUN 752801e4d1 fix(sampling): validate sampling_seed is an int within int64 range (#28960)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
2026-09-24 17:25:13 +02:00
17a7484c06 [chore] surface the cookbook to users who pip install sglang (#40866)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 22:56:41 +08:00
Shangming CaiandClaude Opus 5 cd11037009 [PD] Enable deferred decode-side KV release by default (#41023)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-24 22:35:11 +08:00
黄孝君 4142235c2b [NPU] Update CANN version to 9.1.0 (#40524) 2026-09-24 21:35:31 +08:00
Xiaoyu Zhang ea5baf4022 [Refactor] Retire the model-specific Kimi K3 kernel namespace (#40922) 2026-09-24 21:25:00 +08:00
paulzhang-tm 28ec6700da [HiCache] Make host reclamation independent of transfer order (#40512) 2026-09-24 21:03:25 +08:00
3177d10ca6 Refactor the Cute-DSL AR fusion to support DeepseekV2 archs (GLM-5.3, etc.) (#39816)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
2026-09-24 06:00:58 -07:00
ronnie_zheng ce06a14444 [Diffusion] migrate the whole _register_configs from registry.py to the model own config file (#40612) 2026-09-24 19:29:49 +08:00
Xiaoyu Zhang c6565c1a6d [Diffusion] Enable lossless SANA-Video eager conv fusions for 12.6% lower latency (#40388) 2026-09-24 19:21:32 +08:00
Alan Kao a69583f70f [AMD] Add tuned dsv4 shape (#40996) 2026-09-24 04:06:40 -07:00
AMD-yanfeiwang 0a59830114 [AMD][DSV4] fp8 unified_kv decode: wave-aware split count past 40 tokens (#40878)
Signed-off-by: Yanfei Wang <yanfwang@crsuse2-m2m-v2-017.us-east2-a.compute.internal>
2026-09-24 03:41:23 -07:00
Xiaoyu Zhang 26c63335e2 [Diffusion] Fuse lossless LingBot World FP32 normalization (#40425) 2026-09-24 18:41:10 +08:00
Zhang, Jiejing d94d784441 [AMD] Tune Triton sparse MLA on gfx950 and make split-K workspaces graph-safe (#39059) 2026-09-24 03:21:53 -07:00
Mohammad Miadh Angkad bb1c98baa2 Fix TBO child batch missing dp_spec_prefill_coordination_applied (#41096) 2026-09-24 02:44:55 -07:00
Xiaoyu Zhang c85df5becc [Diffusion] Fuse lossless Wan VAE post-ops for LongLive 2 I2V (#40405) 2026-09-24 17:40:07 +08:00
Xiaoyu Zhang 261cb826e6 [Diffusion] Fuse LongCat GELU+cat and support Edit-Turbo BCG (#40384) 2026-09-24 17:27:56 +08:00
MickandMick Qian 174a5f37b9 [feature] add per-item candidate token scoring and calibration (#40826)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-24 17:00:13 +08:00
Cheng Xin 41812afe43 [NPU] Fix DSV4 hard-coding kv dtype (#40510) 2026-09-24 16:53:44 +08:00
Yuwei An 1417345f5f [Experimental] Preserve speculative decoding during prefill across DP ranks (#40118) 2026-09-24 00:01:12 -07:00
Liangsheng Yin cae4927fbf [DSV4] Budget the ratio-2 pair state pool in DSV4PoolConfigurator (#41048) 2026-09-23 23:53:46 -07:00
YC Yen-Ching Tseng 9f6fc55332 [AMD][Diffusion] FlyDSL fused norm kernels on wave32 targets (gfx1250) (#37751) 2026-09-23 23:06:55 -07:00