   
|
78b382b1ad
|
Support unified memory decode host pools (#39478)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
|
2026-09-24 11:02:15 -07:00 |
|
Trevor Morris
|
984994e3aa
|
fix: Triton 3.8 compatbility to support DSV4.1-Flash in CUDA 13.4 image (Rubin) (#40805)
|
2026-09-24 10:25:35 -07:00 |
|
 Jackey HuaandClaude Fable 5.1
|
5bb24e399d
|
[Quant] ModelOpt mixed precision: dispatch block-FP8 MoE experts and derive the block size (#38726)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-24 12:30:01 -04:00 |
|
 Shangming CaiandClaude Fable 5
|
77173ffb08
|
[HiCache] Demote SWA KV to host on write_back eviction instead of dropping it (#40712)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-24 23:56:17 +08:00 |
|
Ting SUN
|
752801e4d1
|
fix(sampling): validate sampling_seed is an int within int64 range (#28960)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
|
2026-09-24 17:25:13 +02:00 |
|
 
|
17a7484c06
|
[chore] surface the cookbook to users who pip install sglang (#40866)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
2026-09-24 22:56:41 +08:00 |
|
 Shangming CaiandClaude Opus 5
|
cd11037009
|
[PD] Enable deferred decode-side KV release by default (#41023)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-24 22:35:11 +08:00 |
|
黄孝君
|
4142235c2b
|
[NPU] Update CANN version to 9.1.0 (#40524)
|
2026-09-24 21:35:31 +08:00 |
|
Xiaoyu Zhang
|
ea5baf4022
|
[Refactor] Retire the model-specific Kimi K3 kernel namespace (#40922)
|
2026-09-24 21:25:00 +08:00 |
|
paulzhang-tm
|
28ec6700da
|
[HiCache] Make host reclamation independent of transfer order (#40512)
|
2026-09-24 21:03:25 +08:00 |
|
 
|
3177d10ca6
|
Refactor the Cute-DSL AR fusion to support DeepseekV2 archs (GLM-5.3, etc.) (#39816)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
|
2026-09-24 06:00:58 -07:00 |
|
ronnie_zheng
|
ce06a14444
|
[Diffusion] migrate the whole _register_configs from registry.py to the model own config file (#40612)
|
2026-09-24 19:29:49 +08:00 |
|
Xiaoyu Zhang
|
c6565c1a6d
|
[Diffusion] Enable lossless SANA-Video eager conv fusions for 12.6% lower latency (#40388)
|
2026-09-24 19:21:32 +08:00 |
|
Alan Kao
|
a69583f70f
|
[AMD] Add tuned dsv4 shape (#40996)
|
2026-09-24 04:06:40 -07:00 |
|
AMD-yanfeiwang
|
0a59830114
|
[AMD][DSV4] fp8 unified_kv decode: wave-aware split count past 40 tokens (#40878)
Signed-off-by: Yanfei Wang <yanfwang@crsuse2-m2m-v2-017.us-east2-a.compute.internal>
|
2026-09-24 03:41:23 -07:00 |
|
Xiaoyu Zhang
|
26c63335e2
|
[Diffusion] Fuse lossless LingBot World FP32 normalization (#40425)
|
2026-09-24 18:41:10 +08:00 |
|
Zhang, Jiejing
|
d94d784441
|
[AMD] Tune Triton sparse MLA on gfx950 and make split-K workspaces graph-safe (#39059)
|
2026-09-24 03:21:53 -07:00 |
|
Mohammad Miadh Angkad
|
bb1c98baa2
|
Fix TBO child batch missing dp_spec_prefill_coordination_applied (#41096)
|
2026-09-24 02:44:55 -07:00 |
|
Xiaoyu Zhang
|
c85df5becc
|
[Diffusion] Fuse lossless Wan VAE post-ops for LongLive 2 I2V (#40405)
|
2026-09-24 17:40:07 +08:00 |
|
Xiaoyu Zhang
|
261cb826e6
|
[Diffusion] Fuse LongCat GELU+cat and support Edit-Turbo BCG (#40384)
|
2026-09-24 17:27:56 +08:00 |
|
 MickandMick Qian
|
174a5f37b9
|
[feature] add per-item candidate token scoring and calibration (#40826)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
|
2026-09-24 17:00:13 +08:00 |
|
Cheng Xin
|
41812afe43
|
[NPU] Fix DSV4 hard-coding kv dtype (#40510)
|
2026-09-24 16:53:44 +08:00 |
|
Yuwei An
|
1417345f5f
|
[Experimental] Preserve speculative decoding during prefill across DP ranks (#40118)
|
2026-09-24 00:01:12 -07:00 |
|
Liangsheng Yin
|
cae4927fbf
|
[DSV4] Budget the ratio-2 pair state pool in DSV4PoolConfigurator (#41048)
|
2026-09-23 23:53:46 -07:00 |
|
YC Yen-Ching Tseng
|
9f6fc55332
|
[AMD][Diffusion] FlyDSL fused norm kernels on wave32 targets (gfx1250) (#37751)
|
2026-09-23 23:06:55 -07:00 |
|
 
|
32290dda2c
|
[AMD] Small-M MXFP4 fused-MoE kernel for gfx950 (Qwen) (#40204)
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
|
2026-09-23 22:45:33 -07:00 |
|
Shuwen Wang
|
90663ccb41
|
[DSV4] fix: size the C4 state ring by the page it is addressed by (#40337)
|
2026-09-23 22:05:32 -07:00 |
|
 cctryandcctry
|
efff836911
|
[PD] Add decode host receive for custom transfer backends (#40238)
Co-authored-by: cctry <cctry@fb.com>
|
2026-09-23 21:57:33 -07:00 |
|
 fanxingranandThomas Wang
|
0dc8b29e15
|
[ROCm][DSA] Enable AITER fused FP8 indexer writer (#40710)
Co-authored-by: Thomas Wang <thomawan@amd.com>
|
2026-09-23 21:46:35 -07:00 |
|
Nan Jiang
|
c3685dff42
|
[Sampling] Add selected/support sampling logprob modes (#40932)
|
2026-09-23 21:38:19 -07:00 |
|
Jimmy Shong
|
be495a6174
|
[Fix] Use cached prefix lengths for FlashInfer full-attention ragged prefill (#40796)
|
2026-09-23 17:23:25 -10:00 |
|
jacky.cheng
|
e2f4fedf04
|
[AMD] Critical fix enabling Qwen3.8 FP8: restore dropped fused shared-expert weights (#40754)
|
2026-09-23 20:21:33 -07:00 |
|
 Shijin ZhangandBrayden Zhong
|
77983865d8
|
[Moe] Honor swiglu_limit clamped activation in flashinfer_cutlass runner (#39939)
Signed-off-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2026-09-23 20:10:03 -07:00 |
|
 Xia WeiwenandCopilot
|
5c44214d6a
|
[XPU] Qwen3.8-flash-next enablement (#37213)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
2026-09-24 11:06:30 +08:00 |
|
      
|
f4d9d2e670
|
[RL] Add RL weight-update sessions and support updating spec draft runners (#40777)
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Nan Jiang <59716405+nanjiangwill@users.noreply.github.com>
Co-authored-by: maocheng23 <maocheng@berkeley.edu>
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: JD-ETH <jaedon.guo@gmail.com>
Co-authored-by: Letian Ruan <ruanlt420@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-09-23 20:04:57 -07:00 |
|
 Yueming Yuanandzyzshishui
|
f4b90382a2
|
[RL] Keep DSA cuda-graph state and the graph pool intact across TMS pause/resume (#40804)
Co-authored-by: zyzshishui <zyzshishui@gmail.com>
|
2026-09-23 19:55:26 -07:00 |
|
Liangsheng Yin
|
cc6e8b0f25
|
[Fix] Derive per-runner hybrid SWA layer ids on ModelLayerInfo instead of mutating ModelConfig (#40983)
|
2026-09-23 19:52:55 -07:00 |
|
jacky.cheng
|
e98b2f7538
|
[AMD] Speed up Wan2.2 DiT FP8 attention per-tensor quantization (#34695)
|
2026-09-23 19:49:26 -07:00 |
|
jacky.cheng
|
8b5d77c268
|
[AMD] Add a Triton packed sparse decode path for QSA on ROCm (#38876)
|
2026-09-23 18:59:17 -07:00 |
|
hanwlax
|
03fcbe147f
|
[NPU] Support batch invariant FIA graphs for deterministic inference (#39607)
|
2026-09-24 09:53:17 +08:00 |
|
Wang, FangYuan
|
81b81664a6
|
Revert "[AMD] Fix DeepSeek-V4 accuracy by not passing num_token_non_padded to MoE topk" (#40946)
|
2026-09-23 18:38:19 -07:00 |
|
Liangsheng Yin
|
621136e094
|
[mem_cache] Remove unused helpers in mem_cache, storage backends, and metrics (#40963)
|
2026-09-23 18:16:44 -07:00 |
|
Liangsheng Yin
|
81cb895b85
|
[Fix] Recover from stale torch extension locks in every cpp_extension loader (#40989)
|
2026-09-23 17:53:57 -07:00 |
|
     
|
ec75d3d30f
|
MiniMax-M3: allocate the lightning-indexer K cache in fp8 on gfx95 (#36549)
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Kevin Mi <mikevin920@gmail.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai>
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
|
2026-09-23 17:52:02 -07:00 |
|
Yueming Yuan
|
542a043f1b
|
[RL] Release the weight-checker snapshot once compare passes (#37284)
|
2026-09-23 16:06:02 -07:00 |
|
 James LiuandClaude Opus 5.5
|
208f6f7501
|
[Spec] Support DFLASH for Kimi K3 (#40794)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-23 15:58:39 -07:00 |
|
 
|
6fe4b66f5c
|
[RL] Keep pause_generation and weight updates from deadlocking each other (#40779)
Co-authored-by: maocheng23 <maocheng@berkeley.edu>
Co-authored-by: Zhichenzzz <zczeng@uw.edu>
|
2026-09-23 15:13:42 -07:00 |
|
Cheng Wan
|
79fec592a8
|
[Refactor] Read parallel placement in consumers (#40638)
|
2026-09-23 14:57:25 -07:00 |
|
    
|
f2a1366584
|
MiniMax-M3: wave64 histogram-select decode top-k, and raise kMaxNumBlocks for CUDA graphs (#36560)
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Kevin Mi <mikevin920@gmail.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
|
2026-09-23 14:50:13 -07:00 |
|
      
|
4fa2c9c1de
|
MiniMax-M3: run the sparse prefill main attention through AITER Gluon paged attention (#36546)
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Alex Sun <alex.s@amd.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: Kevin Mi <mikevin920@gmail.com>
Co-authored-by: YC Yen-Ching Tseng <yctseng@amd.com>
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai>
|
2026-09-23 14:49:10 -07:00 |
|