Mick and Mick Qian
e01dfd3cbc
[Refactor] multimodal: Replace padding strategies with helpers ( #44845 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-17 20:43:02 +08:00
Shuwen Wang
86e67f7d93
[Fix] unified_cache: Prevent misaligned Mamba checkpoints after speculative truncation ( #43387 )
2026-10-17 20:20:49 +08:00
SuperSong
025655068c
[Fix] unified-memory: Avoid gather scratch during Mamba and Conv compaction ( #44097 )
2026-10-17 04:56:41 -07:00
SuperSong
ee30dc4d40
[Fix] unified-memory: Bound joint reclaim and preserve cached prefixes ( #39294 )
...
Bound joint FULL/SWA/Mamba reclaim so feasible allocations recover capacity without evicting unnecessary cached prefixes, and impossible demands preserve the cache. Use exact unsharded paged-extend demand and update allocation regression fixtures.
2026-10-17 04:46:48 -07:00
Shuwen Wang
ae28306a74
[Fix] cuda_graph: Wait for in-flight layer-wise cache loads before prefill graph replay ( #44220 )
2026-10-17 19:13:57 +08:00
Shuwen Wang and Zhangheng
5fcb07da34
[Fix] hicache: Key MLA Mamba state per TP rank in the HF3FS backend ( #43817 )
...
Co-authored-by: Zhangheng <hzh0425@apache.org >
2026-10-17 11:08:17 +00:00
Woody-Hu and Mick
465ddf9758
[Feat] diffusion/VSA-H3: support sm80 (A100) with a raw-pointer Triton load path ( #43823 )
...
Co-authored-by: Mick <mickjagger19@icloud.com >
2026-10-17 18:47:19 +08:00
400c1dfe6e
[Perf] distributed: skip the DLPack re-wrap of custom all-reduce results ( #44251 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 18:44:31 +08:00
Khoa Pham and Claude Fable 5.1
a126e8d640
[Perf] deepseek_v4: fold the MegaAttention query layout into wq_b for block-FP8-as-MXFP8 checkpoints ( #44838 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-17 02:45:53 -07:00
Mick and Mick Qian
80f40522cd
[UX] diffusion: log LoRA merge decisions only on first evaluation ( #44376 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
2026-10-17 16:47:07 +08:00
56a20d7e34
[Perf] kernels: Enter a JIT kernel's device guard only when the device differs ( #44644 )
...
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-17 16:43:27 +08:00
f75232cb42
[Fix] hicache: Order write-back host copies after the forward that produced the KV ( #44800 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-17 01:31:40 -07:00
0d13efb2b8
[Feat] rust-server: Serve /v1/tokenize and /v1/detokenize from the Rust frontend ( #44767 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-17 01:30:50 -07:00
a2953d7b7f
[Fix] deepseek-v4.1: Serve prompt-logprob requests under decoder SWA bounded replay ( #44850 )
...
Co-authored-by: Pedram Ghazi <pedram.ghazi@amd.com >
Co-authored-by: David Orman <ormandj@corenode.com >
2026-10-17 01:25:33 -07:00
McZyWu
7a8804edb5
[Fix] [NPU] qwen: Fix shared-expert routing with Ascend FuseEP ( #40516 )
2026-10-17 16:19:44 +08:00
Liangsheng Yin
ef26f6ad49
[Fix] deepseek-v4.1: Keep the cached KV of folded encoder replay rows ( #44857 )
2026-10-17 00:56:18 -07:00
Khoa Pham and Claude Fable 5.1
bce7b27aa6
[Perf] deepseek_v4: run FlashInfer autotune for block-FP8 models served as MXFP8 ( #44842 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-17 00:19:35 -07:00
01dc08e23f
[Refactor] dsv41: Shard Engram tables on hash-column boundaries ( #42787 )
...
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-17 00:15:49 -07:00
Khoa Pham and Claude Fable 5.1
14c4dda24e
[Perf] deepseek_v4: fuse block-FP8 shared experts into MegaMoE for any 32-aligned block size ( #44837 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-16 23:47:41 -07:00
Liangsheng Yin
9259553f09
[Fix] megamoe: Support EPLB rebalance on SM100 ( #44847 )
2026-10-16 22:22:40 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
3fc08b939a
[Feat] server-args: Add --enable-hicache alias for --enable-hierarchical-cache ( #44849 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-17 13:22:34 +08:00
Ziang Li
29a7e19de1
[Refactor] nvfp4_online: Use explicit FlashInfer 4over6 config ( #41745 )
2026-10-16 20:56:10 -07:00
3603824541
[Perf] DSv4.1: Enable decoder bounded replay by default ( #44403 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
2026-10-16 19:52:02 -07:00
McZyWu
2d3dd3168d
[Fix] [NPU] attention: Use FIA TND for MLA prefix attention and add Gemma norm fallback ( #40131 )
2026-10-17 10:12:41 +08:00
Khoa Pham
ff979e0816
[Fix] megamoe: Respect request-pool limits in decode validation ( #44816 )
2026-10-16 18:56:38 -07:00
5cbe2371a5
[Feat] rust-server: Report Python's launch-time /server_info fields from the Rust frontend ( #44764 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-16 18:14:50 -07:00
b62a48ee15
[Feat] sglang-server: Serve /flush_cache from the Rust frontend ( #44766 )
...
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-16 17:24:01 -07:00
d8dd49d21d
[Fix] srt: Apply --gc-threshold in every server process and freeze every tokenizer worker ( #44785 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com >
2026-10-16 16:55:54 -07:00
d230f2c35b
[Feat] profiler: Add the forward pass id to detailed step span labels ( #44775 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: charlotte12l <charlotte12l@users.noreply.github.com >
2026-10-16 16:51:20 -07:00
36cf028346
[Perf] unified-memory: Prewarm the compaction move at startup and copy whole page envelopes by page id ( #44592 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com >
2026-10-16 16:47:50 -07:00
Khoa Pham and Claude Fable 5.1
aaf5f397a5
[Feat] dsv4.1: Support encoder SWA bounded replay with CUDA DP attention ( #43813 )
...
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com >
2026-10-16 15:38:33 -07:00
Yuwei An
8e3f11b219
[Perf] DSv4.1: Skip discarded KV rows during window commit ( #44783 )
2026-10-16 15:05:45 -07:00
Aarav and Aarav
d5d87899f9
[Fix] DSA: skip-topk pipeline handoff for owed outputs ( #43832 )
...
Co-authored-by: Aarav <211792885+AaravMehta-07@users.noreply.github.com >
2026-10-16 14:35:23 -07:00
Khoa Pham and Trevor Morris
71b60fa46c
[Fix] engram: Avoid blocking history copies during PD admission ( #44787 )
...
Co-authored-by: Trevor Morris <tmorris@nvidia.com >
2026-10-16 14:29:05 -07:00
2948662026
[Perf] dp-attention: Exchange the scheduler's MLP sync rows with gather + broadcast on gloo ( #44622 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: hanming-lu <hanming-lu@users.noreply.github.com >
2026-10-16 13:53:41 -07:00
Chunan Zeng
07950be745
[Perf] [NVIDIA] custom-all-reduce: Add 2shot_push all-reduce and retune SM100 TP4/TP8 ( #42811 )
2026-10-16 13:43:18 -07:00
Mohammad Miadh Angkad and Mohammad Angkad
21791e7023
[CI] [NVIDIA] DSv4.1-Flash: Split GB300 accuracy test into one file per config ( #44696 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-16 13:40:03 -07:00
+14
456231e251
[Feat] kernels: Add muse spark operators and collective GEMM (2/5) ( #44665 )
...
Co-authored-by: 1am9trash <1am9trash@gmail.com >
Co-authored-by: Alex Nails <alex.nails@radixark.ai >
Co-authored-by: Chunan Zeng <zcnrex@gmail.com >
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
Co-authored-by: Ethan Che <eche@meta.com >
Co-authored-by: HAI <hixiao@gmail.com >
Co-authored-by: Jiajun Li <guapisolo@gmail.com >
Co-authored-by: Jiajun Li <jiajun.li@radixark.ai >
Co-authored-by: Jimmy Shiu <jimmysh341@gmail.com >
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com >
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com >
Co-authored-by: Wookje Han <52435620+wookjeHan@users.noreply.github.com >
Co-authored-by: Wookje Han <wookjeh@nvl72d079-T17.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d094-T17.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d157-T16.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d170-T04.cm.cluster >
Co-authored-by: Wookje Han <wookjeh@nvl72d214-T17.cm.cluster >
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com >
Co-authored-by: Yuhao Yang <yhyang201@users.noreply.github.com >
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com >
Co-authored-by: ethche <ethche@gmail.com >
Co-authored-by: jacky.cheng <yichiche@amd.com >
Co-authored-by: jianan-gu <jianan.gu@intel.com >
2026-10-16 10:49:35 -07:00
ormandj and hnyls2002
80c54e6ac5
[Fix] [AMD] hicache: Serve encoder-replay prefix hits with a paged DSpark draft SWA ( #42716 )
...
Co-authored-by: hnyls2002 <lsyincs@gmail.com >
2026-10-16 10:45:16 -07:00
553068af25
[Fix] rust-server: Honour cache_salt and require_reasoning on /generate ( #44533 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-16 10:38:09 -07:00
453b251dbd
[Fix] hicache: Bound the storage prefetch timeout from the first blocked admission poll ( #44614 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com >
2026-10-16 10:37:11 -07:00
metamergebot and jiayisuse
989ccf4863
[Feat] disaggregation: Let a KV transfer backend skip the checksum for an incomparable peer ( #44605 )
...
Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com >
2026-10-16 10:36:46 -07:00
63a9b7a713
[Feat] http_server: Make the multi-worker ASGI app path configurable ( #44583 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com >
2026-10-16 10:36:35 -07:00
26a2f18c02
[Feat] sglang-server: serve /v1/loads from the Rust frontend ( #44532 )
...
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-16 10:35:37 -07:00
Xinyuan Tong
a038a0dca0
[Fix] dsv41: Preserve tool-result image order and current-turn thinking ( #44751 )
2026-10-16 23:39:50 +08:00
Benjamin Truong and Claude Opus 5.5
7c99ace127
[Fix] dsa: Fix ragged seed indices before MTP draft decode ( #42565 )
...
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-16 19:37:41 +08:00
Xinyi Song and HAI
4afae5568b
[Perf] [AMD] dsv4: Add a separate MegaMoE decode instance ( #44657 )
...
Co-authored-by: HAI <hixiao@gmail.com >
2026-10-16 03:11:02 -07:00
2adc76ad61
[Perf] [AMD] dsv4: reuse the q_lora fp8 quant for the C4 indexer wq_b (4/8) ( #41903 )
...
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com >
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-10-16 02:55:25 -07:00
kk and Duyi-Wang
baf451fcd6
[Perf] [AMD] dsv4: carry aiter DSv4 GEMM tuned rows and mHC tile_m=16 until aiter ships them (7/8) ( #44008 )
...
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-10-16 02:47:41 -07:00
f05da43d44
[Feat] [AMD] disagg: Support DSV4 decode radix cache on ROCm unified KV (bf16/fp8, DSpark) ( #30929 )
...
Co-authored-by: HAI <hixiao@gmail.com >
Co-authored-by: Duyi-Wang <duyi.wang@amd.com >
2026-10-16 02:39:27 -07:00