Commit Graph
14312 Commits
Author SHA1 Message Date
f75232cb42 [Fix] hicache: Order write-back host copies after the forward that produced the KV (#44800)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-17 01:31:40 -07:00
0d13efb2b8 [Feat] rust-server: Serve /v1/tokenize and /v1/detokenize from the Rust frontend (#44767)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-17 01:30:50 -07:00
a2953d7b7f [Fix] deepseek-v4.1: Serve prompt-logprob requests under decoder SWA bounded replay (#44850)
Co-authored-by: Pedram Ghazi <pedram.ghazi@amd.com>
Co-authored-by: David Orman <ormandj@corenode.com>
2026-10-17 01:25:33 -07:00
McZyWu 7a8804edb5 [Fix] [NPU] qwen: Fix shared-expert routing with Ascend FuseEP (#40516) 2026-10-17 16:19:44 +08:00
Liangsheng Yin ef26f6ad49 [Fix] deepseek-v4.1: Keep the cached KV of folded encoder replay rows (#44857) 2026-10-17 00:56:18 -07:00
Khoa PhamandClaude Fable 5.1 bce7b27aa6 [Perf] deepseek_v4: run FlashInfer autotune for block-FP8 models served as MXFP8 (#44842)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-17 00:19:35 -07:00
01dc08e23f [Refactor] dsv41: Shard Engram tables on hash-column boundaries (#42787)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-17 00:15:49 -07:00
Khoa PhamandClaude Fable 5.1 14c4dda24e [Perf] deepseek_v4: fuse block-FP8 shared experts into MegaMoE for any 32-aligned block size (#44837)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-16 23:47:41 -07:00
Liangsheng Yin 9259553f09 [Fix] megamoe: Support EPLB rebalance on SM100 (#44847) 2026-10-16 22:22:40 -07:00
Mohammad Miadh AngkadandMohammad Angkad 3fc08b939a [Feat] server-args: Add --enable-hicache alias for --enable-hierarchical-cache (#44849)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-17 13:22:34 +08:00
Ziang Li 29a7e19de1 [Refactor] nvfp4_online: Use explicit FlashInfer 4over6 config (#41745) 2026-10-16 20:56:10 -07:00
3603824541 [Perf] DSv4.1: Enable decoder bounded replay by default (#44403)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-10-16 19:52:02 -07:00
McZyWu 2d3dd3168d [Fix] [NPU] attention: Use FIA TND for MLA prefix attention and add Gemma norm fallback (#40131) 2026-10-17 10:12:41 +08:00
Khoa Pham ff979e0816 [Fix] megamoe: Respect request-pool limits in decode validation (#44816) 2026-10-16 18:56:38 -07:00
5cbe2371a5 [Feat] rust-server: Report Python's launch-time /server_info fields from the Rust frontend (#44764)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 18:14:50 -07:00
b62a48ee15 [Feat] sglang-server: Serve /flush_cache from the Rust frontend (#44766)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 17:24:01 -07:00
d8dd49d21d [Fix] srt: Apply --gc-threshold in every server process and freeze every tokenizer worker (#44785)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com>
2026-10-16 16:55:54 -07:00
d230f2c35b [Feat] profiler: Add the forward pass id to detailed step span labels (#44775)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: charlotte12l <charlotte12l@users.noreply.github.com>
2026-10-16 16:51:20 -07:00
36cf028346 [Perf] unified-memory: Prewarm the compaction move at startup and copy whole page envelopes by page id (#44592)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com>
2026-10-16 16:47:50 -07:00
Khoa PhamandClaude Fable 5.1 aaf5f397a5 [Feat] dsv4.1: Support encoder SWA bounded replay with CUDA DP attention (#43813)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-16 15:38:33 -07:00
Yuwei An 8e3f11b219 [Perf] DSv4.1: Skip discarded KV rows during window commit (#44783) 2026-10-16 15:05:45 -07:00
AaravandAarav d5d87899f9 [Fix] DSA: skip-topk pipeline handoff for owed outputs (#43832)
Co-authored-by: Aarav <211792885+AaravMehta-07@users.noreply.github.com>
2026-10-16 14:35:23 -07:00
Khoa PhamandTrevor Morris 71b60fa46c [Fix] engram: Avoid blocking history copies during PD admission (#44787)
Co-authored-by: Trevor Morris <tmorris@nvidia.com>
2026-10-16 14:29:05 -07:00
2948662026 [Perf] dp-attention: Exchange the scheduler's MLP sync rows with gather + broadcast on gloo (#44622)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: hanming-lu <hanming-lu@users.noreply.github.com>
2026-10-16 13:53:41 -07:00
Chunan Zeng 07950be745 [Perf] [NVIDIA] custom-all-reduce: Add 2shot_push all-reduce and retune SM100 TP4/TP8 (#42811) 2026-10-16 13:43:18 -07:00
Mohammad Miadh AngkadandMohammad Angkad 21791e7023 [CI] [NVIDIA] DSv4.1-Flash: Split GB300 accuracy test into one file per config (#44696)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-16 13:40:03 -07:00
+14 456231e251 [Feat] kernels: Add muse spark operators and collective GEMM (2/5) (#44665)
Co-authored-by: 1am9trash <1am9trash@gmail.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Ethan Che <eche@meta.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Jiajun Li <guapisolo@gmail.com>
Co-authored-by: Jiajun Li <jiajun.li@radixark.ai>
Co-authored-by: Jimmy Shiu <jimmysh341@gmail.com>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
Co-authored-by: Wookje Han <52435620+wookjeHan@users.noreply.github.com>
Co-authored-by: Wookje Han <wookjeh@nvl72d079-T17.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d094-T17.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d157-T16.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d170-T04.cm.cluster>
Co-authored-by: Wookje Han <wookjeh@nvl72d214-T17.cm.cluster>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Yuhao Yang <yhyang201@users.noreply.github.com>
Co-authored-by: Zijie Chen <300606707+zijiecode@users.noreply.github.com>
Co-authored-by: ethche <ethche@gmail.com>
Co-authored-by: jacky.cheng <yichiche@amd.com>
Co-authored-by: jianan-gu <jianan.gu@intel.com>
2026-10-16 10:49:35 -07:00
ormandjandhnyls2002 80c54e6ac5 [Fix] [AMD] hicache: Serve encoder-replay prefix hits with a paged DSpark draft SWA (#42716)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-16 10:45:16 -07:00
553068af25 [Fix] rust-server: Honour cache_salt and require_reasoning on /generate (#44533)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 10:38:09 -07:00
453b251dbd [Fix] hicache: Bound the storage prefetch timeout from the first blocked admission poll (#44614)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
2026-10-16 10:37:11 -07:00
metamergebotandjiayisuse 989ccf4863 [Feat] disaggregation: Let a KV transfer backend skip the checksum for an incomparable peer (#44605)
Co-authored-by: jiayisuse <jiayisuse@users.noreply.github.com>
2026-10-16 10:36:46 -07:00
63a9b7a713 [Feat] http_server: Make the multi-worker ASGI app path configurable (#44583)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: joshuadeng <joshuadeng@users.noreply.github.com>
2026-10-16 10:36:35 -07:00
26a2f18c02 [Feat] sglang-server: serve /v1/loads from the Rust frontend (#44532)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 10:35:37 -07:00
Xinyuan Tong a038a0dca0 [Fix] dsv41: Preserve tool-result image order and current-turn thinking (#44751) 2026-10-16 23:39:50 +08:00
Benjamin TruongandClaude Opus 5.5 7c99ace127 [Fix] dsa: Fix ragged seed indices before MTP draft decode (#42565)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 19:37:41 +08:00
Xinyi SongandHAI 4afae5568b [Perf] [AMD] dsv4: Add a separate MegaMoE decode instance (#44657)
Co-authored-by: HAI <hixiao@gmail.com>
2026-10-16 03:11:02 -07:00
2adc76ad61 [Perf] [AMD] dsv4: reuse the q_lora fp8 quant for the C4 indexer wq_b (4/8) (#41903)
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:55:25 -07:00
kkandDuyi-Wang baf451fcd6 [Perf] [AMD] dsv4: carry aiter DSv4 GEMM tuned rows and mHC tile_m=16 until aiter ships them (7/8) (#44008)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:47:41 -07:00
f05da43d44 [Feat] [AMD] disagg: Support DSV4 decode radix cache on ROCm unified KV (bf16/fp8, DSpark) (#30929)
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:39:27 -07:00
42eb556e66 [Perf] [AMD] dsv4: optional e8m0 mxscale GEMM for dense 128x128 fp8 linears (6/8) (#44007)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
Co-authored-by: HAI <hixiao@gmail.com>
2026-10-16 02:28:31 -07:00
caihuali95 8d229d96e0 [Fix] unified-memory: enabling trtllm_mha with decode graphs and speculative decoding (#42930) 2026-10-16 02:24:06 -07:00
kkandDuyi-Wang 6efc96abf9 [Perf] [AMD] dsv4: FP4 indexer on aiter's row-group kernel for decode and target-verify (5/8) (#44006)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:16:06 -07:00
kkandDuyi-Wang 9fde8bcc46 [Perf] [AMD] moe: small-batch sort with histogram + bitonic sort for 32 < P <= 256 (3/8) (#44004)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:09:32 -07:00
kkandDuyi-Wang f0ff18353d [Perf] [AMD] dsv4: fp8 verify writes the SWA ring in the fused norm/rope kernel (2/8) (#44003)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 01:58:00 -07:00
kkandDuyi-Wang ab5858ac8a [Perf] [AMD] dsv4: fp8 unified_kv decode/verify on FlyDSL v4 nm decode with a latency-model split plan (1/8) (#44016)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 01:50:16 -07:00
Liangsheng Yin fad7d95a7f [Fix] deepseek-v4.1: Keep the DSpark draft SWA out of the radix tree (#44673) 2026-10-16 01:18:58 -07:00
kkandwunhuang bdc4ec15ce [Perf] [AMD] dspark: No per-step host sync in decode (1/4) (#44405)
Co-authored-by: wunhuang <wunhuang@amd.com>
2026-10-16 01:13:37 -07:00
jiaqiang-dot-liu a448fdf316 [Perf] [AMD] qwen3: Fuse QK-norm + RoPE + KV cache write for dense Qwen3 decode (#44267) 2026-10-16 00:51:55 -07:00
Ziang Li 7098ec387c [Feat] FlashInfer: Add NVFP4 W4A16 and SiTU support to MegaMoE (#44036) 2026-10-16 00:38:50 -07:00
Liangsheng YinandJingshu Chen 3a5d46e946 [Perf] pd: Stage decode-side KV preallocation lengths through pinned memory (#38955)
Co-authored-by: Jingshu Chen <96519051+elmo2019@users.noreply.github.com>
2026-10-16 00:34:56 -07:00