45 Commits
Author SHA1 Message Date
weireweireandweireweire 49a2fbf817 [Fix] docker: Pin sgl-nvidia-mega to 3333e4c so the megakernel starts on main (#46030)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-10-23 17:58:04 -07:00
weireweireandweireweire 96f013b452 [Doc] Cookbook: update GB300 DSV4.1 disagg AgentX launches from InferenceX #4303 (#45495)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-10-22 19:18:55 +08:00
bf145750a6 [Perf] prefill-cuda-graph: Make prefill shared-read completion explicit and default-on (#37958)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-10-20 16:12:55 -07:00
weireweireandweireweire c7687a0e7c [Fix] deepseek-v4: Keep the draft MoE out of the target's expert placement (#45299)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-10-19 22:20:26 -07:00
602c92ccd8 [Perf] DeepSeek-V4.1: Raise the SM100 swap-AB decode cap to 40 rows (#44632)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: nbommi19 <227267412+nbommi19@users.noreply.github.com>
2026-10-19 22:20:10 -07:00
72d886efb8 [Perf] DeepSeek-V4.1: Run the small-batch decode fusions at TP2 (#43838)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: nbommi19 <227267412+nbommi19@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-19 21:17:05 -07:00
06342acad2 [Perf] deepseek-v4.1: Stage late-layer DP row counts through pinned memory (#45062)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-18 20:19:26 -07:00
weireweire 296774da37 [Tokenizer] Require fastokens>=0.3.2 and include it in the all extra (#43104) 2026-10-11 20:41:53 -07:00
weireweireandweireweire 46c2585faa [DSpark] Fold draft sampling for DeepSeek-V4.1 heads under AUTO (#42987)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-10-09 14:40:28 -07:00
78996e3f02 [Performance] Batch Mooncake hybrid-pool metadata and puts (#39696)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:12:42 +08:00
7889fbab9a [PD] Keep EAGLE DP graph and token metadata consistent (#32196)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
2026-09-28 17:15:51 -07:00
335f6aab27 [DSV4] Support raw-index output in TopK v2 (#33672)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-11 07:17:36 -07:00
fb8d7eedda Fix dummy initialization of inverse weight scales (#35491)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-01 14:19:14 -07:00
weireweireandweireweire 3254f9b47c [Blackwell] Reserve SMs for DeepGEMM MegaMoE grid barriers (#36657)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-28 13:37:36 -07:00
weireweireandweireweire ea48cb04cc Pin scheduler metadata before asynchronous H2D copies (#35944)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-26 22:21:47 -07:00
weireweireandweireweire 4d5d506486 Publish gated DSV4 DFLASH-family target-prefill read completion (#35947)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-26 21:33:08 -07:00
weireweireandweireweire 5263568bcb [HiCache] Keep auxiliary load-back out of Full KV pending ownership (#36317)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-27 01:30:38 +08:00
weireweireandweireweire db15976fe9 Fix DSV4 DSpark sample-from-anchor initialization (#36419)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-25 23:18:58 -07:00
weireweireandweireweire 6eb941a34c [Spec] Support MegaMoE for DSpark under dp attention (#34844)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-14 17:20:01 -07:00
weireweireandweireweire 54c44fef73 [Perf] Skip trivial DSV4 nonpaged indexer logits (#33857)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-13 16:21:30 -07:00
weireweireandweireweire 2d5009d130 [DSV4] Avoid host syncs in EAGLE prefill (#33662)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-10 17:09:55 -07:00
weireweireandweireweire 5e58af1503 Fix fractional simulated acceptance in DSpark (#33463)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-07 00:45:07 -07:00
weireweireandweireweire fe55d78b7d Fix MXFP4 scale placeholder initialization (#33500)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-06 01:46:54 -07:00
weireweireandweireweire 23ea7b6481 Prewarm DSV4 MHC post kernel at model load (#30741)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-08-03 23:41:56 -07:00
weireweireandweireweire 55c1963df4 Remove unused draft-extend CUDA graph top-k (#31430)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-07-30 17:45:01 -07:00
weireweireandweireweire bd47ec97ff [EAGLE] Handle NaNs in fused top-k=1 (#32396)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-07-29 00:07:42 -07:00
weireweireandweireweire ac4fa6496c Skip no-op EAGLE sampling renormalization (#31294)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-07-15 22:37:10 -07:00
weireweireandweireweire f2c875d1c8 [PD] Route PD server warmup to every DP rank (#30748)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-07-15 15:59:41 +08:00
weireweireandweireweire a9cf5e68e6 [DSV4] Remove per-step seqlen D2H from speculative to make overlap scheduler work (#30365)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-07-14 18:01:22 -07:00
weireweireandweireweire 5e3dc5dd5f Fix immediate profiler step range boundary (#30215)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-07-09 19:47:23 -07:00
weireweireandweireweire 1c9eb6bb0b [DSV4] Fix draft SWA transfer for disaggregated MTP (#30461)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-07-09 13:18:34 +08:00
weireweireandweireweire c7ca332fb0 Fix DSV4 prefill large Triton recompilation idle across context lengths (#30255)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-07-07 20:35:33 -07:00
weireweireandweireweire a01afdd526 Support real draft tokens to simulated acceptance (#29645)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-06-30 17:22:05 -07:00
weireweire f9a3720e2b Skip empty non-idle output batches (#28504) 2026-06-26 02:40:01 +08:00
weireweireandweireweire 5f30fa258c Support DP-aware PD router dispatch (#26245)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-06-24 15:02:51 -07:00
weireweireandweireweire 7fd63f4cf2 Remove stale load collection from output streaming hot path (#28408)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-06-17 15:02:20 -07:00
weireweire bf38a0b03d Fix disaggregated decode load token accounting (#25736) 2026-06-15 11:37:41 +08:00
weireweireandZhangheng 034dd39189 Support KV events for UnifiedRadixCache (#26387)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-05-27 22:10:40 +08:00
weireweire 629b6c6a85 correct allreduce fusion and dummy_run alignment in SCATTERED MLP mode (moe_dense_tp_size=1) (#19918) 2026-05-22 19:18:25 -07:00
weireweire 684638e053 Fix prefill batch iter logging under overlap (#20845) 2026-05-07 02:10:42 -07:00
weireweire 9191b02eda Fix cuda graph max bs capture upper bound (#21005) 2026-03-31 15:20:56 -07:00
weireweireandWeiliangl User 4455d17619 [PD] Refactor Disagg Conn and Fix Hang with total_request/total_tokens Balancing (#21299)
Co-authored-by: Weiliangl User <weiliangl@login-node.hosted.internal>
2026-03-31 18:01:50 +08:00
weiliang 6559e43f30 Support FP8 MLA prefill and 128k context. (#14395) 2025-12-19 01:24:00 -08:00
weiliang a18161875c Fix Flashinfer Backend for SM120 Usage (#12325) 2025-10-29 11:51:55 -07:00
weiliang 88596739a4 Support running FP4 Deepseek on SM120. (#11708) 2025-10-27 17:37:49 -07:00