Commit Graph
100 Commits
Author SHA1 Message Date
Liangsheng Yin daf6317196 [mem_cache] Add free_full to release the full side of a tombstoned SWA node (#36637) 2026-08-27 19:34:48 -07:00
Liangsheng Yin 5d52f02f22 [misc] Fold the allocator free-group flag into free_group (#36739) 2026-08-27 16:12:47 -07:00
ec4bdbfa4a [Feature] Beam search support (#31626)
Co-authored-by: cswuyg <cswuyg@gmail.com>
Co-authored-by: cswuyg <496090217@qq.com>
Co-authored-by: Vedant Jhaveri <vedantjh2@gmail.com>
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
2026-08-26 16:56:15 -07:00
Liangsheng Yin 45c85c198b [CI] Lower the AWQ Marlin MMLU threshold to 0.80 (#36570) 2026-08-26 14:43:20 -07:00
Liangsheng Yin 896acc8860 [Fix] Clear full-to-SWA mapping with index_fill_ to avoid a blocking H2D copy (#35773) 2026-08-21 01:23:26 -07:00
Liangsheng Yin 0149f56e84 [CI] Gate /rerun-test on commenter trust and remove /rerun-stage (#35750) 2026-08-20 15:05:52 -07:00
Liangsheng Yin 5a100d9086 [misc] Trim restating comments and docstrings in srt/managers (#35622) 2026-08-20 14:18:40 -07:00
Liangsheng Yin a49560ce50 [misc] Add a comment style rule to .claude/rules (#35597) 2026-08-19 18:52:48 -07:00
Liangsheng Yin 1270204d2c Revert "[Feature] Add DeepEPv2 (ElasticBuffer) MoE A2A backend" (#35568) 2026-08-19 14:14:38 -07:00
Liangsheng Yin ccbe380028 [CI] Trim the base-c 4-gpu-h100 stage from 5 shards to 4 (#35407) 2026-08-19 00:48:07 -07:00
Liangsheng Yin e614121866 [Fix] Scale the req_to_token row headroom by attn_dcp_size (#35424) 2026-08-19 00:37:26 -07:00
Liangsheng Yin 7ebaa98f81 [Fix] Assert the page-aligned SWA evict floor on both PD decode prealloc paths (#35396) 2026-08-18 15:48:55 -07:00
Liangsheng Yin 87a09494fa [Refactor] Share the page-aligned decode alloc lens between EAGLE and DFLASH (#35382) 2026-08-18 14:39:41 -07:00
Liangsheng Yin 79dfef390b [Spec] Page-align the DFLASH decode KV reservation (#35265) 2026-08-18 13:36:04 -07:00
Liangsheng Yin aa82229173 [Fix] Assert the page-aligned SWA evict floor at PD decode prealloc (#35286) 2026-08-18 13:32:16 -07:00
Liangsheng Yin 0111b29031 [Perf] Hoist DSv4 draft-extend SWA write locs; unify SWA graph buffer naming (#34890) 2026-08-18 00:13:28 -07:00
Liangsheng Yin c0b6474b43 [Spec] Reduce host-side overhead in ngram draft prep (#35207) 2026-08-17 16:40:06 -07:00
Liangsheng Yin 032fe9c891 [Spec] Relay ngram accept tokens through the FutureMap (#35198) 2026-08-17 14:21:07 -07:00
Liangsheng Yin 711bdacb82 [Spec] Resolve shared-read ends from the backend declaration alone (#35059) 2026-08-17 01:35:29 -07:00
Liangsheng Yin 0d8c850a35 [Fix] Read the DSA prefill CP flag from the parallel config bag in bootstrap (#35110) 2026-08-17 00:21:41 -07:00
Liangsheng Yin 0099107e8b Revert "[AMD] [GLM5] Fuse shared-expert append into aiter grouped-topk (skip per-layer append kernel)" (#35105) 2026-08-16 23:49:49 -07:00
Liangsheng Yin 5e73c89b34 [Spec] Simplify compute_spec_v2_logprobs signature and skip identity gathers (#35058) 2026-08-16 16:01:26 -07:00
Liangsheng Yin 77cadf6b98 [Spec] Point multi-layer eagle's last shared-read runner at the draft runner (#35057) 2026-08-16 15:28:12 -07:00
Liangsheng Yin bae353ba55 [misc] Rename shared-read boundary to shared-read ends and fix wrapper delegation (#34982) 2026-08-16 14:36:31 -07:00
Liangsheng Yin 0f7aaceda5 [misc] Rename the WAR read-done fastpath to shared-read-done (#34916) 2026-08-15 15:02:02 -07:00
Liangsheng Yin 5c9ee86d90 [CI] Move the static ratchets back to CPU unit tests (#34913) 2026-08-14 23:32:00 -07:00
Liangsheng Yin f2c84de022 [Perf] Publish the WAR read-done event at DSPARK verify (#34816) 2026-08-14 00:46:52 -07:00
Liangsheng Yin 151a314829 [Fix] Make the DSpark draft num_token_non_padded host-to-device copy non-blocking (#34782) 2026-08-13 16:10:54 -07:00
Liangsheng Yin 8554d9a5bc [Fix] Carry the backend on Kimi-K3 deferred preprocessing configs (#34766) 2026-08-13 13:30:33 -07:00
Liangsheng Yin 8761b971f1 [CI] Fix nightly test failures (#34637) 2026-08-12 20:53:44 -07:00
Liangsheng Yin 26627e999d [Fix] Snapshot req.prefix_indices when the prefix cache is disabled (#34644) 2026-08-12 20:50:46 -07:00
Liangsheng Yin 50cc1aa241 [CI] Route mmlu and GB300 MMMU-Pro evals through sgl-eval (#34477) 2026-08-12 19:33:21 -07:00
Liangsheng Yin c05eb856f7 [CI] Fix nightly test failures (#34523) 2026-08-12 15:40:20 -07:00
Liangsheng Yin 9d2d737ebf [Benchmark] Remove 22 unmaintained benchmarks (#34520) 2026-08-11 22:24:43 -07:00
Liangsheng Yin 256981ce16 [CI] Align rerun-test environment with the test stages (#34195) 2026-08-11 21:37:10 -07:00
+3 5899674504 [Fix] Make DeepSeek-V4 reasoning and tool-call streaming parsing chunk-invariant (#34458)
Co-authored-by: hao-cyber <89575785+hao-cyber@users.noreply.github.com>
Co-authored-by: Enrico Falco <enrico9034@gmail.com>
Co-authored-by: Svyatoslav <85786374+slivanovich@users.noreply.github.com>
Co-authored-by: Andreas Hassellof <andreas@ombori.com>
Co-authored-by: Leoyzen <leoyzen@gmail.com>
Co-authored-by: Chenglun Hu <chenglunhu@gmail.com>
Co-authored-by: robellliu-dev <robell.liu@huawei.com>
Co-authored-by: Gavin.Zhu <gavin.z@gmicloud.ai>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: tancheng33 <garrytancheng@gmail.com>
Co-authored-by: dineshx29 <dinesh.b.offl@gmail.com>
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
2026-08-11 20:03:28 -07:00
Liangsheng Yin d8a61c26a6 [CI] Add a scheduled workflow to close stale PRs (#34380) 2026-08-11 00:54:02 -07:00
Liangsheng YinandAlex Nails b3c02cbce7 [perf] Collapse the DP attention scheduler sync to a single D2H copy (#34338)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
2026-08-10 21:52:29 -07:00
Liangsheng Yin 585c3c6816 [Refactor] Split the FlashInfer autotune dummy-run flag from the LM-head policy (#34336) 2026-08-10 21:20:05 -07:00
Liangsheng Yin c30872fa00 [Fix] Read trace level via get_global_trace_level() in trace_async (#34325) 2026-08-10 16:46:54 -07:00
Liangsheng Yin c80a38edcd [Fix] Pin cuda-tile to 1.6.0rc5 to unblock Python 3.10 x86_64 installs (#34321) 2026-08-10 15:09:47 -07:00
Liangsheng Yin 4eaaeda004 [CI] Build patched Docker images for both amd64 and arm64 (#34276) 2026-08-10 04:49:22 -07:00
Liangsheng Yin 0b6189d0e8 [CI] Add output_tag input to the Patch Docker Image workflow (#34253) 2026-08-10 02:48:03 -07:00
Liangsheng YinandBrayden Zhong b51bf9ec9e [Spec] Budget the DFLASH draft KV pool from its own attention geometry (#34234)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
2026-08-10 01:28:49 -07:00
Liangsheng Yin 3bb72bc72a [CI] Keep the torch compilation cache instead of wiping it on install (#34231) 2026-08-10 01:21:20 -07:00
Liangsheng YinandBrayden Zhong aea78d1e73 [misc] Pass FP8 scales in FlashInfer SWA prefill, autotune fp8 on SM120, and tighten is_image_understandable_model (#34217)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
2026-08-09 23:45:11 -07:00
Liangsheng YinandBrayden Zhong d6a066131c [Feature] Support NVFP4 token embedding in ModelOpt mixed-precision checkpoints (#34222)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
2026-08-09 23:37:54 -07:00
Liangsheng Yin 7c90840bad [CI] Key scheduled CUDA suites by runner_config instead of hand-written jobs (#34186) 2026-08-09 16:44:53 -07:00
Liangsheng YinandDeleter-D 4a5d7d3c67 [DSV4] Fix silent KV corruption when speculative draft tokens > 4 (#34189)
Co-authored-by: Deleter-D <867909454@qq.com>
2026-08-09 15:54:41 -07:00
Liangsheng Yin 1ebd6fab6c [CI] Gate Kimi-K3 acceptance length on the GSM8K average (#34145) 2026-08-08 20:28:42 -07:00
Liangsheng Yin 4792ab1e90 [CI] Pin the rust frontend parity test to eager prefill (#34146) 2026-08-08 20:07:19 -07:00
Liangsheng Yin a1ca76b24b [Scheduler] Unify WAR read-done gating behind shared-read boundary declarations (#34052) 2026-08-08 03:26:36 -07:00
Liangsheng Yin 2c0188cc78 [Fix] Give the piecewise CUDA graph test stub an hf_config (#34100) 2026-08-08 02:04:16 -07:00
Liangsheng YinandBaizhou Zhang f6a6f5bf1e [CI] Trim redundant nightly test registrations (#34070)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
2026-08-08 01:42:46 -07:00
Liangsheng Yin c59d2b4329 [Fix] Judge the phase-checker device-assert test by its FAIL line, not the exit code (#34017) 2026-08-07 16:19:05 -07:00
3c51e29deb Responses support (#32689)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Harmya Bhatt <harmyacs@gmail.com>
Co-authored-by: harmya <harmya@modal.com>
Co-authored-by: Xinyuan <xinyuan@radixark.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-08-07 13:21:46 -07:00
Liangsheng Yin 7395ee833e [CI] Share VLM engines and prune launch matrices on the per-commit H100/H200 suites (#33944) 2026-08-07 01:22:22 -07:00
Liangsheng Yin afa79330b8 [misc] Remove break-graph debug log; reclaim pid-less /dev/shm leaks in CI (#33929) 2026-08-06 21:18:01 -07:00
Liangsheng Yin 163b739b34 [CI] Refresh the CPU HF cache base only on main-ref runs (#33904) 2026-08-06 19:37:59 -07:00
Liangsheng Yin efc99a86ff [CI] Restore the full prefill CUDA graph capture range in test launches (#33847) 2026-08-06 01:40:10 -07:00
Liangsheng Yin 9bd1461757 [CI] Bound the CUDA graph capture range in test launches and lift the spec fixture's admission cap (#33776) 2026-08-05 20:00:30 -07:00
Liangsheng Yin 3869fe556f [CI] Collapse the EAGLE launch matrix and the scoring engine boots on the per-commit runners (#33756) 2026-08-05 16:43:19 -07:00
Liangsheng Yin c0ef548eef [misc] Unify MLA scaling init and remove dead buffer / scaling code (#33363) 2026-08-05 15:59:33 -07:00
Liangsheng Yin 2d27133fcf [CI] Skip apt-get when the required packages are already installed (#33757) 2026-08-05 15:06:01 -07:00
Liangsheng Yin de34dd11e9 [CI] Fold duplicate-server suites and prune the retract matrix on 1-gpu-5090 (#33745) 2026-08-05 12:41:51 -07:00
Liangsheng Yin 1a045669e4 [CI] Merge tokenizer worker tests and drop redundant triton attention e2e (#33641) 2026-08-05 11:55:11 -07:00
Liangsheng Yin 4a3d6ca88c [CI] Skip sglang-kernel and sgl-deep-gemm reinstall on version match (#33637) 2026-08-05 01:46:16 -07:00
Liangsheng Yin c0d5ebd6c4 [CI] Move CPU-only unit tests to the CPU suite and trim dead 5090 registrations (#33654) 2026-08-05 01:43:29 -07:00
Liangsheng Yin eac1f78568 [CI] Free hosted-runner disk space only when it is low (#33644) 2026-08-04 21:55:06 -07:00
Liangsheng Yin 198a3bc29b [Test] Route GEMM backend UTs through real layer modules and weight loaders (#33615) 2026-08-04 20:53:26 -07:00
Liangsheng Yin 1033cae8d5 [CI] Speed up dependency install: dual-ABI Rust ext cache and prevalidation pruning (#33619) 2026-08-04 20:33:48 -07:00
Liangsheng Yin 76dc89f5aa [Test] Replace NVFP4 MoE runner backend e2e matrix with a layer-level unit test (#33611) 2026-08-04 16:03:29 -07:00
Liangsheng Yin 0d99d91e49 [CI] Make B200 base-b suites single-GPU as prep for 1-gpu B200 runners (#33605) 2026-08-04 15:53:01 -07:00
Liangsheng Yin a0b3f1dde6 [Test] Replace GEMM backend e2e matrices with layer-level unit tests (#33596) 2026-08-04 15:50:41 -07:00
Liangsheng Yin 58da9859c4 [CI] Extract download-rust-ext and give every install step a cache fallback (#33597) 2026-08-04 15:33:34 -07:00
+26 abddb1c7e9 [Kimi] Support kimi-k3 (#32541)
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai>
Co-authored-by: Zijie Xia <37504505+zijiexia@users.noreply.github.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: zhangxiaohao <1024393531@qq.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Hao Phan <htphan@nvidia.com>
Co-authored-by: Thomas Wang <1am9trash@gmail.com>
Co-authored-by: RolaoDenthu <xinyisong0111@gmail.com>
Co-authored-by: pigeonsoup <32922982+pigeonsoup@users.noreply.github.com>
Co-authored-by: HaiShaw <hixiao@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Pranjal Shankhdhar <pranjal.ssh@gmail.com>
Co-authored-by: Lee Nau <lee.nau@gmail.com>
Co-authored-by: HMING <126185151+Hearum@users.noreply.github.com>
Co-authored-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: Xinyi Song <86638975+RolaoDenthu@users.noreply.github.com>
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
Co-authored-by: BBuf <xiaoyu.zhang@radixark.ai>
Co-authored-by: Hanming Lu <hanminglu@meta.com>
Co-authored-by: Xinyi Song <xinyis10@illinois.edu>
2026-08-04 13:22:49 -07:00
Liangsheng Yin 0753663b8e [CI] Trim redundant B200 test registrations (#33586) 2026-08-04 13:22:00 -07:00
Liangsheng Yin b6d548afd7 [Fix] Resolve VLM test image placeholders from the model's own chat template (#33509) 2026-08-04 02:01:39 -07:00
Liangsheng Yin 157401f050 [CI] Build the Rust extensions on the 5090 pool and seed the cache from main (#33460) 2026-08-03 23:33:20 -07:00
Liangsheng YinandMick afc868517b [Perf] Speed up the Kimi-K2.5 vision path and match PIL bicubic in the GPU resize (#33349)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-08-03 21:57:49 -07:00
Liangsheng Yin 7ba393dd15 [CI] Dispatch base-a-test-cpu through its own reusable stage workflow (#33461) 2026-08-03 20:49:23 -07:00
Liangsheng Yin cdff33d738 [CI] Build the Rust extension modules once per run instead of in every CUDA job (#33384) 2026-08-03 18:56:48 -07:00
Liangsheng Yin c949e91f18 [CI] Remove the orphaned site-packages sglang skeleton that shadows the checkout (#33441) 2026-08-03 16:31:25 -07:00
Liangsheng Yin 22c2e2bcad [CI] Persist the cargo build cache across CUDA CI jobs (#33361) 2026-08-03 16:23:09 -07:00
Liangsheng Yin b8f6181bff [CI] Build the Rust extensions with the pinned toolchain instead of the image default (#33437) 2026-08-03 16:16:34 -07:00
Liangsheng Yin 45c00daa1b [misc] Deep-merge nested config overrides and parse request bodies with orjson (#33351) 2026-08-02 23:55:06 -07:00
Liangsheng Yin fcc4de9e5f [CI] Size the CPU stage from the live partition model (#33329) 2026-08-02 22:26:58 -07:00
Liangsheng Yin f5f021672a [Fix] Treat an empty grammar constraint as unset in SamplingParams (#33328) 2026-08-02 21:45:51 -07:00
Liangsheng YinandMick dd6ddc053b [Fix] Drop deprecated multimodal processor residency state (#33308)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-08-02 20:02:53 -07:00
Liangsheng YinandMick 4bc593fdc8 [Perf] Broadcast single-image DP vision embedding instead of pad-to-max all-gather (#33307)
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-08-02 19:39:53 -07:00
Liangsheng Yin 2a7a299c27 [Spec] Support sampling in the DSPARK graph-folded draft proposal (#33298) 2026-08-02 18:46:39 -07:00
Liangsheng Yin 558c9bdcc2 [misc] Improve benchmark determinism and dataset API coverage (#33255) 2026-08-02 01:39:50 -07:00
Liangsheng Yin ca07917c58 [Fix] Bound FULL_MASK verify-mask reuse by the captured max_bs (#33127) 2026-07-31 17:50:40 -07:00
Liangsheng YinandKaixi Matteo Chen 5c6635d8f3 [Spec] Compact the target-verify mask when nothing reads it (#32920)
Co-authored-by: Kaixi Matteo Chen <kaiximatteoc@nvidia.com>
2026-07-30 23:13:21 -07:00
Liangsheng Yin 9f56553408 [Perf] Fast-path chain-style draft token organization in multi-layer EAGLE (#32887) 2026-07-30 02:55:21 -07:00
Liangsheng YinandKaixi 6ab3231b97 [Perf] Skip the target-verify tree mask fill when the backend never reads it (#32886)
Co-authored-by: Kaixi <kaiximatteoc@nvidia.com>
2026-07-30 02:32:38 -07:00
Liangsheng Yin c192145830 [Kernel] Fuse KV-cache writes for asymmetric K/V (head_dim != v_head_dim) (#32813) 2026-07-30 00:26:10 -07:00
Liangsheng Yin 2625fdfe6b [Fix] Count multi-layer draft-extend replays in the fwd-occupancy device timer (#32867) 2026-07-30 00:21:34 -07:00
Liangsheng Yin f4e0ac382e [misc] Remove unused multi_layer_draft_forward_cg module (#32881) 2026-07-29 21:17:16 -07:00
Liangsheng Yin 313a518bee [Spec] Emit step trace span for multi-layer draft-extend graph replays (#32850) 2026-07-29 19:22:58 -07:00