 Liangsheng Yin
|
f5819b09bf
|
Revert "[AMD][DSV4] Fix unified-KV pool sizing and SWA ring accounting" (#38163)
|
2026-09-05 17:28:46 -07:00 |
|
|
Liangsheng Yin
|
0454c074b4
|
[mem_cache] Clean up unified allocator leftovers (#38103)
|
2026-09-04 22:02:50 -07:00 |
|
|
Liangsheng Yin
|
d50e9a9756
|
[Test] Prune redundant unified-memory allocator and pool tests (#38093)
|
2026-09-04 20:47:01 -07:00 |
|
|
Liangsheng Yin
|
0645398a32
|
[mem_cache] Move the unified-memory allocators into allocator/ and split the composites out (#38072)
|
2026-09-04 19:53:15 -07:00 |
|
|
|
67248e04b4
|
[mem_cache] Route hybrid SWA full-side kv-row frees through free_segment (#37876)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
|
2026-09-04 01:48:33 -07:00 |
|
|
Liangsheng Yin
|
6147a54ddf
|
[PD] Bound transfer engine init with SGLANG_DISAGGREGATION_ENGINE_INIT_TIMEOUT (#37874)
|
2026-09-03 17:16:55 -07:00 |
|
|
Liangsheng Yin
|
8b0501399e
|
[CI] Improve Lark CI cards: structured layout, PDT timestamps, slow-only queue digest (#37884)
|
2026-09-03 16:18:56 -07:00 |
|
|
Liangsheng Yin
|
0610a6539d
|
[CI] Add Lark notifications for CUDA CI status, runner health, and queue time (#37881)
|
2026-09-03 15:46:33 -07:00 |
|
|
Liangsheng Yin
|
0e5414fd2f
|
[Test] Allow top-k cutoff ties in test_sampling_mask_matches_topk_logprobs (#37873)
|
2026-09-03 14:13:26 -07:00 |
|
|
Liangsheng Yin
|
2a980cbf10
|
[mem_cache] Require page-aligned starts in free_segment and drop the boundary trim (#37729)
|
2026-09-03 13:28:33 -07:00 |
|
|
Liangsheng Yin
|
54da74e83e
|
[Fix] Broadcast PP dynamic-chunk profiling failures so every rank disables together (#37675)
|
2026-09-02 21:09:04 -07:00 |
|
|
Liangsheng Yin
|
49e6e81830
|
[CI] Accept t64-suffixed apt packages in the install skip check (#37689)
|
2026-09-02 20:51:47 -07:00 |
|
|
Liangsheng Yin
|
a522c8a4b6
|
[misc] Extract PP dynamic chunk sizing into a DynamicChunkSizer scheduler component (#37674)
|
2026-09-02 18:35:43 -07:00 |
|
|
Liangsheng Yin
|
5c46ce37f5
|
[Fix] Apply the attention-CP broadcast result in PP dynamic-chunk profiling (#37669)
|
2026-09-02 17:20:07 -07:00 |
|
|
Liangsheng Yin
|
4bc34117f1
|
[CI] Install lmms-eval from PyPI, drop human-eval install, add clone token fallback (#37672)
|
2026-09-02 16:23:10 -07:00 |
|
|
Liangsheng Yin
|
19c7679e9e
|
[mem_cache] Make free_swa sync-free on page_size == 1 (#36723)
|
2026-09-02 14:18:22 -07:00 |
|
|
Liangsheng Yin
|
2d799c28f4
|
[CI] Authenticate and retry git clones in install scripts (#37647)
|
2026-09-02 13:21:26 -07:00 |
|
|
Liangsheng Yin
|
862a909a08
|
[Fix] Lock PP dynamic-chunk profiling requests before releasing through the tree cache (#37509)
|
2026-09-02 02:09:01 -07:00 |
|
|
Liangsheng Yin
|
ebfd8c60e5
|
[CI] Install sgl-eval from PyPI through the test extra (#37504)
|
2026-09-02 01:45:25 -07:00 |
|
|
Liangsheng Yin
|
01c3a5f54f
|
[misc] Resolve SWA ownership at enqueue time for grouped free() (#36646)
|
2026-09-01 23:06:55 -07:00 |
|
|
Liangsheng Yin
|
832d029870
|
[mem_cache] Split duplicate insert frees at the SWA eviction floor (#37481)
|
2026-09-01 23:01:40 -07:00 |
|
|
Liangsheng Yin
|
83a9b5dd88
|
[mem_cache] Drop the torch.unique sync from the SWA page expansion (#37463)
|
2026-09-01 14:16:47 -07:00 |
|
|
Liangsheng Yin
|
3484f7f836
|
[mem_cache] Add free_kv_row to release a request's kv row by row range (#36721)
|
2026-09-01 01:14:43 -07:00 |
|
|
Liangsheng Yin
|
959ca033eb
|
refactor(hicache): simplify decode offload state bookkeeping (#37299)
|
2026-08-31 23:09:41 -07:00 |
|
|
Liangsheng Yin
|
3b14f37b74
|
[Fix] Use real ReqKvInfo in unit-test req mocks (#37339)
|
2026-08-31 20:25:51 -07:00 |
|
|
Liangsheng Yin
|
2530204502
|
[mem_cache] Make release, row-reuse asserts, and presence checks read the KV record (#37167)
|
2026-08-31 12:46:15 -07:00 |
|
|
Liangsheng YinandBaizhou Zhang
|
95f0f41021
|
[CI] Move tests onto the right CI stages (#34074)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2026-08-31 12:39:59 -07:00 |
|
|
Liangsheng Yin
|
3a6ed55999
|
[Fix] Shut hicache test servers down gracefully before SIGKILL (#37194)
|
2026-08-30 22:45:40 -07:00 |
|
|
Liangsheng Yin
|
5d12ad4fd7
|
[mem_cache] Move mamba state and retraction_backup into ReqKvInfo (#37164)
|
2026-08-30 22:03:01 -07:00 |
|
|
Liangsheng Yin
|
4bb8de34cc
|
[mem_cache] Share one ReqKvInfo between a streaming session slot and its request (#37108)
|
2026-08-30 16:36:48 -07:00 |
|
|
Liangsheng Yin
|
007ef5e23a
|
[mem_cache] Move req_pool_idx into ReqKvInfo (#37094)
|
2026-08-30 14:46:21 -07:00 |
|
|
Liangsheng Yin
|
5ec959965b
|
[mem_cache] Settle extend kv_committed_len inside alloc_for_extend (#37085)
|
2026-08-29 23:14:31 -07:00 |
|
|
Liangsheng Yin
|
0438b16154
|
[mem_cache] Move kv_committed_len into ReqKvInfo (#37078)
|
2026-08-29 22:42:25 -07:00 |
|
|
Liangsheng Yin
|
6be767c2d2
|
[mem_cache] Move cache_protected_len and swa_evict_floor into ReqKvInfo (#36982)
|
2026-08-29 19:37:53 -07:00 |
|
|
Liangsheng Yin
|
9a489f8d2f
|
[Test] Move gpqa and aime25 onto sgl-eval, drop unused eval paths (#36979)
|
2026-08-29 17:36:13 -07:00 |
|
|
Liangsheng Yin
|
24a3a63e6b
|
[misc] Keep req.kv non-optional and key KV ownership on req_pool_idx (#36958)
|
2026-08-28 23:47:42 -07:00 |
|
|
Liangsheng Yin
|
a25df83fe3
|
[Cookbook] Run accuracy benchmarks through sgl-eval (#36977)
|
2026-08-28 23:29:38 -07:00 |
|
|
Liangsheng Yin
|
ec962fd12d
|
[Fix] Fall back to the process-group broadcast for DSA topk when PyNCCL is absent (#36963)
|
2026-08-28 22:20:35 -07:00 |
|
|
Liangsheng Yin
|
0df69849ba
|
[Fix] Drop the duplicated DSpark draft sample_block call (#36934)
|
2026-08-28 18:10:41 -07:00 |
|
|
Liangsheng Yin
|
60f6d77a98
|
[mem_cache] Carry swa_evicted_seqlen into SWARadixCache.cache_unfinished_req (#36909)
|
2026-08-28 14:29:27 -07:00 |
|
|
Liangsheng Yin
|
f65b2b2b15
|
[Fix] Lazy-import aiter in DSv4 paged_decode to unbreak CPU CI (#36914)
|
2026-08-28 14:16:53 -07:00 |
|
|
Liangsheng Yin
|
daf6317196
|
[mem_cache] Add free_full to release the full side of a tombstoned SWA node (#36637)
|
2026-08-27 19:34:48 -07:00 |
|
|
Liangsheng Yin
|
5d52f02f22
|
[misc] Fold the allocator free-group flag into free_group (#36739)
|
2026-08-27 16:12:47 -07:00 |
|
|
|
ec4bdbfa4a
|
[Feature] Beam search support (#31626)
Co-authored-by: cswuyg <cswuyg@gmail.com>
Co-authored-by: cswuyg <496090217@qq.com>
Co-authored-by: Vedant Jhaveri <vedantjh2@gmail.com>
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
|
2026-08-26 16:56:15 -07:00 |
|
|
Liangsheng Yin
|
45c85c198b
|
[CI] Lower the AWQ Marlin MMLU threshold to 0.80 (#36570)
|
2026-08-26 14:43:20 -07:00 |
|
|
Liangsheng Yin
|
896acc8860
|
[Fix] Clear full-to-SWA mapping with index_fill_ to avoid a blocking H2D copy (#35773)
|
2026-08-21 01:23:26 -07:00 |
|
|
Liangsheng Yin
|
0149f56e84
|
[CI] Gate /rerun-test on commenter trust and remove /rerun-stage (#35750)
|
2026-08-20 15:05:52 -07:00 |
|
|
Liangsheng Yin
|
5a100d9086
|
[misc] Trim restating comments and docstrings in srt/managers (#35622)
|
2026-08-20 14:18:40 -07:00 |
|
|
Liangsheng Yin
|
a49560ce50
|
[misc] Add a comment style rule to .claude/rules (#35597)
|
2026-08-19 18:52:48 -07:00 |
|
|
Liangsheng Yin
|
1270204d2c
|
Revert "[Feature] Add DeepEPv2 (ElasticBuffer) MoE A2A backend" (#35568)
|
2026-08-19 14:14:38 -07:00 |
|
|
Liangsheng Yin
|
ccbe380028
|
[CI] Trim the base-c 4-gpu-h100 stage from 5 shards to 4 (#35407)
|
2026-08-19 00:48:07 -07:00 |
|
|
Liangsheng Yin
|
e614121866
|
[Fix] Scale the req_to_token row headroom by attn_dcp_size (#35424)
|
2026-08-19 00:37:26 -07:00 |
|
|
Liangsheng Yin
|
7ebaa98f81
|
[Fix] Assert the page-aligned SWA evict floor on both PD decode prealloc paths (#35396)
|
2026-08-18 15:48:55 -07:00 |
|
|
Liangsheng Yin
|
87a09494fa
|
[Refactor] Share the page-aligned decode alloc lens between EAGLE and DFLASH (#35382)
|
2026-08-18 14:39:41 -07:00 |
|
|
Liangsheng Yin
|
79dfef390b
|
[Spec] Page-align the DFLASH decode KV reservation (#35265)
|
2026-08-18 13:36:04 -07:00 |
|
|
Liangsheng Yin
|
aa82229173
|
[Fix] Assert the page-aligned SWA evict floor at PD decode prealloc (#35286)
|
2026-08-18 13:32:16 -07:00 |
|
|
Liangsheng Yin
|
0111b29031
|
[Perf] Hoist DSv4 draft-extend SWA write locs; unify SWA graph buffer naming (#34890)
|
2026-08-18 00:13:28 -07:00 |
|
|
Liangsheng Yin
|
c0b6474b43
|
[Spec] Reduce host-side overhead in ngram draft prep (#35207)
|
2026-08-17 16:40:06 -07:00 |
|
|
Liangsheng Yin
|
032fe9c891
|
[Spec] Relay ngram accept tokens through the FutureMap (#35198)
|
2026-08-17 14:21:07 -07:00 |
|
|
Liangsheng Yin
|
711bdacb82
|
[Spec] Resolve shared-read ends from the backend declaration alone (#35059)
|
2026-08-17 01:35:29 -07:00 |
|
|
Liangsheng Yin
|
0d8c850a35
|
[Fix] Read the DSA prefill CP flag from the parallel config bag in bootstrap (#35110)
|
2026-08-17 00:21:41 -07:00 |
|
|
Liangsheng Yin
|
0099107e8b
|
Revert "[AMD] [GLM5] Fuse shared-expert append into aiter grouped-topk (skip per-layer append kernel)" (#35105)
|
2026-08-16 23:49:49 -07:00 |
|
|
Liangsheng Yin
|
5e73c89b34
|
[Spec] Simplify compute_spec_v2_logprobs signature and skip identity gathers (#35058)
|
2026-08-16 16:01:26 -07:00 |
|
|
Liangsheng Yin
|
77cadf6b98
|
[Spec] Point multi-layer eagle's last shared-read runner at the draft runner (#35057)
|
2026-08-16 15:28:12 -07:00 |
|
|
Liangsheng Yin
|
bae353ba55
|
[misc] Rename shared-read boundary to shared-read ends and fix wrapper delegation (#34982)
|
2026-08-16 14:36:31 -07:00 |
|
|
Liangsheng Yin
|
0f7aaceda5
|
[misc] Rename the WAR read-done fastpath to shared-read-done (#34916)
|
2026-08-15 15:02:02 -07:00 |
|
|
Liangsheng Yin
|
5c9ee86d90
|
[CI] Move the static ratchets back to CPU unit tests (#34913)
|
2026-08-14 23:32:00 -07:00 |
|
|
Liangsheng Yin
|
f2c84de022
|
[Perf] Publish the WAR read-done event at DSPARK verify (#34816)
|
2026-08-14 00:46:52 -07:00 |
|
|
Liangsheng Yin
|
151a314829
|
[Fix] Make the DSpark draft num_token_non_padded host-to-device copy non-blocking (#34782)
|
2026-08-13 16:10:54 -07:00 |
|
|
Liangsheng Yin
|
8554d9a5bc
|
[Fix] Carry the backend on Kimi-K3 deferred preprocessing configs (#34766)
|
2026-08-13 13:30:33 -07:00 |
|
|
Liangsheng Yin
|
8761b971f1
|
[CI] Fix nightly test failures (#34637)
|
2026-08-12 20:53:44 -07:00 |
|
|
Liangsheng Yin
|
26627e999d
|
[Fix] Snapshot req.prefix_indices when the prefix cache is disabled (#34644)
|
2026-08-12 20:50:46 -07:00 |
|
|
Liangsheng Yin
|
50cc1aa241
|
[CI] Route mmlu and GB300 MMMU-Pro evals through sgl-eval (#34477)
|
2026-08-12 19:33:21 -07:00 |
|
|
Liangsheng Yin
|
c05eb856f7
|
[CI] Fix nightly test failures (#34523)
|
2026-08-12 15:40:20 -07:00 |
|
|
Liangsheng Yin
|
9d2d737ebf
|
[Benchmark] Remove 22 unmaintained benchmarks (#34520)
|
2026-08-11 22:24:43 -07:00 |
|
|
Liangsheng Yin
|
256981ce16
|
[CI] Align rerun-test environment with the test stages (#34195)
|
2026-08-11 21:37:10 -07:00 |
|
|
+3
|
5899674504
|
[Fix] Make DeepSeek-V4 reasoning and tool-call streaming parsing chunk-invariant (#34458)
Co-authored-by: hao-cyber <89575785+hao-cyber@users.noreply.github.com>
Co-authored-by: Enrico Falco <enrico9034@gmail.com>
Co-authored-by: Svyatoslav <85786374+slivanovich@users.noreply.github.com>
Co-authored-by: Andreas Hassellof <andreas@ombori.com>
Co-authored-by: Leoyzen <leoyzen@gmail.com>
Co-authored-by: Chenglun Hu <chenglunhu@gmail.com>
Co-authored-by: robellliu-dev <robell.liu@huawei.com>
Co-authored-by: Gavin.Zhu <gavin.z@gmicloud.ai>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: tancheng33 <garrytancheng@gmail.com>
Co-authored-by: dineshx29 <dinesh.b.offl@gmail.com>
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
|
2026-08-11 20:03:28 -07:00 |
|
|
Liangsheng Yin
|
d8a61c26a6
|
[CI] Add a scheduled workflow to close stale PRs (#34380)
|
2026-08-11 00:54:02 -07:00 |
|
|
Liangsheng YinandAlex Nails
|
b3c02cbce7
|
[perf] Collapse the DP attention scheduler sync to a single D2H copy (#34338)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
|
2026-08-10 21:52:29 -07:00 |
|
|
Liangsheng Yin
|
585c3c6816
|
[Refactor] Split the FlashInfer autotune dummy-run flag from the LM-head policy (#34336)
|
2026-08-10 21:20:05 -07:00 |
|
|
Liangsheng Yin
|
c30872fa00
|
[Fix] Read trace level via get_global_trace_level() in trace_async (#34325)
|
2026-08-10 16:46:54 -07:00 |
|
|
Liangsheng Yin
|
c80a38edcd
|
[Fix] Pin cuda-tile to 1.6.0rc5 to unblock Python 3.10 x86_64 installs (#34321)
|
2026-08-10 15:09:47 -07:00 |
|
|
Liangsheng Yin
|
4eaaeda004
|
[CI] Build patched Docker images for both amd64 and arm64 (#34276)
|
2026-08-10 04:49:22 -07:00 |
|
|
Liangsheng Yin
|
0b6189d0e8
|
[CI] Add output_tag input to the Patch Docker Image workflow (#34253)
|
2026-08-10 02:48:03 -07:00 |
|
|
Liangsheng YinandBrayden Zhong
|
b51bf9ec9e
|
[Spec] Budget the DFLASH draft KV pool from its own attention geometry (#34234)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2026-08-10 01:28:49 -07:00 |
|
|
Liangsheng Yin
|
3bb72bc72a
|
[CI] Keep the torch compilation cache instead of wiping it on install (#34231)
|
2026-08-10 01:21:20 -07:00 |
|
|
Liangsheng YinandBrayden Zhong
|
aea78d1e73
|
[misc] Pass FP8 scales in FlashInfer SWA prefill, autotune fp8 on SM120, and tighten is_image_understandable_model (#34217)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2026-08-09 23:45:11 -07:00 |
|
|
Liangsheng YinandBrayden Zhong
|
d6a066131c
|
[Feature] Support NVFP4 token embedding in ModelOpt mixed-precision checkpoints (#34222)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
|
2026-08-09 23:37:54 -07:00 |
|
|
Liangsheng Yin
|
7c90840bad
|
[CI] Key scheduled CUDA suites by runner_config instead of hand-written jobs (#34186)
|
2026-08-09 16:44:53 -07:00 |
|
|
Liangsheng YinandDeleter-D
|
4a5d7d3c67
|
[DSV4] Fix silent KV corruption when speculative draft tokens > 4 (#34189)
Co-authored-by: Deleter-D <867909454@qq.com>
|
2026-08-09 15:54:41 -07:00 |
|
|
Liangsheng Yin
|
1ebd6fab6c
|
[CI] Gate Kimi-K3 acceptance length on the GSM8K average (#34145)
|
2026-08-08 20:28:42 -07:00 |
|
|
Liangsheng Yin
|
4792ab1e90
|
[CI] Pin the rust frontend parity test to eager prefill (#34146)
|
2026-08-08 20:07:19 -07:00 |
|
|
Liangsheng Yin
|
a1ca76b24b
|
[Scheduler] Unify WAR read-done gating behind shared-read boundary declarations (#34052)
|
2026-08-08 03:26:36 -07:00 |
|
|
Liangsheng Yin
|
2c0188cc78
|
[Fix] Give the piecewise CUDA graph test stub an hf_config (#34100)
|
2026-08-08 02:04:16 -07:00 |
|
|
Liangsheng YinandBaizhou Zhang
|
f6a6f5bf1e
|
[CI] Trim redundant nightly test registrations (#34070)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2026-08-08 01:42:46 -07:00 |
|
|
Liangsheng Yin
|
c59d2b4329
|
[Fix] Judge the phase-checker device-assert test by its FAIL line, not the exit code (#34017)
|
2026-08-07 16:19:05 -07:00 |
|
|
|
3c51e29deb
|
Responses support (#32689)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Harmya Bhatt <harmyacs@gmail.com>
Co-authored-by: harmya <harmya@modal.com>
Co-authored-by: Xinyuan <xinyuan@radixark.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
|
2026-08-07 13:21:46 -07:00 |
|
|
Liangsheng Yin
|
7395ee833e
|
[CI] Share VLM engines and prune launch matrices on the per-commit H100/H200 suites (#33944)
|
2026-08-07 01:22:22 -07:00 |
|
|
Liangsheng Yin
|
afa79330b8
|
[misc] Remove break-graph debug log; reclaim pid-less /dev/shm leaks in CI (#33929)
|
2026-08-06 21:18:01 -07:00 |
|
|
Liangsheng Yin
|
163b739b34
|
[CI] Refresh the CPU HF cache base only on main-ref runs (#33904)
|
2026-08-06 19:37:59 -07:00 |
|