 Liangsheng Yin
|
17afd8421f
|
[Spec] Share the grammar mask build and verify-tree staging across spec workers (#32393)
|
2026-07-25 03:40:21 -07:00 |
|
|
Liangsheng Yin
|
3c5bf1f6d2
|
[Spec] Derive NGRAM grammar tree links on the host instead of reading back retrive_next_token (#32380)
|
2026-07-25 02:25:16 -07:00 |
|
|
Liangsheng Yin
|
cff20a2fbb
|
[Spec] Consolidate the grammar sync decision into ScheduleBatch.grammar_needs_sync (#32353)
|
2026-07-24 20:42:38 -07:00 |
|
|
Liangsheng Yin
|
a31542ebd9
|
[Feature] Add leveled invariant-check primitive for nan/inf/oob validity checks (#32308)
|
2026-07-24 04:27:20 -07:00 |
|
|
Liangsheng Yin
|
f4f15162bc
|
[Fix] Fail fast when a safetensors index references missing shard files (#32279)
|
2026-07-24 02:18:56 -07:00 |
|
|
Liangsheng Yin
|
d059b0f56e
|
[Fix] Clamp degenerate all-sentinel draft rows to token 0 in dspark _online_combine_kernel (#32277)
|
2026-07-24 00:53:56 -07:00 |
|
|
Liangsheng Yin
|
99b29bf188
|
[Fix] Support ENCODER_ONLY target-verify in the trtllm_mha backend (#32178)
|
2026-07-23 20:33:39 -07:00 |
|
|
Liangsheng Yin
|
59ef3b15cc
|
[Fix] Reserve the mamba pool's +1 padding slot in the memory budget solve (#32184)
|
2026-07-23 14:55:35 -07:00 |
|
|
Liangsheng Yin
|
f35411ee81
|
[Spec] Enable grammar overlap scheduling for STANDALONE speculative decoding (#32110)
|
2026-07-23 01:47:27 -07:00 |
|
|
Liangsheng Yin
|
9b853e6832
|
[Scheduler] Enable decode retraction ordering under speculative decoding (#32023)
|
2026-07-23 00:42:56 -07:00 |
|
|
Liangsheng Yin
|
1b63155efe
|
[Perf] Skip blocks past per-request live length in full-width Triton kernels (#32109)
|
2026-07-22 22:00:25 -07:00 |
|
|
Liangsheng Yin
|
98cc8d91cf
|
[Fix] Evict only the KV shortfall in evict_from_tree_cache (#32016)
|
2026-07-22 12:41:06 -07:00 |
|
|
Liangsheng Yin
|
5c6a29b3c3
|
[Fix] Unify pinned host pool release on graceful shutdown (#32029)
|
2026-07-22 12:29:34 -07:00 |
|
|
Liangsheng Yin
|
024639a372
|
[Perf] Stack dspark dense draft per-layer ctx KV projection into one GEMM (#31986)
|
2026-07-21 17:46:13 -07:00 |
|
|
Liangsheng Yin
|
8ae0eb83fc
|
[Perf] Fold dspark dense draft embedding into the draft graph via forward_embed (#31985)
|
2026-07-21 17:00:42 -07:00 |
|
|
Liangsheng Yin
|
08cb081ed0
|
[Perf] Skip page-table columns past kv length in DSA draft-extend metadata kernel (#31981)
|
2026-07-21 16:20:14 -07:00 |
|
|
Liangsheng Yin
|
cb324cc958
|
Add wangfakang to CI_PERMISSIONS.json (#31980)
|
2026-07-21 14:03:00 -07:00 |
|
|
Liangsheng Yin
|
7fe9ad25ac
|
[Spec] Extract DFlash compact draft-cache rebuild helpers (#31677)
|
2026-07-20 15:07:55 -07:00 |
|
|
Liangsheng Yin
|
ff6c755952
|
[Refactor] Unify logprob results into a single LogprobResult and rename chunk env vars (#31733)
|
2026-07-20 12:44:27 -07:00 |
|
|
Liangsheng Yin
|
b15a83983c
|
[Fix] Release hierarchical cache host pool on graceful shutdown (#31746)
|
2026-07-19 22:27:29 -07:00 |
|
|
Liangsheng Yin
|
b3570a4531
|
[Refactor] Unify input logprob processing on a single chunked path (#31655)
|
2026-07-19 16:03:23 -07:00 |
|
|
Liangsheng Yin
|
cce5fe7696
|
[Scheduler] Move the WAR barrier to right after each run_batch launch (#31687)
|
2026-07-18 20:38:34 -07:00 |
|
|
Liangsheng Yin
|
10908a6793
|
[Fix] Respect cache_protected_len in ChunkCache and disabled-radix release paths (#31662)
|
2026-07-18 12:04:25 -07:00 |
|
|
Liangsheng Yin
|
639261f7b2
|
[Refactor] Move output logprob processing into the logprob_processor layer (#31624)
|
2026-07-17 22:33:52 -07:00 |
|
|
Liangsheng Yin
|
19c53c44a0
|
[Fix] Account zero-logprob sequences correctly in chunked logprob stitching (#31639)
|
2026-07-17 22:33:27 -07:00 |
|
|
Liangsheng Yin
|
2c856abbe3
|
[Fix] Enable chunked input-logprob processing by default to cap peak memory (#31498)
|
2026-07-17 13:19:03 -07:00 |
|
|
Liangsheng Yin
|
19f4859b30
|
[CI] Exclude current process memory from GPU idle check (#31571)
|
2026-07-17 02:11:41 -07:00 |
|
|
Liangsheng Yin
|
6e3be088a9
|
[Spec] Allocate the verify tree-mask scratch on the target backend only (#31527)
|
2026-07-16 23:23:21 -07:00 |
|
|
Liangsheng Yin
|
45824c69ca
|
fa3: build the topk>1 verify replay page table on-device (#31381)
|
2026-07-16 21:21:01 -07:00 |
|
|
Liangsheng Yin
|
27ad9d11b1
|
[CI] Wait for GPU memory release before each test class setUpClass (#31509)
|
2026-07-16 21:06:29 -07:00 |
|
|
Liangsheng Yin
|
dfa6278370
|
[Fix] Publish idle scheduler metrics immediately when the running-reqs gauge is stale (#31517)
|
2026-07-16 21:04:10 -07:00 |
|
|
Liangsheng Yin
|
9910ef8167
|
flashmla: sync-free spec via device-side draft-extend (#31090)
|
2026-07-16 19:38:12 -07:00 |
|
|
Liangsheng Yin
|
640101da13
|
[Fix] Deflake score engine determinism tests (#31494)
|
2026-07-16 15:42:47 -07:00 |
|
|
Liangsheng Yin
|
9a4d640244
|
[Perf] Cache uniform ragged-verify layout for DSpark verify-all compact (#31434)
|
2026-07-16 15:00:41 -07:00 |
|
|
Liangsheng Yin
|
fc1e3797b7
|
[Spec] Split the capture width from num_tokens_per_req and gate replay on it (#31255)
|
2026-07-16 14:59:11 -07:00 |
|
|
Liangsheng Yin
|
44f4ea917c
|
fa3: sync-free eagle spec via fixed-window draft-extend metadata (#31364)
|
2026-07-16 12:31:50 -07:00 |
|
|
Liangsheng Yin
|
b55228cfdb
|
[Spec] Consolidate the verify step into eagle_worker_common.run_eagle_verify (#31380)
|
2026-07-16 00:56:54 -07:00 |
|
|
Liangsheng Yin
|
0b372d03de
|
[CI] Guard partition consumers against failed check-changes and degenerate fits (#31416)
|
2026-07-15 22:09:40 -07:00 |
|
|
Liangsheng Yin
|
b871a509e6
|
[Fix] Wire the detokenizer soft watchdog into the multi-http-worker event loop (#31392)
|
2026-07-15 21:55:46 -07:00 |
|
|
Liangsheng Yin
|
1cc9493747
|
[Spec] Converge DP-attention spec width scaling onto num_tokens_per_req (#31244)
|
2026-07-15 17:57:08 -07:00 |
|
|
Liangsheng Yin
|
b0b2dfbda1
|
[Spec] Extract the shared draft() tail into build_eagle_verify_input (#31375)
|
2026-07-15 15:59:31 -07:00 |
|
|
Liangsheng Yin
|
e76cc75cfa
|
[CI] Remove nightly registrations redundant with scheduled stage runs (#31371)
|
2026-07-15 14:12:07 -07:00 |
|
|
Liangsheng Yin
|
dc078ddd2a
|
[Spec] Extract stateless draft prepare helpers into eagle_worker_common (#31257)
|
2026-07-15 10:54:31 -07:00 |
|
|
Liangsheng Yin
|
76dc427806
|
[Spec] Single-source num_tokens_per_req derivation and access (#31013)
|
2026-07-14 18:41:08 -07:00 |
|
|
Liangsheng Yin
|
ca0ee3f1a8
|
[Spec] Consolidate spec-worker weight updates into BaseSpecWorker via draft_runners (#31078)
|
2026-07-14 18:39:46 -07:00 |
|
|
Liangsheng Yin
|
50d1edaa7f
|
[misc] Move SchedulerRecvSkipper into scheduler_components (#31222)
|
2026-07-14 16:35:04 -07:00 |
|
|
Liangsheng Yin
|
e2728ac504
|
[Spec] Remove dead padded_static_len and stale SGLANG_ENABLE_SPEC_V2 references (#30998)
|
2026-07-13 15:30:31 -05:00 |
|
|
Liangsheng Yinandronhuafeng
|
2ab531cfcf
|
fa3/fa4: sync-free for all backends and phases (#29589)
Co-authored-by: ronhuafeng <ronhuafeng@users.noreply.github.com>
|
2026-07-13 15:09:57 -05:00 |
|
|
Liangsheng Yin
|
48fff1f2bd
|
[Spec] Deduplicate spec-v2 worker lifecycle boilerplate into BaseSpecWorker (#31008)
|
2026-07-13 13:48:40 -05:00 |
|
|
Liangsheng Yin
|
c0f1f7e062
|
[Spec] Rename num_tokens_per_bs to num_tokens_per_req (#30977)
|
2026-07-13 13:47:53 -05:00 |
|
|
Liangsheng Yin
|
7a82178277
|
[Fix] Disable FlashInfer allreduce fusion in Nemotron-3-Nano lm-eval test (#30945)
|
2026-07-12 15:39:59 -05:00 |
|
|
Liangsheng Yin
|
5ba3c5147e
|
[Spec] Add kill-switch env for draft-extend CUDA graph capture (#30944)
|
2026-07-12 14:54:36 -05:00 |
|
|
Liangsheng Yin
|
4884f6fbee
|
[Fix] Unify ForwardBatch extend lens cpu fields to their declared list type (#30896)
|
2026-07-11 17:30:53 -05:00 |
|
|
Liangsheng Yin
|
d8ef76682e
|
[Spec] Extract shared draft worker construction and generalize draft sampler capture (#30857)
|
2026-07-11 14:34:09 -05:00 |
|
|
Liangsheng Yin
|
348e6fd29b
|
[Fix] Guard kernel OOB accesses and harden runtime edge cases (#30847)
|
2026-07-11 14:22:13 -05:00 |
|
|
Liangsheng Yin
|
7bac9c8cdb
|
[Spec] Enable draft extend cuda graph for DeepSeek-V4 attention backend (#30853)
|
2026-07-11 04:29:49 -05:00 |
|
|
Liangsheng Yin
|
fef5eda4fb
|
[Refactor] Split DeepSeek-V4 MQALayer into a reusable attention base (#30711)
|
2026-07-10 01:14:03 -07:00 |
|
|
Liangsheng Yin
|
23390589f7
|
[misc] Remove unit test cases that fail the admission criteria (round 3) (#30713)
|
2026-07-09 19:42:51 -07:00 |
|
|
Liangsheng Yin
|
7b9b2e4798
|
[style] Extract init-static values in memory-cache path (#30710)
|
2026-07-09 19:38:15 -07:00 |
|
|
Liangsheng Yin
|
ccd2028def
|
[style] Extract init-static values in tokenizer + multimodal path (#30709)
|
2026-07-09 19:36:34 -07:00 |
|
|
Liangsheng Yin
|
dda61b476e
|
[style] Extract init-static values in forward path (#30708)
|
2026-07-09 19:35:00 -07:00 |
|
|
Liangsheng Yin
|
b5e75b9423
|
[style] Extract init-static values in scheduler hot path (#30707)
|
2026-07-09 19:33:02 -07:00 |
|
|
Liangsheng Yin
|
504570f425
|
[Refactor] Make DeepSeek-V4 attention backend tolerate an absent CPU seq_lens mirror (#30695)
|
2026-07-09 16:39:47 -07:00 |
|
|
Liangsheng Yin
|
a36c873147
|
[misc] Remove unit test cases that fail the admission criteria (round 2) (#30703)
|
2026-07-09 16:35:59 -07:00 |
|
|
Liangsheng Yin
|
c53559ba10
|
[misc] Remove unit test cases that fail the admission criteria (#30690)
|
2026-07-09 15:31:28 -07:00 |
|
|
Liangsheng Yin
|
c46b291c77
|
[misc] Add init-static value extraction to the general code style rule (#30701)
|
2026-07-09 15:25:53 -07:00 |
|
|
Liangsheng Yin
|
078f06fbf4
|
[Refactor] Share chat encoding dispatch between serving and offline tools (#30623)
|
2026-07-09 02:45:42 -07:00 |
|
|
Liangsheng Yin
|
bc5d376c2c
|
[Bench] Add fixed-prompt mode and per-request spec accept length metrics (#30615)
|
2026-07-09 02:06:04 -07:00 |
|
|
Liangsheng Yin
|
61602b95fb
|
[Fix] Prevent silent VLM server crash when /dev/shm is exhausted during multimodal feature transport (#30602)
|
2026-07-09 00:44:42 -07:00 |
|
|
Liangsheng Yin
|
64e2a73c80
|
[Fix] Serialize FanOutCommunicator queueing calls with a FIFO-fair asyncio.Lock (#30606)
|
2026-07-09 00:04:43 -07:00 |
|
|
Liangsheng Yin
|
9e4483d725
|
[misc] Add unit test admission criteria to agent rules (#30608)
|
2026-07-08 23:41:48 -07:00 |
|
|
Liangsheng Yin
|
8f9307736a
|
[misc] Add CI-only guards for the FutureMap seq_lens relay (#30471)
|
2026-07-08 15:29:53 -07:00 |
|
|
Liangsheng Yin
|
10e7f2925f
|
[Fix] Chain the seq_lens publish event records so prebuilt seeding keeps the forward fence (#30435)
|
2026-07-08 13:43:09 -07:00 |
|
|
Liangsheng Yin
|
c0d45cda29
|
[Spec] Add DFLASH basic sanity CI test (#29338)
|
2026-06-28 23:45:05 -07:00 |
|
|
Liangsheng Yin
|
be17475a1b
|
[Spec] Publish DFLASH verify read-done event for fine-grained WAR barrier (#29541)
|
2026-06-28 18:51:38 -07:00 |
|
|
Liangsheng Yin
|
909123ddb8
|
[misc] Use --cuda-graph-max-bs-decode in tests, examples, and docs (#29591)
|
2026-06-28 18:38:28 -07:00 |
|
|
Liangsheng Yin
|
abaee4664e
|
[dflash] fa3/fa4: device-side page table; drop seq_lens_cpu D2H sync (#29343)
|
2026-06-28 18:06:33 -07:00 |
|
|
Liangsheng Yin
|
6582fc1e5d
|
dflash: drop verify_done barrier; rely on scheduler WAR fallback (#29556)
|
2026-06-28 01:47:17 -07:00 |
|
|
Liangsheng Yin
|
593f5ba68f
|
[Spec] Replace shared-infra dflash special-cases with capabilities (WAR barrier + seq_lens_cpu) (#29232)
|
2026-06-28 00:02:32 -07:00 |
|
|
Liangsheng Yin
|
4e6ce8e4ab
|
[Spec] Capture DFLASH draft greedy sampling inside the draft decode cuda graph (#29395)
|
2026-06-27 19:34:26 -07:00 |
|
|
Liangsheng Yin
|
cd6dedf972
|
[router] Count requests/responses at the HTTP edge for true intake (#29467)
|
2026-06-26 21:21:41 -07:00 |
|
|
Liangsheng Yin
|
b6ebdcc92e
|
[test] Split perf table out of fwd occupancy kit report (#29348)
|
2026-06-25 20:23:42 -07:00 |
|
|
Liangsheng Yin
|
e4696ed62d
|
[test] Report token tps in fwd occupancy kit and force ignore_eos (#29332)
|
2026-06-25 15:08:33 -07:00 |
|
|
Liangsheng Yin
|
2812a3c93a
|
[Spec] Unify spec/non-spec decode result handling and overlap relay-payload gating (#29225)
|
2026-06-25 00:44:38 -07:00 |
|
|
Liangsheng Yin
|
8314247d9d
|
[Spec] Merge dflash triton kernels into a single dflash.py (#29228)
|
2026-06-24 18:57:15 -07:00 |
|
|
Liangsheng Yin
|
c7734e6871
|
[Spec] Dissolve EagleDraftInputV2Mixin so spec-info dataclasses hold data only (#29220)
|
2026-06-24 18:08:30 -07:00 |
|
|
Liangsheng Yin
|
85a3f1f75c
|
[Spec] Unify the overlap stash relay behind a RelayPayload dataclass (#29124)
|
2026-06-24 15:36:54 -07:00 |
|
|
Liangsheng Yin
|
c6822f81b1
|
[Spec] Make the overlap bonus-token relay unconditional (#29122)
|
2026-06-24 15:34:02 -07:00 |
|
|
Liangsheng Yin
|
d3dd184e3e
|
[Spec] Fold DFlash verified_id into the shared bonus_tokens relay channel (#29118)
|
2026-06-24 15:30:29 -07:00 |
|
|
Liangsheng Yin
|
8e1988b746
|
[Perf] Overlap result D2H copy with the next forward step (#29075)
|
2026-06-24 01:34:02 -07:00 |
|
|
Liangsheng Yin
|
e3f1fa9d8e
|
[misc] Unify benchmark deprecation shims and one_batch_server CLI entrypoint (#29108)
|
2026-06-23 21:10:25 -07:00 |
|
|
Liangsheng Yin
|
f76c6c95be
|
[misc] Add sglang.bench_offline_throughput deprecation shim (#29085)
|
2026-06-23 18:39:58 -07:00 |
|
|
Liangsheng Yin
|
b448b08401
|
[misc] Move bench_offline_throughput into sglang/benchmark/ with a back-compat shim (#28747)
|
2026-06-23 18:37:52 -07:00 |
|
|
Liangsheng Yin
|
9ef1830701
|
[Scheduler] Extract DFlash prefill refill into a standalone MinFreeSlotsDelayer (#29089)
|
2026-06-23 17:04:55 -07:00 |
|
|
Liangsheng Yin
|
11e7c9e0e6
|
[misc] Add sglang.bench_one_batch deprecation shim (#29082)
|
2026-06-23 14:57:35 -07:00 |
|
|
Liangsheng Yin
|
c864c8d9c2
|
[misc] Move bench_one_batch into sglang/benchmark/ with a back-compat shim (#28687)
|
2026-06-23 14:48:35 -07:00 |
|
|
Liangsheng Yin
|
6c5f466023
|
[server_args] Reland FA4 page_size auto-force for combined --attention-backend fa4 (#28976)
|
2026-06-23 14:21:59 -07:00 |
|
|
Liangsheng Yin
|
ed0a62e4dd
|
[Mem] Add KV-page double-free checks to the invariant checker (#27731)
|
2026-06-23 02:29:40 -07:00 |
|
|
Liangsheng Yin
|
854c688121
|
[Spec] Unify decode KV-commit bookkeeping across spec-v2 workers (#28754)
|
2026-06-23 00:53:05 -07:00 |
|
|
Liangsheng Yin
|
4740f23e1f
|
Revert "[server_args] compute mem_fraction_static after dp chunked-prefill division" (#28991)
|
2026-06-22 20:48:16 -07:00 |
|