 Liangsheng Yin
|
c0d45cda29
|
[Spec] Add DFLASH basic sanity CI test (#29338)
|
2026-06-28 23:45:05 -07:00 |
|
|
Liangsheng Yin
|
be17475a1b
|
[Spec] Publish DFLASH verify read-done event for fine-grained WAR barrier (#29541)
|
2026-06-28 18:51:38 -07:00 |
|
|
Liangsheng Yin
|
909123ddb8
|
[misc] Use --cuda-graph-max-bs-decode in tests, examples, and docs (#29591)
|
2026-06-28 18:38:28 -07:00 |
|
|
Liangsheng Yin
|
abaee4664e
|
[dflash] fa3/fa4: device-side page table; drop seq_lens_cpu D2H sync (#29343)
|
2026-06-28 18:06:33 -07:00 |
|
|
Liangsheng Yin
|
6582fc1e5d
|
dflash: drop verify_done barrier; rely on scheduler WAR fallback (#29556)
|
2026-06-28 01:47:17 -07:00 |
|
|
Liangsheng Yin
|
593f5ba68f
|
[Spec] Replace shared-infra dflash special-cases with capabilities (WAR barrier + seq_lens_cpu) (#29232)
|
2026-06-28 00:02:32 -07:00 |
|
|
Liangsheng Yin
|
4e6ce8e4ab
|
[Spec] Capture DFLASH draft greedy sampling inside the draft decode cuda graph (#29395)
|
2026-06-27 19:34:26 -07:00 |
|
|
Liangsheng Yin
|
cd6dedf972
|
[router] Count requests/responses at the HTTP edge for true intake (#29467)
|
2026-06-26 21:21:41 -07:00 |
|
|
Liangsheng Yin
|
b6ebdcc92e
|
[test] Split perf table out of fwd occupancy kit report (#29348)
|
2026-06-25 20:23:42 -07:00 |
|
|
Liangsheng Yin
|
e4696ed62d
|
[test] Report token tps in fwd occupancy kit and force ignore_eos (#29332)
|
2026-06-25 15:08:33 -07:00 |
|
|
Liangsheng Yin
|
2812a3c93a
|
[Spec] Unify spec/non-spec decode result handling and overlap relay-payload gating (#29225)
|
2026-06-25 00:44:38 -07:00 |
|
|
Liangsheng Yin
|
8314247d9d
|
[Spec] Merge dflash triton kernels into a single dflash.py (#29228)
|
2026-06-24 18:57:15 -07:00 |
|
|
Liangsheng Yin
|
c7734e6871
|
[Spec] Dissolve EagleDraftInputV2Mixin so spec-info dataclasses hold data only (#29220)
|
2026-06-24 18:08:30 -07:00 |
|
|
Liangsheng Yin
|
85a3f1f75c
|
[Spec] Unify the overlap stash relay behind a RelayPayload dataclass (#29124)
|
2026-06-24 15:36:54 -07:00 |
|
|
Liangsheng Yin
|
c6822f81b1
|
[Spec] Make the overlap bonus-token relay unconditional (#29122)
|
2026-06-24 15:34:02 -07:00 |
|
|
Liangsheng Yin
|
d3dd184e3e
|
[Spec] Fold DFlash verified_id into the shared bonus_tokens relay channel (#29118)
|
2026-06-24 15:30:29 -07:00 |
|
|
Liangsheng Yin
|
8e1988b746
|
[Perf] Overlap result D2H copy with the next forward step (#29075)
|
2026-06-24 01:34:02 -07:00 |
|
|
Liangsheng Yin
|
e3f1fa9d8e
|
[misc] Unify benchmark deprecation shims and one_batch_server CLI entrypoint (#29108)
|
2026-06-23 21:10:25 -07:00 |
|
|
Liangsheng Yin
|
f76c6c95be
|
[misc] Add sglang.bench_offline_throughput deprecation shim (#29085)
|
2026-06-23 18:39:58 -07:00 |
|
|
Liangsheng Yin
|
b448b08401
|
[misc] Move bench_offline_throughput into sglang/benchmark/ with a back-compat shim (#28747)
|
2026-06-23 18:37:52 -07:00 |
|
|
Liangsheng Yin
|
9ef1830701
|
[Scheduler] Extract DFlash prefill refill into a standalone MinFreeSlotsDelayer (#29089)
|
2026-06-23 17:04:55 -07:00 |
|
|
Liangsheng Yin
|
11e7c9e0e6
|
[misc] Add sglang.bench_one_batch deprecation shim (#29082)
|
2026-06-23 14:57:35 -07:00 |
|
|
Liangsheng Yin
|
c864c8d9c2
|
[misc] Move bench_one_batch into sglang/benchmark/ with a back-compat shim (#28687)
|
2026-06-23 14:48:35 -07:00 |
|
|
Liangsheng Yin
|
6c5f466023
|
[server_args] Reland FA4 page_size auto-force for combined --attention-backend fa4 (#28976)
|
2026-06-23 14:21:59 -07:00 |
|
|
Liangsheng Yin
|
ed0a62e4dd
|
[Mem] Add KV-page double-free checks to the invariant checker (#27731)
|
2026-06-23 02:29:40 -07:00 |
|
|
Liangsheng Yin
|
854c688121
|
[Spec] Unify decode KV-commit bookkeeping across spec-v2 workers (#28754)
|
2026-06-23 00:53:05 -07:00 |
|
|
Liangsheng Yin
|
4740f23e1f
|
Revert "[server_args] compute mem_fraction_static after dp chunked-prefill division" (#28991)
|
2026-06-22 20:48:16 -07:00 |
|
|
Liangsheng Yin
|
fc0d8fa262
|
[Lint] fix main's lint (#28990)
|
2026-06-22 20:40:46 -07:00 |
|
|
Liangsheng Yin
|
d70726a31a
|
Revert "[server_args] fix FA4 page_size auto-force for combined --attention-backend fa4" (#28972)
|
2026-06-22 17:01:37 -07:00 |
|
|
Liangsheng Yin
|
a32f7111e4
|
[CI] Disable async-assert for base-a tests in rerun-test (#28965)
|
2026-06-22 15:29:45 -07:00 |
|
|
Liangsheng Yin
|
770d6b2825
|
[Spec] Add sync-free fast_prefill_plan for EAGLE draft-extend CUDA graph (#28854)
|
2026-06-22 15:15:15 -07:00 |
|
|
Liangsheng Yinandthanhhao98
|
106d2930a6
|
[core] Gate the overlap WAR barrier on forward reads to recover decode throughput (#28363)
Co-authored-by: thanhhao98 <31717833+thanhhao98@users.noreply.github.com>
|
2026-06-22 00:40:56 -07:00 |
|
|
Liangsheng Yin
|
64e455d4bf
|
Fix lint break on main (#28886)
|
2026-06-21 22:46:08 -07:00 |
|
|
Liangsheng Yin
|
4f5ff39bc9
|
[Spec] Enable FR-Spec in EAGLE draft-extend CUDA graph by sizing logits buffer from the draft head (#28856)
|
2026-06-21 15:35:57 -07:00 |
|
|
Liangsheng Yin
|
e6722c751b
|
[Feature] Add graceful scheduler shutdown; free hisparse host buffer on exit (#28779)
|
2026-06-21 15:08:10 -07:00 |
|
|
Liangsheng Yin
|
8e890391f5
|
[Spec] Support FlashInfer CUDA graph for EAGLE draft-extend (#28782)
|
2026-06-21 14:46:25 -07:00 |
|
|
Liangsheng Yin
|
d271de64fe
|
[misc] Move bench_one_batch_server into sglang/benchmark/ with a back-compat shim (#28625)
|
2026-06-19 14:19:29 -07:00 |
|
|
Liangsheng Yin
|
c9c2445146
|
[CI] Bump actions/github-script to v8 (Node 24 runtime) (#28741)
|
2026-06-19 14:18:37 -07:00 |
|
|
Liangsheng Yin
|
9bb9d17e1a
|
[Spec] Unify speculative grammar token-accept path in decode processing (#28682)
|
2026-06-19 02:49:40 -07:00 |
|
|
Liangsheng Yin
|
0146692cc9
|
[CI] Fail fast on empty install_script in Rerun Test workflow (#28721)
|
2026-06-19 02:07:52 -07:00 |
|
|
Liangsheng Yin
|
8f6d9ef9a5
|
[misc] Drop redundant req_pool_indices_cpu guards; fold hisparse into GLM-5.1 e2e (#28607)
|
2026-06-18 12:51:23 -07:00 |
|
|
Liangsheng Yin
|
cf93712937
|
[misc] Share bench HTTP-client base-URL resolution with IPv6-compatible formatting (#28598)
|
2026-06-17 20:50:49 -07:00 |
|
|
Liangsheng Yin
|
d2b5488392
|
[misc] Centralize bench launch-vs-connect into a reusable acquire_endpoint (#28592)
|
2026-06-17 19:40:43 -07:00 |
|
|
Liangsheng Yin
|
9888b7b42b
|
[misc] Trim dead code in trtllm_mha page-table backend; reuse eager page-table buffer (#28578)
|
2026-06-17 18:36:22 -07:00 |
|
|
Liangsheng Yin
|
a663500ea9
|
[Test] Fold EAGLE return_hidden_states regression into spec triton suite (#28577)
|
2026-06-17 16:42:02 -07:00 |
|
|
Liangsheng Yin
|
b88bada64e
|
[misc] Unify bench seed default to 42 and rename --profile-filename-prefix to --profile-prefix (#28576)
|
2026-06-17 16:41:23 -07:00 |
|
|
Liangsheng Yin
|
e053890b6f
|
[Fix] Reuse an already-running server in bench_one_batch_server instead of forking an orphan (#28563)
|
2026-06-17 15:14:44 -07:00 |
|
|
Liangsheng Yin
|
f86e9b48e8
|
[Perf] Make spec-decode penalty H2D non-blocking and share decode cumulate path (#28500)
|
2026-06-17 00:59:58 -07:00 |
|
|
Liangsheng Yin
|
3bc618485a
|
[Perf] Make latest_output_ids H2D non-blocking in prepare_for_decode (#28491)
|
2026-06-16 23:46:35 -07:00 |
|
|
Liangsheng Yin
|
556cf54d47
|
[Perf] Avoid per-decode-step host sync in min_new_tokens penalty (#28397)
|
2026-06-15 23:32:28 -07:00 |
|
|
Liangsheng Yin
|
14f6348524
|
[Fix] Demote OpenAIServingResponses init failure log to one-line WARNING (#28349)
|
2026-06-15 16:58:33 -07:00 |
|
|
Liangsheng Yin
|
bb48405c31
|
Unify NVTX annotation helpers and split the enable gate per subsystem (#28165)
|
2026-06-14 00:04:59 -07:00 |
|
|
Liangsheng Yin
|
32ef040618
|
[Spec] Move eagle verify prepare_for_verify/sample to eagle_utils free helpers (#28117)
|
2026-06-12 20:37:33 -07:00 |
|
|
Liangsheng Yin
|
5633ca8599
|
[Spec] Move prepare_for_draft to EagleDraftWorkerBase (#28105)
|
2026-06-12 17:42:43 -07:00 |
|
|
Liangsheng Yin
|
5f315c74da
|
[CI] Enforce modern stage=/runner_config= form for dispatchable test suites (#28108)
|
2026-06-12 17:32:54 -07:00 |
|
|
Liangsheng Yin
|
87554c7855
|
[Spec] Move draft-extend prep to EagleDraftWorkerBase; unify prepare_for_* names (#28093)
|
2026-06-12 16:44:04 -07:00 |
|
|
Liangsheng Yin
|
fd977adbd6
|
Fix PR-close cancellation skipping workflows beyond the first 30 (#27785)
|
2026-06-12 15:39:33 -07:00 |
|
|
Liangsheng Yin
|
caf59759ea
|
[Spec] Centralize dummy verify-input capture; add carries_draft_hidden_states (#28032)
|
2026-06-12 14:23:29 -07:00 |
|
|
Liangsheng Yin
|
a52ccd2179
|
[Spec] Install EagleDraftExtendInput as the V2 draft-extend spec_info (#24860)
|
2026-06-12 00:46:16 -07:00 |
|
|
Liangsheng Yin
|
e1164a6dfc
|
[Spec] Remove dead prepare_for_verify / prepare_extend_after_decode + extend-decode kernel (#27761)
|
2026-06-11 20:59:30 -07:00 |
|
|
|
7074704c0c
|
[Spec] Dedup post-verify mamba state commit into shared spec_utils helpers (#27966)
Co-authored-by: xbfs <xuebf1@lenovo.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-06-11 18:33:40 -07:00 |
|
|
Liangsheng Yin
|
3ffe72517f
|
[Spec] Remove the dead spec V1 scheduler paths (#27977)
|
2026-06-11 18:31:13 -07:00 |
|
|
Liangsheng Yin
|
c0480a88be
|
[Spec] Retire Spec V1 (#27964)
|
2026-06-11 16:15:15 -07:00 |
|
|
Liangsheng Yin
|
fee717f303
|
[Spec] Fold the DFLASH worker base into DFlashWorkerV2 on BaseSpecWorker (#27950)
|
2026-06-11 14:54:28 -07:00 |
|
|
Liangsheng Yin
|
acdb39edd2
|
[Spec] Remove the DFLASH V1 worker path (#27959)
|
2026-06-11 14:37:58 -07:00 |
|
|
Liangsheng Yin
|
df5055e00f
|
Bump spec logprob match delta for the bf16 eagle fixture (#27952)
|
2026-06-11 14:21:07 -07:00 |
|
|
Liangsheng Yin
|
1ac75c3463
|
[CI] Fix GB300 TestDummyWithSBO crash: disable NaN assert and coredump for dummy weights (#27955)
|
2026-06-11 14:17:03 -07:00 |
|
|
Liangsheng Yin
|
493bb6ae0d
|
Fix fp16 NaN flake in spec CI: bf16 eagle fixture; sanitize NaN logits in sampler (#27883)
|
2026-06-11 01:16:12 -07:00 |
|
|
Liangsheng Yin
|
16124fc9b2
|
[Metrics] Fix fwd_occupancy reading NaN on every decode log line; probe-free base-a (#27836)
|
2026-06-10 15:42:19 -07:00 |
|
|
Liangsheng Yin
|
5fefe91289
|
[Spec] NGRAMWorker on BaseSpecWorker; algo-owned verify-tree shape params (#27799)
|
2026-06-10 12:38:16 -07:00 |
|
|
Liangsheng Yin
|
d1895cb60d
|
[Spec] Extract move_accept_tokens_to_target_kvcache into spec_utils (#27764)
|
2026-06-09 21:55:26 -07:00 |
|
|
Liangsheng Yin
|
f332e52611
|
Add UT guarding per-request bookkeeping clock ownership (#27710)
|
2026-06-09 17:11:49 -07:00 |
|
|
|
decb88e0e3
|
Support spec v2 for Frozen-KV MTP; remove v1 worker (#27607)
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-09 15:30:20 -07:00 |
|
|
Liangsheng Yin
|
186f1e300a
|
[CI] Move JIT kernel tests + benchmarks to test/registered/jit; add in-package guard (#27644)
|
2026-06-09 12:37:39 -07:00 |
|
|
Liangsheng Yin
|
cae022aa88
|
[JIT] Reuse JIT kernel build cache across CI runs (#27605)
|
2026-06-08 22:14:48 -07:00 |
|
|
Liangsheng Yin
|
3fe6bc390b
|
[Spec] Naming cleanup: contiguous draft-loc kernel + accepted->accept (#27599)
|
2026-06-08 15:04:58 -07:00 |
|
|
Liangsheng Yin
|
b5c64b94d5
|
[Spec] Rename token resolver to _resolve_spec_v2_tokens; remove dead V1 helpers (#27552)
|
2026-06-08 14:42:21 -07:00 |
|
|
Liangsheng Yin
|
28c1a3cb45
|
[Spec] Deprecate Spec V1 (#25464)
|
2026-06-08 13:10:27 -07:00 |
|
|
Liangsheng Yin
|
57ea09badb
|
Fix NaN in triton EAGLE spec-v2 draft-extend CUDA graph at topk>1 (wrong qo_indptr stride) (#27545)
|
2026-06-08 02:28:48 -07:00 |
|
|
Liangsheng Yin
|
6365d6faee
|
[spec] Misc defensive guards for EAGLE draft KV indexing (#27486)
|
2026-06-07 21:27:46 -07:00 |
|
|
Liangsheng Yin
|
303757ccd8
|
[Attn] Fix aiter MLA verify kv_indices under-alloc + shared assert_buffer_fits guard (#27485)
|
2026-06-07 17:06:25 -07:00 |
|
|
Liangsheng Yin
|
f68c79675f
|
Support topk > 1 tree drafting for mamba/hybrid-linear models on spec v2 (#27463)
|
2026-06-07 17:04:09 -07:00 |
|
|
Liangsheng Yin
|
70db73afce
|
[Spec] Clamp multimodal pad sentinels in spec-v2 draft prefill embedding (#27512)
|
2026-06-07 17:00:49 -07:00 |
|
|
Liangsheng Yin
|
eab2e02fa0
|
[spec] Dedup draft kv_indices sizing into spec_utils helpers (#27475)
|
2026-06-07 00:42:29 -07:00 |
|
|
Liangsheng Yin
|
5e2e0d5b49
|
[spec] Make spec_utils module-importable: type-only imports under TYPE_CHECKING (#27484)
|
2026-06-07 00:01:32 -07:00 |
|
|
Liangsheng Yin
|
0ce3db3c0a
|
[Bug] Fix out-of-range token id crashing tp=1 VocabParallelEmbedding (#27482)
|
2026-06-06 23:00:17 -07:00 |
|
|
Liangsheng Yin
|
80eee2d92c
|
[Spec] Guard async-assert probes against None tensor (#27478)
|
2026-06-06 22:14:00 -07:00 |
|
|
Liangsheng Yin
|
066b4a2180
|
[core] Probe set_kv_buffer / set_mla_kv_buffer slot ids for OOB (#27459)
|
2026-06-06 19:51:09 -07:00 |
|
|
Liangsheng Yin
|
5160f7914e
|
Fix MLA EAGLE draft CUDA-graph kv_indices under-allocation for topk > 1 (#27460)
|
2026-06-06 16:28:34 -07:00 |
|
|
Liangsheng Yin
|
032c9efb46
|
Enable async-assert invariant probes by default in CI (#27461)
|
2026-06-06 16:23:21 -07:00 |
|
|
Liangsheng Yin
|
1c7acba579
|
[spec] Consolidate the per-decode KV alloc reserve into one helper (#27458)
|
2026-06-06 14:58:50 -07:00 |
|
|
Liangsheng Yin
|
4989d6691d
|
ci: show partition fit window as a date range in the step summary (#27457)
|
2026-06-06 14:39:36 -07:00 |
|
|
Liangsheng Yin
|
84ca0ffb8c
|
Spec v2 tree drafting (topk>1) with page_size>1 (#26972)
|
2026-06-06 12:00:27 -07:00 |
|
|
Liangsheng Yin
|
aa5213abb1
|
[debug] Register #27338 EAGLE draft kv_indices revert in pr_fix_toggle (#27428)
|
2026-06-06 00:30:13 -07:00 |
|
|
Liangsheng Yin
|
26b9053dcc
|
[Spec] Fix fa3 EAGLE draft-decode expand page_table scatter OOB for topk>1 + page_size>1 (#27360)
|
2026-06-06 00:24:34 -07:00 |
|
|
Liangsheng Yin
|
58a05d3dd2
|
[CI] Isolate CUDA coredump dir per run to fix tracker mis-attribution (#27344)
|
2026-06-05 21:31:22 -04:00 |
|
|
Liangsheng Yin
|
2c2a4f243a
|
[Bug] Fix EAGLE draft CUDA-graph kv_indices under-allocation for topk > 1 (#27338)
|
2026-06-05 16:22:04 -04:00 |
|
|
Liangsheng Yin
|
cd98d97037
|
Use level-1 (quiet) busy memory check in chunked-prefill and streaming tests (#27303)
|
2026-06-04 17:04:27 -04:00 |
|
|
Liangsheng Yin
|
687cfe9198
|
Enable runtime busy memory check for speculation topk>1 (#27228)
|
2026-06-04 16:46:20 -04:00 |
|
|
Liangsheng Yin
|
ef170c27b6
|
Add quiet mode for busy mem check (level 1: buffer + dump on leak) (#27238)
|
2026-06-04 04:31:57 -04:00 |
|