minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
16,125 Commits 3 Branches 0 Tags
ba12a16a62f1702fbb316bc1969113dfddbe8ef4
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Liangsheng Yin 3869fe556f [CI] Collapse the EAGLE launch matrix and the scoring engine boots on the per-commit runners (#33756) 2026-08-05 16:43:19 -07:00
Liangsheng Yin c0ef548eef [misc] Unify MLA scaling init and remove dead buffer / scaling code (#33363) 2026-08-05 15:59:33 -07:00
Liangsheng Yin 2d27133fcf [CI] Skip apt-get when the required packages are already installed (#33757) 2026-08-05 15:06:01 -07:00
Liangsheng Yin de34dd11e9 [CI] Fold duplicate-server suites and prune the retract matrix on 1-gpu-5090 (#33745) 2026-08-05 12:41:51 -07:00
Liangsheng Yin 1a045669e4 [CI] Merge tokenizer worker tests and drop redundant triton attention e2e (#33641) 2026-08-05 11:55:11 -07:00
Liangsheng Yin 4a3d6ca88c [CI] Skip sglang-kernel and sgl-deep-gemm reinstall on version match (#33637) 2026-08-05 01:46:16 -07:00
Liangsheng Yin c0d5ebd6c4 [CI] Move CPU-only unit tests to the CPU suite and trim dead 5090 registrations (#33654) 2026-08-05 01:43:29 -07:00
Liangsheng Yin eac1f78568 [CI] Free hosted-runner disk space only when it is low (#33644) 2026-08-04 21:55:06 -07:00
Liangsheng Yin 198a3bc29b [Test] Route GEMM backend UTs through real layer modules and weight loaders (#33615) 2026-08-04 20:53:26 -07:00
Liangsheng Yin 1033cae8d5 [CI] Speed up dependency install: dual-ABI Rust ext cache and prevalidation pruning (#33619) 2026-08-04 20:33:48 -07:00
Liangsheng Yin 76dc89f5aa [Test] Replace NVFP4 MoE runner backend e2e matrix with a layer-level unit test (#33611) 2026-08-04 16:03:29 -07:00
Liangsheng Yin 0d99d91e49 [CI] Make B200 base-b suites single-GPU as prep for 1-gpu B200 runners (#33605) 2026-08-04 15:53:01 -07:00
Liangsheng Yin a0b3f1dde6 [Test] Replace GEMM backend e2e matrices with layer-level unit tests (#33596) 2026-08-04 15:50:41 -07:00
Liangsheng Yin 58da9859c4 [CI] Extract download-rust-ext and give every install step a cache fallback (#33597) 2026-08-04 15:33:34 -07:00
+26
Liangsheng YinDarkSharpnessXiaoyu ZhangMickYuhao YangCheng WanKe BaoBaizhou ZhangChunan ZengKhoa PhamZiyi XuZijie XiaYuwei AnzhangxiaohaoYangmin LiJulien LinHao PhanThomas WangRolaoDenthupigeonsoupHaiShawXinyuan TongPranjal ShankhdharLee NauHMINGelvischenvByron HsuByron HsuClaude Opus 5Thomas WangXinyi SongMohammad Miadh AngkadCheng WanBBufHanming LuXinyi Song
 		abddb1c7e9 [Kimi] Support kimi-k3 (#32541) 
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai>
Co-authored-by: Zijie Xia <37504505+zijiexia@users.noreply.github.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: zhangxiaohao <1024393531@qq.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Hao Phan <htphan@nvidia.com>
Co-authored-by: Thomas Wang <1am9trash@gmail.com>
Co-authored-by: RolaoDenthu <xinyisong0111@gmail.com>
Co-authored-by: pigeonsoup <32922982+pigeonsoup@users.noreply.github.com>
Co-authored-by: HaiShaw <hixiao@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Pranjal Shankhdhar <pranjal.ssh@gmail.com>
Co-authored-by: Lee Nau <lee.nau@gmail.com>
Co-authored-by: HMING <126185151+Hearum@users.noreply.github.com>
Co-authored-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
Co-authored-by: Xinyi Song <86638975+RolaoDenthu@users.noreply.github.com>
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
Co-authored-by: BBuf <xiaoyu.zhang@radixark.ai>
Co-authored-by: Hanming Lu <hanminglu@meta.com>
Co-authored-by: Xinyi Song <xinyis10@illinois.edu>
 2026-08-04 13:22:49 -07:00
Liangsheng Yin 0753663b8e [CI] Trim redundant B200 test registrations (#33586) 2026-08-04 13:22:00 -07:00
Liangsheng Yin b6d548afd7 [Fix] Resolve VLM test image placeholders from the model's own chat template (#33509) 2026-08-04 02:01:39 -07:00
Liangsheng Yin 157401f050 [CI] Build the Rust extensions on the 5090 pool and seed the cache from main (#33460) 2026-08-03 23:33:20 -07:00
Liangsheng YinandMick afc868517b [Perf] Speed up the Kimi-K2.5 vision path and match PIL bicubic in the GPU resize (#33349) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-08-03 21:57:49 -07:00
Liangsheng Yin 7ba393dd15 [CI] Dispatch base-a-test-cpu through its own reusable stage workflow (#33461) 2026-08-03 20:49:23 -07:00
Liangsheng Yin cdff33d738 [CI] Build the Rust extension modules once per run instead of in every CUDA job (#33384) 2026-08-03 18:56:48 -07:00
Liangsheng Yin c949e91f18 [CI] Remove the orphaned site-packages sglang skeleton that shadows the checkout (#33441) 2026-08-03 16:31:25 -07:00
Liangsheng Yin 22c2e2bcad [CI] Persist the cargo build cache across CUDA CI jobs (#33361) 2026-08-03 16:23:09 -07:00
Liangsheng Yin b8f6181bff [CI] Build the Rust extensions with the pinned toolchain instead of the image default (#33437) 2026-08-03 16:16:34 -07:00
Liangsheng Yin 45c00daa1b [misc] Deep-merge nested config overrides and parse request bodies with orjson (#33351) 2026-08-02 23:55:06 -07:00
Liangsheng Yin fcc4de9e5f [CI] Size the CPU stage from the live partition model (#33329) 2026-08-02 22:26:58 -07:00
Liangsheng Yin f5f021672a [Fix] Treat an empty grammar constraint as unset in SamplingParams (#33328) 2026-08-02 21:45:51 -07:00
Liangsheng YinandMick dd6ddc053b [Fix] Drop deprecated multimodal processor residency state (#33308) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-08-02 20:02:53 -07:00
Liangsheng YinandMick 4bc593fdc8 [Perf] Broadcast single-image DP vision embedding instead of pad-to-max all-gather (#33307) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-08-02 19:39:53 -07:00
Liangsheng Yin 2a7a299c27 [Spec] Support sampling in the DSPARK graph-folded draft proposal (#33298) 2026-08-02 18:46:39 -07:00
Liangsheng Yin 558c9bdcc2 [misc] Improve benchmark determinism and dataset API coverage (#33255) 2026-08-02 01:39:50 -07:00
Liangsheng Yin ca07917c58 [Fix] Bound FULL_MASK verify-mask reuse by the captured max_bs (#33127) 2026-07-31 17:50:40 -07:00
Liangsheng YinandKaixi Matteo Chen 5c6635d8f3 [Spec] Compact the target-verify mask when nothing reads it (#32920) 
Co-authored-by: Kaixi Matteo Chen <kaiximatteoc@nvidia.com>
 2026-07-30 23:13:21 -07:00
Liangsheng Yin 9f56553408 [Perf] Fast-path chain-style draft token organization in multi-layer EAGLE (#32887) 2026-07-30 02:55:21 -07:00
Liangsheng YinandKaixi 6ab3231b97 [Perf] Skip the target-verify tree mask fill when the backend never reads it (#32886) 
Co-authored-by: Kaixi <kaiximatteoc@nvidia.com>
 2026-07-30 02:32:38 -07:00
Liangsheng Yin c192145830 [Kernel] Fuse KV-cache writes for asymmetric K/V (head_dim != v_head_dim) (#32813) 2026-07-30 00:26:10 -07:00
Liangsheng Yin 2625fdfe6b [Fix] Count multi-layer draft-extend replays in the fwd-occupancy device timer (#32867) 2026-07-30 00:21:34 -07:00
Liangsheng Yin f4e0ac382e [misc] Remove unused multi_layer_draft_forward_cg module (#32881) 2026-07-29 21:17:16 -07:00
Liangsheng Yin 313a518bee [Spec] Emit step trace span for multi-layer draft-extend graph replays (#32850) 2026-07-29 19:22:58 -07:00
Liangsheng Yin e5c46ff07d [Fix] Route asymmetric-KV models to fa4 on SM100 and pin MiMoV2 FP8 MoE to flashinfer_trtllm (#32818) 2026-07-29 16:37:43 -07:00
Liangsheng Yin cce5873513 [CI] Fail lint when a registered file's TestCase classes never run (#32735) 2026-07-29 00:45:01 -07:00
Liangsheng Yin c4fc241fd3 [Perf] Free KV pages by segment in the paged allocator without a device sync (#32701) 2026-07-28 22:24:16 -07:00
Liangsheng Yin 14bd315d6e [Refactor] Remove dead allocator backup_state / restore_state (#32709) 2026-07-28 19:49:47 -07:00
Liangsheng Yin 85618cc798 docs: update sglang cookbook (#32654) 2026-07-28 15:33:24 -07:00
Liangsheng Yinkpham-sglZijie XiaispobockMickBaizhou ZhangthomawanBBuf
 		7dafacca49 docs(cookbook): add the Kimi-K3 serving cookbook (#32542) 
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
Co-authored-by: ispobock <ispobaoke@gmail.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: thomawan <thomawan@amd.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2026-07-27 08:37:30 -07:00
Liangsheng Yin 3da1071d56 [Spec] Hold the grammar bitmask in one GrammarMask type across all decode paths (#32409) 2026-07-25 15:27:43 -07:00
Liangsheng Yin 17afd8421f [Spec] Share the grammar mask build and verify-tree staging across spec workers (#32393) 2026-07-25 03:40:21 -07:00
Liangsheng Yin 3c5bf1f6d2 [Spec] Derive NGRAM grammar tree links on the host instead of reading back retrive_next_token (#32380) 2026-07-25 02:25:16 -07:00
Liangsheng Yin cff20a2fbb [Spec] Consolidate the grammar sync decision into ScheduleBatch.grammar_needs_sync (#32353) 2026-07-24 20:42:38 -07:00
Liangsheng Yin a31542ebd9 [Feature] Add leveled invariant-check primitive for nan/inf/oob validity checks (#32308) 2026-07-24 04:27:20 -07:00
Liangsheng Yin f4f15162bc [Fix] Fail fast when a safetensors index references missing shard files (#32279) 2026-07-24 02:18:56 -07:00
Liangsheng Yin d059b0f56e [Fix] Clamp degenerate all-sentinel draft rows to token 0 in dspark _online_combine_kernel (#32277) 2026-07-24 00:53:56 -07:00
Liangsheng Yin 99b29bf188 [Fix] Support ENCODER_ONLY target-verify in the trtllm_mha backend (#32178) 2026-07-23 20:33:39 -07:00
Liangsheng Yin 59ef3b15cc [Fix] Reserve the mamba pool's +1 padding slot in the memory budget solve (#32184) 2026-07-23 14:55:35 -07:00
Liangsheng Yin f35411ee81 [Spec] Enable grammar overlap scheduling for STANDALONE speculative decoding (#32110) 2026-07-23 01:47:27 -07:00
Liangsheng Yin 9b853e6832 [Scheduler] Enable decode retraction ordering under speculative decoding (#32023) 2026-07-23 00:42:56 -07:00
Liangsheng Yin 1b63155efe [Perf] Skip blocks past per-request live length in full-width Triton kernels (#32109) 2026-07-22 22:00:25 -07:00
Liangsheng Yin 98cc8d91cf [Fix] Evict only the KV shortfall in evict_from_tree_cache (#32016) 2026-07-22 12:41:06 -07:00
Liangsheng Yin 5c6a29b3c3 [Fix] Unify pinned host pool release on graceful shutdown (#32029) 2026-07-22 12:29:34 -07:00
Liangsheng Yin 024639a372 [Perf] Stack dspark dense draft per-layer ctx KV projection into one GEMM (#31986) 2026-07-21 17:46:13 -07:00
Liangsheng Yin 8ae0eb83fc [Perf] Fold dspark dense draft embedding into the draft graph via forward_embed (#31985) 2026-07-21 17:00:42 -07:00
Liangsheng Yin 08cb081ed0 [Perf] Skip page-table columns past kv length in DSA draft-extend metadata kernel (#31981) 2026-07-21 16:20:14 -07:00
Liangsheng Yin cb324cc958 Add wangfakang to CI_PERMISSIONS.json (#31980) 2026-07-21 14:03:00 -07:00
Liangsheng Yin 7fe9ad25ac [Spec] Extract DFlash compact draft-cache rebuild helpers (#31677) 2026-07-20 15:07:55 -07:00
Liangsheng Yin ff6c755952 [Refactor] Unify logprob results into a single LogprobResult and rename chunk env vars (#31733) 2026-07-20 12:44:27 -07:00
Liangsheng Yin b15a83983c [Fix] Release hierarchical cache host pool on graceful shutdown (#31746) 2026-07-19 22:27:29 -07:00
Liangsheng Yin b3570a4531 [Refactor] Unify input logprob processing on a single chunked path (#31655) 2026-07-19 16:03:23 -07:00
Liangsheng Yin cce5fe7696 [Scheduler] Move the WAR barrier to right after each run_batch launch (#31687) 2026-07-18 20:38:34 -07:00
Liangsheng Yin 10908a6793 [Fix] Respect cache_protected_len in ChunkCache and disabled-radix release paths (#31662) 2026-07-18 12:04:25 -07:00
Liangsheng Yin 639261f7b2 [Refactor] Move output logprob processing into the logprob_processor layer (#31624) 2026-07-17 22:33:52 -07:00
Liangsheng Yin 19c53c44a0 [Fix] Account zero-logprob sequences correctly in chunked logprob stitching (#31639) 2026-07-17 22:33:27 -07:00
Liangsheng Yin 2c856abbe3 [Fix] Enable chunked input-logprob processing by default to cap peak memory (#31498) 2026-07-17 13:19:03 -07:00
Liangsheng Yin 19f4859b30 [CI] Exclude current process memory from GPU idle check (#31571) 2026-07-17 02:11:41 -07:00
Liangsheng Yin 6e3be088a9 [Spec] Allocate the verify tree-mask scratch on the target backend only (#31527) 2026-07-16 23:23:21 -07:00
Liangsheng Yin 45824c69ca fa3: build the topk>1 verify replay page table on-device (#31381) 2026-07-16 21:21:01 -07:00
Liangsheng Yin 27ad9d11b1 [CI] Wait for GPU memory release before each test class setUpClass (#31509) 2026-07-16 21:06:29 -07:00
Liangsheng Yin dfa6278370 [Fix] Publish idle scheduler metrics immediately when the running-reqs gauge is stale (#31517) 2026-07-16 21:04:10 -07:00
Liangsheng Yin 9910ef8167 flashmla: sync-free spec via device-side draft-extend (#31090) 2026-07-16 19:38:12 -07:00
Liangsheng Yin 640101da13 [Fix] Deflake score engine determinism tests (#31494) 2026-07-16 15:42:47 -07:00
Liangsheng Yin 9a4d640244 [Perf] Cache uniform ragged-verify layout for DSpark verify-all compact (#31434) 2026-07-16 15:00:41 -07:00
Liangsheng Yin fc1e3797b7 [Spec] Split the capture width from num_tokens_per_req and gate replay on it (#31255) 2026-07-16 14:59:11 -07:00
Liangsheng Yin 44f4ea917c fa3: sync-free eagle spec via fixed-window draft-extend metadata (#31364) 2026-07-16 12:31:50 -07:00
Liangsheng Yin b55228cfdb [Spec] Consolidate the verify step into eagle_worker_common.run_eagle_verify (#31380) 2026-07-16 00:56:54 -07:00
Liangsheng Yin 0b372d03de [CI] Guard partition consumers against failed check-changes and degenerate fits (#31416) 2026-07-15 22:09:40 -07:00
Liangsheng Yin b871a509e6 [Fix] Wire the detokenizer soft watchdog into the multi-http-worker event loop (#31392) 2026-07-15 21:55:46 -07:00
Liangsheng Yin 1cc9493747 [Spec] Converge DP-attention spec width scaling onto num_tokens_per_req (#31244) 2026-07-15 17:57:08 -07:00
Liangsheng Yin b0b2dfbda1 [Spec] Extract the shared draft() tail into build_eagle_verify_input (#31375) 2026-07-15 15:59:31 -07:00
Liangsheng Yin e76cc75cfa [CI] Remove nightly registrations redundant with scheduled stage runs (#31371) 2026-07-15 14:12:07 -07:00
Liangsheng Yin dc078ddd2a [Spec] Extract stateless draft prepare helpers into eagle_worker_common (#31257) 2026-07-15 10:54:31 -07:00
Liangsheng Yin 76dc427806 [Spec] Single-source num_tokens_per_req derivation and access (#31013) 2026-07-14 18:41:08 -07:00
Liangsheng Yin ca0ee3f1a8 [Spec] Consolidate spec-worker weight updates into BaseSpecWorker via draft_runners (#31078) 2026-07-14 18:39:46 -07:00
Liangsheng Yin 50d1edaa7f [misc] Move SchedulerRecvSkipper into scheduler_components (#31222) 2026-07-14 16:35:04 -07:00
Liangsheng Yin e2728ac504 [Spec] Remove dead padded_static_len and stale SGLANG_ENABLE_SPEC_V2 references (#30998) 2026-07-13 15:30:31 -05:00
Liangsheng Yinandronhuafeng 2ab531cfcf fa3/fa4: sync-free for all backends and phases (#29589) 
Co-authored-by: ronhuafeng <ronhuafeng@users.noreply.github.com>
 2026-07-13 15:09:57 -05:00
Liangsheng Yin 48fff1f2bd [Spec] Deduplicate spec-v2 worker lifecycle boilerplate into BaseSpecWorker (#31008) 2026-07-13 13:48:40 -05:00
Liangsheng Yin c0f1f7e062 [Spec] Rename num_tokens_per_bs to num_tokens_per_req (#30977) 2026-07-13 13:47:53 -05:00
Liangsheng Yin 7a82178277 [Fix] Disable FlashInfer allreduce fusion in Nemotron-3-Nano lm-eval test (#30945) 2026-07-12 15:39:59 -05:00
Liangsheng Yin 5ba3c5147e [Spec] Add kill-switch env for draft-extend CUDA graph capture (#30944) 2026-07-12 14:54:36 -05:00
Liangsheng Yin 4884f6fbee [Fix] Unify ForwardBatch extend lens cpu fields to their declared list type (#30896) 2026-07-11 17:30:53 -05:00
Liangsheng Yin d8ef76682e [Spec] Extract shared draft worker construction and generalize draft sampler capture (#30857) 2026-07-11 14:34:09 -05:00