minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
18,577 Commits 2 Branches 0 Tags
ab03a8e7eb82d35907bfbeb645bf0af8b4ce290e
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Liangsheng Yin 76a9065bef [Fix] Raise on undelivered embeddings in send_with_url, fix broken tests (#40502) 2026-09-20 17:35:27 -07:00
Liangsheng Yin acd20a516e [CI] Give the kernel lane a 5090 suite and move kernel-only tests off the general lane (#40496) 2026-09-20 16:46:53 -07:00
Liangsheng Yin 2e2d8a2fda [CI] Drive per-commit stage jobs from a runner table instead of copied job blocks (#40495) 2026-09-20 15:43:46 -07:00
Liangsheng Yin f6483e479f [Test] Drop cause-less disabled tests, fix XPU lane, demote quality gates off base-c (#40288) 2026-09-20 14:36:02 -07:00
Liangsheng Yin 0024efa0de [CI] Derive registered-test kind from the registry call instead of the path (#40294) 2026-09-20 02:01:22 -07:00
Liangsheng Yin dc002c85fc [Test] Fix OOT DFlash hook test resolving the draft config over the network (#40427) 2026-09-20 01:25:34 -07:00
Liangsheng Yin d507accadc [Test] Drop dead and strictly-subsumed CI test registrations (#40264) 2026-09-18 17:49:13 -07:00
Liangsheng Yin b876213548 [Test] Add a ci-test-audit skill cataloging CI and test audit patterns (#40257) 2026-09-18 16:35:49 -07:00
Liangsheng Yin 6cc9090d1f [mem_cache] Release up to owned_kv_len on radix cache insert (#40075) 2026-09-18 15:37:52 -07:00
Liangsheng YinBBufClaude Opus 5Xiaoyu ZhangYuwei AnKhoa PhamYuhao YangZhichen ZengKe Bao
 		a6cf05817f dsv4.1: remaining model and runtime integration (#38798) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2026-09-18 02:55:30 -07:00
Liangsheng Yin 1b200ffaaa [Quant] Serve 32-wide-K ue8m0 block-FP8 linears through the FlashInfer MXFP8 GEMMs (#40039) 2026-09-18 02:51:12 -07:00
Liangsheng Yin f65c70bb7d [Kernel] Move CUDA and ROCm speculative kernels to JIT (#40033) 2026-09-17 17:27:26 -07:00
Liangsheng Yin 1f0c73e9bd [DSV4] Generalize attention metadata, sparse prefill, and KV pool over compress ratios (#39921) 2026-09-17 15:55:19 -07:00
Liangsheng YinandBBuf 3401b75240 dsv4.1: Engram module and request history support (#39666) 
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-16 23:49:47 -07:00
Liangsheng Yin 1c4b130bf7 [CI] Unify basic and speculative sanity accuracy checks with MMLU (#39906) 2026-09-16 22:19:05 -07:00
Liangsheng Yin 923e4a56d4 [CI] Fix sanity evaluation and diffusion test suite blockers (#39892) 2026-09-16 21:47:41 -07:00
Liangsheng YinandBBuf 464fffbec8 dsv4.1: chat encoding and tool parsing (#39665) 
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-16 20:49:42 -07:00
Liangsheng YinBBufYuhao Yang
 		c89c63fa38 dsv4.1: vision tower and image preprocessing (#39668) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
 2026-09-16 18:11:16 -07:00
Liangsheng Yin 408d2334c3 dsv4.1: mHC computation and compensated projections (#39664) 
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
  Co-authored-by: BBuf <1182563586@qq.com>
  Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
 2026-09-16 17:11:55 -07:00
Liangsheng YinBBufYuhao Yang
 		46ae84df15 dsv4.1: Hopper FP8 matmul kernels and tuning (#39657) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
 2026-09-16 16:54:19 -07:00
Liangsheng Yin dc067c7d8c [Fix] Allow closed object schemas in Outlines prevalidation (#39869) 2026-09-16 16:42:17 -07:00
Liangsheng YinDarkSharpnessXiaoyu Zhang
 		c2443458e1 dsv4.1: RoPE and FP4 packing kernels (#39656) 
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-16 16:17:12 -07:00
Liangsheng YinDarkSharpnessXiaoyu Zhang
 		35b7589e1a dsv4.1: candidate indexer library (#39671) 
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
 2026-09-16 15:15:18 -07:00
Liangsheng YinBBufClaude Opus 5DarkSharpness
 		13d593b6cf dsv4.1: compression, KV I/O, and metadata kernels (#39652) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
 2026-09-16 13:54:07 -07:00
+4
Liangsheng YinCheng WanClaude Opus 5CursorDarkSharpnessDarkSharpnessKe BaoKhoa PhamXiaoyu ZhangXiaoyu ZhangYuhao YangYuwei AnZhichen ZengZiyi Xu
 		7d5696b3a1 dsv4.1: communication kernels and wrappers (#39653) 
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai>
 2026-09-16 02:09:40 -07:00
Liangsheng YinandBBuf fc5a979f21 [misc] Merge FlashInfer autotune caches across spec workers, pad MXFP4 TP shards, drop dead ngram attrs (#39678) 
Co-authored-by: BBuf <1182563586@qq.com>
 2026-09-16 01:50:29 -07:00
+4
Liangsheng YinBBufCheng WanClaude Opus 5CursorDarkSharpnessDarkSharpnessKe BaoKhoa PhamXiaoyu ZhangYuhao YangYuwei AnZhichen ZengZiyi Xu
 		faaff1eca8 dsv4.1: Top-k kernels and candidate selection (#39648) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: DarkSharpness <76582120+DarkSharpness@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ziyi Xu <ziyi.xu@radixark.ai>
 2026-09-15 23:54:35 -07:00
Liangsheng Yin 91f691c490 dsv4.1: standalone kernels and Python wrappers (#39646) 2026-09-15 22:31:18 -07:00
Liangsheng Yin 2f7d04da5d dsv4.1: Rust extension modules for image preprocessing, KV pool names, and PD bootstrap (#39677) 2026-09-15 18:11:48 -07:00
Liangsheng Yin 0dabef3d30 [misc] Fix tool-call index, graph padded-row count, and prefill-graph input_embeds refresh (#39574) 2026-09-15 16:37:22 -07:00
Liangsheng Yin 87c9f78e99 [Fix][PD] Give prefill and decode their own RDMA NICs in disaggregation tests (#39584) 2026-09-15 15:07:42 -07:00
Liangsheng Yin d58342deab [Fix] Release NCCL on scheduler exit and let the ASGI server own shutdown (#39560) 2026-09-15 13:39:15 -07:00
Liangsheng Yin 8565b11003 [misc] Trim redundant variants from the 8-gpu-h20 disaggregation test suite (#39544) 2026-09-15 01:24:46 -07:00
Liangsheng Yin 860fa83a9f [CI] Wait for a killed test server's GPU memory before the next launch (#39545) 2026-09-14 23:24:37 -07:00
Liangsheng Yin 66c7bc838e [misc] Revert #38346, #33426, #39061 and #39219 (#39405) 2026-09-14 03:04:07 -07:00
Liangsheng Yin 17fa5ad327 [Refactor] Generalize attention graph variants in the decode runner (#38993) 2026-09-11 00:45:58 -07:00
Liangsheng Yin 94ce940ff8 [Bug] Include DSA variant in exact-bucket graph admission (#38992) 2026-09-10 21:03:03 -07:00
Liangsheng Yin 41da06adca [Refactor] Generalize DeepSeek V4 compressed pool management (#38954) 2026-09-10 17:29:03 -07:00
Liangsheng Yin dc5f59c3a2 [Refactor] Clarify DeepSeek V4 metadata names for V4.1 (#38947) 2026-09-10 15:42:52 -07:00
Liangsheng Yin 28ebede865 [mem_cache] Free hybrid SWA pages by one representative per page on page_size > 1 (#38159) 2026-09-07 20:15:54 -07:00
Liangsheng Yin b23d835048 [Scheduler] Unify per-iteration request intake into ingest_requests() (#38389) 2026-09-07 20:02:02 -07:00
Liangsheng Yin b8a81f055d [CI] Replace the Lark queue-digest card with a daily queue-timeline chart (#38380) 2026-09-07 18:28:34 -07:00
Liangsheng Yin 4dcecc7891 Revert "[kernel] add fused silu mul quant fp8" (#38381) 2026-09-07 17:36:10 -07:00
Liangsheng Yin f5819b09bf Revert "[AMD][DSV4] Fix unified-KV pool sizing and SWA ring accounting" (#38163) 2026-09-05 17:28:46 -07:00
Liangsheng Yin 0454c074b4 [mem_cache] Clean up unified allocator leftovers (#38103) 2026-09-04 22:02:50 -07:00
Liangsheng Yin d50e9a9756 [Test] Prune redundant unified-memory allocator and pool tests (#38093) 2026-09-04 20:47:01 -07:00
Liangsheng Yin 0645398a32 [mem_cache] Move the unified-memory allocators into allocator/ and split the composites out (#38072) 2026-09-04 19:53:15 -07:00
Liangsheng YinweireweireSam Shleifer
 		67248e04b4 [mem_cache] Route hybrid SWA full-side kv-row frees through free_segment (#37876) 
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2026-09-04 01:48:33 -07:00
Liangsheng Yin 6147a54ddf [PD] Bound transfer engine init with SGLANG_DISAGGREGATION_ENGINE_INIT_TIMEOUT (#37874) 2026-09-03 17:16:55 -07:00
Liangsheng Yin 8b0501399e [CI] Improve Lark CI cards: structured layout, PDT timestamps, slow-only queue digest (#37884) 2026-09-03 16:18:56 -07:00
Liangsheng Yin 0610a6539d [CI] Add Lark notifications for CUDA CI status, runner health, and queue time (#37881) 2026-09-03 15:46:33 -07:00
Liangsheng Yin 0e5414fd2f [Test] Allow top-k cutoff ties in test_sampling_mask_matches_topk_logprobs (#37873) 2026-09-03 14:13:26 -07:00
Liangsheng Yin 2a980cbf10 [mem_cache] Require page-aligned starts in free_segment and drop the boundary trim (#37729) 2026-09-03 13:28:33 -07:00
Liangsheng Yin 54da74e83e [Fix] Broadcast PP dynamic-chunk profiling failures so every rank disables together (#37675) 2026-09-02 21:09:04 -07:00
Liangsheng Yin 49e6e81830 [CI] Accept t64-suffixed apt packages in the install skip check (#37689) 2026-09-02 20:51:47 -07:00
Liangsheng Yin a522c8a4b6 [misc] Extract PP dynamic chunk sizing into a DynamicChunkSizer scheduler component (#37674) 2026-09-02 18:35:43 -07:00
Liangsheng Yin 5c46ce37f5 [Fix] Apply the attention-CP broadcast result in PP dynamic-chunk profiling (#37669) 2026-09-02 17:20:07 -07:00
Liangsheng Yin 4bc34117f1 [CI] Install lmms-eval from PyPI, drop human-eval install, add clone token fallback (#37672) 2026-09-02 16:23:10 -07:00
Liangsheng Yin 19c7679e9e [mem_cache] Make free_swa sync-free on page_size == 1 (#36723) 2026-09-02 14:18:22 -07:00
Liangsheng Yin 2d799c28f4 [CI] Authenticate and retry git clones in install scripts (#37647) 2026-09-02 13:21:26 -07:00
Liangsheng Yin 862a909a08 [Fix] Lock PP dynamic-chunk profiling requests before releasing through the tree cache (#37509) 2026-09-02 02:09:01 -07:00
Liangsheng Yin ebfd8c60e5 [CI] Install sgl-eval from PyPI through the test extra (#37504) 2026-09-02 01:45:25 -07:00
Liangsheng Yin 01c3a5f54f [misc] Resolve SWA ownership at enqueue time for grouped free() (#36646) 2026-09-01 23:06:55 -07:00
Liangsheng Yin 832d029870 [mem_cache] Split duplicate insert frees at the SWA eviction floor (#37481) 2026-09-01 23:01:40 -07:00
Liangsheng Yin 83a9b5dd88 [mem_cache] Drop the torch.unique sync from the SWA page expansion (#37463) 2026-09-01 14:16:47 -07:00
Liangsheng Yin 3484f7f836 [mem_cache] Add free_kv_row to release a request's kv row by row range (#36721) 2026-09-01 01:14:43 -07:00
Liangsheng Yin 959ca033eb refactor(hicache): simplify decode offload state bookkeeping (#37299) 2026-08-31 23:09:41 -07:00
Liangsheng Yin 3b14f37b74 [Fix] Use real ReqKvInfo in unit-test req mocks (#37339) 2026-08-31 20:25:51 -07:00
Liangsheng Yin 2530204502 [mem_cache] Make release, row-reuse asserts, and presence checks read the KV record (#37167) 2026-08-31 12:46:15 -07:00
Liangsheng YinandBaizhou Zhang 95f0f41021 [CI] Move tests onto the right CI stages (#34074) 
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-08-31 12:39:59 -07:00
Liangsheng Yin 3a6ed55999 [Fix] Shut hicache test servers down gracefully before SIGKILL (#37194) 2026-08-30 22:45:40 -07:00
Liangsheng Yin 5d12ad4fd7 [mem_cache] Move mamba state and retraction_backup into ReqKvInfo (#37164) 2026-08-30 22:03:01 -07:00
Liangsheng Yin 4bb8de34cc [mem_cache] Share one ReqKvInfo between a streaming session slot and its request (#37108) 2026-08-30 16:36:48 -07:00
Liangsheng Yin 007ef5e23a [mem_cache] Move req_pool_idx into ReqKvInfo (#37094) 2026-08-30 14:46:21 -07:00
Liangsheng Yin 5ec959965b [mem_cache] Settle extend kv_committed_len inside alloc_for_extend (#37085) 2026-08-29 23:14:31 -07:00
Liangsheng Yin 0438b16154 [mem_cache] Move kv_committed_len into ReqKvInfo (#37078) 2026-08-29 22:42:25 -07:00
Liangsheng Yin 6be767c2d2 [mem_cache] Move cache_protected_len and swa_evict_floor into ReqKvInfo (#36982) 2026-08-29 19:37:53 -07:00
Liangsheng Yin 9a489f8d2f [Test] Move gpqa and aime25 onto sgl-eval, drop unused eval paths (#36979) 2026-08-29 17:36:13 -07:00
Liangsheng Yin 24a3a63e6b [misc] Keep req.kv non-optional and key KV ownership on req_pool_idx (#36958) 2026-08-28 23:47:42 -07:00
Liangsheng Yin a25df83fe3 [Cookbook] Run accuracy benchmarks through sgl-eval (#36977) 2026-08-28 23:29:38 -07:00
Liangsheng Yin ec962fd12d [Fix] Fall back to the process-group broadcast for DSA topk when PyNCCL is absent (#36963) 2026-08-28 22:20:35 -07:00
Liangsheng Yin 0df69849ba [Fix] Drop the duplicated DSpark draft sample_block call (#36934) 2026-08-28 18:10:41 -07:00
Liangsheng Yin 60f6d77a98 [mem_cache] Carry swa_evicted_seqlen into SWARadixCache.cache_unfinished_req (#36909) 2026-08-28 14:29:27 -07:00
Liangsheng Yin f65b2b2b15 [Fix] Lazy-import aiter in DSv4 paged_decode to unbreak CPU CI (#36914) 2026-08-28 14:16:53 -07:00
Liangsheng Yin daf6317196 [mem_cache] Add free_full to release the full side of a tombstoned SWA node (#36637) 2026-08-27 19:34:48 -07:00
Liangsheng Yin 5d52f02f22 [misc] Fold the allocator free-group flag into free_group (#36739) 2026-08-27 16:12:47 -07:00
Liangsheng YincswuygcswuygVedant JhaveriVedant Jhaveri
 		ec4bdbfa4a [Feature] Beam search support (#31626) 
Co-authored-by: cswuyg <cswuyg@gmail.com>
Co-authored-by: cswuyg <496090217@qq.com>
Co-authored-by: Vedant Jhaveri <vedantjh2@gmail.com>
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
 2026-08-26 16:56:15 -07:00
Liangsheng Yin 45c85c198b [CI] Lower the AWQ Marlin MMLU threshold to 0.80 (#36570) 2026-08-26 14:43:20 -07:00
Liangsheng Yin 896acc8860 [Fix] Clear full-to-SWA mapping with index_fill_ to avoid a blocking H2D copy (#35773) 2026-08-21 01:23:26 -07:00
Liangsheng Yin 0149f56e84 [CI] Gate /rerun-test on commenter trust and remove /rerun-stage (#35750) 2026-08-20 15:05:52 -07:00
Liangsheng Yin 5a100d9086 [misc] Trim restating comments and docstrings in srt/managers (#35622) 2026-08-20 14:18:40 -07:00
Liangsheng Yin a49560ce50 [misc] Add a comment style rule to .claude/rules (#35597) 2026-08-19 18:52:48 -07:00
Liangsheng Yin 1270204d2c Revert "[Feature] Add DeepEPv2 (ElasticBuffer) MoE A2A backend" (#35568) 2026-08-19 14:14:38 -07:00
Liangsheng Yin ccbe380028 [CI] Trim the base-c 4-gpu-h100 stage from 5 shards to 4 (#35407) 2026-08-19 00:48:07 -07:00
Liangsheng Yin e614121866 [Fix] Scale the req_to_token row headroom by attn_dcp_size (#35424) 2026-08-19 00:37:26 -07:00
Liangsheng Yin 7ebaa98f81 [Fix] Assert the page-aligned SWA evict floor on both PD decode prealloc paths (#35396) 2026-08-18 15:48:55 -07:00
Liangsheng Yin 87a09494fa [Refactor] Share the page-aligned decode alloc lens between EAGLE and DFLASH (#35382) 2026-08-18 14:39:41 -07:00
Liangsheng Yin 79dfef390b [Spec] Page-align the DFLASH decode KV reservation (#35265) 2026-08-18 13:36:04 -07:00
Liangsheng Yin aa82229173 [Fix] Assert the page-aligned SWA evict floor at PD decode prealloc (#35286) 2026-08-18 13:32:16 -07:00
Liangsheng Yin 0111b29031 [Perf] Hoist DSv4 draft-extend SWA write locs; unify SWA graph buffer naming (#34890) 2026-08-18 00:13:28 -07:00