minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
14,208 Commits 3 Branches 0 Tags
61a8b42c001442b947f7b114ed232dfd65764e2e
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Lianmin Zheng 7cead0fb8f Add JonnyKong to CI_PERMISSIONS.json (#28550) 2026-06-17 11:42:45 -07:00
Lianmin ZhengandIan O'Connell 7e629a2f8c Allow overriding tokenizer path in benchmark harness (#28280) 
Co-authored-by: Ian O'Connell <ianoc@meta.com>
 2026-06-15 13:07:50 -07:00
Lianmin Zhengandgemini-code-assist[bot] 3b419f66da [JIT] Track angle-bracket includes in source hash (#28273) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-06-15 02:58:14 -07:00
Lianmin Zheng f18d38d040 Revert "[AMD][Quantization] Online MXFP4 quantization 2/N - FP8 to MXFP4 requantization on AMD GPUs" (#28213) 2026-06-14 13:34:06 -07:00
Lianmin Zheng b001d3e815 Add prajjwal1 to CI_PERMISSIONS.json (#28120) 2026-06-12 20:26:38 -07:00
Lianmin Zheng 165331a200 Share BCG output buffers across capture sizes (#27659) 2026-06-09 20:33:46 -07:00
Lianmin Zheng ca716f4734 Add TP server GPU process regression test (#27721) 2026-06-09 16:25:27 -07:00
Lianmin Zhengandlmzheng eb8dceda44 Defer DeepGEMM PDL setup to worker init (#27671) 
Co-authored-by: lmzheng <lmzheng@fb.com>
 2026-06-09 13:52:30 -07:00
Lianmin Zheng fca4ef9d69 Fix SWA pool resolution for EAGLE draft workers (#27491) 2026-06-08 11:00:29 -07:00
Lianmin Zheng bcb5645629 Fix TRTLLM target verify query metadata (#27495) 2026-06-08 10:58:48 -07:00
Lianmin Zheng a07d813ec8 Revert "Fix TRTLLM target verify query metadata (#27473)" (#27494) 2026-06-07 02:48:52 -07:00
Lianmin ZhengandMing Yang db58e76c33 Add all_to_all_single to GroupCoordinator (#27492) 
Co-authored-by: Ming Yang <minos.future@gmail.com>
 2026-06-07 02:48:47 -07:00
Lianmin Zheng 5be0b0c8c0 Fix TRTLLM target verify query metadata (#27473) 2026-06-07 02:46:01 -07:00
Lianmin Zhengcctrycctry
 		88a7b0fd30 Classify malformed-multimodal rejects as invalid_request (#27451) 
Co-authored-by: cctry <cctry@meta.com>
Co-authored-by: cctry <cctry@fb.com>
 2026-06-06 10:19:22 -07:00
Lianmin Zheng 448d3afb76 fix(spec): complete CustomSpecAlgo duck-typing interface and guard against drift (#27300) 2026-06-04 15:52:46 -07:00
Lianmin Zheng ff5c4d7b57 Add zx3xyy to CI_PERMISSIONS.json (#27213) 2026-06-03 22:12:00 -07:00
Lianmin ZhengHanming LuGrigory Sizov
 		14ed9b448e Add ZMQ IPv6 support, bench_serving sampling params, and reduce routed_dp_rank log noise (#27180) 
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: Grigory Sizov <grisha.sizov@gmail.com>
 2026-06-03 17:49:34 -07:00
Lianmin Zheng 53b8378307 Fix weights_checker checksum for 0-dim tensors and multi-GPU (#26863) 2026-05-31 21:27:03 -07:00
Lianmin ZhengandJaewon a779791b3f Add random-ids dataset, round-robin expert simulation, and kill_process_tree logging (#26862) 
Co-authored-by: Jaewon <52840625+jaewonlee-fb@users.noreply.github.com>
 2026-05-31 20:50:29 -07:00
Lianmin Zhengandcctry 4ff1296f5e Optimize get load calls (/v1/loads) using shared-memory load snapshots (#26348) 
Co-authored-by: cctry <cctry@meta.com>
 2026-05-29 13:40:26 -07:00
Lianmin Zheng dc4e7bc479 Fix TRTLLM MHA draft decode cache seqlens replay (#26655) 2026-05-28 20:58:16 -07:00
Lianmin Zheng 93fa577bb9 Clean up server startup log noise (#26205) 2026-05-24 14:35:15 -07:00
Lianmin ZhengandJaewon 8fa56a0ab1 Fix FlashInfer A2A token cap sizing (#25907) 
Co-authored-by: Jaewon <52840625+jaewonlee-fb@users.noreply.github.com>
 2026-05-20 23:01:28 -07:00
Lianmin Zheng c4a7d12092 Enable breakable CUDA graph for eagle (#25795) 2026-05-20 18:02:04 -07:00
Lianmin Zheng 61ac6792e6 Add DevashishLal-CB to CI_PERMISSIONS.json (#25908) 2026-05-20 11:19:43 -07:00
Lianmin Zheng b45b52ee8f Add spec_verify_calls_total metric for speculative decoding (#25689) 2026-05-18 18:35:11 -07:00
Lianmin ZhengIan O'Connellianoc
 		314dedf7c6 Use SGLANG_CACHE_DIR env for gpu_p2p_access_cache path (#25686) 
Co-authored-by: Ian O'Connell <ianoc@meta.com>
Co-authored-by: ianoc <ianoc@fb.com>
 2026-05-18 16:00:54 -07:00
Lianmin ZhengandHanming Lu 745abd6cc0 Add no_combine support to cutlass_moe_fp4 (#25688) 
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2026-05-18 15:59:55 -07:00
Lianmin ZhengMing YangYinghai Lu
 		878e6b8886 [SP] Fix runtime_max_tokens_per_rank for sequence parallelism (#25685) 
Co-authored-by: Ming Yang <minos.future@gmail.com>
Co-authored-by: Yinghai Lu <yinghai@meta.com>
 2026-05-18 15:59:33 -07:00
Lianmin Zheng f21fe6ad4d Fix PD disaggregation warmup: set request_name and improve error logging (#25542) 2026-05-18 09:49:56 -07:00
Lianmin ZhengandHanming Lu 21e420b4c2 [Test] Set default temperature to 0.0 in kl_test_utils (#25429) 
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
 2026-05-15 18:20:10 -07:00
Lianmin Zhengandcctry 5fb6bde6c0 Add sglang:get_loads_duration_seconds metric (#25163) 
Co-authored-by: cctry <cctry@meta.com>
 2026-05-13 18:38:49 -07:00
Lianmin Zheng 1ae3218d03 Add jasonjk-park and charlotte12l to CI_PERMISSIONS.json (#25104) 2026-05-13 01:51:05 -07:00
Lianmin Zheng 48fc26a814 Fix Eagle draft decode positions (#25015) 2026-05-12 14:04:58 -07:00
Lianmin Zheng 34555aee9c Add minosfuture to CI_PERMISSIONS.json (#25019) 2026-05-11 16:54:32 -07:00
Lianmin Zheng 893dfb7b74 Add libertyeagle to CI_PERMISSIONS.json (#25011) 2026-05-11 14:44:07 -07:00
Lianmin Zheng e40e339c72 Filter non-int token ids in benchmark and observe decode-side bootstrap/alloc metrics (#24684) 2026-05-08 11:45:37 -07:00
Lianmin Zheng 3c3f0bd55e Cache empty MatchResult in RadixCache (#24470) 2026-05-07 17:13:20 -07:00
Lianmin Zheng b859f7ffba Improve metrics, observability, and PD deploy tooling (#24521) 2026-05-06 11:27:35 -07:00
Lianmin Zheng d4d4b04d66 [PD] Fix missing update_status call in abort() across all KV backends (#24522) 2026-05-06 05:30:11 -07:00
Lianmin Zheng b91b05ae27 Add --random-input-len to send_one.py (#24464) 2026-05-05 17:49:33 -07:00
Lianmin Zheng 64f80eabbe Register aten::rms_norm and aten::mm.dtype in batch invariant mode (#24459) 2026-05-05 17:21:34 -07:00
Lianmin Zheng 46bde1f426 Add fwd_occupancy metric to SchedulerStats and Prometheus collector (#24458) 2026-05-05 17:04:34 -07:00
Lianmin Zheng 710fed10fb Revert "[fix] /pause_generation and /continue_generation wrong for --tokenizer-worker-num > 1" (#24461) 2026-05-05 16:44:34 -07:00
Lianmin Zheng 29dd3a36c0 Refactor device timer installation and rename prefill prealloc to bootstrap (#24341) 2026-05-04 13:57:13 -07:00
Lianmin Zheng c7566a860b Add CI permissions for user luccafong, 842974287 (#24238) 2026-05-01 12:50:36 -07:00
Lianmin Zheng ece8a1a788 Refactor device timer, clean up metrics collector, and add fwd occupancy metric (#24197) 2026-05-01 10:25:25 -07:00
Lianmin ZhengXiaozhu MengYinghai Lu
 		d9e8a4a7f8 [SWA] Ensure we use pre-computed SWA cache location during prefill (#24138) 
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Yinghai Lu <yinghai@meta.com>
 2026-05-01 00:01:49 -07:00
Lianmin Zheng b1ef99f65f [CI] Remove orphaned test/srt/ascend and test/srt/configs (#24145) 2026-04-30 04:43:11 -07:00
Lianmin Zheng e60c60eff0 [SWA] Fix missing mamba_indices parameter in cpu copy interface (#24026) 2026-04-29 17:33:38 -07:00
Lianmin Zheng 2a771a40ac Add engine_type label to tokenizer manager metrics (#23978) 2026-04-28 19:52:58 -07:00
Lianmin Zheng d66eb3a91b docs: update contribution guide with coding style guidelines (#23977) 2026-04-28 19:51:58 -07:00
Lianmin Zheng 14b4e6fa69 Support --model as alias for --model-path in CLI (#23894) 2026-04-28 17:23:27 -07:00
Lianmin Zheng 8536d4b402 Clean up noisy startup warnings from third-party deps (#23669) 2026-04-27 03:10:46 -07:00
Lianmin Zheng a4facdf3f6 [CI] Refactor ci_install_dependency.sh into standalone functions (#23592) 2026-04-24 17:39:39 -07:00
Lianmin Zheng 6344b546c8 Deprecate --collect-tokens-histogram, auto-collect with --enable-metrics (#23595) 2026-04-24 12:00:16 -07:00
Lianmin Zheng 95d021b523 Pre-set SWA cache location in CudaGraphRunner (#23552) 2026-04-23 16:51:29 -07:00
Lianmin Zheng bb962b0046 Fix MoE no_combine: skip router weight in down projection (#23545) 2026-04-23 16:47:58 -07:00
Lianmin Zheng 6a3c070ee3 Add 'allready' to ignore words list in .codespellrc (#23465) 2026-04-22 02:39:04 -07:00
Lianmin Zheng 9c47bbad13 Clean up bench_one_batch warning and simplify norm dispatch (#23110) 2026-04-17 17:42:20 -07:00
Lianmin Zheng 44e67c6835 Remove deprecated double sparsity feature (#23009) 2026-04-17 13:33:12 -07:00
Lianmin Zheng 35da90cb76 [misc] Configure logging before ServerArgs.__post_init__ (#22926) 2026-04-15 23:53:15 -07:00
Lianmin Zheng ccff59254c Update .codespellrc (#22912) 2026-04-15 16:25:55 -07:00
Lianmin Zheng 43925d179d [Speculative] Fix Eagle3/DFLASH aux hidden state capture during CUDA graph init (#22836) 2026-04-15 14:04:54 -07:00
Lianmin Zheng adb310b976 Cleanup server_args.py and minor code tidying (#22820) 2026-04-14 18:52:41 -07:00
Lianmin Zheng 222eda1598 [Misc] Use cache_once for is_arch_support_pdl in sgl-kernel (#22725) 2026-04-14 15:22:10 -07:00
Lianmin Zheng 9fb00ede15 Clean up TokenizerManager and req_time_stats: reduce overhead and simplify (#21646) 2026-04-13 16:47:32 -07:00
Lianmin Zheng 8f9553bccb [Misc] Migrate SGLANG_SET_CPU_AFFINITY to envs and refactor model config building (#22730) 2026-04-13 16:10:31 -07:00
Lianmin Zheng f81b6e8f51 [Misc] Add @cache_once to is_arch_support_pdl in jit_kernel (#22724) 2026-04-13 14:42:49 -07:00
Lianmin Zheng ba7bcca6b3 Use reshape instead of contiguous().view() in TRTLLMHAAttnBackend (#22517) 2026-04-13 14:29:12 -07:00
Lianmin Zheng ddc8ef1038 Lazy import flash_attention_v4 to avoid loading flash_attn.cute at startup (#22306) 2026-04-08 20:40:25 -07:00
Lianmin ZhengandClaude Opus 4.6 494bb86169 Cache sub-objects in __getitem__ to ensure identity stability (#22184) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-06 18:53:38 -07:00
Lianmin Zheng a80961333b Clean up req_time_stats: reduce overhead and simplify (#22186) 2026-04-06 14:20:51 -07:00
Lianmin Zheng e835601fb7 Cache gfx95 quant format detection in DeepseekV2DecoderLayer (#22143) 2026-04-05 20:20:54 -07:00
Lianmin Zheng eb407b80f3 [Kernel] Make FA3/FA4 imports lazy in FlashAttentionBackend (#22028) 2026-04-03 13:49:00 -07:00
Lianmin Zheng fe38410c3e Remove logging for subprocess watchdog start (#21968) 2026-04-02 11:30:33 -07:00
Lianmin Zheng afb32d7622 README: coding agent sponsorship for long-term contributors (#21642) 2026-03-29 16:02:51 -07:00
Lianmin ZhengandClaude Opus 4.6 9f7792415a Clean up TokenizerManager: remove dead code and improve rid validation (#21639) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 15:12:49 -07:00
Lianmin ZhengandClaude Opus 4.6 f3970b17ef [Cleanup] Remove unused BatchMultimodalOutput and BatchMultimodalDecodeReq (#21640) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 14:54:25 -07:00
Lianmin ZhengandClaude Opus 4.6 1d9c8e8c9e Simplify routed experts test and move base64 encoding to tokenizer manager (#21634) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 12:44:01 -07:00
Lianmin ZhengandClaude Opus 4.6 ba6b501f3a Clean up detokenizer and remove dead multimodal_gen code (#21588) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 21:44:40 -07:00
Lianmin Zheng 83997080a6 docs: flesh out MAINTAINER.md oncall lists and link GitHub profiles (#21575) 2026-03-27 17:39:16 -07:00
Lianmin Zheng 9a91323c9f test: point DSV3 int8 MLA CI models to lmsys Hugging Face org (#21561) 2026-03-27 13:04:01 -07:00
Lianmin Zheng 3dfaa47d5e Change cron schedule to run every 6 hours (#21285) 2026-03-24 00:04:23 -07:00
Lianmin Zheng d6ce1b8dce docs: update add-jit-kernel skill for run_suite CI registration (#21264) 2026-03-23 21:31:22 -07:00
Lianmin Zheng 260abe1fb1 Refactor JIT kernel CI to use run_suite.py registration system (#21239) 2026-03-23 21:17:27 -07:00
Lianmin Zheng 4779755eb9 Split pr-test.yml: extract sgl-kernel, jit-kernel, and multimodal-gen tests into separate workflow files (#21219) 2026-03-23 13:17:05 -07:00
Lianmin ZhengandClaude Opus 4.6 27ac831a84 docs: improve CI and testing documentation (#21202) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 10:48:50 -07:00
Lianmin Zheng 814202704b ci: unify PR test suite naming (#21187) 2026-03-23 00:18:45 -07:00
Lianmin Zheng 7757a9ddd0 ci: remove IS_BLACKWELL env var; auto-detect Blackwell (#21118) 2026-03-22 23:44:48 -07:00
Lianmin Zheng 7050011dee Enable JIT clamp_position and resolve_future_token_ids on ROCm (#21116) 2026-03-22 22:33:54 -07:00
Lianmin Zheng 3343a79466 Fix jit-kernel unit test CI dependency install in pr-test workflow (#21089) 2026-03-21 22:13:06 -07:00
Lianmin Zheng 1ff0ee95e3 Update pr-test.yml to remove the temp hack - cron: '15 8 * * *' (#21115) 2026-03-21 21:53:31 -07:00
Lianmin ZhengandClaude Opus 4.6 76e4a8662c Replace clamp_position with JIT kernel + platform dispatch (#20999) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 21:26:26 -07:00
Lianmin Zheng 9b98d55ba4 Change PR test schedule time to 7:50 AM UTC (#21087) 2026-03-21 00:36:05 -07:00
Lianmin Zheng f2c9d5696f Change daily cron job time in pr-test.yml (#21084) 2026-03-21 00:24:53 -07:00
Lianmin Zheng dba6fb3d30 Fix streaming logprobs corruption caused by shared mutable list reference (#21030) 2026-03-21 00:18:48 -07:00
Lianmin Zheng d089db0563 ci: add daily PR Test schedule at 12:18am PDT (#21083) 2026-03-21 00:15:48 -07:00
Lianmin Zheng 182dd20d57 ci: bypass maintenance gate for PR Test on main (#21082) 2026-03-21 00:08:47 -07:00
Lianmin Zheng 507db41504 Add CI maintenance mode gate to nightly-test-nvidia.yml (#21079) 2026-03-20 23:44:37 -07:00