minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity
8,918 Commits 3 Branches 0 Tags
3f44268fe57ebd70ef52963ea9fb7621e99f185c
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiaoyu Zhang 0d904ef44c [diffusion] fix: fix fsdp tp load make param miss parallel meta data (#17058) 2026-01-15 00:12:33 +08:00
Xiaoyu Zhang 2ab3ed3e9e Fix sgl-kernel per_token_quant fp8 kernel scale shared_memory bug (#16886) 2026-01-13 23:22:05 +08:00
Xiaoyu Zhang 740d3c0b39 [Diffusion] Remove useless dependency in diffusion (#16967) 2026-01-13 17:25:53 +08:00
Xiaoyu Zhangandgemini-code-assist[bot] 9d4d57dbfa [Diffusion] Tiny rename parallel_groups (#16743) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-09 15:08:25 +08:00
Xiaoyu ZhangandMick 294ff71d18 [Diffusion] Avoid cpu2gpu sync in flashinfer rope and apply flashinfer rope to wanvideo (#16668) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-01-08 22:44:38 +08:00
Xiaoyu Zhang 5a5cece561 [Diffusion] clean useless and buggy set_seq_parallel_pg in yunchang (#16669) 2026-01-08 11:08:34 +08:00
Xiaoyu ZhangandMick 32a6540afc [Diffusion] Fix Ulysses/Ring process group construction under TP to enable correct Wan2.2 tensor parallelism (#16532) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-01-07 20:52:36 +08:00
Xiaoyu Zhang 62d0280f62 Tiny fix readme (#16654) 2026-01-07 20:41:07 +08:00
Xiaoyu Zhang 5e5b1183ed [Diffusion] Ring Attention support sage backend (#16496) 2026-01-06 14:45:53 +08:00
Xiaoyu Zhang 4ea6a11c83 [CI] Fail wheel build when sgl-kernel artifacts are missing (#16450) 2026-01-04 21:46:40 -08:00
Xiaoyu Zhang 520c048d55 [diffusion] CI: add script for automatically generation ci perf baseline (#16389) 2026-01-05 13:18:35 +08:00
Xiaoyu Zhang 0fee6bc632 [JIT kernel] Apply jit per_tensor_quant_fp8 kernel (#15836) 2026-01-05 10:15:00 +08:00
Xiaoyu ZhangandMick d0fb24ee7b [Diffusion] Flux2 tp support (#16219) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-01-03 17:02:27 +08:00
Xiaoyu Zhang 1cfd2b2ded [diffusion] chore: remove redundant ulysses nccl warmup (#16301) 2026-01-03 00:19:35 +08:00
Xiaoyu Zhang bd48ad5e6b [Diffusion] Fix broken ring_attention when use upstream fa3 (#16270) 2026-01-02 14:33:12 +08:00
Xiaoyu Zhangandgemini-code-assist[bot] 733a0c1a37 [Diffusion] Zimage opt with qknorm and flashinfer rope (#16161) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-30 23:39:32 +08:00
Xiaoyu Zhangandgithub-actions[bot] b369aaa23f [Diffusion] Refine diffusion profling doc (#16163) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-12-30 23:38:46 +08:00
Xiaoyu Zhang 88f3de2514 [JIT kernel] Jit kernel add codeowners (#16085) 2025-12-29 22:32:55 +08:00
Xiaoyu Zhang 24616c5234 [Diffusion] Qwen image edit support qknorm optimization (#16062) 2025-12-29 21:37:24 +08:00
Xiaoyu Zhang f3d73b0199 [Diffusion] Refactor qwen_image's rope in a single helper func (#16047) 2025-12-29 17:24:26 +08:00
Xiaoyu Zhang 8305dc1718 [Diffusion] Disable packed QKV for FLUX & Z-Image (#16038) 2025-12-29 14:33:05 +08:00
Xiaoyu Zhang 7d02c8e59f [JIT kernel] CI support jit kernel tests (#15939) 2025-12-28 23:11:02 +08:00
Xiaoyu ZhangandMick 51dbdb2202 [diffusion] improve: improve qwen-image-edit performance to align with LightX2V (#15812) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-26 22:25:10 +08:00
Xiaoyu ZhangandMick e6ce16a4c2 [diffusion] feat: support TP for Flux.1.dev (#15666) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-25 18:00:44 +08:00
Xiaoyu Zhang de2f2880b5 [JIT sgl-kernel] Jit support per tensor quant (#15709) 2025-12-25 16:24:37 +08:00
Xiaoyu Zhang d77f3fccbf [Diffusion] Support peak memory record in offline generate and serving (#15610) 2025-12-22 21:21:21 +08:00
Xiaoyu Zhangandgemini-code-assist[bot] 42bff706df [diffusion] profiling: simplify --perf-dump-path JSON output (remove duplicate denoise steps) (#15537) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-21 12:56:46 +08:00
Xiaoyu Zhang 4b351f6b95 Apply new moe align block size kernel (#14134) 2025-12-21 10:13:32 +08:00
Xiaoyu Zhang 7fa4906f4f [sgl-kernel] Streamline kernel size report (Top 20 only) and clean up (#15552) 2025-12-21 10:00:47 +08:00
Xiaoyu Zhang bee8ac5b88 [diffusion] doc: add --perf-dump-path section to profiling doc (#15533) 2025-12-21 00:15:27 +08:00
Xiaoyu Zhang 8999ce754f [diffusion] perf: support zero-cost weight offload and overlap with compute for wan-series (#15511) 2025-12-20 22:52:40 +08:00
Xiaoyu Zhangandgemini-code-assist[bot] f3705b0115 [diffusion] doc: add doc for attention backends (#15408) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-19 22:10:29 +08:00
Xiaoyu Zhang 9a7641d7bf [diffusion] profiling: include per-denoising-step timings in perf-dump-path (#15397) 2025-12-18 18:43:33 +08:00
Xiaoyu Zhang 56d12b4aea Fix warp illegal instruction in kimi k2 thinking PCG (#15306) 2025-12-18 16:58:23 +08:00
Xiaoyu ZhangMickgemini-code-assist[bot]github-actions[bot]
 		6c4bf8a0be [diffusion] profiling: enhance trace export with gzip and integrity check (#15326) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-12-17 20:44:55 +08:00
Xiaoyu Zhang 533851fbcb [diffusion] ci: add flux2 tp2 test into ci to avoid breaking tensor parallel (#15237) 2025-12-17 20:44:21 +08:00
Xiaoyu Zhang 6292d97135 [diffusion] fix: fix pack qkv opt break tensor parallel (#15225) 2025-12-16 14:33:49 +08:00
Xiaoyu ZhangMickgemini-code-assist[bot]
 		4901693110 [diffusion] perf: support FFN pack gate and up proj for Z-Image(#15201) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-16 01:18:47 +08:00
Xiaoyu Zhang c0d94440b7 [diffusion] perf: support pack qkv for Z-Image (#15191) 2025-12-16 00:22:24 +08:00
Xiaoyu ZhangandMick 7bc8b1532e [diffusion] fix: fix AttributeError in _build_parallelism_config when accessing tp_group.device_group (#15196) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-15 23:41:06 +08:00
Xiaoyu ZhangandMick 92c29d43ac [diffusion] fix: cache dit with parallel (#15163) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-15 19:15:51 +08:00
Xiaoyu Zhang 4513f549ee [diffusion] fix: fix default resolution 720p width from 1080 to 1280 (#15058) 2025-12-15 09:16:47 +08:00
Xiaoyu ZhangandMick 64b5c3ab90 [diffusion] refactor: refactor fuse qkv with QKVParallelLinear linear (#15090) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-15 00:33:29 +08:00
Xiaoyu ZhangandMick e3f51e823e [diffusion] feat: add support for additional sampling parameters in video generation API (#15062) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-14 19:44:03 +08:00
Xiaoyu Zhanggithub-actions[bot]Mick
 		fdfabb7afc [diffusion] fix: tiny fix _templated_ring_attention bug (#15053) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-14 19:41:53 +08:00
Xiaoyu ZhangMickgemini-code-assist[bot]
 		0c23331e2e [diffusion] doc: add multimodal-gen profiling doc (#15069) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-13 22:26:20 +08:00
Xiaoyu ZhangMickgemini-code-assist[bot]
 		3e1e71575c [diffusion] docker: Tiny fix Docker Hub link in installation documentation (#14987) 
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-12-12 20:25:36 +08:00
Xiaoyu Zhang 12b7a4fab0 [diffusion] performance: refactor diffusion fuse qkv and apply to qwen-image (#14793) 2025-12-10 18:55:41 +08:00
Xiaoyu Zhang 53d170883a Add fuse_marlin_moe test to ci and add new ep test (#14686) 2025-12-09 20:17:38 +08:00
Xiaoyu Zhang 03b835e7d1 Refactor tuning block wise kernel and opt Qwen/Qwen3-VL-32B-Instruct-FP8 (#14141) 2025-12-08 09:24:58 +08:00
Xiaoyu Zhang ae6a6630e4 Add Expert Parallelism (EP) support for kimi-k2-thinking (#13725) 2025-12-07 20:28:57 +08:00
Xiaoyu Zhang e5135b73f4 Add CUDA kernel size analysis tool for sgl-kernel optimization (#14544) 2025-12-07 15:29:41 +08:00
Xiaoyu ZhangandMick 6d41791823 [diffusion] perf: add QKV fusion optimization for Flux models (#14505) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-12-06 20:44:16 +08:00
Xiaoyu Zhang 5347732219 [diffusion] fix: Fix profiler trace missing Python stack in diffusion pipeline (#14499) 2025-12-05 12:12:35 +00:00
Xiaoyu Zhang c5947ecd85 Opt moe align block size kernel (#14133) 2025-12-02 19:13:55 +08:00
Xiaoyu Zhang 3de09aadbc Add new moe wna16 marlin gemm (#14122) 2025-12-01 23:07:53 +08:00
Xiaoyu Zhang fa9021b21f fix: Increase FlashInfer workspace size for Qwen3VL models (#14173) 2025-12-01 17:54:23 +08:00
Xiaoyu Zhangandgithub-actions[bot] 9c80072845 Add peak output tokens per second in bench_serving (#14165) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-12-01 17:47:54 +08:00
Xiaoyu Zhangandgithub-actions[bot] 407cb3ce1e [CI tiny fix] Enhance robustness of vision chunked prefill test with ROUGE-L metric (#13793) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-11-25 15:41:14 +08:00
Xiaoyu Zhang ecefc7904f [sgl-kernel Code Clean] Remove useless lightning_attention kernel (#13819) 2025-11-24 18:26:25 +08:00
Xiaoyu Zhang 9ea1953331 [Doc] Refine fused_moe_triton configs doc (#13820) 2025-11-23 19:09:41 -08:00
Xiaoyu Zhang b964ce61d6 [DeepEP] Add SGLANG_DEEPEP_BF16_DISPATCH env var in Normal mode (#13787) 2025-11-23 17:32:43 +08:00
Xiaoyu Zhang a34d3abb54 [Clean code] Compressed_tensors_moe code clean (#13719) 2025-11-21 18:15:46 +08:00
Xiaoyu Zhang bfcf15a129 [opt kimi k2 4 / n] Delete useless pad kernel in sgl_moe_align_block_size (#13587) 2025-11-21 13:16:42 +08:00
Xiaoyu Zhanggemini-code-assist[bot]github-actions[bot]
 		fb04d43428 [kimi k2 thinking] Avoid useless torch.zeros_ (#13596) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-11-21 13:15:27 +08:00
Xiaoyu Zhang dc69462456 [CI fix] Fix image download failures in VLM CI tests (#13613) 2025-11-20 11:18:06 +08:00
Xiaoyu Zhang 820e13c9c1 [opt kimi k2 3/n] opt kimi_k2 moe_fused_gate kernel (#13374) 2025-11-18 15:36:31 +08:00
Xiaoyu Zhang 7b44526038 [Ci tiny fix] Lower score threshold in evaluation test (#13443) 2025-11-18 00:45:20 +08:00
Xiaoyu Zhang 8b5e2c5368 [Tiny fix] Fix bench_speculative.py run bug (#13416) 2025-11-17 18:58:19 +08:00
Xiaoyu ZhangandYuan Luo 95f43669b5 [2 / 2] apply sgl-kernel weak_ref_tensor (#12978) 
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
 2025-11-16 21:24:35 +08:00
Xiaoyu Zhang 50691d7b49 [opt kimi k2 2/n] apply kimi k2 thinking moe_fused_gate (#13332) 2025-11-16 21:20:08 +08:00
Xiaoyu Zhang 1d3d42bda0 [opt kimi k2 1 / n] Add kimi k2 moe fused gate (#13287) 2025-11-15 17:14:19 +08:00
Xiaoyu Zhang a1cb717d0b Opt kimi_k2_thinking biased topk module (#13150) 2025-11-12 16:52:06 -08:00
Xiaoyu Zhang fe92d4d88e [CI] Auto format code (#13053) 2025-11-10 23:30:29 -08:00
Xiaoyu Zhang 9caca6a45c [PieceWise CUDA Graph] Support awq/gptq model in piecewise cudagraph (#12518) 2025-11-11 11:56:15 +08:00
Xiaoyu Zhang f18ec927f3 fix tuning_fused_moe_triton_sep tool per_channel_quant bug (#13027) 2025-11-11 10:33:54 +08:00
Xiaoyu Zhang 547de8c774 [1 / 2] register weak_ref_tensor in sgl-kernel (#12999) 2025-11-10 22:12:59 +08:00
Xiaoyu Zhang 05559a4a90 Support hidden_dim % 4 == 0 in per_token_quant_fp8 (#12883) 2025-11-10 17:13:14 +08:00
Xiaoyu Zhangxu-yfeiYongfei Xu
 		fc84b0730c [Refactor] Refactor fused_moe_triton tuning tools: extract shared utils, add EP/MLLM support, reduce overhead (#12440) 
Co-authored-by: xu-yfei <xu-yfei@users.noreply.github.com>
Co-authored-by: Yongfei Xu <xuyongfei.xyf@antgroup.com>
 2025-11-06 20:54:42 +08:00
Xiaoyu Zhang 95191ebdca Migrate weak_ref_tensor to sgl-kernel (#12505) 2025-11-02 10:55:39 +08:00
Xiaoyu Zhang d8fcbaa38d [CI Monitor] Fix ci_monitor perf analyzer bug (#12281) 2025-10-30 09:47:12 -07:00
Xiaoyu Zhang 04e5b6faa7 Revert "Triton fused_moe_kernel support ep moe tuning" (#12377) 2025-10-30 07:12:06 -07:00
Xiaoyu Zhang 52694b60da Triton fused_moe_kernel support ep moe tuning (#12343) 2025-10-29 23:16:09 +08:00
Xiaoyu Zhang 334543ff3b Add continuous_usage_stats support for streaming responses (#12241) 2025-10-29 10:01:23 +08:00
Xiaoyu Zhang d0cff78f54 [CI] Add ci monitor balance workflow (#11962) 2025-10-25 12:14:36 -07:00
Xiaoyu Zhang 8470133852 [b200] fix piecewise cuda graph launch bug (#12067) 2025-10-24 22:36:39 +08:00
Xiaoyu Zhang 8374a96e49 piecewise cuda graph support qwen3-moe (#11845) 2025-10-21 10:55:49 +08:00
Xiaoyu Zhang 984fbeb16b Revert "[CI Monitor] Ci monitor only deal with main branch in default" (#11846) 2025-10-19 22:06:40 -07:00
Xiaoyu Zhang 24ed3f32c0 fix(ci): Fix CI Monitor limit parameter and add CI Analysis to summary (#11832) 2025-10-19 18:08:34 -07:00
Xiaoyu Zhang 88a6f9dab5 bench_serving support PD Disaggregation (#11542) 2025-10-13 19:43:26 -07:00
Xiaoyu Zhang 8e51049f56 [CI Monitor] Ci monitor only deal with main branch in default (#11538) 2025-10-13 13:50:04 -07:00
Xiaoyu Zhang 6806c4e63e [CI monitor] Improve CI analyzer: fix job failure tracking and add CUDA-focused filtering (#11505) 2025-10-13 13:31:09 +08:00
Xiaoyu Zhang 6f16bf9d9d [Ci Monitor] Auto uploaded performance data to sglang_ci_data repo (#10976) 2025-09-29 16:17:27 +08:00
Xiaoyu Zhang 11965b0daf Fix sgl-kernel benchmark dead code (#11022) 2025-09-29 15:06:40 +08:00
Xiaoyu Zhang 2387c22b56 Ci monitor support performance (#10965) 2025-09-27 09:11:21 +08:00
Xiaoyu ZhangLianmin Zhengsglang-bot
 		05a3526654 Restruct gpu_memory_settings in a unify function and relax max_cuda_graph_bs (#10372) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: sglang-bot <sglangbot@gmail.com>
 2025-09-26 15:10:49 -07:00
Xiaoyu Zhang c4197e99bb [ci] add ci-monitor workflow (#10898) 2025-09-25 19:29:47 -07:00
Xiaoyu Zhang c1f39013b7 [ci feature] add ci monitor (#10872) 2025-09-24 23:16:29 -07:00
Xiaoyu Zhang c4e314f986 Restruct sgl-kernel benchmark (#10861) 2025-09-25 07:45:25 +08:00
Xiaoyu Zhang 37367da639 [fix CI] Fix logical condition in fused MoE layer for compressed tensor quantization (#10299) 2025-09-10 23:54:09 -07:00