sglang

minke.yu/sglang

Fork 0

6f85957ff2 [AMD] Fix aiter checkout (rocm dockerfile) (#26544) Bingxu Chenandyctseng0211 2026-05-28 14:25:01 +08:00
8ff66b707f feat: convert mm_hashes to str in encode_server for Mooncake key compat (#26487) Mike QiuandMike_Qiu 2026-05-28 14:16:29 +08:00
505f37a63d [AMD] force AITER checkout to bypass CSV CRLF/LF smudge dirty state (#26535) YC Yen-Ching Tseng 2026-05-28 14:05:53 +08:00
e60f799b40 Enable Kimi-K2.5 piecewise CUDA graph (#26382) Xiaoyu Zhang 2026-05-28 13:51:33 +08:00
5018e5c969 [CI] /rerun-test: support glob wildcard patterns (#26422) Alison Shao 2026-05-27 21:55:58 -07:00
9040feebd8 [Gemma4] Add test for MTP models (#24552) Khoa PhamandClaude Opus 4.7 2026-05-27 21:36:28 -07:00
b437a0d066 Fix PD decode radix cache double-counting cached_tokens (#25973) Jay Chun 2026-05-28 12:50:03 +09:00
9fea20a078 update XPU Dockerfile (#25174)
Yao Matrix MingxuZh Ma Mingfei
2026-05-27 19:58:09 -07:00
714fdd9723 Fix MiniMax-M2.7 on CPU (#25061) Chunyuan WU 2026-05-28 10:53:13 +08:00
21ba329dac [Xeon] CPU CI enhancement for Intel Xeon platforms (#24649) MingxuZh 2026-05-27 19:49:04 -07:00
81663cb5f1 [AMD] [CI] Register MI35x GSM8K nightly tests (#26478) Bingxu ChenandCursor 2026-05-28 10:31:43 +08:00
14c1bb2721 [Feat][LMCache] Support LMCache mp mode (#24089) Shaoting 2026-05-27 19:15:09 -07:00
421bda6d85 [Bug Fix] Remove H20 device check for FlashInfer AllReduce Fusion (#26470) jvzibro 2026-05-28 08:45:55 +08:00
eae03ce3b2 refactor(dsv4): route MHC prenorm through DeepGEMM wrapper (#26238) YAMY 2026-05-27 17:45:45 -07:00
68e5b4fdd6 [NemotronH] Fix weight-loading unit test broken by Puzzle support (#26522) Baizhou ZhangandClaude Opus 4.7 2026-05-27 17:44:45 -07:00
0abe6a85a5 Support NemotronHPuzzleForCausalLM (#24429) Netanel Haber 2026-05-28 02:12:44 +03:00
7c421ed3ec [CI] runner-utilization: count in-flight queue waits + per-job status/links (#26509) Alison Shao 2026-05-27 16:06:55 -07:00
561e54f803 Update kimi k25 launch command in cookbook (#26511) Qiaolin Yu 2026-05-27 16:04:04 -07:00
24bcb37efb [diffusion] fix: fix diffusion LoRA consistency cases (#26327) Mick 2026-05-28 06:29:35 +08:00
deaba74745 [AMD][DSV4] DSV4 MTP graph + sparse triton attn optimizations (#26383) +2
kk wunhuang Thomas Wang Xinyi Song HaiShaw amd-danli103 Lin, Soga Raiden-Makoto Hubert Lu yichiche@amd.com yctseng0211 Bingxu Chen
2026-05-28 06:23:35 +08:00
e06058ed62 [Kernel] Import flash_mla kernels from sglang kernel for deepseek v4 (#26499) Chunan Zeng 2026-05-27 14:32:44 -07:00
19663aafcd Support batch size > 1 when enable CP (#23269)
Shunkangz Shunkang Khoa Pham Baizhou Zhang
2026-05-28 05:11:17 +08:00
ddf0627254 [NVIDIA] [GDN] Add FlashInfer prefill support for SM100+ (Blackwell) (#22921) Kaixi Hou 2026-05-27 13:58:05 -07:00
14f81a67d9 chore: bump sglang-kernel version to 0.4.3 (#26421) sglang-botandsglang-bot 2026-05-27 11:31:11 -07:00
d6e1692410 Allow custom speculative algorithm to support disaggregation (#26195) jasonjk-park 2026-05-27 09:54:53 -07:00
a95b4e2e09 [Feature] WebSocket streaming audio input for ASR (#22848) Sam HandYihao Wang 2026-05-27 10:44:55 -04:00
034dd39189 Support KV events for UnifiedRadixCache (#26387) weireweireandZhangheng 2026-05-27 22:10:40 +08:00
5f8911183b [UnifiedTree]: Update Unified Radix Cache README (#26485) Zhangheng 2026-05-27 21:50:53 +08:00
d9d719b270 [npu] [bugfix] Add contiguous operation during quantized weight loading. (#26309) gjsheu 2026-05-27 19:55:58 +08:00
83d5f4604c [NPU]add decord2 for npu (#26308) zhaozx-cn 2026-05-27 19:48:16 +08:00
51840ca459 Add xutizhou as code owner for eplb directory (#26479) Cheng Wan 2026-05-27 03:32:35 -07:00
a1ebc4917a [NPU][DOCS]Add faq and feature Compatibilit (#26464) loading66 2026-05-27 17:48:47 +08:00
3afc80d781 [diffusion] Fix multi image input for GLM-Image (#26311) Makcum888eandronnie_zheng 2026-05-27 12:33:20 +03:00
98bc6f3c22 API Perf: Replace pydantic per-element validation with C loop validation (#26355) Jialin Ouyang 2026-05-27 02:04:07 -07:00
d44584e8d8 [AMD] [CI] Add GLM-5.1 MXFP4 TP2 accuracy gate (#26396) Jacob0226 2026-05-27 16:49:48 +08:00
bf5bc23431 [AMD] [CI] Add DeepSeek-R1-0528 FP8 HiCache GSM8K test on MI35x (#26395) Jacob0226andCursor 2026-05-27 16:48:38 +08:00
216ed270e5 refresh resolve_seq_lens_cpu comments (#26463) Liangsheng Yin 2026-05-27 00:51:43 -07:00
f70e604101 [diffusion] fix: fix diffusion serve warmup defaults (#26247) Mick 2026-05-27 15:43:47 +08:00
163b970127 [core] WAR barrier for overlap schedule buffer writes, without fwd occupancy cost (#26380) Liangsheng Yin 2026-05-26 23:58:32 -07:00
dea85c30f4 Add Ling_2_6 (#23837) ant-yyand得泽 2026-05-27 14:57:23 +08:00
d6032c04b6 [docs] Fix V4 Pro balanced recipe (#26451) Baizhou ZhangandClaude Opus 4.7 2026-05-26 23:50:26 -07:00
9060509214 [NPU] fix CI (#26390) Makcum888e 2026-05-27 09:45:42 +03:00
f32ca1e0e5 [AMD] AＭD CI - temporarily change to mi325 (#26443) YC Yen-Ching Tseng 2026-05-27 13:55:03 +08:00
21d0e74aff Disable torch.compile for NPU in speculative overlap utils (#26403) xdtbynd 2026-05-27 12:30:53 +08:00
0c34fc5ace [Misc] Update CI Permission (#26435) Baizhou Zhang 2026-05-26 19:57:17 -07:00
d45ee3f6c5 [HiCache] fix: Mooncake Dummy Client mode for hybrid Mamba models (#25278)
Teng Ma Cursor Agent Teng Ma
2026-05-27 10:29:52 +08:00
1051a8456f [core] Maintain req_pool_indices_cpu host mirror (like seq_lens_cpu) (#26425) Liangsheng Yin 2026-05-26 18:59:54 -07:00
6076066e38 Add mooncake_tcp transfer backend (mooncake over TCP) (#26346) Liangsheng Yin 2026-05-26 18:15:55 -07:00
737c6cd6d1 [XPU] Add registry mechanism for XPU CI tests (#25405) vikram singh shekhawatandMa Mingfei 2026-05-27 06:26:59 +05:30
87c3171aaa [CPU] Add support for Qwen3-vl and Qwen3-omni (#12662)
blzheng Ma Mingfei Xinyuan Tong
2026-05-27 08:56:09 +08:00
c317beda99 [diffusion] model: support a new model (#24994) Dawid Majchrowski 2026-05-27 02:51:03 +02:00
468c565168 Wire YARN rope_parameters through LFM2 and LFM2-MoE attention (#26187) Piotr Mazurek 2026-05-26 16:00:22 -07:00
6989fede3c Purge usage of pytorch named tensors (#25911) Joel Schlosser 2026-05-26 17:58:57 -04:00
1a05b511e4 dp: refactor idle batch logic (#25025) Yilong Zhaoandhappierpig 2026-05-26 14:22:25 -07:00
dd6f073377 Reland "[perf][spec decoding] Skip full-vocab softmax in EAGLE draft when topk == 1 (#26235)" (#26397) Qiaolin Yu 2026-05-26 14:14:48 -07:00
499eecce22 [NemotronH] V3 Omni wrapper: WeightsMapper + config round-trip (#25023) Serge PanevandYihao Wang 2026-05-26 13:34:05 -07:00
2b1e53c98d [RL] Fix FP8 skip matching for trailing-dot prefixes (#26287) Ziang Li 2026-05-26 13:30:08 -07:00
47617cc4df [CPU Doc]Add Xeon CPU info in Qwen3 Cookbook (#25971) Zaili Wang 2026-05-27 03:14:07 +08:00
0753182b50 chore: bump sgl-kernel version to 0.4.3 (#26414) sglang-botandsglang-bot 2026-05-26 12:10:16 -07:00
6afebc278a [docs] DeepSeek-V4 cookbook: note cu129 image for GB200 Pro DeepEP backend (#26413) zijiexiaandClaude Opus 4.7 2026-05-26 12:08:48 -07:00
b66f8e0b96 Sgl flashmla (#26132) Chunan Zeng 2026-05-26 12:00:23 -07:00
ec6f8d61f7 [Spec] Async-assert probes across EAGLE/MTP; zero tgt_cache_loc (#26335) Liangsheng Yin 2026-05-26 11:34:03 -07:00
6c8128650e [Bugfix] Fix flashinfer_cutlass MoE crash when intermediate_size_per_partition is not 16-aligned (#22627) Venkatesh GuduruandvguduruTT 2026-05-26 22:28:33 +05:30
38f32c38ab [UnifiedRadixTree]: Support L3 HiStorage framework (#26062) Zhangheng 2026-05-26 22:38:00 +08:00
c47f0e7cdd [PD] Fix top logprobs crash in prefill path (#26299) Jun Liu 2026-05-26 23:01:10 +09:00
c8c1aed5e9 [PD] Fix cross-rank queue divergence by gating metadata readiness before all-reduce (#26394) Shangming Cai 2026-05-26 21:59:42 +08:00
98eb84497d [PP] Skip PP output communication for pure chunked prefill batches (#26148) Yongfei Xu 2026-05-26 21:59:18 +08:00
a26913158b fix(ci): enforce legacy docs/ gate in Lint workflow (#26322) zijiexia 2026-05-26 05:06:54 -07:00
9409969fd5 Revert "[perf][spec decoding] Skip full-vocab softmax in EAGLE draft when topk == 1 (#26235)" (#26358) Michael 2026-05-26 17:47:52 +08:00
d9c82934c8 Extract Scheduler init methods and add skills to enforce the splitting requirements (#26271) fzyzcjy 2026-05-26 17:45:09 +08:00
48f3264807 [HiCache]: Check return code of cudaHostRegister (#26301) Chao Shi 2026-05-26 17:44:15 +08:00
d25a220fdb [AMD] Relaxing timeout for AMD CI (#26392) YC Yen-Ching Tseng 2026-05-26 17:25:05 +08:00
e958f4561f [feat] Support extra_buffer in Mamba2-based models (#15829) roikoren755 2026-05-26 11:03:29 +03:00
7e6e5efe51 Revert "fix(tool_call): normalize non-standard JSON Schema types in tool params" (#26379) Liangsheng Yin 2026-05-26 00:49:41 -07:00
dabdd91ef3 [EPD] Cross-request batching for image/audio encoder (#25964) Zhonghua Deng 2026-05-26 15:38:59 +08:00
d34d4d9f5f [GDN] Support SM100 CuTeDSL GDN Prefill Kernel (#26200) Yuan Luoandluoyuan.luo 2026-05-26 15:38:29 +08:00
64c7c6851b fix(tool_call): normalize non-standard JSON Schema types in tool params (#23476) Xinyuan Tong 2026-05-26 08:23:59 +01:00
137168539a [Perf][Moe]improve cutlass_moe_fp4 performance by using apply_router_weight_on_i… (#19493) chengchao23 2026-05-26 15:07:49 +08:00
3f5e2c7688 [AMD] Dsv4/pr2 compressor opt (#26208) +2
kk wunhuang Thomas Wang Xinyi Song HaiShaw amd-danli103 Lin, Soga Raiden-Makoto Hubert Lu yichiche@amd.com yctseng0211 Bingxu Chen
2026-05-26 14:54:40 +08:00
7c0fbc8c2e fix: fix fa3 cross-attention batched-decode for per-request varlen encoder (#25045) Shaojun Zhou 2026-05-26 14:26:15 +08:00
f0ba651d66 [Doc] Update pip install commands for Cuda12 (#26344) Baizhou Zhang 2026-05-25 22:28:27 -07:00
63a89bf1a9 [kernel] reuse wna16 marlin moe workspace (#26112) Mick 2026-05-26 13:19:13 +08:00
7ef06bfc06 GLM-4.7-Flash: standalone MLA impl and MLA NextN/MTP (#26088) Yuxuan Zhang 2026-05-26 13:17:39 +08:00
59cad671e2 Support DeepSeek V4 DeepEP Waterfill (#25391) xutizhou 2026-05-26 12:04:26 +08:00
3142278c5f [diffusion] feat: layerwise NVTX markers for Nsight Systems profiling (#25683) mispa-msandMick 2026-05-25 20:24:27 -07:00
1953565ba1 Signal CUDA coredumps to tracker issue (#26338) Liangsheng Yin 2026-05-25 20:21:19 -07:00
156d1af23a [Intel GPU] Fix incorrect KV-cache page table for local attention when page_size > 1 (#23757) Chetan Kumar Verma 2026-05-26 08:32:21 +05:30
29e245e6a7 [misc] Update permission (#26336) Baizhou Zhang 2026-05-25 19:50:06 -07:00
8f2a4e70f8 [SRT] minor: reuse req input id array for unpadded ids (#26232) Mick 2026-05-26 08:58:32 +08:00
8805f4cf16 Fail-fast on PD subprocess exit and scheduler exception (#26298) Liangsheng Yin 2026-05-25 16:42:50 -07:00
e7b12fe6fa Fix stale forward_metadata leak in DP attn unpadded idle batch (#26313) Ke Bao 2026-05-26 07:04:00 +08:00
2b9dd9c8b3 [FlashInfer v0.6.10] [RL] [DSv32] [GLM-5] Add --dsa-topk-backend and integrate FlashInfer and pytorch topk (#22851) Ziang Li 2026-05-25 13:08:03 -07:00
b13d3d18c6 Refactor HiCache stack dispatch into strategies (#26295) Ke Bao 2026-05-26 00:06:17 +08:00
2aa6995308 [CI] Enable EPD CI for EPD architecture enhancements (#26281) Shangming Cai 2026-05-25 23:52:58 +08:00
121cc09405 [diffusion] Add CFG gating for denoising (#25848) Xiaoyu ZhangandBBuf 2026-05-25 22:57:09 +08:00
85f9522e36 [diffusion] Cache fp32 layernorm params (#25847) Xiaoyu ZhangandBBuf 2026-05-25 22:56:39 +08:00
0801cc05ed [Diffusion][NPU] Disaggregation diffusion stages support for NPU (#25895) Makcum888eandronnie_zheng 2026-05-25 13:51:25 +03:00
3e67398a96 Zero req_pool_indices padding in cuda-graph populate (#26292) Liangsheng Yin 2026-05-25 03:29:05 -07:00
533ef41112 [Diffusion] Default NVFP4 backend to FlashInfer TRTLLM (#25523) Xiaoyu Zhang 2026-05-25 18:14:06 +08:00
c05756da7a [SRT] fix flashInfer allreduce fusion not used on blackwell (#26197) Mick 2026-05-25 18:07:15 +08:00

... 54 55 56 57 58 ...

Commit Graph Select branches Hide Pull Requests dsv41-pd dsv41-pd-visioncp main Mono Color

Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main