Commit Graph
6055 Commits
Author SHA1 Message Date
Rita Brugarolas de2d01c8da [AMD] Fuse shared-expert append + DeepEP remap into one Triton kernel (#28450)
Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com>
2026-06-25 01:22:57 -07:00
Raiden MakotoandRaiden-Makoto 9215da2515 [AMD] [GLM5] skip redundant -inf pre-fill of HIP indexer MQA-logits (#28757)
Co-authored-by: Raiden-Makoto <Raiden-Makoto@users.noreply.github.com>
2026-06-24 23:21:39 -07:00
Liangsheng Yin c7734e6871 [Spec] Dissolve EagleDraftInputV2Mixin so spec-info dataclasses hold data only (#29220) 2026-06-24 18:08:30 -07:00
fzyzcjy 2b64fc7a2c Remove Req.extend_logprob_start_len field and make it pure (#27625) 2026-06-25 09:06:14 +08:00
Ma Mingfei 2c3f007a65 [CPU] optimize GDN prefill performance (#29117) 2026-06-25 09:04:34 +08:00
fzyzcjy 7002a37ea1 Inline extend_range accessors and remove the extend_input_len/fill_len properties (#27611) 2026-06-25 08:56:44 +08:00
fzyzcjy d0524d6433 Avoid scattered assignment of extend_input_len and fill_len by merging them into Req.extend_range (#27610) 2026-06-25 08:33:28 +08:00
fzyzcjy 563c3418a7 Avoid dual semantics of extend_input_len by computing the candidate on the fly (#27616) 2026-06-25 08:17:37 +08:00
Brayden ZhongandBrayden Zhong 5d4e63d49e Sync the changes in #23402 (#29063)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
2026-06-24 17:01:02 -07:00
Lianmin ZhengandRain Jiang fd87a85388 Convert SamplingParams to msgspec Struct (#29198)
Co-authored-by: Rain Jiang <96632942+rainj-me@users.noreply.github.com>
2026-06-24 13:42:45 -07:00
Kaixi d5e9176f65 [BCG][GLM5] perf: BCG support and prefill enhancements (#27053) 2026-06-24 13:13:35 -07:00
Lianmin Zheng d5c566e59b Extract profile request cleanups (#29098) 2026-06-24 11:22:58 -07:00
qiaozpandishandhanani 03773ae35b [HiCache] Add NIXL FILE cache cleaner (#28258)
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-06-24 08:13:55 -07:00
hirakunaramuka2andClaude Fable 5 7430c56b20 [Misc] Use logger instead of print() in utils/common.py (#29004)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-24 09:45:01 +00:00
Yuhao Yang 0fc815aa2c [CI] Temporarily disable openbmb MiniCPM tests (#29095) 2026-06-24 01:39:44 -07:00
jundu e04ed05193 [CI] reduce CPU CI scope with base-c suite (#28623) 2026-06-24 16:18:06 +08:00
Even Zhou 4da23fdc35 [NPU] removes deprecated pr testing files (#27859) 2026-06-24 16:06:45 +08:00
zijiexiaandClaude Opus 4.8 a707b2054d [CI] Fix pre-commit failures in MLX backend tests (#29141)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 15:56:52 +08:00
5f76736427 kimik2_detector fix the normal text detection before tool call. (#25071)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Rohit Harkhani <rharkhani@gmail.com>
2026-06-24 15:53:22 +08:00
Yuan Luoandluoyuan.luo 0df796473b [VLM] Qwen3-VL / Moss-VL ViT preprocessing optimizations (#28940)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-06-24 14:36:29 +08:00
Stella-17andxinyue.fan 6842335fcf [MUSA][24/N] CI:Fix LLM server smoke test (#28934)
Co-authored-by: xinyue.fan <xinyue.fan@mthreads.com>
2026-06-23 22:58:59 -07:00
toufupiand“toufupi” c394f812d1 [MLX] Fix Apple Silicon server startup; align MLX tests with upstream (#28770)
Co-authored-by: “toufupi” <“byte2016@outlook.com”>
2026-06-23 22:58:49 -07:00
Alex Tumanov 33373cbb12 [misc] Move bench_serving into sglang.benchmark (#28996) 2026-06-23 19:34:11 -07:00
Liangsheng Yin b448b08401 [misc] Move bench_offline_throughput into sglang/benchmark/ with a back-compat shim (#28747) 2026-06-23 18:37:52 -07:00
karverma-amdandsogalin_codegen 20b2817bdf [AMD] Enable BCG on ROCm + route aiter prefill via MHA during PCG/BCG capture for Kimi-2.5 (#27833)
Co-authored-by: sogalin_codegen <39478626+sogalin@users.noreply.github.com>
2026-06-23 18:22:41 -07:00
Polisetty V R K Jyothendra VarmaandMa Mingfei 5338e44483 [Intel GPU] fix triton-mla attention on XPU by limiting max_kv_splits to 8 which is default (#28646)
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-06-24 09:06:13 +08:00
b2c8f7a22e [AMD] Support triton backend decode context parallel for Qwen3.5 (#25090)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Hubert Lu <55214931+hubertlu-tw@users.noreply.github.com>
Co-authored-by: zhengyao <zayao@amd.com>
2026-06-23 17:35:56 -07:00
Liangsheng Yin 9ef1830701 [Scheduler] Extract DFlash prefill refill into a standalone MinFreeSlotsDelayer (#29089) 2026-06-23 17:04:55 -07:00
Jackey Hua f444b5897b [Spec][1/N] Decoupled speculative decoding: IPC protocol + cross-process request id + server flags (#27634) 2026-06-23 17:04:45 -07:00
Lianmin Zheng 34dd9c28ca [Refactor] Introduce sock_send/sock_recv wrappers for zmq IPC (#29012) 2026-06-23 15:54:36 -07:00
Liangsheng Yin c864c8d9c2 [misc] Move bench_one_batch into sglang/benchmark/ with a back-compat shim (#28687) 2026-06-23 14:48:35 -07:00
Liangsheng Yin 6c5f466023 [server_args] Reland FA4 page_size auto-force for combined --attention-backend fa4 (#28976) 2026-06-23 14:21:59 -07:00
Xinyuan Tong 0c6e8e9477 Expand parser auto detection coverage (#28449) 2026-06-23 12:26:37 -07:00
karverma-amdandCursor e0dc8b7137 [AMD] Fuse topk padded-token masking into a single Triton kernel (#28084)
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-23 11:15:12 -07:00
ed26a109ee [Fix] Return streaming logprobs when reasoning/tool parser is active (#28601)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-06-23 09:56:09 -07:00
Shangming Cai 0460f277b7 Fix manual chunked-prefill test to use req.fill_len after fill_ids refactor (#29032) 2026-06-23 17:52:26 +08:00
Liangsheng Yin ed0a62e4dd [Mem] Add KV-page double-free checks to the invariant checker (#27731) 2026-06-23 02:29:40 -07:00
Shangming Cai a65c68c1fb Revert "[CI] Fix flaky optimistic test by adding contention handling" (#29023) 2026-06-23 16:53:32 +08:00
ishandhanani e67b228d4c feat: session radix cache (#27058)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
2026-06-23 01:51:41 -07:00
ybyang 349a6af6b8 [HiCache] Fix hicache host memory leak by bounding PP-sync work_list (#28916) 2026-06-23 16:39:03 +08:00
Mohammad Miadh Angkad 7b1a20344c Re-enable SM90 FlashInfer allreduce fusion with safe backend defaults (#28789) 2026-06-23 01:29:19 -07:00
Liangsheng Yin 854c688121 [Spec] Unify decode KV-commit bookkeeping across spec-v2 workers (#28754) 2026-06-23 00:53:05 -07:00
Yuan Luoandluoyuan.luo abb0717174 [CI] Fix lint brought by #27527 (#28988)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-06-22 20:40:06 -07:00
Chetan Kumar Verma 6cd8d2869b Vectorize _create_custom_4d_mask in CustomQwen2Decoder (#27527) 2026-06-23 10:56:35 +08:00
62f7ffc492 feat: add Mooncake group semantics (#26574)
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
Co-authored-by: Teng Ma <sima.mt@alibaba-inc.com>
2026-06-22 19:38:58 -07:00
Michael 28d5627fd8 [AMD] register kv_canary + mock_model e2e tests to extra-a (1-gpu-small + 2-gpu-large) (#28850) 2026-06-22 19:06:31 -07:00
Terry-UVandhnyls2002 a17753e449 Fix EAGLE draft graph seq_lens_sum padding (#26880)
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-06-22 18:00:25 -07:00
Liangsheng Yin d70726a31a Revert "[server_args] fix FA4 page_size auto-force for combined --attention-backend fa4" (#28972) 2026-06-22 17:01:37 -07:00
Ting SUN e00703bb2a fix(frontend): return 400 for missing completions json_schema (#28090)
Signed-off-by: Ting Sun <suntcrick@gmail.com>
2026-06-22 15:47:05 -07:00
6c212a5d6b [server_args] fix FA4 page_size auto-force for combined --attention-backend fa4 (#28825)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
2026-06-22 15:35:10 -07:00