sglang

minke.yu/sglang

Fork 0

0d0254c9de Fix port overflow in DP attention path when base port is near 65535 (#20260) Jan BernlöhrandClaude Opus 4.6 2026-06-08 09:39:29 +02:00
71a0b10462 Fix the _chunked_req_scheduled_last_iter flag with a content-based stash gate (#26938) fzyzcjy 2026-06-08 14:55:41 +08:00
f746e4a608 Fix fill_len asymmetric assignment statement in ignore-eos branch (#26999) fzyzcjy 2026-06-08 14:53:21 +08:00
9034c2f9ae Fix Req fill_len (fill_ids) having dual semantics by restricting to truncated/committed semantics (#26659) fzyzcjy 2026-06-08 14:52:48 +08:00
259a2da3e0 Refactor Req.fill_ids into full_untruncated_fill_ids + fill_len with equivalence (#26637) fzyzcjy 2026-06-08 14:52:18 +08:00
4201de11de Extract release_req and retract_all as module-level free functions (#26548) fzyzcjy 2026-06-08 14:51:45 +08:00
3197808283 Avoid calling filter_batch with chunked_req_to_exclude being things unrelated to chunked reqs (#26547) fzyzcjy 2026-06-08 14:51:21 +08:00
f5fdf9c5d8 Speed up dump comparator percentile computation using numpy (#26874) fzyzcjy 2026-06-08 14:50:31 +08:00
995e649190 Add parallel-rank dump filenames and pipeline-global layer remapping to dumper (#26850) fzyzcjy 2026-06-08 14:49:22 +08:00
3d2165a286 Fix dual-chunk sparse fallback index overflow (#27361) Han Yu 2026-06-07 23:15:37 -07:00
1aa5040c74 Update code owners (AMD) (#27530) HAI 2026-06-07 22:43:51 -07:00
d1777d1f6d Cookbook renovation (#26885) zijiexiaandClaude Opus 4.8 2026-06-07 22:04:54 -07:00
6365d6faee [spec] Misc defensive guards for EAGLE draft KV indexing (#27486) Liangsheng Yin 2026-06-07 21:27:46 -07:00
6c2770149b [diffusion] Fix native text-encoder loading for T5/UMT5 encoder-decoder models (#27432) Xiaoyu Zhangandxiaoyu.zhang 2026-06-08 12:23:16 +08:00
5bf7dd8e4a Update SGLang diffusion skills (#27496) Xiaoyu Zhang 2026-06-08 12:22:45 +08:00
bf7fb6b925 fix dflash rope config parsing for updated transformers (#27477) David Wang 2026-06-07 23:52:07 -04:00
acf65c1c68 [XPU CI] Pull prebuilt nightly image instead of building per-stage (#27282) ashwini rathiandClaude Opus 4.7 2026-06-08 08:38:41 +05:30
1c73ff8ad3 [AMD] Optimize gpt-oss-120B performance (#27063) kkandwunhuang 2026-06-08 11:06:30 +08:00
6bc3953b48 feat(agentic router): add sticky-session routing policy (#27394) Kangyan-ZhouandClaude Opus 4.8 2026-06-07 20:05:27 -07:00
12d47fa78b ci(xpu): push latest tag and use 7-char SHA in nightly image tags (#27299) Jitendra Patil 2026-06-07 21:49:54 -05:00
2d1856bf45 Support encoder_decoder on cpu_graph_runner (#10950) Cao E 2026-06-08 10:30:11 +08:00
303757ccd8 [Attn] Fix aiter MLA verify kv_indices under-alloc + shared assert_buffer_fits guard (#27485) Liangsheng Yin 2026-06-07 17:06:25 -07:00
f68c79675f Support topk > 1 tree drafting for mamba/hybrid-linear models on spec v2 (#27463) Liangsheng Yin 2026-06-07 17:04:09 -07:00
70db73afce [Spec] Clamp multimodal pad sentinels in spec-v2 draft prefill embedding (#27512) Liangsheng Yin 2026-06-07 17:00:49 -07:00
10d33bd77e [AMD] Enable Piecewise CUDA Graph for AMD GPUs (#22299) Hubert Lu 2026-06-07 16:28:20 -07:00
02be2e7189 [diffusion] support tp for ideogram4 (#27393) Mick 2026-06-07 23:14:58 +08:00
a07d813ec8 Revert "Fix TRTLLM target verify query metadata (#27473)" (#27494) Lianmin Zheng 2026-06-07 02:48:52 -07:00
db58e76c33 Add all_to_all_single to GroupCoordinator (#27492) Lianmin ZhengandMing Yang 2026-06-07 02:48:47 -07:00
5be0b0c8c0 Fix TRTLLM target verify query metadata (#27473) Lianmin Zheng 2026-06-07 02:46:01 -07:00
0a190d1c97 Fix PP is_fully_idle missing in-flight microbatches (#27446) fzyzcjy 2026-06-07 17:38:26 +08:00
14b8f98a21 Complete server warmup before scripted runtime scripts start (#27445) fzyzcjy 2026-06-07 17:37:37 +08:00
a39c428d3f [UnifiedTree][CI]: Reduce HiCache PP KL test concurrency to avoid decode OOM (#27483) Zhangheng 2026-06-07 17:12:42 +08:00
eab2e02fa0 [spec] Dedup draft kv_indices sizing into spec_utils helpers (#27475) Liangsheng Yin 2026-06-07 00:42:29 -07:00
5e2e0d5b49 [spec] Make spec_utils module-importable: type-only imports under TYPE_CHECKING (#27484) Liangsheng Yin 2026-06-07 00:01:32 -07:00
ff8b97406d [diffusion] optimize: precompute ideogram4 denoising metadata (#27443) Xiaoyu Zhang 2026-06-07 14:27:41 +08:00
0ce3db3c0a [Bug] Fix out-of-range token id crashing tp=1 VocabParallelEmbedding (#27482) Liangsheng Yin 2026-06-06 23:00:17 -07:00
80eee2d92c [Spec] Guard async-assert probes against None tensor (#27478) Liangsheng Yin 2026-06-06 22:14:00 -07:00
52a5c01eba [plugin] enable OOT platforms to provide custom quant configs (#25347)
DevashishLal-CB Devashish Lal Devashish Lal
2026-06-06 21:48:55 -07:00
857ecb2dbc [HiSparse & HiCache]Support mooncake store layer first layout (#27454) huangtingwei 2026-06-07 12:46:42 +08:00
52f221cce0 Fix Req array token-id concatenation (#26182) Mohammad Miadh AngkadandLianmin Zheng 2026-06-07 10:59:51 +08:00
066b4a2180 [core] Probe set_kv_buffer / set_mla_kv_buffer slot ids for OOB (#27459) Liangsheng Yin 2026-06-06 19:51:09 -07:00
fe548f36b0 [UnifiedTree]: Fix SWA admission budget under-counts HiCache load-back consumption (#27391) Zhangheng 2026-06-07 10:47:08 +08:00
e57323cae9 [mem_cache][4/N] refactor: extract MambaTokenToKVPoolAllocator into allocator/ (#27256) shuwenn 2026-06-07 10:46:29 +08:00
5da265de30 [NVIDIA] Fix FP8 gemm performance with fp16 models (MInimax-M2.5) (#22300) Trevor Morris 2026-06-06 19:45:00 -07:00
2c3e84affe [Diffusion] Enable Cosmos3 denoising profiling (#27439) Xiaoyu Zhang 2026-06-07 10:32:28 +08:00
4c8a022f38 Fix DeepSeek V4 DP reduce scatter when use attention DP + MoE TP (#27191) Yueming Yuan 2026-06-06 18:24:33 -07:00
5160f7914e Fix MLA EAGLE draft CUDA-graph kv_indices under-allocation for topk > 1 (#27460) Liangsheng Yin 2026-06-06 16:28:34 -07:00
032c9efb46 Enable async-assert invariant probes by default in CI (#27461) Liangsheng Yin 2026-06-06 16:23:21 -07:00
4b0f629082 [perf] reduce radix cache match overhead by changing the match algorithm (#27364) Qiaolin Yu 2026-06-06 15:40:28 -07:00
1c7acba579 [spec] Consolidate the per-decode KV alloc reserve into one helper (#27458) Liangsheng Yin 2026-06-06 14:58:50 -07:00
4989d6691d ci: show partition fit window as a date range in the step summary (#27457) Liangsheng Yin 2026-06-06 14:39:36 -07:00
9097647090 Route the eager forward path through the CUDA graph input-buffer registry (#27407) Cheng WanandClaude Opus 4.8 2026-06-06 14:35:53 -07:00
84ca0ffb8c Spec v2 tree drafting (topk>1) with page_size>1 (#26972) Liangsheng Yin 2026-06-06 12:00:27 -07:00
88a7b0fd30 Classify malformed-multimodal rejects as invalid_request (#27451)
Lianmin Zheng cctry cctry
2026-06-06 10:19:22 -07:00
bd7fea0740 [diffusion] Fix LingBot-World crash on camera control with ulysses>1 (#27437)
Xiaoyu Zhang BBuf Claude Opus 4.8
2026-06-06 22:02:29 +08:00
7c6f9542c7 [Diffusion] Avoid GPU syncs in UniPC scheduler (#27440) Xiaoyu Zhang 2026-06-06 22:01:41 +08:00
4e14b50c48 [NPU]Support torch_npu profiler patch API drift (#26356)
L4 leland17 OmX ronnie_zheng
2026-06-06 21:27:51 +08:00
280280ace9 [SPEC] fix: import copy module for eagle sampling info clone (#27390) shuwenn 2026-06-06 20:12:46 +08:00
9f28512737 [NPU] Update documentation for software version upgrades (#26731) Артем Савкин 2026-06-06 15:06:13 +03:00
99cf0a4399 Fix flaky test_self_e2e_pd_perturb (#27426) fzyzcjy 2026-06-06 19:31:08 +08:00
9a48bf75f5 Add GB300 base C CI suite (#27427) Baizhou Zhang 2026-06-06 02:27:30 -07:00
42fe025280 [HiCache] Fix the compatibility between PP and HiCache (L2). (#27285)
Chao Shi ybyang hzh0425 shangmingc 晟海
2026-06-06 16:57:23 +08:00
aa5213abb1 [debug] Register #27338 EAGLE draft kv_indices revert in pr_fix_toggle (#27428) Liangsheng Yin 2026-06-06 03:30:13 -04:00
26b9053dcc [Spec] Fix fa3 EAGLE draft-decode expand page_table scatter OOB for topk>1 + page_size>1 (#27360) Liangsheng Yin 2026-06-06 03:24:34 -04:00
e9dbbd19e9 [model] Apertus Tool/Function and Reasoning parser (#25100) EduardDurech 2026-06-06 09:04:31 +02:00
8c47b7678a [attn backend] clean legacy init_mha_chunk_metadata in trtllm_mla backend (#27403) Qiaolin Yu 2026-06-05 23:30:21 -07:00
163fafba51 fix test_qwen3_next_models flaky (#27419) Hanming Lu 2026-06-05 23:26:32 -07:00
f57f8a8afd Optimize Gemma4 H200 MoE and extend attention (#26588) Xiaoyu ZhangandBBuf 2026-06-06 14:14:25 +08:00
e513c13e2e Optimize ngram decode token table update (#24756)
Xiaoyu Zhang Codex BBuf
2026-06-06 14:13:45 +08:00
9da88e32e0 [Cohere2Moe] Enable flashinfer_trtllm NVFP4 fused-MoE via SigmoidRenorm routing (#27401) zijiexiaandClaude Opus 4.8 2026-06-05 22:49:20 -07:00
caeb449cd6 [Doc][CPU]Update Cookbook with Xeon support info (#27248) Zaili Wangandgemini-code-assist[bot] 2026-06-06 13:39:40 +08:00
38ae22e08c Nemotron perf changes (#26733) Brayden ZhongandBrayden Zhong 2026-06-05 22:31:46 -07:00
393d0e169e [Bugfix] Restore overridden HF config fields and support index_skip_topk_offset for DSA topk sharing (#27114) Yuxuan Zhang 2026-06-06 13:26:04 +08:00
aa55657e9e [AMD][WA] force to use gate_mode interleaved to fix tp2/tp4/tp8 acc issue (#27201) kkandwunhuang 2026-06-06 11:18:11 +08:00
bf66b7b6da [diffusion] model: support Ideogram4 NVFP4 (#27379) Mick 2026-06-06 11:14:28 +08:00
e8668508d1 [diffusion] optimize: optimize LingBot realtime sp cache path (#27383) Mick 2026-06-06 09:37:29 +08:00
58a05d3dd2 [CI] Isolate CUDA coredump dir per run to fix tracker mis-attribution (#27344) Liangsheng Yin 2026-06-05 21:31:22 -04:00
3030119ef7 [bugfix][AMD] AttributeError and warp mask bugs in DeepSeek V4 FP4 indexer (#27152) Xinyi SongandHaiShaw 2026-06-05 18:26:38 -07:00
25d8f431d1 [diffusion] optimize: cosmos3 fused qknorm rope (#27096) Chi McIsaac 2026-06-05 18:15:42 -07:00
bf4f2ccc78 Add scripted-runtime unit, core integration, and chunked-prefill tests (#27413) fzyzcjy 2026-06-06 09:08:35 +08:00
5a82db85f0 Add scripted-runtime KV-pool and lock-ref exhauster primitives (#27412) fzyzcjy 2026-06-06 09:07:40 +08:00
dd176387c6 Add scripted-runtime harness core and wire scheduler/IPC hooks (#27411) fzyzcjy 2026-06-06 09:07:01 +08:00
21201ef718 Add kv_canary PP self-test fixture and SWA divergence coverage (#27410) fzyzcjy 2026-06-06 09:06:19 +08:00
b3e4c204fd Don't write crash dump on graceful exit (#27405) cctry 2026-06-05 17:06:39 -07:00
6a3316dd1e [plugin] default device detection fixes for OOT platform plugins (#25337) DevashishLal-CBandDevashish Lal 2026-06-05 16:55:50 -07:00
e69cc07a60 Remove DeepSeek V4 release Docker workflow (#27404) Baizhou Zhang 2026-06-05 16:49:05 -07:00
2e7523ddbe fix(spec-dec): treat num_nextn_predict_layers=0 the same as absent for EAGLE3 drafts (#26726) ThanhhaoandHao Phan 2026-06-06 06:08:07 +07:00
29591594f5 Support Waterfill with dynamic EPLB (#27150) xutizhou 2026-06-06 07:01:16 +08:00
6b180959a8 [SPEC][5/N] feat: batchsize-aware support for adaptive speculative_num_steps (#24055)
maoyuhan 坤钧 alphabetc1 EanWang211123 shuwenn shuwenn
2026-06-05 18:43:02 -04:00
c9f582a272 [LoRA] Experimental fast LoRA path with experimental_sgl_trtllm MoE backend for FP8 and NVFP4 models (#27329)
Yanbin Jiang fzyzcjy Chunan Zeng Ethan Su
2026-06-05 14:45:59 -07:00
d381ec7997 [CI] Fix Nemotron nightly mixed precision checkpoints test (#27284) Brayden ZhongandBrayden Zhong 2026-06-05 13:51:26 -07:00
3b62286fca Reland "Support NextN = 2/4 in DSV32" (#27166) Brayden ZhongandBrayden Zhong 2026-06-05 13:43:28 -07:00
2c2a4f243a [Bug] Fix EAGLE draft CUDA-graph kv_indices under-allocation for topk > 1 (#27338) Liangsheng Yin 2026-06-05 16:22:04 -04:00
632a3d480e docs: add Tencent Hunyuan and Poolside cards to autoregressive cookbook (#27400) zijiexiaandClaude Opus 4.8 2026-06-05 13:17:13 -07:00
bf172c492a Cookbook for QAT (#27396) Khoa PhamandClaude Opus 4.8 2026-06-05 11:18:11 -07:00
57909f731b [PD] Fix KV cache corruption on abort by notifying ongoing prefill (#27372) Shangming Cai 2026-06-06 00:56:13 +08:00
86b9bf5812 [CI]: Fix CI Stage for 3fs backend test (#27388) Zhangheng 2026-06-06 00:38:03 +08:00
e4a7388187 feat(agentic router, 1/N): Add LoadBasedPolicy (#26480) Zhangheng 2026-06-06 00:13:27 +08:00
d01cf27b7d [diffusion] model: support Ideogram 4 FP8 (#27279) xlyandMick 2026-06-05 23:54:48 +08:00
e7f94d0d40 [Bug Fix] Fix activation.cuh JIT compilation failure on CUDA 13 due to template type/value mismatch (#26444) Michael_miaoande01448 2026-06-05 22:59:24 +08:00

... 49 50 51 52 53 ...

Commit Graph Select branches Hide Pull Requests dsv41-pd dsv41-pd-visioncp main Mono Color

Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main