sglang

minke.yu/sglang

Fork 0

7c02ca7882 cancel pr ci: cover closed-no-merge; widen workflows; rerun-test opt-in (#26035) Liangsheng Yin 2026-05-21 19:38:24 -07:00
4374789abf fix(mm): make multimodal data loading non-blocking to prevent health check stalls (#24751) longguoandYang 2026-05-22 10:08:03 +08:00
b2631a9a4d [NPU] Docs op performance optimize (#25830) McZyWu 2026-05-22 09:20:13 +08:00
16b3edc84f [diffusion] feat: enable warmup for sglang serve by default (#25988) Mick 2026-05-22 08:54:48 +08:00
f6d98a17ba [diffusion] optimize: reuse cached dynamic lora weights (#25893) Mick 2026-05-22 08:54:01 +08:00
7cf193fe1f feat: support HybridLinearKVPool in chunked prefix cache handling (#25753) nohup 2026-05-22 07:17:26 +08:00
d765dfd043 refactor(attn): init hisparse_coordinator before attn_backend; replace lazy property with init-time capture (#26012) Cheng WanandClaude Sonnet 4.6 2026-05-21 16:03:42 -07:00
c5251a98a9 feat(model_runner): remove pool/backend refs from ForwardBatch via ForwardContext (#25983) Cheng WanandClaude Sonnet 4.6 2026-05-21 14:01:49 -07:00
44ec2ee18d [core] Unify output_tokens_buf in FutureMap (#25922) Liangsheng Yin 2026-05-21 13:56:44 -07:00
c9a0e55eb5 [Spec] Polish FutureMap after #25879: rename callback, async guard, cleanup (#25962) Liangsheng Yin 2026-05-21 13:56:22 -07:00
17dadebd4e [Docs] DeepSeek-V4: switch H200 FP4 Pro to flashinfer_mxfp4, Flash Balanced too (#25923) zijiexiaandClaude Opus 4.7 2026-05-21 13:51:40 -07:00
1a85586738 [Fix]: Restrict Kimi-K2.5 shared-experts fusion to Quark MXFP4 checkpoints (#25974) Jimmy Shong 2026-05-21 13:07:45 -07:00
81d686d9fa Default MegaMoE to W4A8 for Max-Throughput recipe (#26004) Yuhao Yang 2026-05-22 02:54:16 +08:00
b765faee30 [MoE Refactor] deprecate forward_npu and NpuFuseEPMoE (#25678) Cheng WanandClaude Opus 4.7 2026-05-21 10:25:17 -07:00
a24c374f84 [lora] Remove synchronous .any().item() guard in LoRA MoE prefill path (#25531) Ethan (Yusheng) SuandCursor 2026-05-21 23:58:57 +08:00
ca9dc17be4 [diffusion] chore: adjust layer wise-offload strategy (#25930) Mick 2026-05-21 23:48:58 +08:00
049bb83134 [CI] bot-cherry-pick: surface created PR number/URL in job summary (#26001) Kangyan-ZhouandClaude Opus 4.7 2026-05-21 22:58:21 +08:00
32f996b75a Avoiding the problem of printing a large number of compatibility warn… (#25956) liuxianglong17 2026-05-21 22:10:23 +08:00
caa9f08294 [CI] Force-reinstall nvidia-cutlass-dsl-libs-cu13 last to avoid wheel-mix TypeError (#25958) Kangyan-ZhouandLiangsheng Yin 2026-05-21 22:01:42 +08:00
ac83d8a339 docs: delete deprecated args from npu supported features (#25995) amote-i 2026-05-21 20:20:29 +08:00
32352f7edf [CI] Fix bot-cherry-pick: use state == "MERGED" instead of invalid merged field (#25987) Kangyan-Zhou 2026-05-21 20:10:12 +08:00
fbebdd5105 [CI] Enable nixl disaggregation test for decode radix cache (#25990) Shangming Cai 2026-05-21 19:29:24 +08:00
2e0d2d4c18 [NPU][DOCS]Add best practice and benchmark result parameter description (#25875) loading66 2026-05-21 19:08:10 +08:00
64f21b1589 [CI] Improve bot-cherry-pick: accept PR number, require merged, explicit title (#25981) Kangyan-Zhou 2026-05-21 18:32:57 +08:00
8562d5ae94 [AMD] Relaxing Timeout for AMD stage-a (#25978)
YC Yen-Ching Tseng Cursor Agent Bingxu Chen bingxche
2026-05-21 17:32:51 +08:00
f66881f03c [NPU]Ascend NPU Performance Profiling Guide and Ascend NPU Operator Development Guide (#25384) jianzhao-xu 2026-05-21 17:32:25 +08:00
e72e3145a0 [AMD] Upgrade AITER (#25896) Bingxu ChenandCursor 2026-05-21 17:10:43 +08:00
baeac179f7 [Spec] Route seq_lens through FutureMap; drop verify_done.wait (#25879) Liangsheng Yin 2026-05-21 01:51:40 -07:00
19f55c0e6d [Refactor] major JIT kernel clean up for dsv4 (#25884) DarkSharpnessandClaude 2026-05-21 16:14:31 +08:00
1b3d8da827 cap API quota for runner-utilization / amd-ci-job-monitor (#25965) Liangsheng Yin 2026-05-21 01:13:37 -07:00
c3f9bc9818 Fix nixl mla key and backup skipping (#24376) hxie 2026-05-21 00:48:28 -07:00
b9ae8353d2 [NPU] Support model DeepSeek-OCR and DeepSeek-OCR-2 (#25257) 看海的人 2026-05-21 15:21:20 +08:00
190488e9a8 [NPU] Support chunk prefill for Qwen3.5/Qwen3.6 models (#25839) Liwansi 2026-05-21 14:44:25 +08:00
4ea8282cb7 [Revert] nvidia-cutlass-dsl[cu13] 4.5.1 -> 4.5.0 (#25938) Kangyan-Zhou 2026-05-21 14:36:56 +08:00
40faf44f7a [auto-detect] match Ring-2.6/Ling XML kv tool-call format via vocab signature (#25366) Xinyuan Tong 2026-05-21 07:34:52 +01:00
45cadc215f [AMD][CI] Clean up AMD nightly + pr-test workflows (#25266) Bingxu ChenandCursor Agent 2026-05-21 14:30:26 +08:00
b5b9c809e1 fix(model-gateway): rustfmt nightly in conversations/handlers.rs (#25947) ybyang 2026-05-21 14:29:51 +08:00
c4f14650b9 fix act fun for xpu (#23809) sushil Dubey 2026-05-21 11:32:21 +05:30
8fa56a0ab1 Fix FlashInfer A2A token cap sizing (#25907) Lianmin ZhengandJaewon 2026-05-20 23:01:28 -07:00
e8608bdcb5 Fix EPLB redundant experts with shared expert fusion and Waterfill (#25367) xutizhou 2026-05-21 13:58:08 +08:00
847cbada9c Support Gemma4 MoE NVFP4 (#25054) Charles Chen 2026-05-20 22:45:15 -07:00
888a8794ef [Fix] DSV4 cached_loc invalidated when SWA mapping is rebuilt (#25889) Cheng WanandClaude Sonnet 4.6 2026-05-20 22:38:12 -07:00
3a6de13cd8 perf(dsv4): add MHC token-count prewarm (#25810) YAMY 2026-05-20 22:22:41 -07:00
1ac3e33622 [diffusion] optimize: enable inference mode in pipeline executor (#25891) Mick 2026-05-21 13:20:24 +08:00
e56db8bd24 fix: use base GPU ID CUDA device for multimodal processor (#21191) Brilliant Hanabi 2026-05-21 13:18:37 +08:00
84ea47eb22 [CPU] Fix issues when running llama3.2-11B vision model with image tasks (#8666)
blzheng JieXin Liang Yineng Zhang jianan-gu
2026-05-21 13:09:18 +08:00
79b937aefb [Refactor] Encapsulate SWA loc translation inside SWAKVPool with per-batch cache invalidation (#25824) Cheng WanandClaude Sonnet 4.6 2026-05-20 21:26:32 -07:00
90efa9c83f [AMD] Fix AMD stage-a-test-small-1-gpu (#25932) YC Yen-Ching Tseng 2026-05-21 11:51:49 +08:00
791a2f057f Add overridable hooks for custom chat serving implementations (#25807) Randall LinandCursor 2026-05-20 20:21:25 -07:00
74c6294ba9 [BugFix][EPD]Fix Qwen3VLMoe encoder-only AttributeError (#25759) Zheng Wengang 2026-05-21 11:07:21 +08:00
a449ee4822 [Deps] Use cu13 extra for nvidia cutlass dsl (#25576) Mohammad Miadh Angkad 2026-05-21 10:31:27 +08:00
34479c19bd [XPU] upgrade triton-xpu version to 3.7.1 (#25730) jiayisunxandMa Mingfei 2026-05-21 10:29:20 +08:00
4868b92d47 [CI] Fix bot-cherry-pick auth: GITHUB_TOKEN for push, dedicated PAT for PR (#25926) Kangyan-ZhouandClaude Opus 4.7 2026-05-21 10:28:42 +08:00
a528eb7564 fix: rustfmt service_discovery.rs warn! line length (#25927) Cheng WanandClaude Sonnet 4.6 2026-05-20 19:25:11 -07:00
e603beab55 [NPU] Add Qwen3.5-397B-A17B best practice doc (#25594) silencejade 2026-05-21 10:02:12 +08:00
ddf3817924 Revert "[AMD]fix: use CUDA event for targeted draft-to-verify sync in… (#25917) Hanming Lu 2026-05-20 18:49:01 -07:00
cf1fd26d16 benchmark/lora: make number of LoRA adapters configurable (#25363) Pai Liu 2026-05-21 11:23:13 +10:00
643d44d699 [BugFix] fix(hicache): fix two slot-reuse races in DecodeKVCacheOffloadManager (#24226) Kevin Flansburg 2026-05-20 19:20:19 -06:00
c4a7d12092 Enable breakable CUDA graph for eagle (#25795) Lianmin Zheng 2026-05-20 18:02:04 -07:00
f9f82d238c fix deepseek v4 hisparse (#25646) huangtingweiandZhiqiang Xie 2026-05-21 08:28:38 +08:00
512d164916 Address overlap future token map by request-pool index (#25862) Liangsheng Yin 2026-05-20 16:34:00 -07:00
b7d0df4b6f [SMG] Support regular worker discovery alongside PD workers in IGW mode (#25294) grunerandAmit Gruner 2026-05-21 02:23:55 +03:00
9f2bc24b35 Fix/dsv4 flash eagle dummy ima (#25892) Kaixi 2026-05-21 00:41:28 +02:00
ce7141ef98 add git gemm warpper for dispatch_bf16_fp32_backend (#25860) BingjiaWang 2026-05-21 06:24:10 +08:00
371b6c9ea0 [Scheduler] fix chunked prefill not always being full (#25741) Hanming Lu 2026-05-20 15:03:11 -07:00
1a17d753f1 [perf] prepare_prefill_qkv hook + fp8 quantize jit kernel (#25460) Qiaolin Yu 2026-05-20 14:20:49 -07:00
dac78768f0 [RL][TITO] Preserve whitespace in reasoning parser outputs (#24251) Yuzhen Zhou 2026-05-20 12:45:09 -07:00
801d7e3eed [DSA] Make MQA logits free memory ratio configurable (#25859) YAMY 2026-05-20 12:27:16 -07:00
5e7bf73757 Fix bench_serving non-stream reasoning content (#25298) Ratish P 2026-05-21 00:11:46 +05:30
1f209b4433 Add support for generic num_tokens_per_bs in TARGET_VERIFY (#25681) jasonjk-park 2026-05-20 11:34:45 -07:00
61ac6792e6 Add DevashishLal-CB to CI_PERMISSIONS.json (#25908) Lianmin Zheng 2026-05-20 11:19:43 -07:00
33c57b8716 [Bug][RadixTree] Fix LRU list reference cycle leak in radix_cache (#25770) Mark Smithandzhangjiadong1@corp.netease.com 2026-05-21 01:06:05 +08:00
6e0b7f35ad [radix cache] pluggable RadixCache factory (--radix-cache-backend) (#25101) Jialin Ouyang 2026-05-20 10:05:04 -07:00
ccbbae00ea [codex] Reland Wan2.2 ModelOpt CI checkpoints (#25857) Xiaoyu Zhang 2026-05-20 22:15:25 +08:00
55ba03db6a [NPU]use triton split_qkvgate_gemma_rmsnorm_rope for Qwen3.5 and Qwen3_next (#23925) Liwansi 2026-05-20 20:22:10 +08:00
34d3e23232 spec_v2: consolidate seq_lens_cpu/sum maintenance into helper (#25818) Liangsheng Yin 2026-05-20 04:42:26 -07:00
9b005d3608 disagg prebuilt: drop dead prepare_for_extend shift (#25819) Liangsheng Yin 2026-05-20 04:39:47 -07:00
eccd5c8253 pr-test: schedule 3x -> 2x; fix extra gate skipped on schedule (#25872) Liangsheng Yin 2026-05-20 03:58:54 -07:00
1bd4f94598 [Test] Add fwd_occupancy sanity kit (#25886) Liangsheng Yin 2026-05-20 03:34:37 -07:00
47979fb252 [diffusion] fix: fix GLM-Image /v1/images/edits support (#25697) Chi McIsaac 2026-05-20 05:11:51 -04:00
614672fea5 [Test] Stage-a sanity kits; consolidate core/ + models_e2e/ tests (#25831) Liangsheng Yin 2026-05-20 01:58:48 -07:00
24d27c2035 [BugFix] Fix rid_to_state leak for aborted queued requests (#24070) Yuhong Guo 2026-05-20 16:32:44 +08:00
e99f87c974 fix: add missing distro dependency to runtime docker image (#25817) Matt Van HornandMatt Van Horn 2026-05-20 00:46:32 -07:00
044649c23a feat: Support flashinfer_cutedsl MoE runner with flashinfer alltoall backend (#22669)
Sam (Kesen Li)Trevor Morris Claude Opus 4.7
2026-05-20 15:36:26 +08:00
bdacb1be4d Update CODEOWNERS to replace 'nsa' with 'dsa' (#25861) Cheng Wan 2026-05-20 00:31:23 -07:00
a1843524a5 ci(sgl-router): add PR test workflow (pre-positioned for feature PR) (#25854) Kangyan-ZhouandClaude Opus 4.7 2026-05-20 15:20:03 +08:00
8131641bc6 [Refactor] Rename NSA → DSA: user-facing aliases, file/class/import rename (#25821) Cheng WanandClaude Sonnet 4.6 2026-05-20 00:18:04 -07:00
da6d549ab2 Update GLM-5 H200 FP8 (#25814) Faradawn Yang 2026-05-20 01:44:54 -05:00
52eebc82ae [Docs] MiMo-V2.5 cookbook: B200 benchmarks + multi-layer EAGLE acceptance profile + long-context reference (#25359) Xinyuan TongandXinyuan Tong 2026-05-20 07:15:21 +01:00
7fda3caea4 [AMD] test(sgl-kernel): seed RNG on ROCm in test_moe_topk_sigmoid to fix tie-break flake (#25356) Michael 2026-05-20 14:01:13 +08:00
0d3a94f643 [Bug] Correct Weight Offloader's Attribute Name for torch.nn.Parameter (#25786) Xiaobaoandgithub-actions[bot] 2026-05-20 13:28:59 +08:00
b7085d3860 [fp8] SM90 swap-AB scaled_mm dispatch (~1.16x kernel geomean, +5.8-18.5% end-to-end) (#25532) Yuan Luoandluoyuan.luo 2026-05-20 13:20:37 +08:00
052abcc0dd [Refactor] Pass PP start_layer via model constructor instead of forward_batch.token_to_kv_pool (#25825) Cheng WanandClaude Sonnet 4.6 2026-05-19 22:16:07 -07:00
a4b51d35ef Revert "[codex] Update Wan2.2 ModelOpt CI checkpoints" (#25845) Cheng Wan 2026-05-19 21:45:20 -07:00
0aedc5678b loader: yield filtered MTP weights lazily to avoid OOM hang on multi-layer EAGLE (#25748) Xinyuan Tong 2026-05-20 05:33:54 +01:00
af22390af7 [codex] Align diffusion skills with nightly Nvidia benchmarks (#25842) Xiaoyu Zhang 2026-05-20 12:18:05 +08:00
579fed2090 Reduce excessively long logs caused by transformer version updates. (#25737) liuxianglong17andAdarsh Shirawalmath 2026-05-20 11:39:03 +08:00
1fbee74fb6 [PD] Clean early abort logic in PD module (#25677) Shangming Cai 2026-05-20 11:17:42 +08:00
3b2178c412 [PD] Un-blacklist mooncake sessions when probe succeeds (#25287) Kevin Flansburg 2026-05-19 21:16:28 -06:00
5fe655bf26 vlm: fix shared memory bug of deep copy mm_items (#24231) mingyue300 2026-05-20 11:06:51 +08:00

... 56 57 58 59 60 ...

Commit Graph Select branches Hide Pull Requests dsv41-pd dsv41-pd-visioncp main Mono Color

Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main