sglang

minke.yu/sglang

Fork 0

a9e804623e Allow EPLB manual test to use FlashInfer A2A (#30641) Lianmin Zheng 2026-07-09 03:46:07 -07:00
8e54517f02 [Feat][GLM5.2] Add DSA Cache Layer Split under Prefill CP (#29421) Shijin Zhang 2026-07-09 18:03:56 +08:00
336b64ecce [AMD] fix dsv4 indexer dtype dispatch on gfx950 (#29479) billishyahaoandHAI 2026-07-09 17:54:00 +08:00
078f06fbf4 [Refactor] Share chat encoding dispatch between serving and offline tools (#30623) Liangsheng Yin 2026-07-09 02:45:42 -07:00
c02b032da0 [fix] Repoint the prefetch-dispatch test at the loader's current config binding (#30631) Cheng Wan 2026-07-09 02:21:04 -07:00
d8791fb5e3 [docs] Add the sglang-runtime-context skill (#30494) Cheng Wan 2026-07-09 02:11:36 -07:00
1f15308dca [refactor] Retire the legacy config accessor and the remaining process singletons (#30493) Cheng Wan 2026-07-09 02:10:47 -07:00
122b3266a2 Add opt-in SGLANG_ROPE_CACHE_FP32 to keep RoPE cache in fp32 on non-CUDA (#29729) Chengze Fan 2026-07-09 02:10:39 -07:00
e703f9e566 [refactor] Adopt get_parallel() everywhere and close out the parallel wrapper surface (#30492) Cheng Wan 2026-07-09 02:09:39 -07:00
06eb1b1838 [refactor] Split the DP gathered-buffer state between flags.dp and ctx.forward (#30491) Cheng Wan 2026-07-09 02:09:21 -07:00
fef2128e19 [refactor] Add the per-forward flags tier: ctx.forward (#30490) Cheng Wan 2026-07-09 02:09:03 -07:00
65b14881c5 [refactor] Move the EP dispatcher and fusion-workspace manager state onto ctx.resources (#30489) Cheng Wan 2026-07-09 02:08:35 -07:00
bc5d376c2c [Bench] Add fixed-prompt mode and per-request spec accept length metrics (#30615) Liangsheng Yin 2026-07-09 02:06:04 -07:00
bd7e54d737 [AMD] Fix AITER custom all-gather CUDA-graph capture crash under torch_memory_saver (#30557) Zhiyao JiangandXinyu Jiang 2026-07-09 04:30:51 -04:00
d74619b373 [AMD] Enable JIT staged HiCache write-back and fix CPU-index crash (#28534) AMD-yanfeiwangandDuyi-Wang 2026-07-09 16:22:37 +08:00
61602b95fb [Fix] Prevent silent VLM server crash when /dev/shm is exhausted during multimodal feature transport (#30602) Liangsheng Yin 2026-07-09 00:44:42 -07:00
866ae6848f [PDD] Add true request retraction for PDD (#25372) Ata Fatahi 2026-07-09 00:33:01 -07:00
666a09fe2a Disable multi-threaded load by default when prefetch is on (#30146) Mohammad Miadh Angkad 2026-07-09 15:28:53 +08:00
64e2a73c80 [Fix] Serialize FanOutCommunicator queueing calls with a FIFO-fair asyncio.Lock (#30606) Liangsheng Yin 2026-07-09 00:04:43 -07:00
9e4483d725 [misc] Add unit test admission criteria to agent rules (#30608) Liangsheng Yin 2026-07-08 23:41:48 -07:00
fdf6c70e3a fix arm test_norm.py error (#30597) Ma Mingfei 2026-07-09 14:36:45 +08:00
0562ccb1a8 Move breakable CUDA graph back into model_executor/runner_backend_utils (#30586) Yuwei AnandClaude Fable 5 2026-07-08 23:19:16 -07:00
1c9eb6bb0b [DSV4] Fix draft SWA transfer for disaggregated MTP (#30461) weireweireandweireweire 2026-07-09 13:18:34 +08:00
69ddbf9ef6 [NPU] [DOC] fix model name error on Ascend NPU (#30577) amote-i 2026-07-09 11:28:29 +08:00
3b43df5b6d Support speculative decoding on CPU (#27862) Haotong ZouandValentine233 2026-07-09 10:27:09 +08:00
177c048c68 [Diffusion][CPU] Adding AMX optimizations for CPU platform (#28527) jianan-guandMa Mingfei 2026-07-09 10:26:22 +08:00
395a2201e4 [diffusion] rl: enable RL rollout path for LTX-2.3 post-training (#28926) WenhaoZhang 2026-07-09 10:24:06 +08:00
6bce72d968 [Intel GPU] xpu_piecewise: fall back to eager when PCG capture stream is unset (#30235)
Chetan Kumar Verma Claude Opus 4.8 gemini-code-assist[bot]Ma Mingfei
2026-07-09 07:51:43 +05:30
2b3d9ad375 [Tiny] Fix docstring in CP abstractions (#30579) Baizhou Zhang 2026-07-08 18:41:40 -07:00
0ffed946f2 [NPU] Add extra topk_weights input in deepep ll dispatch (#29480) cen121212 2026-07-09 09:23:29 +08:00
cf8f1df6e8 [AMD] add dedicated jit-kernel-benchmark-test-amd stage + register portable JIT benches (#30307) Michael 2026-07-08 18:09:05 -07:00
074bb928f0 Move template manager files under parser; update CODEOWNERS (#26052) Xinyuan Tong 2026-07-09 07:47:36 +08:00
bc607ff650 Enhance mechanical refactor proof construction and verification skill (#30483) fzyzcjy 2026-07-09 07:39:11 +08:00
6af1d5ff2d Enhance large class styles and code styles (#28978) fzyzcjy 2026-07-09 07:32:03 +08:00
937734d3ed ci(nightly): add force_baseline_update dispatch input for precision job (#30495) Xinyuan Tong 2026-07-09 07:27:41 +08:00
cc13e2eae7 [CI] Move piecewise CUDA graph (pcg) tests to nightly (#30563) Cheng WanandClaude Opus 4.8 2026-07-08 16:07:08 -07:00
8f9307736a [misc] Add CI-only guards for the FutureMap seq_lens relay (#30471) Liangsheng Yin 2026-07-08 15:29:53 -07:00
096551eed6 Make CUDA graph disabling PD-role-aware (prefill/decode) (#30409) cctry 2026-07-08 15:08:22 -07:00
07ef650ef7 [AMD] Fix GLM-5.2 MTP Quark excludes (#30265)
wangjiaxin99 zhaolin Raiden-Makoto HAI
2026-07-09 05:57:10 +08:00
cc7d6659fd feat(grpc): support disaggregated generation requests (#30440) Connor CarpenterandIshan Dhanani 2026-07-08 14:53:56 -07:00
ca8f15cd70 Fix FlashInfer A2A IMA by DP-synchronizing the decode graph bucket (#30242) (#30450) Lianmin Zheng 2026-07-08 14:34:49 -07:00
c55190a638 [AMD] Register 2 CPU-bound 1-GPU tests (phase_checker, scripted_runtime_core) for AMD PR CI (#30446) Michael 2026-07-08 14:04:23 -07:00
10e7f2925f [Fix] Chain the seq_lens publish event records so prebuilt seeding keeps the forward fence (#30435) Liangsheng Yin 2026-07-08 13:43:09 -07:00
47a6dfd708 [AMD] Register 3 ROCm-portable JIT kernel tests for AMD CI (#30212) Michael 2026-07-08 13:38:17 -07:00
45019b56ce [Bugfix] Map reasoning_effort=low to Nemotron-3 Super low_effort + warn on unsupported levels (#30463) Xinyuan TongandEazyReal 2026-07-09 03:19:44 +08:00
8d2b66fd90 Fix gfx95 bpreshuffle FP8 activation scale layout (#29275) sonle5andsunxxuns 2026-07-09 00:55:09 +07:00
fc378f843e [AMD] Temporarily reduce AMD scheduled test frequency to save resource (#30534) YC Yen-Ching Tseng 2026-07-09 00:30:59 +08:00
04e4fadff3 Use FP32 logits in MoEGate fallbacks (#30323) Mohammad Miadh Angkad 2026-07-08 23:43:45 +08:00
b8ca06fdad Fix zero expert routed ids for MoE backends (#30387) Xiaoyu Zhang 2026-07-08 21:23:25 +08:00
108a183f6b [mem_cache][6/N] refactor: move MHA host-pool into pool_host/mha.py (#30249) shuwenn 2026-07-08 20:12:52 +08:00
4c5fe42be4 [DSA] Fix IMA in fused top-k v2: write all output slots on tie overflow (#30512) Jimmy ShongandClaude Opus 4.8 2026-07-08 04:49:26 -07:00
3d96bb9721 [diffusion] chore: rename lingbot world v2 (#30518) HuangJi 2026-07-08 19:06:44 +08:00
042228a195 [NPU] [DOC] Remove unsupported options of features on Ascend NPU (#30504) amote-i 2026-07-08 17:35:06 +08:00
96368a5f77 [AMD] Fuse shared-expert sigmoid + bf16->fp32 cast into the MoE append kernel (3 kernels -> 1) (#28658) jacky.cheng 2026-07-08 17:34:20 +08:00
669b4bc72b [AMD] Gate stage-c on stage-b-test-1-gpu-large-amd (#30496) YC Yen-Ching Tseng 2026-07-08 15:41:21 +08:00
8a868f8c00 [NVIDIA] Allow modelopt_mixed quantization with flashinfer_cutedsl MoE runner (#30443) YAMY 2026-07-07 23:50:52 -07:00
fda87173ab Revert "Increase the KV cache pool when using indexShare by 15% (#30310)" (#30472) Mohammad Miadh Angkad 2026-07-08 14:48:35 +08:00
33c3dfd7e0 [diffusion] Enable breakable CUDA graph (BCG) for diffusion DiTs (#27436)
Xiaoyu Zhang BBuf Claude Opus 4.8 BBuf
2026-07-08 14:45:48 +08:00
c9303a08da Fix scheduler crash on prefill-unreachable decode abort (#29834) amote-i 2026-07-08 14:06:37 +08:00
7709a1f358 [refactor] ctx.resources: named slots, stream leases, and workspace buffer leases (#30348) Cheng Wan 2026-07-07 21:30:10 -07:00
b7cca0bf8f [refactor] Collect MoE and DP-attention runtime state into typed flag groups (#30347) Cheng Wan 2026-07-07 21:29:27 -07:00
be32c57598 [refactor] Read resolved config from server_args fields; retire the flags mirror tier (#30346) Cheng Wan 2026-07-07 21:28:34 -07:00
b14f7b4f75 [refactor] Move model-capability adjustments into the resolution pipeline (#30299) Cheng Wan 2026-07-07 21:26:55 -07:00
d4963f5c55 Fix prefill CUDA graph disabled for deeply-nested multimodal models (#30006) Rahul VijayaraghavanandMa Mingfei 2026-07-08 09:41:58 +05:30
455ab36eeb Increase the KV cache pool when using indexShare by 15% (#30310) Brayden ZhongandBrayden Zhong 2026-07-07 20:59:13 -07:00
c7ca332fb0 Fix DSV4 prefill large Triton recompilation idle across context lengths (#30255) weireweireandweireweire 2026-07-08 11:35:33 +08:00
fa278a762c Fix FA3 prefill CP NaNs (#30439) Baizhou Zhang 2026-07-07 20:16:09 -07:00
f3c3eea608 ci: make multi-GPU jit test hangs attributable from the CI log (#29925) Alison Shao 2026-07-07 19:56:32 -07:00
7bc343470f [Spec] DFlash: support pure-MLA targets with an fp8 KV cache (Kimi-K2.x-NVFP4) (#29218)
Thanhhao Hao Phan Claude Opus 4.8
2026-07-08 09:52:14 +07:00
db40fd83d2 [diffusion] model: support LingBot-World 2.0 (#30361) HuangJi 2026-07-08 10:42:50 +08:00
68901ba387 [diffusion] Support SP for Krea-2 (#29777) Yihao WangandClaude Opus 4.8 2026-07-07 19:39:00 -07:00
9bf122a455 [AMD] Run MI355X disaggregation Nightly Test with runtime checkout code mechanism (#30386) YC Yen-Ching Tsengandbingxche 2026-07-08 09:27:49 +08:00
9f5948c391 feat(mem_cache): add client-side metadata cache for HiCacheFile storage (#29716) Yuchen Tian 2026-07-07 18:21:46 -07:00
49109d4267 [Tiny] Fix Import Error for Pure TP config with flashinfer_mxfp4 (#30426) Yuwei AnandClaude Opus 4.8 2026-07-07 18:17:52 -07:00
fa185ed84d [diffusion] fix: fix z-Image accuracy (#29742) Chi McIsaacandMick 2026-07-07 18:08:32 -07:00
d7dcdf3efd [DSV4] perf: Make FP8 quant output tensor contiguous (#27926)
Kaixi liqichao chenbong
2026-07-08 02:40:59 +02:00
b363249423 [AMD] ci: run multimodal_gen unit suite on AMD (#30309) Michael 2026-07-07 16:02:58 -07:00
48ad6a83cf [DeepSeek-V4] Enable non-paged indexer by default for large prefill chunks (#30140) YAMY 2026-07-07 15:51:24 -07:00
631213c3bf Add DeepReinforce Ornith-1.0 to cookbook (#29404)
averyjones4 averyjones4 Zijie Xia Claude Opus 4.8
2026-07-08 06:14:37 +08:00
3d2e7cc601 [gRPC] Native server: launcher + HTTP + server args wiring (3/4) (#23508) Alex NailsandClaude Opus 4.7 2026-07-08 03:27:25 +05:30
801571e949 [spec decoding] support rejection sampling in multi layer eagle (#30303) Qiaolin Yu 2026-07-07 14:50:24 -07:00
60f502a4fd [AMD] Register 2 hardware-agnostic 1-GPU PR tests for AMD CI (#30207) Michael 2026-07-07 14:46:22 -07:00
090efa27a2 [AMD] Register 5 CI-verified 1-GPU kernel/attention unit tests for AMD PR CI (#30290) Michael 2026-07-07 14:44:45 -07:00
40a68521c9 [AMD] Fix DeepSeekV4 server cutlass error (#30374) Wang, FangYuan 2026-07-08 05:43:05 +08:00
bbc537035a [DSA] Re-enable fused top-k v2 for MTP: clamp padded-row seq_lens to >= 0 (#30378)
DarkSharpness ziyi.xu Claude Fable 5
2026-07-08 04:44:01 +08:00
6875df3378 [Cherry pick to release/v0.5.15] Fix NVILA weight loading (#30400) Brayden ZhongandBrayden Zhong 2026-07-07 13:21:55 -07:00
e2ea7aafad [Cherry pick to release/v0.5.15] Fix NVFP4 online quantization (#30397) Brayden ZhongandBrayden Zhong 2026-07-07 13:20:36 -07:00
d88644b430 docs: sync LMSYS SGLang blog cards (#30395) sglang-botandsglang-bot 2026-07-07 12:30:47 -07:00
0bf7ddb481 docs(install): add nightly install + docker tag guidance, and auto-bump version on release tag (#30308) zijiexia 2026-07-07 12:10:05 -07:00
2ad9a243f5 Size KV pool after CUDA graph capture (opt-in) (#30157) cctry 2026-07-07 12:05:01 -07:00
ead1e490b5 [Doc] Add LongCat 2.0 FP8 cookbook (#30320) Xiaoyu ZhangandZijie Xia 2026-07-08 02:48:13 +08:00
11cea29c90 [diffusion][cache-dit] add dual-transformer Cache-DiT adapter specs (#30150) Jzz1943andYihao Wang 2026-07-07 23:10:52 +08:00
e339c83f82 [Model] Support LongCat 2.0 FP8 (#30275)
Xiaoyu Zhang sunjiaqi11 Ke Bao
2026-07-07 19:51:12 +08:00
cfd3fdc54f [NPU] [DOC] Update features and mainstream models on ascend npu (#30370) amote-i 2026-07-07 19:03:43 +08:00
efdf02a38a [NPU]Add support --pre-warm-nccl (#30312) loading66 2026-07-07 17:17:29 +08:00
7fdc1cef17 [fix] Fix two trunk test regressions due to flexkv change (#29701) (#30372) Jialin Ouyang 2026-07-07 01:58:45 -07:00
946804e042 Disable FA3 sparse mask kernels by default (#30356) Baizhou Zhang 2026-07-07 01:46:44 -07:00
f32b4ecd26 [Docs] Use trtllm_mha for Qwen3.6 B300 (#29964) Mohammad Miadh Angkad 2026-07-07 16:44:01 +08:00
9ddea8d9ef [AMD] [MORI-EP] Skip LocalExpertCount kernel in decode graph when not recording (#30302) Rita Brugarolas 2026-07-07 01:07:07 -07:00
5e9032c527 [NPU]Modify LoRA heading in ascend_npu_support_features.mdx to specify Qwen model limitations. (#30358) qinsir5522 2026-07-07 16:01:51 +08:00

... 36 37 38 39 40 ...

Commit Graph Select branches Hide Pull Requests dsv41-pd dsv41-pd-visioncp main Mono Color

Commit Graph

Select branches

Hide Pull Requests

dsv41-pd

dsv41-pd-visioncp

main