minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
17,021 Commits 2 Branches 0 Tags
61981e1fcd43c91cf8f70669fa43e91f774c5116
Commit Graph
17021 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
MickandClaude Fable 5 61981e1fcd [diffusion] optimization: keep vae decoder weights in their decode dtype from load (#35967) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-22 21:31:06 +08:00
Shangming Cai 3c69a4c744 fix(test): unbreak test_kv_transfer_replica_metric after #35950 (#35974) 2026-08-22 19:23:46 +08:00
MickandClaude Opus 5 a8c16b2e55 [diffusion] feature: use the directory for the vae mapping gate (#35946) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-22 19:12:48 +08:00
Mick 489e605b35 [diffusion] feat: admit compatible quantized native encoders (#35962) 2026-08-22 18:51:01 +08:00
MickandClaude Opus 5 382343f860 [diffusion] optimization: transfer mapped layers through a courier thread (#35882) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-22 17:17:14 +08:00
siyuandliusy58 cb10ca16dd [FEAT] Weight Daemon abstraction (#33279) 
Co-authored-by: liusy58 <liusy58@smail.nju.edu.cn>
 2026-08-22 02:13:46 -07:00
Shangming Cai 5c03069d4b refactor(disagg): drop dead placeholder overrides in Common KV sender/receiver (#35950) 2026-08-22 15:45:18 +08:00
Shangming Cai 15a4398320 refactor(disagg): hoist duplicated _handle_staging_req into a mixin (#35948) 2026-08-22 15:35:55 +08:00
Mick b391ef171f [diffusion] feat: load serialized bnb4 components with transformers (#35945) 2026-08-22 15:33:53 +08:00
Mohammad Miadh Angkad 90354326c7 [VLM] Split Pixtral multi-image features before the CUDA IPC wrap (#35463) 2026-08-22 15:17:55 +08:00
Lianmin ZhengandYichao Fu 6fd0384d42 Make draft attention backends extensible (#35932) 
Co-authored-by: Yichao Fu <yichaofu@meta.com>
 2026-08-21 23:54:44 -07:00
Jianfei Wangandmiraclezqc af39ad9349 [Model] Complete dots.note.omni support with native encoders, video preprocessing, and MTP decoding (#33829) 
Co-authored-by: miraclezqc <dysania@pku.edu.cn>
 2026-08-22 14:19:14 +08:00
Zhiqiang Xie c35683fda0 [HiCache] Clamp tombstoned SWA locs in UnifiedSWAKVPool translation (#35933) 2026-08-21 23:08:56 -07:00
Jimmy ShongandClaude Fable 5 4cb5aebfe0 [docs] Re-measure the Qwen3.8-27B RTX 5090, RTX PRO 6000 and DGX Spark grids on 1cf2b8c (#35825) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-22 13:03:18 +08:00
Xiaoyu Zhang 96bfd2476c [diffusion] Enable SANA-Video breakable CUDA graphs (#35729) 2026-08-22 12:57:09 +08:00
Xiaoyu Zhang 83e9ece672 [diffusion] Fuse SANA-Video interleaved RoPE (#35695) 2026-08-22 12:56:45 +08:00
Mohammad Miadh AngkadandMohammad Angkad 3e096629cf [CI] Re-enable B300 jobs (#35607) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-22 12:34:21 +08:00
Mick 5290327025 [diffusion] feat: resolve hub component subfolders (#35939) 2026-08-22 12:01:44 +08:00
Shangming Cai ac179eec11 fix(disagg): PD transfer-failure injection was silently inert (#35890) 2026-08-22 11:51:16 +08:00
Zhiqiang Xie 0db2bdfec5 Fix buffer-mode HiCache load-back ownership races; add optional prefetch anchor lock (#35769) 
Signed-off-by: Zhiqiang Xie <zqx@meta.com>
 2026-08-21 20:20:36 -07:00
Alec SandAlec Solder fbafd1b123 Add sampling observer auxiliary output hooks (#35747) 
Co-authored-by: Alec Solder <alecs@fb.com>
 2026-08-21 19:28:18 -07:00
Ke Bao 5662c03363 Support CPU offload for mxfp8 KV cache (#35888) 2026-08-22 10:19:06 +08:00
R0CKSTARandAlex Nails d90318b3e2 [MLX] Upgrade to Torch 2.13/MLX 0.32+ and redesign the Torch-MLX tensor bridge (#32984) 
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
 2026-08-21 18:51:42 -07:00
Baizhou Zhang 3b5909de0e [DeepSeek V4] Add W4A4 MegaMoE server flag (#35918) 2026-08-21 18:44:18 -07:00
MickandClaude Opus 5 0be2a209ac [diffusion] refactor: hand out pinned host memory per layer (#35867) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-22 09:37:31 +08:00
Mick b26695a26e [diffusion] feat: reject unsupported quantized component checkpoints (#35873) 2026-08-22 09:33:43 +08:00
MickandClaude Opus 5 22dafbcbd9 [diffusion] feat: keep a cpu-started vae weights on the checkpoint mapping (#35862) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-22 09:31:30 +08:00
Khoa PhamandClaude Opus 5 0ee3749e07 [Fix] Read the granite sinks dtype from the exec bag, not the legacy global shim (#35921) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-21 18:13:18 -07:00
DarkSharpnessandClaude Opus 5 7fd5454335 [DSA] Route the ragged prefill top-k to the v2 kernel (#35175) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-21 16:59:41 -07:00
Baizhou Zhang 60ff1e33a5 [DeepSeek V4] Default FP4 checkpoints to FlashInfer MXFP4 MoE (#35919) 2026-08-21 16:48:58 -07:00
Rain Jiang 7d7ab4b5c6 Rainj me/rust server refactor2 (#35239) 2026-08-21 16:37:02 -07:00
zijiexiaandClaude Opus 5 fe8f9d7457 [Docs] Add --prerelease=allow to cookbook uv install commands (#35920) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-21 15:51:48 -07:00
Alison Shao 2440820528 docs: add website link to README header (#35210) 2026-08-21 15:50:52 -07:00
YAMY 834400705f perf: overlap Qwen shared expert with DeepEP routed experts (#34938) 2026-08-21 15:39:44 -07:00
Yanbin Jiang 7d893255c3 [Spec][LoRA] Support multi-adapter LoRA with EAGLE/NEXTN/DFLASH/DSPARK speculative decoding (#34337) 2026-08-21 14:21:53 -07:00
Hank HanandTRAE CLI 590b11a5ef [Runtime] Don't override CUDA_MODULE_LOADING (#35711) 
Co-authored-by: TRAE CLI <traecli@bytedance.com>
 2026-08-21 14:09:27 -07:00
sglang-botandsglang-bot 4d42deff0a chore: bump docs install version to 0.5.18 (#35911) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2026-08-21 13:24:03 -07:00
Xinyuan Tong 05c584c44f docs: add DSPARK speculative decoding option to Ling-3.0-flash cookbook (#35861) 2026-08-22 02:50:55 +08:00
Shangming Cai 729a050ea3 refactor(disagg): extract _all_reduce_polls helper (#35886) 2026-08-22 02:30:33 +08:00
Connor Carpenter c3735625de fix(grpc): derive choice count before normalization (#35778) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
 2026-08-21 11:29:52 -07:00
Mohammad Miadh AngkadandMohammad Angkad 61c2da42bb [Fix] Pass Anthropic thinking history as reasoning_content for custom chat encoders (#35480) 
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
 2026-08-22 02:22:11 +08:00
Davis Wertheimer 70983bd7db Add SGLang Granite SWA support via existing Granite models (#35794) 
Signed-off-by: Davis Wertheimer <davis.wertheimer@ibm.com>
 2026-08-21 11:13:15 -07:00
Shuwen Wangandispobock 0bdd28d487 [mem_cache] docs: add a layer map and placement rules (#35643) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-08-22 00:35:07 +08:00
Shuwen Wangandispobock a7ec6b97f7 Restructure mem_cache auto-labels by layer (#25122) 
Co-authored-by: ispobock <ispobaoke@gmail.com>
 2026-08-21 23:29:59 +08:00
Mick 8658d00764 [diffusion] feat: support loading peft lora (#35868) 2026-08-21 22:58:59 +08:00
li_maxandMick 0447ade326 [diffusion] fix: fall back to a component's default attention backend (#35796) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-08-21 22:56:44 +08:00
王鹤男 932f632158 [diffusion] fix: do not warn that the recommended short edge is unverified (#35745) 2026-08-21 22:56:03 +08:00
Shangming Cai a41da991c8 refactor(disagg): collapse duplicated branches in get_kv_class (#35847) 2026-08-21 22:11:31 +08:00
王鹤男 5ecd6d794d [diffusion] fix: fix quantized qkv scales and missing-param policy for minimax-h3 (#35740) 2026-08-21 22:10:50 +08:00
Shangming Cai 4f343abc13 refactor(disagg): remove unreferenced dead code (#35838) 2026-08-21 22:09:25 +08:00