minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
16,954 Commits 3 Branches 0 Tags
3efa0574496b53984bad2d485573a9400e013bdc
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Mick 6127d1daee [diffusion] feat: allow offloaded weights stay on the checkpoint mapping (#35701) 2026-08-21 10:54:28 +08:00
MickandClaude Opus 5 e0cf75d9bd [doc] standardize diffusion cookbook model pages (#34247) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-21 10:25:40 +08:00
Mick 5bb981dce1 [diffusion] chore: read the cgroup this process is actually in (#35707) 2026-08-21 08:48:50 +08:00
Mick be373395b4 [diffusion] feat: support out-of-tree models and pipelines (#35713) 2026-08-21 00:33:34 +08:00
Mick 7f8f030000 [diffusion] feat: let every layerwise component be configurable (#35688) 2026-08-20 22:38:05 +08:00
Mick 82c6fc2db9 [diffusion] quant: support pruned safetensors checkpoints for minimax-h3 (#35418) 2026-08-20 19:34:14 +08:00
Mick 97efc0507c [diffusion] feat: plan pinned host memory against the cgroup cap not the machine (#35641) 2026-08-20 19:32:29 +08:00
Mick cf3813f4ce [diffusion] feat: add weight source reader (#35668) 2026-08-20 18:37:29 +08:00
Mick 17313cf4b2 [diffusion] CI: add minimax-h3 ref2va audio consistency coverage and guard peak vram (#35511) 2026-08-20 17:22:05 +08:00
Mick f1b9a1f42a [diffusion] feat: support unverified short edge instead of rejecting it for minimax-h3 (#35664) 2026-08-20 16:52:44 +08:00
Mick b8996a5ab2 [diffusion] fix: keep large vocab tables in host memory under layerwise offload (#35626) 2026-08-20 15:20:06 +08:00
Mick f744607567 [diffusion] UX: report where a component's weights are (#35618) 2026-08-20 13:05:14 +08:00
Mick db2eb47500 [diffusion] fix: keep cosmos3 T=1 fusion on blackwell only (#35612) 2026-08-20 12:13:04 +08:00
Mick a3e4592f89 [diffusion] CI: use canonical residency selector in nightly (#35615) 2026-08-20 10:50:29 +08:00
Mick 3b22f4f000 [diffusion] UX: reduce per-request log noise (#35614) 2026-08-20 10:49:54 +08:00
Mick ab203663c4 [diffusion] fix: reject unsupported modelopt checkpoint algorithms (#35182) 2026-08-20 09:13:24 +08:00
Mick 1f87d8f512 [diffusion] fix: stop reserving nccl device buffers for single-rank groups (#35538) 2026-08-20 09:04:35 +08:00
MickandClaude Opus 5 23f2320c95 [Docs] PaddleOCR-VL: update which stage of the pipeline this serves and show real output (#35458) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 21:27:58 +08:00
Mick e3445ed2bd [diffusion] fix: route quantized vae component repos safely (#35184) 2026-08-19 21:02:46 +08:00
Mick 29f5d1c7c3 [diffusion] fix: fix multi-group layerwise offload startup memory (#35509) 2026-08-19 20:32:31 +08:00
MickandYiqi Yang 73e5fa4724 [diffusion] refactor: gate native encoder quantized checkpoints (#35183) 
Co-authored-by: Yiqi Yang <yangyiqi8787@gmail.com>
 2026-08-19 16:18:36 +08:00
MickandClaude Opus 5 c0c87e0547 VLM: feed the packed qkv projection output to vision backends uncopied (#35336) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 14:39:52 +08:00
Mick a5c96362b6 [diffusion] chore: reuse shared checkpoint quant metadata resolver (#35174) 2026-08-19 13:36:55 +08:00
Mick 593b1a9b8a [diffusion] optimization: reduce minimax h3 mps memory pressure (#33880) 2026-08-19 13:35:01 +08:00
Mick 4cef72faee [diffusion] refactor: reuse srt qwen vision and text modules (#35006) 2026-08-19 10:12:44 +08:00
Mick baa2251847 [diffusion] chore: make --vae-tiling honest, fix the decode oom advice, gate nvfp4 on blackwell (#35353) 2026-08-19 09:27:54 +08:00
Mick ef490853bb quant: extract shared checkpoint quant metadata resolver (#35172) 2026-08-19 08:26:41 +08:00
MickandClaude Fable 5 e73201e462 [diffusion] feat: support cache-dit, cfg gating, attention backend override as per-request param (#35339) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-19 08:24:03 +08:00
MickandClaude Opus 5 77fc5c128e [perf] overlap page preprocessing, pack the vit, enable prefill CUDA graph for paddle-ocr (#35318) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 08:20:55 +08:00
MickandClaude Opus 5 3f26febaff [diffusion] fix: decouple encoder parallelism from the dit parallel layout (#34713) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 00:16:09 +08:00
MickandYiqi Yang d55f1c28e2 [diffusion] feat: load quantized H3 text encoder checkpoints (#34986) 
Co-authored-by: Yiqi Yang <yangyiqi8787@gmail.com>
 2026-08-18 09:10:54 +08:00
Mick d97b796c16 [diffusion] chore: reuse SRT CLIP encoder blocks (#35004) 2026-08-17 19:51:51 +08:00
Mick e9ad8102a2 [diffusion] chore: reuse SRT SigLIP in Pi0.5 (#34992) 2026-08-17 19:33:36 +08:00
Mick 0e178c3d22 [diffusion] chore: reuse srt siglip vision model (#34988) 2026-08-17 09:16:17 +08:00
Mick d3589a7251 [diffusion] CI: tighten NVIDIA perf baselines (#35016) 2026-08-16 20:54:50 +08:00
Mick 3d3194f6c3 vlm: cache kimi-k3 per-image processor artifacts (#34404) 2026-08-16 19:51:13 +08:00
Mick 968b355f12 vlm: streamline vision sdpa reshapes (#34991) 2026-08-16 19:05:53 +08:00
Mick 2ee0d38a85 [diffusion] chore: refresh docs, retire stale knobs, and fix nightly attribution (#34663) 2026-08-16 15:41:08 +08:00
Mick a54de989c8 [diffusion] chore: speed up minimax-h3 vae decode on 2×h100 (#34817) 2026-08-16 15:38:21 +08:00
Mick e9fe58139f [diffusion] refactor: unify component residency controls (#34736) 2026-08-16 11:24:48 +08:00
Mick d269a28b47 [diffusion] refactor: route minimax h3 vae attention through native backends (#34949) 2026-08-16 10:07:26 +08:00
Mick 4f9da62547 [diffusion] chore: use native hunyuan3d paint and delight models (#34980) 2026-08-16 10:03:48 +08:00
Mick d106e8b23a [diffusion] chore: use native ernie prompt enhancer (#34951) 2026-08-16 09:59:51 +08:00
Mick 19e3bd6391 [diffusion] chore: use native qwen3-vl vision encoder (#34945) 2026-08-16 09:58:46 +08:00
Mick eb6b773149 [diffusion] chore: use native qwen2.5-vl generation (#34896) 2026-08-16 09:57:39 +08:00
Mick 4beb157e87 [diffusion] doc: define native diffusion model integration contract (#34952) 2026-08-15 23:37:34 +08:00
Mick e331baaaa8 [diffusion] chore: scope attention backend fallback (#34891) 2026-08-15 22:00:46 +08:00
Mick a64791b312 test: restore GLM-4.1V nightly latency threshold (#34811) 2026-08-15 18:13:19 +08:00
Mick 35cefd1c51 feat: add safeguards for remote media URLs (#34892) 2026-08-15 18:12:15 +08:00
Mick 46d84f4b48 feat(cli): add extensible serve backend plugins (#34753) 2026-08-14 13:57:59 +08:00
MickandClaude Opus 5 969921b32d [diffusion] chore: track minimax-h3 in the nightly diffusion benchmark (#34655) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-13 15:12:26 +08:00
Mick e8c7dddfa0 [VLM] add content-addressed preprocessing cache infrastructure (#34398) 2026-08-13 14:09:07 +08:00
Mick 69bf601e3c fix: restore VLM nightly regression coverage (#34662) 2026-08-12 21:47:16 -07:00
MickandClaude Opus 5 a318e16956 [diffusion] feat: publish an index of nightly comparison runs (#34652) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-13 11:50:10 +08:00
Mick ad47dde65c [diffusion] optimize: fuse cosmos qk norm, rope, and kv packing (#34275) 2026-08-12 23:18:16 +08:00
Mick dc5f6c4883 [diffusion] optimize: stream and parallelize bit-exact video output saves (#34564) 2026-08-12 20:03:37 +08:00
Mick 9701cc138c [diffusion] optimize: optimize bit-exact h3 reference video ingress (#34563) 2026-08-12 20:02:49 +08:00
Mick b3bffef70a [diffusion] UX: suppress noisy worker startup warnings (#34512) 2026-08-12 17:53:45 +08:00
Mick 644d55ebfa [diffusion] feat: support native and peft minimax h3 loras (#34359) 2026-08-12 17:52:40 +08:00
Mick a9a355774a [diffusion] feat: support dynamically cpu offload components (#34391) 2026-08-12 11:38:27 +08:00
Mick 2be9773a21 [diffusion] doc: update cosmos3 edge and distilled cookbook (#34497) 2026-08-12 10:51:11 +08:00
MickandClaude Opus 5 81c88da1ab [diffusion] fix: nightly diffusion benchmark passes the retired --warmup flag (#34423) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-12 10:25:14 +08:00
Mick a2d723820e [diffusion] fix: fix model-driven dit layerwise offload auto policy (#34401) 2026-08-12 09:08:31 +08:00
Mick 8267d76c2c [VLM] replace deprecated image processor use_fast (#34175) 2026-08-12 00:14:07 +08:00
Mick dd8c5849af [diffusion] refactor: move dit execution capabilities to runtime models (#34249) 2026-08-11 18:20:43 +08:00
Mick aeab1de1de [diffusion] optimization: support cuda graph for Pi-0.5 prefix encoding (#34256) 2026-08-11 09:14:53 +08:00
Mick ba5183fe10 [diffusion] UX: fix CI server warmup progress logging (#34301) 2026-08-11 09:09:32 +08:00
Mick 418975ba64 [EPD] feat: pipeline owner-only multimodal preprocessing (#34206) 2026-08-11 09:08:05 +08:00
Mick ec9babe36c [diffusion] fix: fix h3 rank-local fsdp qkv loading (#34294) 2026-08-10 23:59:00 +08:00
Mick 3e2a26708b [diffusion] chore: expose architecture config at the dit runtime boundary (#34248) 2026-08-10 20:08:45 +08:00
Mick 8ba9385097 [diffusion] chore: optimize model weight loading (#34064) 2026-08-10 20:07:48 +08:00
Mick 443b62db57 fix(vlm): stream-order cuda-ipc feature pool lifecycle and streamline multimodal transport module (#33949) 2026-08-10 18:47:19 +08:00
Mick 955569a2dc [diffusion] feat: expose cosmos3 policies through the Action API (#34243) 2026-08-10 18:16:20 +08:00
Mick 169783d42f [diffusion] chore: make torch.compile opt-in for speed mode (#34173) 2026-08-10 10:22:16 +08:00
Mick c20e99bd22 fix(vlm): preserve Kimi-K3 GPU JPEG accuracy (#34163) 2026-08-10 09:42:52 +08:00
Mick 22e003580b [Kimi K3] optimize: preprocess cpu-transport images on the vision owner (#33921) 2026-08-09 16:13:20 +08:00
Mick db75dfe10f fix: always capture default prefill CUDA graph (#33352) 2026-08-08 19:24:49 +08:00
Mick cf2d4fd679 docs: clarify K3 VLM feature transport (#34099) 2026-08-08 19:23:00 +08:00
Mick d747bd052e feat(vlm): auto-select cuda vmm on multi-node mnnvl (#33936) 2026-08-08 16:00:58 +08:00
Mick db3898fec1 fix: avoid piecewise prefill graph for trtllm_mla (#32785) 2026-08-08 16:00:10 +08:00
MickandClaude Fable 5 24c84dfa68 [diffusion] doc: add parallelism overview (#33704) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-08 11:36:06 +08:00
MickandClaude Fable 5 a25c330eb1 [diffusion] feat: cross-node sequence parallelism (Ulysses x Ring) (#33327) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-08 10:56:28 +08:00
MickandClaude Fable 5 52afe87a08 [diffusion] fix: stop runai-model-streamer's rank-discovery collective from firing on independent per-rank loads (#33969) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-08 08:54:08 +08:00
Mick 5ca734fc3d [diffusion] UX: speed up tp and fsdp checkpoint loading (#33960) 2026-08-07 19:23:30 +08:00
MickandClaude Fable 5 13938fed3f [diffusion] feat: make ring admission a backend capability (#33928) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 17:55:44 +08:00
MickandClaude Fable 5 28b43bf693 [diffusion] perf: build qwen's masked varlen metadata host-side (#33954) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 17:55:15 +08:00
MickandClaude Fable 5 85d611a055 [diffusion] fix: scope the masked-path replicated guard to sp runs (#33953) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 15:45:50 +08:00
Mick 9aadacfc53 [diffusion] CI: exercise the default sp selection in CI (#33931) 2026-08-07 14:20:19 +08:00
Mick c2657cc4bf [diffusion] refactor: gate fast vae paths by quality (#33849) 2026-08-07 12:39:55 +08:00
MickandClaude Fable 5 6dc77e490d [diffusion] chore: route zimage and hunyuanvideo attention through USPAttention (#33923) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 12:36:23 +08:00
MickandClaude Fable 5 698f019a5a [diffusion] chore: derive h3 attention admission from backend capabilities (#33707) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 12:35:15 +08:00
MickandClaude Fable 5 1e08b865f9 [diffusion] feat: support K/V-gather style sequence parallel (CP-like) attention (#32667) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-07 09:39:28 +08:00
Mick 9ee658d4f6 [diffusion] CI: fix output-rank test fixture (#33878) 2026-08-07 09:33:50 +08:00
Mick 7195b8e4c7 [diffusion] refactor: validate and document spectrum controls (#33851) 2026-08-06 23:23:11 +08:00
Mick 44bde3911a [diffusion] fix: resolve IPC A2A peers from process groups (#33848) 2026-08-06 23:22:41 +08:00
Mick c212a6938c [diffusion] chore: retire released warmup and decoder flags (#33850) 2026-08-06 23:02:32 +08:00
Mick 183bd80add [diffusion] chore: centralize entrypoint API hygiene (#33845) 2026-08-06 21:53:40 +08:00
Mick 2132cdef16 [diffusion] chore: consolidate pipeline core hygiene (#33843) 2026-08-06 21:51:34 +08:00
Mick 45dfd80674 [diffusion] refactor: simplify disaggregation transport hygiene (#33844) 2026-08-06 21:50:32 +08:00
MickandClaude Fable 5 bfce378e5f [diffusion] feat: capture-safe pynccl all-to-all (#33775) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-06 15:41:05 +08:00