minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
17,121 Commits 3 Branches 0 Tags
d8433868ce8f0e525648426c13c8df9e4f798af4
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Mick 51b27f747a [diffusion] feat: support loading pruned minimax h3 components natively (#36070) 2026-08-24 16:39:37 +08:00
Mick adc09a1f63 [diffusion] feat: support loading self-describing quanto int8 encoders (#36052) 2026-08-24 16:37:09 +08:00
Mick 5081ad5d4e [diffusion] feat: add per-component quantization overrides (#36084) 2026-08-24 16:35:48 +08:00
MickandClaude Fable 5 c8e1ddc707 [diffusion] feat: cache LoRA-merged weights in files the page cache can hold (#36062) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-24 14:18:55 +08:00
Mick 3fe18f13cd [diffusion] feat: add plain component weight overrides (#36086) 2026-08-24 13:33:23 +08:00
Mick 8df3b9eff9 [diffusion] feat: support loading mixed w4a8 text encoders (#36037) 2026-08-24 13:31:27 +08:00
Mick 7a7b655ddf [quantization] share bounded post-load device staging (#35180) 2026-08-24 11:48:30 +08:00
Mick 5ce700aee8 [diffusion] feat: infer LoRA alpha from safetensors metadata (#36082) 2026-08-24 11:46:18 +08:00
Mick 230c052ebc [diffusion] chore: reuse srt AutoRound for quantized DiTs (#36068) 2026-08-24 11:21:44 +08:00
MickandClaude Opus 5 f294d51a71 [diffusion] fix: fix a refit key error on mapped weights, and stop claiming strides the reload discards (#35832) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-24 11:11:35 +08:00
Mick 6ca872a11f [diffusion] chore: fetch metadata beside nested lora weights (#36057) 2026-08-24 11:02:23 +08:00
Mick 1a368eca1c [diffusion] optimization: reuse minimax h3 prompt refinement across outputs (#36027) 2026-08-24 11:01:21 +08:00
Mick f6fff25756 [diffusion] feat: support vae weight-file overrides (#36085) 2026-08-24 10:33:14 +08:00
Mick 2d84de5e69 [diffusion] feat: support loading serialized comfy w4a8 checkpoints (#36036) 2026-08-24 10:32:21 +08:00
Mick fee00a41db [diffusion] feat: add composable component weight path cli (#36078) 2026-08-24 09:31:08 +08:00
Mick 1c1c9d9b4e [diffusion] refactor: reuse srt quantization contracts and mxfp8 kernels (#36063) 2026-08-24 09:26:54 +08:00
Mick d1af3c8923 [diffusion] feat: support loading native diffusers miniMax h3 components (#36067) 2026-08-23 22:13:59 +08:00
Mick de6a1dbd7a [diffusion] feat: support hybrid conditioning for minimax h3 (#36080) 2026-08-23 21:51:25 +08:00
Mick 939c00a7e3 [diffusion] feat: support compact qwen3-vl conditioning for minimax h3 (#36076) 2026-08-23 21:25:11 +08:00
MickandClaude Fable 5 886e37a649 [diffusion] CI: guard the anonymous-host budget alongside peak VRAM (#36051) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-23 20:13:46 +08:00
Mick dd15fb57b5 [diffusion] feat: automatically infer comfy fp8 activation scaling (#36060) 2026-08-23 18:43:33 +08:00
Mick e3a008a9db [diffusion] UX: clean up startup and offload logs (#36034) 2026-08-23 16:50:25 +08:00
MickandClaude Fable 5 bd3cc97e7e [diffusion] CI: let the 5090 consumer case runs two warm requests on the full recipe (#36032) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-23 15:39:32 +08:00
Mick 97ae27893a [diffusion] feat: release a layerwise component's non-layer weights between uses (#35734) 2026-08-23 12:05:24 +08:00
Mick 70319a0881 [diffusion] feat: support loading serialized comfy convrot int8 native encoders (#36023) 2026-08-23 12:00:38 +08:00
Mick bbbcbf9418 [diffusion] fix: stabilize ltx-2.3 two-stage cold requests (#35997) 2026-08-23 10:04:00 +08:00
MickandClaude Fable 5 59cdc9de7d [diffusion] chore: re-home decode-dtype vae weights to a file-backed store (#35986) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-23 09:32:04 +08:00
Mick a36c0746b9 [diffusion] feat: support serialized comfy convrot int8 dits (#35994) 2026-08-23 09:29:59 +08:00
MickandClaude Opus 5 7d22b7a875 [diffusion] docs: add tuning guide for h3 on consumer-level gpu (#35816) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-22 23:48:40 +08:00
Mick 0b064e3739 [diffusion] comfyui: add a minimax-h3 node and a generic extra-fields passthrough (#35352) 2026-08-22 23:11:49 +08:00
Mick 453b98c490 [diffusion] feat: support single-file component weight overrides (#35979) 2026-08-22 23:09:35 +08:00
Mick 46cb12ab45 [diffusion] fix: fix hunyuan3d stale extension lock hangs (#35989) 2026-08-22 23:08:44 +08:00
MickandClaude Opus 5 db570fe619 [diffusion] chore: let the auto policy select h3's dit for layerwise offload (#35812) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-22 21:36:46 +08:00
MickandClaude Fable 5 61981e1fcd [diffusion] optimization: keep vae decoder weights in their decode dtype from load (#35967) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-22 21:31:06 +08:00
MickandClaude Opus 5 a8c16b2e55 [diffusion] feature: use the directory for the vae mapping gate (#35946) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-22 19:12:48 +08:00
Mick 489e605b35 [diffusion] feat: admit compatible quantized native encoders (#35962) 2026-08-22 18:51:01 +08:00
MickandClaude Opus 5 382343f860 [diffusion] optimization: transfer mapped layers through a courier thread (#35882) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-22 17:17:14 +08:00
Mick b391ef171f [diffusion] feat: load serialized bnb4 components with transformers (#35945) 2026-08-22 15:33:53 +08:00
Mick 5290327025 [diffusion] feat: resolve hub component subfolders (#35939) 2026-08-22 12:01:44 +08:00
MickandClaude Opus 5 0be2a209ac [diffusion] refactor: hand out pinned host memory per layer (#35867) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-22 09:37:31 +08:00
Mick b26695a26e [diffusion] feat: reject unsupported quantized component checkpoints (#35873) 2026-08-22 09:33:43 +08:00
MickandClaude Opus 5 22dafbcbd9 [diffusion] feat: keep a cpu-started vae weights on the checkpoint mapping (#35862) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-22 09:31:30 +08:00
Mick 8658d00764 [diffusion] feat: support loading peft lora (#35868) 2026-08-21 22:58:59 +08:00
Mick 5a46d657b7 [diffusion] refactor: resolve lora weight sources deterministically (#35774) 2026-08-21 21:05:40 +08:00
Mick 5206f11543 [diffusion] fix: stop the mapped-weight store from holding the parameter itself (#35813) 2026-08-21 21:02:21 +08:00
Mick 6127d1daee [diffusion] feat: allow offloaded weights stay on the checkpoint mapping (#35701) 2026-08-21 10:54:28 +08:00
MickandClaude Opus 5 e0cf75d9bd [doc] standardize diffusion cookbook model pages (#34247) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-21 10:25:40 +08:00
Mick 5bb981dce1 [diffusion] chore: read the cgroup this process is actually in (#35707) 2026-08-21 08:48:50 +08:00
Mick be373395b4 [diffusion] feat: support out-of-tree models and pipelines (#35713) 2026-08-21 00:33:34 +08:00
Mick 7f8f030000 [diffusion] feat: let every layerwise component be configurable (#35688) 2026-08-20 22:38:05 +08:00
Mick 82c6fc2db9 [diffusion] quant: support pruned safetensors checkpoints for minimax-h3 (#35418) 2026-08-20 19:34:14 +08:00
Mick 97efc0507c [diffusion] feat: plan pinned host memory against the cgroup cap not the machine (#35641) 2026-08-20 19:32:29 +08:00
Mick cf3813f4ce [diffusion] feat: add weight source reader (#35668) 2026-08-20 18:37:29 +08:00
Mick 17313cf4b2 [diffusion] CI: add minimax-h3 ref2va audio consistency coverage and guard peak vram (#35511) 2026-08-20 17:22:05 +08:00
Mick f1b9a1f42a [diffusion] feat: support unverified short edge instead of rejecting it for minimax-h3 (#35664) 2026-08-20 16:52:44 +08:00
Mick b8996a5ab2 [diffusion] fix: keep large vocab tables in host memory under layerwise offload (#35626) 2026-08-20 15:20:06 +08:00
Mick f744607567 [diffusion] UX: report where a component's weights are (#35618) 2026-08-20 13:05:14 +08:00
Mick db2eb47500 [diffusion] fix: keep cosmos3 T=1 fusion on blackwell only (#35612) 2026-08-20 12:13:04 +08:00
Mick a3e4592f89 [diffusion] CI: use canonical residency selector in nightly (#35615) 2026-08-20 10:50:29 +08:00
Mick 3b22f4f000 [diffusion] UX: reduce per-request log noise (#35614) 2026-08-20 10:49:54 +08:00
Mick ab203663c4 [diffusion] fix: reject unsupported modelopt checkpoint algorithms (#35182) 2026-08-20 09:13:24 +08:00
Mick 1f87d8f512 [diffusion] fix: stop reserving nccl device buffers for single-rank groups (#35538) 2026-08-20 09:04:35 +08:00
MickandClaude Opus 5 23f2320c95 [Docs] PaddleOCR-VL: update which stage of the pipeline this serves and show real output (#35458) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 21:27:58 +08:00
Mick e3445ed2bd [diffusion] fix: route quantized vae component repos safely (#35184) 2026-08-19 21:02:46 +08:00
Mick 29f5d1c7c3 [diffusion] fix: fix multi-group layerwise offload startup memory (#35509) 2026-08-19 20:32:31 +08:00
MickandYiqi Yang 73e5fa4724 [diffusion] refactor: gate native encoder quantized checkpoints (#35183) 
Co-authored-by: Yiqi Yang <yangyiqi8787@gmail.com>
 2026-08-19 16:18:36 +08:00
MickandClaude Opus 5 c0c87e0547 VLM: feed the packed qkv projection output to vision backends uncopied (#35336) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 14:39:52 +08:00
Mick a5c96362b6 [diffusion] chore: reuse shared checkpoint quant metadata resolver (#35174) 2026-08-19 13:36:55 +08:00
Mick 593b1a9b8a [diffusion] optimization: reduce minimax h3 mps memory pressure (#33880) 2026-08-19 13:35:01 +08:00
Mick 4cef72faee [diffusion] refactor: reuse srt qwen vision and text modules (#35006) 2026-08-19 10:12:44 +08:00
Mick baa2251847 [diffusion] chore: make --vae-tiling honest, fix the decode oom advice, gate nvfp4 on blackwell (#35353) 2026-08-19 09:27:54 +08:00
Mick ef490853bb quant: extract shared checkpoint quant metadata resolver (#35172) 2026-08-19 08:26:41 +08:00
MickandClaude Fable 5 e73201e462 [diffusion] feat: support cache-dit, cfg gating, attention backend override as per-request param (#35339) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-08-19 08:24:03 +08:00
MickandClaude Opus 5 77fc5c128e [perf] overlap page preprocessing, pack the vit, enable prefill CUDA graph for paddle-ocr (#35318) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 08:20:55 +08:00
MickandClaude Opus 5 3f26febaff [diffusion] fix: decouple encoder parallelism from the dit parallel layout (#34713) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-19 00:16:09 +08:00
MickandYiqi Yang d55f1c28e2 [diffusion] feat: load quantized H3 text encoder checkpoints (#34986) 
Co-authored-by: Yiqi Yang <yangyiqi8787@gmail.com>
 2026-08-18 09:10:54 +08:00
Mick d97b796c16 [diffusion] chore: reuse SRT CLIP encoder blocks (#35004) 2026-08-17 19:51:51 +08:00
Mick e9ad8102a2 [diffusion] chore: reuse SRT SigLIP in Pi0.5 (#34992) 2026-08-17 19:33:36 +08:00
Mick 0e178c3d22 [diffusion] chore: reuse srt siglip vision model (#34988) 2026-08-17 09:16:17 +08:00
Mick d3589a7251 [diffusion] CI: tighten NVIDIA perf baselines (#35016) 2026-08-16 20:54:50 +08:00
Mick 3d3194f6c3 vlm: cache kimi-k3 per-image processor artifacts (#34404) 2026-08-16 19:51:13 +08:00
Mick 968b355f12 vlm: streamline vision sdpa reshapes (#34991) 2026-08-16 19:05:53 +08:00
Mick 2ee0d38a85 [diffusion] chore: refresh docs, retire stale knobs, and fix nightly attribution (#34663) 2026-08-16 15:41:08 +08:00
Mick a54de989c8 [diffusion] chore: speed up minimax-h3 vae decode on 2×h100 (#34817) 2026-08-16 15:38:21 +08:00
Mick e9fe58139f [diffusion] refactor: unify component residency controls (#34736) 2026-08-16 11:24:48 +08:00
Mick d269a28b47 [diffusion] refactor: route minimax h3 vae attention through native backends (#34949) 2026-08-16 10:07:26 +08:00
Mick 4f9da62547 [diffusion] chore: use native hunyuan3d paint and delight models (#34980) 2026-08-16 10:03:48 +08:00
Mick d106e8b23a [diffusion] chore: use native ernie prompt enhancer (#34951) 2026-08-16 09:59:51 +08:00
Mick 19e3bd6391 [diffusion] chore: use native qwen3-vl vision encoder (#34945) 2026-08-16 09:58:46 +08:00
Mick eb6b773149 [diffusion] chore: use native qwen2.5-vl generation (#34896) 2026-08-16 09:57:39 +08:00
Mick 4beb157e87 [diffusion] doc: define native diffusion model integration contract (#34952) 2026-08-15 23:37:34 +08:00
Mick e331baaaa8 [diffusion] chore: scope attention backend fallback (#34891) 2026-08-15 22:00:46 +08:00
Mick a64791b312 test: restore GLM-4.1V nightly latency threshold (#34811) 2026-08-15 18:13:19 +08:00
Mick 35cefd1c51 feat: add safeguards for remote media URLs (#34892) 2026-08-15 18:12:15 +08:00
Mick 46d84f4b48 feat(cli): add extensible serve backend plugins (#34753) 2026-08-14 13:57:59 +08:00
MickandClaude Opus 5 969921b32d [diffusion] chore: track minimax-h3 in the nightly diffusion benchmark (#34655) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-13 15:12:26 +08:00
Mick e8c7dddfa0 [VLM] add content-addressed preprocessing cache infrastructure (#34398) 2026-08-13 14:09:07 +08:00
Mick 69bf601e3c fix: restore VLM nightly regression coverage (#34662) 2026-08-12 21:47:16 -07:00
MickandClaude Opus 5 a318e16956 [diffusion] feat: publish an index of nightly comparison runs (#34652) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-13 11:50:10 +08:00
Mick ad47dde65c [diffusion] optimize: fuse cosmos qk norm, rope, and kv packing (#34275) 2026-08-12 23:18:16 +08:00