Commit Graph
1824 Commits
Author SHA1 Message Date
WenhaoZhang 44fac689f5 [diffusion] optimization: decode json pixel lists to numpy at the entrypoint (#41895) 2026-10-04 22:20:20 +08:00
ba68306256 [diffusion] feat: let requests choose the video libx264 preset (#41797)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 16:22:36 +08:00
Mohammad Miadh AngkadandMohammad Angkad 0bdd0cc8dc [Deps] Upgrade the CUDA PyTorch stack to 2.14 (#38641)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-04 00:48:02 -07:00
Baizhou Zhang fccc61c1c5 [CP] Support DP x TP x CP for attention with interleave CP strategy (#42038) 2026-10-03 20:57:36 -07:00
sglang-botandsglang-bot 8afef58af4 docs: sync LMSYS SGLang blog cards (#41296)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-10-04 02:26:05 +00:00
Cheng Wan 10edb05e72 [Refactor] Let the remaining placement consumers read the parallel context (#42348) 2026-10-03 14:59:32 -07:00
aaf7ebe077 Advertise the KV-event replay endpoint in /server_info (#42274)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 04:42:33 +08:00
5b5d721239 [PD] Add opt-in prefill-complete decode KV allocation (#40703)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-03 21:03:09 +08:00
Cheng Wan 7be5e3473c [Refactor] Drop the reduction-skip mechanisms stage boundaries no longer use (#42312) 2026-10-03 02:21:55 -07:00
Cheng Wan b157d548bb [Refactor] Let stage boundaries complete every stage-output sum (#42301) 2026-10-03 02:17:48 -07:00
Richard WangandRichard Wang 482b062d28 [Feature] serve pplx-decider decision checkpoints on /v1/systemone (#42183)
Co-authored-by: Richard Wang <11150595+rwang5203@users.noreply.github.com>
2026-10-03 10:18:53 +08:00
MickandMick Qian 65c7425a56 [diffusion] fix: respect explicit residency over pipeline preload hints (#41622)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-03 09:54:21 +08:00
Zhang, Jiejing cbe070b6bf [Docs][AMD] Update GLM-5.2 MI355X daily image to 20260930 (#42278) 2026-10-02 15:39:07 -07:00
Alex O. P.andPo-Han Huang e667ab10d5 [diffusion] quantization: ConvRot INT8 online W8A8 for DiTs (sgl-kernel + convrot_int8_customkernel) (#38040)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-10-02 23:26:28 +08:00
Mohammad Miadh AngkadandMohammad Angkad e7d9c0299b [Docs] Keep the last CUDA 12 image tag fixed during install version bumps (#42201)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-02 00:17:13 -07:00
sglang-botandsglang-bot d61a9c28ba chore: bump docs install version to 0.5.21 (#42013)
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
2026-10-01 23:16:26 -07:00
MingyiandMick Qian b906cd3f45 doc: refresh README (#41967)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-02 12:11:33 +08:00
Kedar PotdarandClaude Opus 5.5 8acd36b56b [Docs] GLM-5.2 GB300 NVFP4: add env vars from InferenceX AgentX recipe (#42109)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 09:11:29 +08:00
cdadb1c79a [unified-memory] Token-major dense views for the unified memory pool (3/7) (#38592)
Co-authored-by: Caihua Li <caihua.li@bytedance.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
2026-10-01 17:09:38 -07:00
f17f7705a5 [AMD] [GLM-5.3-Flash] Enable FP8 and MXFP4 serving on gfx950 (#39273)
Co-authored-by: Byron Hsu <byron+per@periodiclabs.ai>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: long10024070 <long10024070@users.noreply.github.com>
Co-authored-by: Kevin Mi <45493463+kevin-mii@users.noreply.github.com>
Co-authored-by: Kevin Mi <mikevin920@yahoo.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: sunxxuns <126995791+sunxxuns@users.noreply.github.com>
Co-authored-by: Long Luong <long.luong2+personal@onemount.com>
2026-10-01 15:59:44 -07:00
Mohammad Miadh AngkadandMohammad Angkad 40e1bb0f36 [Docs] Add NVIDIA NVFP4 checkpoint to GLM-5.3-Flash cookbook (#41986)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-01 12:15:30 +08:00
MickandMick Qian 2ab1b537f8 [diffusion] docs: recommend docker for gpu deployments (#41966)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-10-01 09:52:31 +08:00
Yu-Yun Changandkk 6af651ea0c [AMD][Quark] Serve the Kimi-K3 MXFP4 checkpoint on ROCm (#40811)
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
2026-09-30 18:14:15 -07:00
Cheng Wan a9c97c9f69 [Feature] Add --attn-dp-size and deprecate --enable-dp-attention (#41818) 2026-09-30 13:34:18 -07:00
Kevin MiandKevin Mi 3a398442bf dsv4.1-amd: serve DeepSeek-V4.1 on gfx950 (#41308)
Co-authored-by: Kevin Mi <kevin.mi@radixark.ai>
2026-09-30 11:47:14 -07:00
jacky.cheng 47dcde70f4 [Docs] Add MI355X FP8 agentic recipe to the Qwen3.5 cookbook (#41849) 2026-09-30 17:44:19 +08:00
amote-i 964c45cf31 [NPU] [DOC]: add Kimi-K3 NPU PD disaggregation recipes (#41717) 2026-09-30 16:42:46 +08:00
24ee67d2fa Readme refresh (#41787)
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
2026-09-29 23:48:38 -07:00
Cheng Wan faa5064f6e [Refactor] Send producer-written residual streams through to_pp (#41753) 2026-09-29 21:11:02 -07:00
Cheng Wan 01b5f572b7 [Refactor] Rename layer boundary protocols, contracts and helpers (#41750) 2026-09-29 21:05:48 -07:00
zijiexiaandClaude Sonnet 5.5 f884231f5a [Docs] Add IQuestLab card logo to cookbook landing page (#41746)
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 12:47:21 -07:00
51ace48387 [Metrics] Add request-level TPOT histogram (sglang:request_time_per_output_token_seconds) (#40275)
Co-authored-by: Gilford Ting <gilford@modal.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 12:20:22 -07:00
8f2a9638f7 [metrics] Fix non-streaming TTFT by flushing the first output through detokenization (#38600)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-09-29 11:24:52 -07:00
MickandMick Qian dc1bd46802 [diffusion] feat: support bounded exact conditioning cache across native models (#40470)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-29 15:27:22 +08:00
eeecho c7be3e935b [SM120] Add optional FlashInfer PCIe-IPC all-reduce for switch-free hosts (#34528) 2026-09-28 23:08:44 -07:00
Cheng Wan 6dad152490 [Refactor] Document layer boundary contracts and integration (#41557) 2026-09-28 20:18:27 -07:00
Cheng Wan de9f850b36 [Refactor] Rename the module to layer_boundary (#41555) 2026-09-28 20:17:50 -07:00
Cheng Wan 701213c867 [Refactor] Retire the layer facade and simplify boundary internals (#41554) 2026-09-28 20:17:33 -07:00
Cheng Wan 9b89b96230 [Refactor] Migrate specialized decoder and overlap boundaries (#41553) 2026-09-28 20:17:17 -07:00
Cheng Wan 49b5e76e45 [Refactor] Construct independent decoder stage boundaries (#41552) 2026-09-28 20:17:00 -07:00
77091cea68 [Radix Cache] Sync Rust TreeCore and make it the default (#39627)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com>
2026-09-29 10:18:57 +08:00
WenhaoZhang 49d26ff0c2 [diffusion] model: support flux 3 action robot policies (#41066) 2026-09-29 10:17:19 +08:00
79cafec013 [Model] Add IQuest Q1 support and MTP draft (#41590)
Co-authored-by: zelong huang <yzhu@ubiquant.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: zelong518 <zelonghuang05@gmail.com>
Co-authored-by: chenwenxiaolive <cocoshirolive@gmail.com>
2026-09-28 17:21:31 -07:00
Stanislav PetrovandStanislav Petrov b95329a74d [Docs] Add GigaChat 3.5 and GigaChat 3.5 Reasoning cookbook pages (#41118)
Co-authored-by: Stanislav Petrov <stapetrov@sberbank.ru>
2026-09-29 08:16:22 +08:00
Zhang, Jiejing 26d7539799 [Docs][AMD] Update GLM-5.2 MI355X daily image (#41597) 2026-09-28 15:42:54 -07:00
MickandMick Qian 090439bf8f [diffusion] docs: consolidate Qwen-Image 2.1 guidance in its cookbook (#41540)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-28 20:04:32 +08:00
Baizhou Zhang 7ee7bef79d docs: add prefill context parallelism guide and design draft (#39354) 2026-09-28 00:26:02 -07:00
Yihao WangandMick Qian 8b2ca8ecc2 [diffusion] feat: support multiple task types for pipelines (#38762)
Co-authored-by: Mick Qian <mickqian@users.noreply.github.com>
2026-09-28 10:00:45 +08:00
78eee88113 [Spec] Add LiLiCorr: a candidate-lattice reranker for DFlash drafts (#37462)
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 15:36:48 -07:00
Thomas Wang b252aceffe [AMD] Update v4 cookbook for megamoe, fp8 kv attn, BCG (#41458) 2026-09-27 10:53:21 -07:00