Commit Graph
100 Commits
Author SHA1 Message Date
113f6f080e [PD] Enter the custom mem pool once when allocating DCP pack buffers (#40284)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
2026-09-19 19:34:58 -07:00
df6424967a [docs] Add the NVIDIA NVFP4 export to the Qwen3.8-27B cookbook (#38611)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Jiminator <jimmysh341@gmail.com>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
2026-09-11 11:58:30 -07:00
zijiexiaandClaude Opus 5 4b7331fb77 Make the remaining DeepSeek-V4.1 NVIDIA cells start (#38861)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 11:04:53 -07:00
zijiexiaandClaude Opus 5 5caafd2118 Fix the DeepSeek-V4.1 reasoning example and mark the B300 cells verified (#38839)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-10 01:49:44 -07:00
zijiexiaandClaude Opus 5 69777c4d36 Add DeepSeek-V4.1 Flash cookbook (#38802)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-09 23:15:30 -07:00
zijiexiaandXinyuan Tong b5c9b68f03 [Kimi-K3] Recover the reply when the model skips the think channel (#37743)
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
2026-09-07 14:08:42 +00:00
zijiexiaandClaude Opus 5 e4008de757 Add MiniCPM5-2B cookbook (#38295)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 21:30:17 +08:00
zijiexiaandClaude Opus 5 3b64169f9d [Cookbook] Kimi-K3: add measured B300 1x8 Unified 8k/1k speed numbers (#37878)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 17:44:00 -07:00
zijiexia 6d34a4d3ce [Cookbook] Verify DeepSeek-V4 Flash Vision on GB300 (#37492) 2026-09-01 17:11:41 -07:00
zijiexiaandClaude Fable 5 0f18d389b4 [Cookbook] Verify DeepSeek-V4 Flash Vision balanced and high-throughput on B200 (#37468)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 13:09:14 -07:00
zijiexiaandClaude Opus 5 dc1ae02684 [Cookbook] Add the DFlash2 speculative option to GLM-5.3 (#37392)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 09:08:28 +00:00
zijiexia 6c72b49a57 Revert "[AMD] Add GLM-5.3-Flash recipes for MI300X, MI325X, and MI355X (#36608)" (#37380) 2026-09-01 01:25:13 -07:00
zijiexiaandClaude Fable 5 379e33d87e [Cookbook] Add NVFP4 options for DeepSeek-V4 Flash Official (0731) and Pro Official (0813) (#37351)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-01 08:03:39 +00:00
zijiexia 455232de6e [Cookbook] Enable DSpark on the DeepSeek-V4 Flash Vision low-latency recipes (#37301) 2026-08-31 16:43:54 -07:00
zijiexia 88cf5c9541 [Cookbook] Add DeepSeek-V4-Flash-Vision-Exp to the DeepSeek-V4 page (#37293) 2026-08-31 14:52:28 -07:00
zijiexiaandClaude Fable 5 2960d69622 [Cookbook] Hy4-Preview follow-ups: runtime-accurate recipes + released-model info (#36808)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-28 00:19:07 -07:00
zijiexiaandClaude Fable 5 1948b61ad4 [Cookbook] Add the Hy4-Preview model page (Tencent) (#36804)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 23:26:13 -07:00
zijiexiaandClaude Fable 5 43b5a57dbb [Docs] Feature GLM-5.3-Flash in the popular-models banner (#36784)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 21:56:10 -07:00
zijiexiaandClaude Opus 5 6ccfeb59bc cookbook: add a Speculative card to the GLM-5.3-Flash playground (#36740)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 16:03:59 -07:00
zijiexiaandClaude Opus 5 46a544e0a0 [Docs] GLM-5.3-Flash: point at compute-mamba-ratio for the KDA/KV pool split (#36719)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 12:16:35 -07:00
zijiexia 636a6f7dba cookbook: fix GLM-5.3-Flash speculative flag, size Hopper memory, record GSM8K (#36660) 2026-08-27 02:19:47 -07:00
zijiexiaandClaude Opus 5 fe8f9d7457 [Docs] Add --prerelease=allow to cookbook uv install commands (#35920)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-21 15:51:48 -07:00
21c88f8625 [diffusion] quant: support gguf (#35370)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
2026-08-20 15:46:34 +08:00
zijiexiaandClaude Opus 5 f019f0b064 [Docs] Feature MiniMax-H3 in the popular-models banner (#35068)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-17 02:09:26 +00:00
70e291b70f [Docs] Add GB300 cells and benchmarks for Qwen3.8-27B (#34863)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 15:42:50 +00:00
29c6be15a4 [Docs] Add Qwen3.8-27B cookbook page (#34860)
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-14 15:23:39 +00:00
zijiexiaandClaude Opus 5 463981922c [Cookbook] Add DeepSeek-V4-Pro-0813 (Pro Official) serving recipes (#34809)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-13 22:53:41 -07:00
zijiexia abdef3c38e [Qwen] Update Docker image tag for MI300X to v0.5.17-rocm700-mi30x-20… (#34770) 2026-08-13 15:59:42 -07:00
zijiexiaandClaude Opus 5 8e7c07fae7 [Docs] Add Qwen3.8 cookbook (#34587)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 08:07:46 -07:00
zijiexiaandClaude Opus 4.8 b819d2fb5b [Docs] Rename docs_new/ to docs/ (#32123)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-03 16:51:00 -07:00
zijiexiaandClaude Opus 5 c2dbdfd218 [Docs] Fix overlapping quality-profile table headers on MiniMax-H3 page (#33345)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-03 05:22:03 +00:00
zijiexiaandClaude Opus 5 21d932069b docs: drop unreachable inkling LoRA benchmark entry (#33251)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 23:58:35 -07:00
zijiexiaandClaude Opus 5 5efbb18a6f [docs] Rotate popular models on the landing pages, lead the Cookbook nav with Kimi (#32835)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 17:46:50 -07:00
zijiexiaandClaude Opus 5 3c9efaf3e1 [docs] Kimi-K3: widen the H200 High-Throughput recipe to 4x8 TP32/EP32 (#32834)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 17:18:56 -07:00
zijiexiaandClaude Opus 5 b79388f338 docs(cookbook): mark every Kimi-K3 cell in-progress; land Playground "Switch base" on the configurator (#32586)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-27 18:59:57 -07:00
zijiexia 8905cbd42f Fix MiniMax-M3 crash on ROCm by making its override fields resolvable (#31837) 2026-07-20 16:14:12 -07:00
zijiexiaandClaude Opus 4.8 0a2d3ca071 [cookbook] Inkling: add measured accuracy numbers to benchmark cards (#31823)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 22:43:35 +00:00
zijiexiaandClaude Fable 5 8f765bc1c9 [Docs] Inkling cookbook: mark B300/GB300 recipes verified, tune B300 MTP mem fractions (#31550)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 16:07:09 +08:00
zijiexiaandClaude Fable 5 40a3bd7659 [Docs] Mistral Medium 3.5 cookbook: replace stale day-0 dev images with latest (#31507)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 20:19:01 -07:00
zijiexiaandClaude Fable 5 68f4de162d [Docs] Remove Inkling H200 LoRA BF16 cookbook command (#31489)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-16 19:57:37 +00:00
zijiexia 1b9f228838 [Docs] Playground: migrate CP knob to canonical prefill-CP flags, align gating with runtime semantics (#31411) 2026-07-16 01:07:34 -07:00
zijiexiaandClaude Fable 5 7e0b29ac03 docs: add VLA card image to cookbook overview (#31132)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 07:08:49 +00:00
zijiexiaandClaude Fable 5 96c2ebc58b [docs] Note the default dsa-topk-backend on all DSA-model cookbook pages (#31124)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 00:04:47 -07:00
zijiexia 50ed4c011f Remove legacy Sphinx docs/ and finish the Mintlify cutover (#28964) 2026-07-13 15:06:08 -07:00
zijiexia 0bf7ddb481 docs(install): add nightly install + docker tag guidance, and auto-bump version on release tag (#30308) 2026-07-07 12:10:05 -07:00
zijiexiaandClaude Opus 4.8 7c9bb316cf docs(cookbook): total (input+output) throughput per GPU + percentile latency labels (#30214)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 00:39:48 -07:00
zijiexiaandClaude Opus 4.8 cba3801f52 docs: add PD disaggregation to GLM-5.2 cookbook playground (#29544)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 09:57:38 -07:00
zijiexiaandClaude Opus 4.8 1c75243f5e docs: add Qwen3.6-27B-NVFP4 variant to cookbook (#29905)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 00:05:29 -07:00
zijiexiaandClaude Opus 4.8 5106b42cbd [cookbook] GLM-5.2 NVFP4 B300: TP8 recipe + 3 strategies (#29557)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 14:42:19 -07:00
zijiexiaandClaude Opus 4.8 11b7ed7c9e [Docs] Add --prerelease=allow so uv installs the latest sglang (#29676)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 14:13:34 -07:00
zijiexiaandClaude Opus 4.8 74a197af9d docs: add B200 NVFP4 recipes + benchmarks to GLM-5.2 cookbook (#29674)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 14:06:30 -07:00
zijiexiaandClaude Opus 4.8 e0c0c0a45c [Cookbook] GLM-5.2: tune GB300 NVFP4 recipes + fill benchmarks (#29486)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 01:01:28 -07:00
zijiexiaandClaude Opus 4.8 c21c2d9421 [Docs] Cookbook: match playground docker image resolution to deployment (hw|quant) (#29400)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 00:56:34 -07:00
zijiexiaandClaude Opus 4.8 dd56a9f069 [Docs] Add NVFP4 quantization to GLM-5.2 cookbook (#29380)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 00:40:13 -07:00
zijiexiaandClaude Opus 4.8 dd2d919e21 [Docs] Fix mem-fraction-static default and document how it is computed (#29135)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 08:05:06 +00:00
zijiexiaandClaude Opus 4.8 a707b2054d [CI] Fix pre-commit failures in MLX backend tests (#29141)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 15:56:52 +08:00
zijiexiaandClaude Opus 4.8 52a90c9a36 docs(minimax-m3): use published AMD ROCm images (#28777)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 07:48:16 +01:00
zijiexiaandClaude Opus 4.8 669be5448b [cuda graph] Enable prefill piecewise CUDA graph for Cohere2Vision (text path) (#28686)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 19:20:14 +00:00
zijiexiaandClaude Opus 4.8 3f66873304 [Docs] DeepSeek-V4 cookbook: drop --disable-flashinfer-autotune from GB300 Flash low-latency (#28590)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 00:05:21 -07:00
zijiexia 74e2e48c82 Introduce CpuDeviceMixin and CpuSRTPlatform (#26385) 2026-06-17 17:41:17 -07:00
zijiexiaandClaude Opus 4.8 7221be2cec feat(cookbook): MTP --max-running-requests callout + skill sync (#28340)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 15:53:01 -07:00
zijiexiaandClaude Opus 4.8 d988d5d681 feat(cookbook): generic config-declared flagSelects playground axis (#28128)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 11:46:34 -07:00
zijiexiaandClaude Fable 5 29ac249be1 docs: add cookbook-migrate-model skill from the Qwen3.5 pilot (#27845)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 18:31:52 -07:00
zijiexiaandClaude Fable 5 43835b5ba5 docs: cookbook benchmark accuracy labels come from the model config (no engine default) (#27842)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 18:27:38 -07:00
zijiexiaandClaude Fable 5 73d0989d9b docs: make playground issue template model field a free-form input (#27827)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 14:04:16 -07:00
zijiexiaandClaude Opus 4.8 99258b2f1e [Docs] Restore right-hand ToC on the DeepSeek-V4 cookbook page (#27830)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 11:58:02 -07:00
zijiexiaandClaude Opus 4.8 1a5775a9df [Docs] Remove the legacy release-docs.yml deploy workflow (#27766)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 23:18:41 -07:00
zijiexia 6565b7c464 [Docs] Update MegaMoE handling and rerun benchmarks (#27726) 2026-06-09 19:00:43 -07:00
zijiexiaandClaude Opus 4.8 d1777d1f6d Cookbook renovation (#26885)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 22:04:54 -07:00
zijiexiaandClaude Opus 4.8 9da88e32e0 [Cohere2Moe] Enable flashinfer_trtllm NVFP4 fused-MoE via SigmoidRenorm routing (#27401)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 22:49:20 -07:00
zijiexiaandClaude Opus 4.8 632a3d480e docs: add Tencent Hunyuan and Poolside cards to autoregressive cookbook (#27400)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 13:17:13 -07:00
zijiexiaandClaude Opus 4.8 c6c1f1a29a docs: sync legacy docs/-only updates into docs_new (Mintlify) (#27308)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 19:45:13 -07:00
zijiexiaandClaude Opus 4.8 088f70d0c0 ci: open the LMSYS blog-sync PR with the repo sglang-bot (#27318)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 17:35:33 -07:00
zijiexiaandClaude Opus 4.8 47377525cb ci: fix LMSYS blog sync to open a PR via gh and only run on main (#27179)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 16:47:46 -07:00
zijiexia b89686710d [Docs] re-organize nemotron cookbook (#27240) 2026-06-04 00:40:13 -07:00
zijiexiaandYihao Wang 1ebc7438ac model: support Command A plus (#26106)
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com>
2026-06-03 11:23:04 +08:00
7fb7b41a3e [docs] Qwen3.5 cookbook: multi-node, MTP TP overrides, dense mamba flag (#26695)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-05-29 12:10:14 -07:00
zijiexiaandClaude Opus 4.7 6afebc278a [docs] DeepSeek-V4 cookbook: note cu129 image for GB200 Pro DeepEP backend (#26413)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 12:08:48 -07:00
zijiexia a26913158b fix(ci): enforce legacy docs/ gate in Lint workflow (#26322) 2026-05-26 20:06:54 +08:00
zijiexiaandClaude Opus 4.7 81cd338fcc [docs] DeepSeek-V4 cookbook: balanced MegaMoE cap, H200 Pro FP4 mem-frac, nsa-* compat, PD-disagg fixes (#26164)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 02:42:41 -07:00
zijiexiaandClaude Opus 4.7 88a37d7405 [docs] DeepSeek-V4 cookbook: split Quantization axis, add H100 SGLang FP8 (#26057)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 00:52:10 -07:00
zijiexiaandClaude Opus 4.7 17dadebd4e [Docs] DeepSeek-V4: switch H200 FP4 Pro to flashinfer_mxfp4, Flash Balanced too (#25923)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 13:51:40 -07:00
zijiexia 9f26697d6a [Docs] Update DeepSeek V4 cookbook to use the latest docker image (#25410) 2026-05-16 11:17:51 -07:00
zijiexiaandClaude Opus 4.7 1f119f6a44 [Docs] update dsv4 cookbook with H100 deployment commands (#25243)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 23:45:29 -07:00
zijiexiaandClaude Opus 4.7 7d515c6d1f docs: prepend SGLANG_JIT_DEEPGEMM_PRECOMPILE=0 for H200 FP8 Flash max-throughput (#25152)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 00:50:15 -07:00
zijiexiaandClaude Opus 4.7 b0018ad015 [Doc]: refactor Intern-S2-Preview cookbook with interactive command generator (#25134)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 22:26:18 -07:00
zijiexia 83b48fd523 [codex] update Nemotron3 Nano Omni cookbook benchmarks (#23998) 2026-05-05 14:53:15 -07:00
zijiexia e0faed8ebc [Docs] Add B200, GB200, GB300 NVIDIA hardware platform support for Kimi-K2.6 (#24441) 2026-05-05 14:53:05 -07:00
zijiexia 8b23d32ec1 feat: implement workflow to sync LMSYS SGLang blog (#23438) 2026-04-30 16:17:29 -07:00
zijiexia 387c932dfc [Docs] update Docker image for Nemotron 3 Nano Omni (#23968) 2026-04-28 15:08:34 -07:00
zijiexia ad785a2299 [Docs] add Nemotron 3 Nano Omni cookbook (#23907) 2026-04-28 10:24:40 -07:00
zijiexia 7ce2352323 [docs] fix sglang serve --model-path in cookbooks (#23905) 2026-04-27 21:54:21 -07:00
zijiexia c2ec64f243 docs: verify GB300 Pro DeepSeek V4 recipes (#23817) 2026-04-27 00:21:26 -07:00
zijiexia c74b345e3a [docs] Update FA4 support SWA (#23793) 2026-04-26 20:40:36 -07:00
zijiexia 3d95ca7546 docs(DeepSeek-V4): mark gb200|big|low-latency verified (#23737) 2026-04-26 11:15:59 -07:00
zijiexia 1a37e57fb1 [codex] docs: note H200 DeepSeek-V4 checkpoint (#23628) 2026-04-24 00:06:30 -07:00
zijiexiaandgemini-code-assist[bot] 9b2f7f8a91 docs: split MI300X and MI325X options in GLM-5.1 generator (#23540)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-04-23 12:01:58 -07:00
zijiexia 6490afe36e [docs] add deprecation notice banner to legacy documentation site (#23516) 2026-04-22 20:17:53 -07:00
zijiexia 9b142df334 Docs/add sglang omni redirect (#23437) 2026-04-22 11:37:27 -07:00
zijiexia 1408d97408 [Docs] Improve SGLang Diffusion docs navigation and compatibility table (#23411) 2026-04-21 16:59:42 -07:00