diff --git a/docs_new/cookbook/autoregressive/Moonshotai/Kimi-K3.mdx b/docs_new/cookbook/autoregressive/Moonshotai/Kimi-K3.mdx index 84203f6f8..b22057c36 100644 --- a/docs_new/cookbook/autoregressive/Moonshotai/Kimi-K3.mdx +++ b/docs_new/cookbook/autoregressive/Moonshotai/Kimi-K3.mdx @@ -30,7 +30,7 @@ For how to launch the image, see [Install → Method 3: Using Docker](../../../d -If you do not want to use a Docker image, reproduce the dependency installation steps from the [CUDA 13 Dockerfile](https://github.com/DarkSharpness/sglang-kimi/blob/kimi-k3/docker/kimi_k3/kimi_k3_cu13.Dockerfile) or [CUDA 12 Dockerfile](https://github.com/DarkSharpness/sglang-kimi/blob/kimi-k3/docker/kimi_k3/kimi_k3_cu12.Dockerfile). +If you do not want to use a Docker image, reproduce the dependency installation steps from the [CUDA 13 Dockerfile](https://github.com/sgl-project/sglang/blob/kimi-k3/docker/kimi_k3/kimi_k3_cu13.Dockerfile) or [CUDA 12 Dockerfile](https://github.com/sgl-project/sglang/blob/kimi-k3/docker/kimi_k3/kimi_k3_cu12.Dockerfile). @@ -96,8 +96,8 @@ K3 **always runs with thinking enabled**, with reasoning depth controlled by `re Kimi-K3 is Moonshot AI's first open-source model in the trillion-plus class; **full model weights -are scheduled to release by July 27, 2026**. The recipes on this page were validated on the -[`DarkSharpness/sglang-kimi`](https://github.com/DarkSharpness/sglang-kimi) fork — the HuggingFace +are scheduled to release by July 27, 2026**. The recipes on this page were validated on the public +[`sgl-project/sglang` `kimi-k3` branch](https://github.com/sgl-project/sglang/tree/kimi-k3) — the HuggingFace repository (`moonshotai/Kimi-K3`) and a public `lmsysorg/sglang` image with K3 support will be available at launch. @@ -247,6 +247,8 @@ K3's hybrid HiCache tiers the paged MLA KV **and** the KDA/mamba state across L1 - On the DCP recipes (Blackwell Balanced / High-Throughput), the host tiers are not fully DCP-aware yet: **L3 always, and L1+L2 with Spec Decode on, drop the DCP flags and run plain TP** (the command hints call it out — per-request KV capacity shrinks accordingly). L1+L2 NOSPEC keeps DCP. - Low-Latency and the Hopper recipes take all tiers unchanged. + + ### 3.4 PD Disaggregation PD splits prefill and decode into separate server groups; because K3 is hybrid, the transfer moves **both** the paged MLA KV and the KDA recurrent state. diff --git a/docs_new/src/snippets/configs/moonshotai/kimi-k3.jsx b/docs_new/src/snippets/configs/moonshotai/kimi-k3.jsx index c9a603ad6..15f21354c 100644 --- a/docs_new/src/snippets/configs/moonshotai/kimi-k3.jsx +++ b/docs_new/src/snippets/configs/moonshotai/kimi-k3.jsx @@ -5,7 +5,7 @@ // (benchmark/H200/script/v1/launch-k3.sh, benchmark/B300/script/v1/launch-k3.sh) // and the B200 2×8 / GB200 4×4 / H100 4×8 / MI35x 1×8 reference launches. // Kimi-K3 is a hybrid MoE VLM: 93 layers = 69 KDA (linear) + 24 MLA, 896 routed -// experts + 1 shared. Served today from the DarkSharpness/sglang-kimi fork. +// experts + 1 shared. Served today from the public sgl-project/sglang kimi-k3 branch. export const config = { modelName: "Kimi-K3",