docs: point Kimi-K3 references to public branch (#32547)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
This commit is contained in:
co-authored by
Xinyuan Tong
parent
7dafacca49
commit
3ebb7c2d07
@@ -30,7 +30,7 @@ For how to launch the image, see [Install → Method 3: Using Docker](../../../d
|
||||
|
||||
</Tabs>
|
||||
|
||||
If you do not want to use a Docker image, reproduce the dependency installation steps from the [CUDA 13 Dockerfile](https://github.com/DarkSharpness/sglang-kimi/blob/kimi-k3/docker/kimi_k3/kimi_k3_cu13.Dockerfile) or [CUDA 12 Dockerfile](https://github.com/DarkSharpness/sglang-kimi/blob/kimi-k3/docker/kimi_k3/kimi_k3_cu12.Dockerfile).
|
||||
If you do not want to use a Docker image, reproduce the dependency installation steps from the [CUDA 13 Dockerfile](https://github.com/sgl-project/sglang/blob/kimi-k3/docker/kimi_k3/kimi_k3_cu13.Dockerfile) or [CUDA 12 Dockerfile](https://github.com/sgl-project/sglang/blob/kimi-k3/docker/kimi_k3/kimi_k3_cu12.Dockerfile).
|
||||
|
||||
</Accordion>
|
||||
|
||||
@@ -96,8 +96,8 @@ K3 **always runs with thinking enabled**, with reasoning depth controlled by `re
|
||||
|
||||
<Note>
|
||||
Kimi-K3 is Moonshot AI's first open-source model in the trillion-plus class; **full model weights
|
||||
are scheduled to release by July 27, 2026**. The recipes on this page were validated on the
|
||||
[`DarkSharpness/sglang-kimi`](https://github.com/DarkSharpness/sglang-kimi) fork — the HuggingFace
|
||||
are scheduled to release by July 27, 2026**. The recipes on this page were validated on the public
|
||||
[`sgl-project/sglang` `kimi-k3` branch](https://github.com/sgl-project/sglang/tree/kimi-k3) — the HuggingFace
|
||||
repository (`moonshotai/Kimi-K3`) and a public `lmsysorg/sglang` image with K3 support will be
|
||||
available at launch.
|
||||
|
||||
@@ -247,6 +247,8 @@ K3's hybrid HiCache tiers the paged MLA KV **and** the KDA/mamba state across L1
|
||||
- On the DCP recipes (Blackwell Balanced / High-Throughput), the host tiers are not fully DCP-aware yet: **L3 always, and L1+L2 with Spec Decode on, drop the DCP flags and run plain TP** (the command hints call it out — per-request KV capacity shrinks accordingly). L1+L2 NOSPEC keeps DCP.
|
||||
- Low-Latency and the Hopper recipes take all tiers unchanged.
|
||||
|
||||
<a id="pd-disaggregation" />
|
||||
|
||||
### 3.4 PD Disaggregation
|
||||
|
||||
PD splits prefill and decode into separate server groups; because K3 is hybrid, the transfer moves **both** the paged MLA KV and the KDA recurrent state.
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
// (benchmark/H200/script/v1/launch-k3.sh, benchmark/B300/script/v1/launch-k3.sh)
|
||||
// and the B200 2×8 / GB200 4×4 / H100 4×8 / MI35x 1×8 reference launches.
|
||||
// Kimi-K3 is a hybrid MoE VLM: 93 layers = 69 KDA (linear) + 24 MLA, 896 routed
|
||||
// experts + 1 shared. Served today from the DarkSharpness/sglang-kimi fork.
|
||||
// experts + 1 shared. Served today from the public sgl-project/sglang kimi-k3 branch.
|
||||
|
||||
export const config = {
|
||||
modelName: "Kimi-K3",
|
||||
|
||||
Reference in New Issue
Block a user