diff --git a/docs/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next.mdx b/docs/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next.mdx
index 28df6bee3e..9fd95259df 100644
--- a/docs/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next.mdx
+++ b/docs/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next.mdx
@@ -218,3 +218,45 @@ Pending update — a sample transcript will be added here once the weights are p
```
+
+### 2.3 GB300 long-context serving with HiCache and MTP
+
+Use this single-GPU, aggregated TP1/EP1 configuration for long-context, multi-turn workloads on GB300. It combines the NVIDIA NVFP4 checkpoint's embedded MTP head, CPU DRAM KV offload through HiCache, and pinned-memory PLE embedding offload.
+
+This serving configuration comes from [InferenceX #3816](https://github.com/SemiAnalysisAI/InferenceX/pull/3816). [Run 37714835620, attempt 1](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/37714835620) passed concurrency 1, 2, 4, 8, 16, 32, 48 and 64 on commit `e67ca6fc9ecf4cd57f27654bda8a3e9059923820`. Its [matching GSM8K eval](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/37714835620/job/113109083376) used real verification and scored 97.35% on 1,319 questions.
+
+The throughput sweep used InferenceX's simulated acceptance; the generated command uses normal MTP verification and does not claim those throughput numbers for production traffic.
+
+#### Prerequisites and memory budget
+
+- Use a Linux ARM64 GB300 host with Docker, NVIDIA Container Toolkit, and a driver compatible with the image's CUDA 13.0 runtime. See [Docker installation](/docs/get-started/install#method-3-using-docker).
+- Reserve one GB300 GPU. The recipe keeps TP, PP, DP and EP at 1.
+- Budget 224 GiB of CPU memory for the combined target/draft KV and Mamba HiCache host pool, plus approximately 48 GiB for the offloaded FP8 PLE table and additional headroom for checkpoint loading and host processes. `--hicache-size` sets the combined cache budget; it is separate from PLE weight offload. Keep sufficient free host memory for both.
+- The Docker command mounts your Hugging Face cache to retain downloaded model files across restarts. The image digest and both target/draft checkpoint revisions are pinned to the validated configuration.
+
+#### Generate the launch command
+
+In the [Deployment panel](#deployment), select **GB300 → NVFP4 (NVDA) → Low Latency → Single Node**, and leave **PLE Offload** at **Auto** or **On**. The matched cell includes all five serving environment variables and the full argument set, including ReplaySSM, HiCache and pinned PLE offload. Choose **Docker** to use the pinned image, or **Python** for a matching local installation. Changing PLE Offload to **Off** produces an unverified configuration.
+
+The Docker wrapper comes from the shared command builder and is adapted from the validated Slurm/Enroot launch; it has not been run separately on a GB300 host. Use the command generated by the panel rather than a separate static copy.
+
+
+
+The validated run loaded the target and embedded draft through the stock image's mixed-precision loader. This excerpt is from the real-verification eval; timings and memory availability vary by host.
+
+```text Output
+[2026-10-08 02:22:36] Load weight end. elapsed=1011.56 s, type=Qwen4ExpForConditionalGeneration, quant=modelopt_mixed, quant_algo=MIXED_PRECISION, avail mem=202.03 GB, mem usage=73.64 GB.
+[2026-10-08 02:23:03] Load weight end. elapsed=25.97 s, type=Qwen4ExpForCausalLMMTP, quant=modelopt_mixed, quant_algo=MIXED_PRECISION, avail mem=197.09 GB, mem usage=4.89 GB.
+```
+
+
+
+#### Serving choices
+
+- **Draft precision:** revision `fc694b54` stores the MTP routed experts in block-scaled FP8 and the rest of the head in BF16. The pinned image, built from SGLang `e00930c5`, preserves this per-layer `modelopt_mixed` configuration. The draft uses the same checkpoint revision as the target. `SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0` disables the optional draft precision conversion path; target and draft KV caches stay in BF16.
+- **Hybrid attention:** the recipe selects FlashInfer for linear-attention prefill/decode, Triton for linear verification, and `trtllm_mha` as the attention backend. ReplaySSM speculation, `extra_buffer_lazy`, a tracking interval of 256 and 480 Mamba cache slots retain the qualified state-cache configuration.
+- **Admission and graphs:** the server admits up to 96 requests, uses 16,384-token prefill chunks, and captures full decode graphs at the listed batch sizes up to 96. Prefill graphs remain disabled. The qualification varied client concurrency while retaining these server settings.
+- **CPU offload:** HiCache uses the `kernel` I/O backend, `page_first` layout and `write_back` policy. PLE uses a separate pinned-memory embedding table. See [HiCache design](/docs/advanced_features/hicache_design) for the cache tiers and policies.
+- **API and telemetry:** send requests through the chat-completions API at `http://localhost:30000/v1`, using model name `nvidia/Qwen3.8-Flash-Next-NVFP4`. `--reasoning-parser auto` separates reasoning from answer content. Engine metrics and cache reporting remain enabled.
+
+The generated command omits arguments that match the pinned image's defaults, including TP/PP/DP/EP=1, the inherited served-model name and draft MoE backend, full decode graphs, and default Mamba tracking, HiCache layout, PLE backend, tokenizer and scheduler settings. It retains the qualified lane's non-default serving settings. InferenceX-only trace loading, simulated-acceptance controls, client concurrency overrides, result collection and Slurm allocation settings are excluded.
diff --git a/docs/src/snippets/configs/Qwen/qwen3.8-flash-next.jsx b/docs/src/snippets/configs/Qwen/qwen3.8-flash-next.jsx
index 326de3ba68..ea06946634 100644
--- a/docs/src/snippets/configs/Qwen/qwen3.8-flash-next.jsx
+++ b/docs/src/snippets/configs/Qwen/qwen3.8-flash-next.jsx
@@ -103,16 +103,22 @@ export const config = {
disableReason: (sel) => sel.hw === "rtx6000"
? "RTX PRO 6000 (96 GB) only fits this checkpoint with the 47.7 GiB FP8 N-gram table in pinned host RAM; the verified cells pass --ple-offload-embedding explicitly, so On is the only pick."
: "DGX Spark is unified memory: PLE offload to RAM frees nothing (the pinned table shares the 128 GB pool with the weights). The verified settings are Off for the two-node cells and On (NVMe file) for a single Spark.",
- hints: ["PLE Offload: auto-enabled for BF16 on CUDA, off otherwise"] },
+ hints: (sel) => sel.hw === "gb300" && sel.quant === "nvfp4-nvda" && sel.strategy === "low-latency"
+ ? ["PLE Offload: enabled in the GB300 base recipe; reserve approximately 48 GiB of host RAM in addition to the 224 GiB HiCache pool."]
+ : ["PLE Offload: auto-enabled for BF16 on CUDA, off otherwise"] },
{ id: "on", label: "On",
disabled: (sel) => sel.hw === "dgx-spark",
disableReason: "DGX Spark is unified memory: PLE offload to RAM frees nothing (the pinned table shares the 128 GB pool with the weights). The verified settings are Off for the two-node cells and On (NVMe file) for a single Spark.",
+ stripPrefixes: (sel) => sel.hw === "gb300" && sel.quant === "nvfp4-nvda" && sel.strategy === "low-latency"
+ ? ["--ple-offload-embedding"] : [],
flags: ["--ple-offload-embedding"] },
{ id: "off", label: "Off",
disabled: (sel) => sel.hw === "rtx6000" || (sel.hw === "dgx-spark" && sel.nodes === "single"),
disableReason: (sel) => sel.hw === "dgx-spark"
? "A single DGX Spark cannot hold the 126 GiB checkpoint in its 128 GB of unified memory; the verified single-Spark cells keep the 47.7 GiB FP8 N-gram table in a file on the local NVMe (On (NVMe file))."
: "RTX PRO 6000 (96 GB) cannot hold the 47.7 GiB FP8 N-gram table alongside the other 78 GiB of the checkpoint; the table must be offloaded to pinned host RAM (On).",
+ stripPrefixes: (sel) => sel.hw === "gb300" && sel.quant === "nvfp4-nvda" && sel.strategy === "low-latency"
+ ? ["--ple-offload-embedding", "--ple-offload-backend"] : [],
flags: ["--no-ple-offload-embedding"] },
// File-backed table (sgl-project/sglang#37068, merged into qwen4-main-squashed):
// a sparse 47.7 GiB file under $SGLANG_CACHE_DIR/ple/ (override
@@ -206,6 +212,7 @@ export const config = {
"b300|nvfp4-nvda": "lmsysorg/sglang:latest",
gb300: "lmsysorg/sglang:qwen38flashnext",
"gb300|nvfp4-nvda": "lmsysorg/sglang:latest",
+ "gb300|nvfp4-nvda|low-latency": "lmsysorg/sglang:v0.5.21-cu130@sha256:1f176b3a1dd1205d2a462ab80ea9fc17769a32e53cb6f3c690efa2dc2c384977",
mi350x: "lmsysorg/sglang-rocm:qwen38flashnext",
mi355x: "lmsysorg/sglang-rocm:qwen38flashnext",
},
@@ -803,20 +810,60 @@ export const config = {
},
{
match: { hw: "gb300", variant: "default", quant: "nvfp4-nvda", strategy: "low-latency", nodes: "single" },
- verified: false,
- warn: "Requires SGLang v0.5.20 or later for this NVIDIA ModelOpt MIXED_PRECISION export. Quantization and MoE backends are selected automatically from the checkpoint.",
- env: [],
+ verified: true,
+ verificationStatus: (sel) => sel.pleOffload === "off" ? "unverified" : "verified",
+ warn: "Long-context recipe: reserve 224 GiB of host RAM for HiCache, approximately 48 GiB for pinned PLE offload, and loading headroom. See [GB300 configuration notes](#23-gb300-long-context-serving-with-hicache-and-mtp). The Docker wrapper is adapted from the qualified Slurm/Enroot launch.",
+ env: [
+ "PYTHONNOUSERSITE=1",
+ "PYTHONUNBUFFERED=1",
+ "NO_COLOR=1",
+ "SGLANG_TIMEOUT_KEEP_ALIVE=1800",
+ "SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0",
+ ],
+ // Omit defaults from the pinned v0.5.21 build: TP/PP/DP/EP=1,
+ // model-name and draft-MoE inheritance, full decode graphs, and
+ // the standard Mamba, HiCache, PLE, tokenizer and scheduler settings.
flags: [
"--model-path {{MODEL_NAME}}",
- "--tp 1",
+ "--moe-runner-backend flashinfer_trtllm",
+ "--disable-shared-experts-fusion",
+ "--revision fc694b54fb0174e0913e6adf86691ef85a4ead47",
+ "--speculative-draft-model-revision fc694b54fb0174e0913e6adf86691ef85a4ead47",
+ "--trust-remote-code",
+ "--quantization modelopt_mixed",
+ "--context-length 262144",
+ "--attention-backend trtllm_mha",
"--linear-attn-prefill-backend flashinfer",
"--linear-attn-decode-backend flashinfer",
+ "--linear-attn-verify-backend triton",
"--mamba-ssm-dtype bfloat16",
+ "--mamba-radix-cache-strategy extra_buffer_lazy",
+ "--max-mamba-cache-size 480",
+ "--mem-fraction-static 0.88",
+ "--max-running-requests 96",
+ "--page-size 64",
+ "--chunked-prefill-size 16384",
"--speculative-algorithm NEXTN",
"--speculative-num-steps 3",
"--speculative-eagle-topk 1",
"--speculative-num-draft-tokens 4",
+ "--enable-linear-replayssm-spec",
+ "--cuda-graph-backend-prefill disabled",
+ "--cuda-graph-max-bs-decode 96",
+ "--cuda-graph-bs-decode 1 2 4 8 16 24 32 40 48 56 64 72 80 88 96",
+ "--kv-cache-dtype bfloat16",
+ "--speculative-draft-kv-cache-dtype bfloat16",
+ "--enable-hierarchical-cache",
+ "--hicache-size 224",
+ "--hicache-write-policy write_back",
+ "--ple-offload-embedding",
+ "--enable-metrics",
+ "--enable-cache-report",
"--reasoning-parser auto",
+ "--decode-log-interval 10",
+ "--stream-interval 20",
+ "--watchdog-timeout 1800",
+ "--random-seed 35",
"--host {{HOST_IP}}",
"--port {{PORT}}",
],