From 91144797c517b5962612bcfdd90de5230735283b Mon Sep 17 00:00:00 2001
From: Faradawn Yang <73060648+faradawn@users.noreply.github.com>
Date: Mon, 17 Aug 2026 17:12:07 -0700
Subject: [PATCH] Update Qwen3.5 H200 FP8 for AgentX HiCache MTP (#35194)

---
 docs/cookbook/autoregressive/Qwen/Qwen3.5.mdx | 39 +++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/docs/cookbook/autoregressive/Qwen/Qwen3.5.mdx b/docs/cookbook/autoregressive/Qwen/Qwen3.5.mdx
index 11b10d0f0..f5cfe5f0b 100644
--- a/docs/cookbook/autoregressive/Qwen/Qwen3.5.mdx
+++ b/docs/cookbook/autoregressive/Qwen/Qwen3.5.mdx
@@ -989,6 +989,45 @@ Max ITL (ms):                            1220.68
 ==================================================
 ```
 
+#### 5.2.3 Agentic Long-Context with HiCache DRAM Offload (H200 FP8, MTP)
+
+For agentic workloads, here is how to enable HiCache and MTP.
+
+Container image (pinned for reproducibility):
+`lmsysorg/sglang:nightly-dev-cu13-20260815-a5ba081f`.
+
+Server Launch Command:
+```bash Command
+SGLANG_ENABLE_SPEC_V2=1 \
+python3 -m sglang.launch_server \
+  --model-path Qwen/Qwen3.5-397B-A17B-FP8 \
+  --served-model-name Qwen/Qwen3.5-397B-A17B-FP8 \
+  --trust-remote-code \
+  --tensor-parallel-size 8 \
+  --data-parallel-size 1 \
+  --expert-parallel-size 1 \
+  --quantization fp8 \
+  --kv-cache-dtype fp8_e4m3 \
+  --mamba-ssm-dtype bfloat16 \
+  --attention-backend flashinfer \
+  --enable-flashinfer-allreduce-fusion \
+  --mem-fraction-static 0.8 \
+  --stream-interval 50 \
+  --scheduler-recv-interval 10 \
+  --tokenizer-worker-num 6 \
+  --enable-metrics \
+  --speculative-algorithm EAGLE \
+  --speculative-num-steps 3 \
+  --speculative-eagle-topk 1 \
+  --speculative-num-draft-tokens 4 \
+  --page-size 64 \
+  --enable-hierarchical-cache \
+  --hicache-size 77 \
+  --hicache-io-backend kernel \
+  --hicache-mem-layout page_first \
+  --hicache-write-policy write_through_selective
+```
+
 ### 5.3 Vision Speed Benchmark
 
 We use SGLang's built-in benchmarking tool to conduct performance evaluation with random images. Each request has 128 input tokens, two 720p images, and 1024 output tokens.