[docs] Note the default dsa-topk-backend on all DSA-model cookbook pages (#31124)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
2ced88238a
commit
96c2ebc58b
@@ -38,6 +38,10 @@ This section provides deployment configurations optimized for different hardware
|
||||
|
||||
<DeepSeekMathV2Deployment />
|
||||
|
||||
<Warning>
|
||||
DeepSeek-Math-V2 is built on DeepSeek-V3.2 and uses DSA sparse attention. All recipes here run the DSA indexer top-k on the default `--dsa-topk-backend sgl-kernel`. Other top-k backend choices have not been fully validated on this model.
|
||||
</Warning>
|
||||
|
||||
### 3.2 Configuration Tips
|
||||
|
||||
**Hardware Requirements:**
|
||||
|
||||
@@ -57,6 +57,10 @@ import { DeepSeekV32Deployment } from "/src/snippets/autoregressive/deepseek-v32
|
||||
|
||||
<DeepSeekV32Deployment />
|
||||
|
||||
<Warning>
|
||||
All recipes here run the DSA indexer top-k on the default `--dsa-topk-backend sgl-kernel`. Other top-k backend choices have not been fully validated on DeepSeek-V3.2.
|
||||
</Warning>
|
||||
|
||||
### 3.2 Configuration Tips
|
||||
|
||||
- **Short-sequence MHA prefill (adaptive):** For prefill sequences shorter than 2048 tokens (default threshold), the DSA backend automatically switches to standard MHA (using FlashAttention variable-length on SM90, TRT-LLM ragged MHA on SM100). To extend this to longer sequences set env var `SGLANG_DSA_PREFILL_DENSE_ATTN_KV_LEN_THRESHOLD` to a larger value (potential minor accuracy trade-off).
|
||||
|
||||
@@ -30,6 +30,10 @@ import { GLM51Deployment } from '/src/snippets/autoregressive/glm-51-deployment.
|
||||
|
||||
<GLM51Deployment />
|
||||
|
||||
<Warning>
|
||||
All recipes here run the DSA indexer top-k on the default `--dsa-topk-backend sgl-kernel`. Other top-k backend choices have not been fully validated on GLM-5.1.
|
||||
</Warning>
|
||||
|
||||
### 3.2 Configuration Tips
|
||||
|
||||
- Speculative decoding (MTP) can significantly reduce latency for interactive use cases.
|
||||
|
||||
@@ -52,6 +52,10 @@ import { benchmarks } from "/src/snippets/configs/zai-org/glm-5.2-benchmarks.jsx
|
||||
|
||||
<Deployment config={config} benchmarks={benchmarks} />
|
||||
|
||||
<Warning>
|
||||
All recipes here run the DSA indexer top-k on the default `--dsa-topk-backend sgl-kernel`. Other top-k backend choices have not been fully validated on GLM-5.2.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
Speed numbers are measured with `--random-range-ratio 1.0`, `--flush-cache`, on `main @ 09ca4fc`. Spec cells pin the EAGLE acceptance length via the serve env `SGLANG_SIMULATE_ACC_LEN` (low-latency 5-1-6 = 3.5, balanced 2-1-3 = 2); high-throughput has no spec.
|
||||
</Note>
|
||||
|
||||
@@ -41,6 +41,10 @@ import { GLM5Deployment } from '/src/snippets/autoregressive/glm-5-deployment.js
|
||||
|
||||
<GLM5Deployment />
|
||||
|
||||
<Warning>
|
||||
All recipes here run the DSA indexer top-k on the default `--dsa-topk-backend sgl-kernel`. Other top-k backend choices have not been fully validated on GLM-5.
|
||||
</Warning>
|
||||
|
||||
### 3.2 Configuration Tips
|
||||
|
||||
- Speculative decoding (MTP) can significantly reduce latency for interactive use cases.
|
||||
|
||||
@@ -59,6 +59,10 @@ import { benchmarks } from "/src/snippets/configs/meituan-longcat/longcat-2.0-be
|
||||
|
||||
<Deployment config={config} benchmarks={benchmarks} />
|
||||
|
||||
<Warning>
|
||||
All recipes here run the LongCat sparse-attention indexer top-k on the default `--dsa-topk-backend sgl-kernel`. Other top-k backend choices have not been fully validated on LongCat-2.0.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
The B300 single-node recipe was validated end-to-end with CUDA graph capture enabled. H200, B200, and H20 are shown as 2-node recipes because LongCat-2.0-FP8 needs 16 ranks for those GPU memory profiles.
|
||||
</Note>
|
||||
|
||||
Reference in New Issue
Block a user