diff --git a/docs_new/cookbook/autoregressive/DeepSeek/DeepSeek-V4.mdx b/docs_new/cookbook/autoregressive/DeepSeek/DeepSeek-V4.mdx index 74608fe50..e4070a464 100644 --- a/docs_new/cookbook/autoregressive/DeepSeek/DeepSeek-V4.mdx +++ b/docs_new/cookbook/autoregressive/DeepSeek/DeepSeek-V4.mdx @@ -156,6 +156,18 @@ MegaMoE is not supported with this DeepEP Waterfill recipe yet. Waterfill routes the shared expert through DeepEP for load balancing, so `--enable-deepep-waterfill` requires `--moe-a2a-backend deepep`. +**FP4 Indexer (Experimental)** + +DeepSeek-V4 uses the default indexer path unless `--enable-deepseek-v4-fp4-indexer` is set. Enable this flag to use the experimental FP4 C4 indexer on SM100 GPUs with DeepGEMM FP4 indexer support. This path is intended for decode-heavy long-context workloads where reducing indexer cache bandwidth is beneficial. + +```bash Command +# Please use latest main branch for this feature +sglang serve deepseek-ai/DeepSeek-V4-Flash \ + --tp 4 \ + --moe-runner-backend flashinfer_mxfp4 \ + --enable-deepseek-v4-fp4-indexer +``` + **Hopper (H200) note** diff --git a/docs_new/docs/advanced_features/server_arguments.mdx b/docs_new/docs/advanced_features/server_arguments.mdx index f63cf7b52..90a8817fa 100644 --- a/docs_new/docs/advanced_features/server_arguments.mdx +++ b/docs_new/docs/advanced_features/server_arguments.mdx @@ -1218,6 +1218,12 @@ Please consult the documentation below and [server_args.py](https://github.com/s `sgl-kernel` sgl-kernel, torch, flashinfer + + `--enable-deepseek-v4-fp4-indexer` + Enable the experimental FP4 C4 indexer path for DeepSeek V4. When unset, SGLang keeps the default DeepSeek V4 indexer path. Requires SM100 GPUs with DeepGEMM FP4 indexer support. + `False` + bool flag (set to enable) + `--fp8-gemm-backend` Choose the runner backend for Blockwise FP8 GEMM operations. Options: 'auto' (default, auto-selects based on hardware), 'deep_gemm' (JIT-compiled; enabled by default on NVIDIA Hopper (SM90) and Blackwell (SM100) when DeepGEMM is installed), 'flashinfer_trtllm' (FlashInfer TRTLLM backend; SM100/SM103 only), 'flashinfer_cutlass' (FlashInfer CUTLASS backend, SM120 only), 'flashinfer_deepgemm' (Hopper SM90 only, uses swapAB optimization for small M dimensions in decoding), 'cutlass' (optimal for Hopper/Blackwell GPUs and high-throughput), 'triton' (fallback, widely compatible), 'aiter' (ROCm only).