Files
sglang/benchmark/kernels/deepseek
minke.yu ef1a5955f3 PR #39306: H20-3e block-FP8 GEMM configs + adapt tables to NVIDIA H20
Net diff of PR #39306 vs upstream/main (kernel path already landed via #39657).
Also duplicated the six #41098 quantization tables and the six #39306 gemm
tables under device_name=NVIDIA_H20: config lookup is exact-filename and our
GPUs report 'NVIDIA H20' (96GB), not H20-3e. H20-3e shares the SM90 compute
configuration, so the tuned tables transfer.
2026-10-20 10:26:58 +08:00
..

DeepSeek kernels benchmark

Prerequisites

  • You should install DeepGemm from source before run benchmark_deepgemm_fp8_gemm.py and benchmark_deepgemm_fp8_group_gemm.py.

Benchmark

  • benchmark_deepgemm_fp8_gemm.py

    python benchmark_deepgemm_fp8_gemm.py --run_correctness --tp_size 1
    
  • benchmark_deepgemm_fp8_group_gemm.py

    python benchmark_deepgemm_fp8_group_gemm.py --run_correctness --tp_size 1
    
  • You can use the --run_correctness parameter to verify all kernels results's correctness.

    • You can use the --tp_size parameter to benchmark all FP8 w8a8 block-wise matrix multiplications involved in DeepSeek V3/R1 under the current tensor parallelism (TP) setting. This benchmark compares DeepSeek's open-source DeepGemm implementation with SGLang's and VLLM Triton implementation.