Net diff of PR #39306 vs upstream/main (kernel path already landed via #39657). Also duplicated the six #41098 quantization tables and the six #39306 gemm tables under device_name=NVIDIA_H20: config lookup is exact-filename and our GPUs report 'NVIDIA H20' (96GB), not H20-3e. H20-3e shares the SM90 compute configuration, so the tuned tables transfer.
DeepSeek kernels benchmark
Prerequisites
- You should install DeepGemm from source before run
benchmark_deepgemm_fp8_gemm.pyandbenchmark_deepgemm_fp8_group_gemm.py.
Benchmark
-
benchmark_deepgemm_fp8_gemm.pypython benchmark_deepgemm_fp8_gemm.py --run_correctness --tp_size 1 -
benchmark_deepgemm_fp8_group_gemm.pypython benchmark_deepgemm_fp8_group_gemm.py --run_correctness --tp_size 1 -
You can use the
--run_correctnessparameter to verify all kernels results's correctness.- You can use the
--tp_sizeparameter to benchmark all FP8 w8a8 block-wise matrix multiplications involved in DeepSeek V3/R1 under the current tensor parallelism (TP) setting. This benchmark compares DeepSeek's open-source DeepGemm implementation with SGLang's and VLLM Triton implementation.
- You can use the