minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[Perf] Occupancy tuning for DSA indexer fp8-quant Q kernel (#32755)

Browse Source 
Signed-off-by: Rainchar9119 <1134601163@qq.com>
This commit is contained in:
Rainchar9119
2026-08-13 01:15:27 -07:00
committed by GitHub
parent fad376d3ee
commit dbebc1deb4
2 changed files with 193 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/kernels/jit/csrc/deepseek_v4/main_norm_rope.cuh
+4 -2
View File
@@ -35,8 +35,10 @@ SGL_DEVICE uint8_t quant_fp4_e2m1(float x) {
return idx;
}
// 4 warps per block: warp-per-(token, head) work-item dispatch (Q kernel).
constexpr uint32_t kFusedQBlockSize = 128;
// 8 warps per block: warp-per-(token, head) work-item dispatch (Q kernel).
// 256 threads lifts scheduler occupancy (~38% -> ~86%) on the fp8-quant path;
// math is unchanged, output is bitwise-identical.
constexpr uint32_t kFusedQBlockSize = 256;
constexpr uint32_t kFusedQNumWarps = kFusedQBlockSize / device::kWarpThreads;
// 8 warps per block: block-per-token work-item dispatch (K kernel).