[Doc] layer-boundary/moe: Correct guides and comments that drifted from the code (#44896)

This commit is contained in:
Cheng Wan
2026-10-20 16:26:40 -07:00
committed by GitHub
parent 299a75ac5d
commit 027208ae02
4 changed files with 23 additions and 15 deletions
@@ -136,15 +136,13 @@ def _fused_rmsnorm_fp8_per_token_quant(
return (out_fp8, scale.unsqueeze(1))
# TODO: According to the discussion in https://github.com/flashinfer-ai/flashinfer/issues/1223#issuecomment-3047256465
# We set the max token num to 128 for allreduce fusion with min-latency case(use_oneshot=True).
# Largest batch the non-CuTe-DSL FlashInfer all-reduce fusion runs on;
# it also sizes that fusion's workspace, pre-allocated before graph capture.
FUSE_ALLREDUCE_MAX_BATCH_SIZE = 2048
def flashinfer_ar_fusion_applies(batch_size: int):
return (
# NOTE: flashinfer 0.6.1 caused performance regression on sm100 for allreduce fusion
# Ref: https://github.com/sgl-project/sglang/issues/17237
(_is_sm90_supported or _is_sm100_supported)
and _is_flashinfer_available
and not is_dp_attention_enabled()
+4 -3
View File
@@ -716,9 +716,10 @@ def is_moe_input_scattered_across_dp_ranks() -> bool:
def should_use_dp_reduce_scatterv():
"""
Use reduce_scatterv in the standard dispatcher's combine() for DP attention
with EP, replacing the default all-reduce + dp_scatter path.
Only changes the combine (post-kernel) communication; dispatch is unchanged.
Whether the parallel configuration allows reduce_scatterv for returning an
FFN output over attention DP (DP attention with EP), replacing the default
all-reduce + dp_scatter path; a stage boundary also needs boundary_reduction
rsv or rs+rsv. Only that post-expert return changes; dispatch is unchanged.
The reduce_scatterv group is the global TP group, while its variable split
sizes are one entry per attention-DP rank. Therefore this optimization is