Fix Qwen3 MoE: also guard EP all-reduce with not use_reduce_scatter (follow-up to #23731) (#23734)

Co-authored-by: Byron Hsu <byron@periodiclabs.ai> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 20:35:52 -07:00
parent 714173555c
commit 71029abd64
1 changed files with 1 additions and 0 deletions
@@ -335,6 +335,7 @@ class Qwen3MoeSparseMoeBlock(nn.Module):
        if (
            self.ep_size > 1
            and not should_allreduce_fusion
+            and not use_reduce_scatter
            and not should_use_dp_reduce_scatterv()
        ):
            final_hidden_states = moe_expert_parallel_all_reduce(final_hidden_states)