[Docs] Update MegaMoE handling and rerun benchmarks (#27726)

2026-06-09 19:00:43 -07:00
parent 6110ed671f
commit 6565b7c464
8 changed files with 133 additions and 193 deletions
@@ -135,7 +135,8 @@ this table (RTX PRO 6000, GH200, future chips) goes in the model's own `config.h
   - `supportedHardware` + the EXAMPLE cells: keep your tested families; **delete the
     `mi*` ids + AMD example cell if no AMD recipe**, etc. A GPU not in the shared catalog
     (e.g. RTX PRO 6000) → declare it in `config.hardware` and add its id here.
-   - `playgroundFeatures` axes: delete `megamoe` (non-Blackwell-MoE), `hisparse`
+   - `playgroundFeatures` axes: remove the `megamoe` backend option + the
+     `megamoeQuant` block from the `moe` axis (non-Blackwell-MoE), delete `hisparse`
     (non-DSA), `pdDisagg`/`router` (no PD), the `parsers` axis (no parsers), etc.
   - `quantizations` / `variants`: drop what the model doesn't ship; collapse `variants`
     to single `default` if there's no variant axis (then drop the `variant` half of
@@ -109,13 +109,12 @@ the `_playground.jsx` header):
 | Axis key | Widget | Use when |
 |---|---|---|
 | `attention` | TP / CP / DP-Attention sub-knobs (DP-Attention is a combined knob: its value is the DP degree AND toggles `--enable-dp-attention`) | Model exposes parallelism knobs in its cells (§2.2) and you want users to override them. |
-| `moe` | Backend select + EP knob | Model is MoE and supports multiple `--moe-*-backend` choices. |
+| `moe` | Backend select (incl. MegaMoE) + EP knob; picking the MegaMoE backend reveals a Quantization sub-select (W4A8/W4A4) | Model is MoE and supports multiple `--moe-*-backend` choices. For Blackwell MoE kernel-fusion, give the `megamoe` backend option a `requiresHw` (and optional `excludesStrategy`) gate, then add a sibling `megamoeQuant` block (`{stripEnv, options}`): W4A8 = `NUM_MAX` only, W4A4 adds the FP4-activations env vars; both strip the DeepEP dispatch env. |
 | `parsers` | Multi-toggle | Model has reasoning / tool-call parsers. |
 | `speculative` | Single-select chip group | Model has spec-decoding presets you want to expose. |
 | `pdDisagg` | Mode + transfer backend (+ optional per-backend env via `envWhen` hw-gate) + IB device + optional `router{port, command}` | Model supports prefill/decode disaggregation. When a PD role is active and `router` is set, the playground shows the router (SGLang Model Gateway) launch command as a separate companion block and retargets the cURL modal to `router.port` (clients hit the router, not the role servers). |
 | `hicache` | Enable + storage + write policy | Model is large enough that hierarchical KV cache matters. |
 | `hisparse` | Enable + host-ratio select; whole card gated on the live PD-Disagg mode being `decode` | DSA-style model (DeepSeek-V3.2 / V4, GLM-5) that supports decode-side hierarchical sparse attention. |
-| `megamoe` | Single-select with hw/strategy gating | Blackwell-only kernel fusion variant. |

 **Per-chip constraints**: any chip entry in any axis can be wrapped with
 `hide` / `disable` constraint objects:
@@ -1,11 +1,12 @@
 # Engine extension: add a new playground feature axis

 Loaded on demand by the `cookbook-add-model` skill. **Rare** — adding a model
-cookbook is data-only and never needs this. The current 8 built-in axes
+cookbook is data-only and never needs this. The current 7 built-in axes
 (`attention`, `moe`, `parsers`, `speculative`, `pdDisagg`, `hicache`,
-`hisparse`, `megamoe`) already cover the SGLang feature surface most cookbooks
-need. Only add a new axis if a real cookbook needs it and the feature does not
-fit any existing axis. Touches `_playground.jsx` only.
+`hisparse`) already cover the SGLang feature surface most cookbooks need.
+(MegaMoE is not its own axis — it lives inside `moe` as a backend option +
+a `megamoeQuant` sub-select.) Only add a new axis if a real cookbook needs it
+and the feature does not fit any existing axis. Touches `_playground.jsx` only.

 For the per-model config/cells/MDX reference see [authoring-reference.md](authoring-reference.md).

@@ -94,7 +95,7 @@ Template:
  // also receives the derived
  // value (as `derived`) and may use it as a no-op shortcut when the
  // user's pick matches base. Skip when your axis owns flags that never
-  // appear in base cells (PD-Disagg / HiCache / MegaMoE).
+  // appear in base cells (PD-Disagg / HiCache).
  // deriveFromBase: (cell, fc, h) => ({ ... }) | null,

  // Optional: hints for the renderer. Currently only pdDisagg uses this
@@ -103,7 +104,7 @@ Template:

  // Returns the axis card JSX. The outer div MUST have key={axisId} so
  // React can track it in the engine's map loop. Return null for
-  // axis-level gating (e.g. MegaMoE on Hopper). Lay out as a single
+  // axis-level gating (e.g. HiSparse when the live PD mode isn't `decode`). Lay out as a single
  // compact horizontal row: title on the left, fields after.
  render: ({ axisId, value, setValue, fc, base, s, h, renderChip, renderSelect, derived }) => {
    if (/* axis-level gating fails */) return null;
@@ -147,7 +148,7 @@ Template:
    the **default** compact control (a `<select>` dropdown). It filters
    hidden chips and disables greyed-out ones internally — no per-chip
    `evaluateChip` loop needed in the render body. Most axes use it
-    (attention, moe, pdDisagg, hisparse, hicache, megamoe). Pass
+    (attention, moe, pdDisagg, hisparse, hicache). Pass
    `{ hideValues: [<sentinel>] }` when your `deriveFromBase` resolved to
    a real value, so the inherit-sentinel ("Auto" / "Inherited" /
    "current") doesn't clutter the dropdown.
@@ -206,7 +207,8 @@ emits flags that should land somewhere specific, include the most likely
 anchor prefixes in your call. Order doesn't matter (set semantics).

 **Conditional strips** — Some axes strip ONLY when overridden
-(`attention.tp`, `moe.backend`, `speculative`, `megamoe`). Others strip
+(`attention.tp`, `moe.backend` (incl. the MegaMoE quant env), `speculative`).
+Others strip
 UNCONDITIONALLY whenever declared (`parsers`, `pdDisagg`, `hicache`). The
 header comment in `AXIS_HANDLERS` documents which policy each axis uses;
 follow the same pattern when adding a new axis. If unsure, prefer
@@ -214,9 +216,9 @@ conditional strip — it preserves base behavior when the user does not
 opt in.

 **Closure of `AXIS_HANDLERS`** — Inside a handler method, you can
-reference `AXIS_HANDLERS.<otherAxis>` for cross-handler calls (megamoe
-does this for `_gateOpen`). This works because `AXIS_HANDLERS` is in
-lexical scope. Do NOT use this for general logic — it tightly couples
+reference `AXIS_HANDLERS.<otherAxis>` for cross-handler calls (no built-in
+axis currently needs this, but it works because `AXIS_HANDLERS` is in
+lexical scope). Do NOT use this for general logic — it tightly couples
 handlers. Reserve it for one handler's helpers shared between its own
 `render` and `revertHidden`.

@@ -178,10 +178,32 @@ sgl-eval run gsm8k \\
        options: [
          { id: null,               label: "Inherited" },
          { id: "deepep",           label: "DeepEP",            flags: ["--moe-a2a-backend deepep"] },
+          // KEEP the MegaMoE option + the megamoeQuant block below ONLY for Blackwell
+          // MoE kernel-fusion models; DELETE both otherwise. requiresHw gates it to
+          // Blackwell (the engine hides it elsewhere); add excludesStrategy: [...] too
+          // for a strategy gate. Selecting MegaMoE reveals the Quantization sub-select.
+          { id: "megamoe",          label: "MegaMoE",           flags: ["--moe-a2a-backend megamoe"],
+            requiresHw: ["b200", "b300", "gb200", "gb300"] },
          { id: "flashinfer_mxfp4", label: "FlashInfer (MXFP4)", flags: ["--moe-runner-backend flashinfer_mxfp4"] },
          { id: "marlin",           label: "Marlin (W4A16)",    flags: ["--moe-runner-backend marlin"] },
        ],
      },
+      // MegaMoE quantization sub-select — shown only when backend === "megamoe".
+      // W4A4 adds the FP4-activations env vars; both strip the DeepEP dispatch env.
+      // DELETE this block if there's no MegaMoE backend option above.
+      megamoeQuant: {
+        stripEnv: ["SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK"],
+        options: [
+          { id: "w4a8", label: "W4A8",
+            env: ["SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320"] },
+          { id: "w4a4", label: "W4A4",
+            env: [
+              "SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320",
+              "SGLANG_OPT_DEEPGEMM_MEGA_MOE_USE_FP4_ACTS=1",
+              "SGLANG_OPT_DEEPGEMM_MEGA_MOE_USE_MXF4_KIND=1",
+            ] },
+        ],
+      },
      ep: { label: "EP", values: [
        null, 1, 2, 4, 8,
        { value: 16, disable: { nodes: ["single"] },
@@ -275,19 +297,6 @@ sgl-eval run gsm8k \\
      ],
      defaultHostRatio: 10,
    },
-
-    // ----- Card: "MegaMoE" -----  KEEP only for Blackwell MoE kernel-fusion models.
-    megamoe: {
-      requiresHw: ["b200", "b300", "gb200", "gb300"],
-      excludesStrategy: ["low-latency", "balanced"],
-      stripEnv: ["SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK"],
-      options: [
-        { id: "disabled", label: "Disabled" },
-        { id: "w4a8",     label: "W4A8",
-          flags: ["--moe-a2a-backend megamoe"],
-          env: ["SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK=8320"] },
-      ],
-    },
  },

  // EXAMPLE cells — one per hardware family to show the shape. REPLACE each with
@@ -53,8 +53,8 @@ than restating.
  NVL72) — NOT every `multi-N` hw (standard-IB DeepEP / Marlin multi-node don't need it).
 - `github.cookbookModel` matches the issue-template `model` dropdown value.
 - `playgroundFeatures` axes are pruned to what the model supports — no empty/stub axes
-  (`megamoe` only on Blackwell MoE; `hisparse` only DSA-style; `pdDisagg.router` only with
-  a PD topology).
+  (the `moe` axis's MegaMoE backend option + `megamoeQuant` block only on Blackwell MoE,
+  gated by `requiresHw`; `hisparse` only DSA-style; `pdDisagg.router` only with a PD topology).
 - **No leftover `__TOKEN__`** — the config was stamped from the template and every
  placeholder is filled (`grep -rn '__[A-Z_]*__'` on the new config/benchmarks/MDX returns
  nothing).