diff --git a/docs/cookbook/autoregressive/MiniMax/MiniMax-M2.5.mdx b/docs/cookbook/autoregressive/MiniMax/MiniMax-M2.5.mdx
index 77a7851f03..b3b8319a0a 100644
--- a/docs/cookbook/autoregressive/MiniMax/MiniMax-M2.5.mdx
+++ b/docs/cookbook/autoregressive/MiniMax/MiniMax-M2.5.mdx
@@ -1,5 +1,6 @@
---
title: MiniMax-M2.5
+description: "Deploy MiniMax-M2.5 with SGLang on NVIDIA, AMD, or Ascend A2, with an Ascend A3 reference configuration."
metatags:
description: "Deploy MiniMax-M2.5 with SGLang - community contribution guide for MiniMax M2.5 model deployment."
---
@@ -23,7 +24,7 @@ For more details, please refer to the [official MiniMax-M2.5 announcement](https
SGLang offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.
-Please refer to the [official SGLang installation guide](../../../docs/get-started/install) for installation instructions.
+Please refer to the [official SGLang installation guide](/docs/get-started/install) for installation instructions.
**For AMD MI300X/MI325X/MI355X GPUs:**
@@ -35,19 +36,75 @@ docker pull lmsysorg/sglang:v0.5.9-rocm720-mi30x
docker pull lmsysorg/sglang:v0.5.9-rocm720-mi35x
```
+### Ascend A2 prerequisites (smoke tested)
+
+Select **Ascend A2** for the tested **8 × Ascend 910B4, 32 GiB HBM per device** configuration. It loads the W8A8 main model with TP=8 and serves one request at a time, without a draft model. The final run passed six simple chat checks and six synthetic generation probes, including 3500 input tokens plus 128 output tokens; see [the validation record](#5-4-ascend-a2-smoke-validation) for the runtime versions and test limits.
+
+1. Prepare a compatible Ascend NPU runtime using the [NPU installation guide](/docs/hardware-platforms/ascend-npus/getting-started/installation). The tested runtime was SGLang `0.5.14.dev172+gf308abc05`, TorchNPU 2.10.0, and sgl-kernel-npu 2026.5.1. Follow-up inspection recorded CANN 9.0.0 and driver 25.5.2. An image digest was not captured; the A3 image below was not validated for this A2 run.
+2. Run `npu-smi info` inside the container and confirm all eight 910B4 devices are visible and healthy. Reserve space for the approximately 230.82 GB main weights, download cache, and runtime.
+3. Download the complete main model repository, including tokenizer and configuration files:
+
+```bash Command
+python -m pip install modelscope
+modelscope download --model Eco-Tech/MiniMax-M2.5-w8a8-QuaRot \
+ --local_dir /models/MiniMax-M2.5-w8a8-QuaRot
+```
+
+Enter absolute paths inside your container in the selector. Both directories default to the complete model directory. If you keep tokenizer files separately, use files from the same model revision and set **Tokenizer Directory** accordingly. The original experiment used separate metadata and weights directories, as recorded in section 5.4.
+
+### Ascend A3 prerequisites (unvalidated reference)
+
+The A3 selector provides **one Ascend A3 node with 8 cards / 16 NPU dies**, running prefill and decode together (PD co-located). It uses W8A8 weights and EAGLE3 speculative decoding. Each A3 card contains two dies; a cloud allocation labelled "1 NPU" is not a full node. The [minimum recommended W8A8 configuration](/docs/hardware-platforms/ascend-npus/model-deployment/tutorials/minimax_m2_5#model-weights) is 4 cards / 8 dies with 64 GB per die, while this preset requires all 16 dies.
+
+
+This recipe is based on the upstream Ascend best practice. Hardware validation of the generated A3 cookbook command is pending; the reference accuracy and performance values below are not measurements of this recipe. The tested A2 selector is a separate low-concurrency configuration and does not validate the A3 preset.
+
+
+1. Set up the Ascend runtime using the [MiniMax-M2.5 installation instructions](/docs/hardware-platforms/ascend-npus/model-deployment/tutorials/minimax_m2_5#installation). The reference environment uses CANN 9.0.0 and `quay.io/ascend/sglang:cann9.0.0-a3-v0.5.16`. Map `/dev/davinci0` through `/dev/davinci15` and mount your model directory into the container, for example `-v /path/to/models:/models`.
+2. Check device health and allocation with `npu-smi info` inside the container. Confirm that all 16 dies are visible. For source builds, follow [installation with NPU support](/docs/hardware-platforms/ascend-npus/getting-started/installation), keep the CANN, TorchNPU, and kernel versions compatible, and record the SGLang commit you install.
+3. Reserve space for the **230.82 GB** main weights, the draft model, download cache, and at least **30 GB** for the container image. Download both models below, or reuse complete copies from shared storage.
+
+| Role | Model | Directory inside the container |
+| --- | --- | --- |
+| W8A8 main model | [Eco-Tech/MiniMax-M2.5-w8a8-QuaRot](https://www.modelscope.cn/models/Eco-Tech/MiniMax-M2.5-w8a8-QuaRot) | `/models/MiniMax-M2.5-w8a8-QuaRot` |
+| EAGLE3 draft model | [sgl-npu/MiniMax-M2.5-eagel-model-0318](https://www.modelscope.cn/models/sgl-npu/MiniMax-M2.5-eagel-model-0318) | `/models/MiniMax-M2.5-eagel-model-0318` |
+
+The draft repository identifier contains `eagel`; use that exact spelling. Keep its `custom_eagle3` Python package alongside the draft weights. The generated command adds this directory to `PYTHONPATH` and sets `SGLANG_EXTERNAL_MODEL_PACKAGE=custom_eagle3`.
+
+You can download the weights with the [ModelScope CLI](https://github.com/modelscope/modelscope/blob/master/docs/source/command.md#download):
+
+```bash Command
+python3 -m pip install modelscope
+modelscope download --model Eco-Tech/MiniMax-M2.5-w8a8-QuaRot \
+ --local_dir /models/MiniMax-M2.5-w8a8-QuaRot
+modelscope download --model sgl-npu/MiniMax-M2.5-eagel-model-0318 \
+ --local_dir /models/MiniMax-M2.5-eagel-model-0318
+```
+
+Before launching, run `ip -brief address` and select an interface available inside the container for both HCCL and Gloo. The selector defaults to `lo` for this single-node setup.
+
+For comparable performance, use the host settings from the [selected 8-card reference](/docs/hardware-platforms/ascend-npus/model-deployment/best-practices/minimax_m2_5#minimax-m2-5-w8a8-8p-in3k5-out1k5-50ms). These affect the whole host; apply them on the dedicated benchmark server before running the generated command:
+
+```bash Command
+echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
+sysctl -w vm.swappiness=0
+sysctl -w kernel.numa_balancing=0
+sysctl -w kernel.sched_migration_cost_ns=50000
+```
+
## 3. Model Deployment
This section provides deployment configurations optimized for different hardware platforms and use cases.
### 3.1 Basic Configuration
-**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform, deployment strategy, and feature capabilities.
+**Interactive Command Generator**: Select your hardware to generate the deployment command. For **Ascend A2**, complete the A2 prerequisites, expand **Deployment paths**, and enter the model and tokenizer directories; the command reproduces the tested single-request settings on port 31216. For **Ascend A3**, complete its separate prerequisites, expand **Deployment paths and network**, enter both model directories as absolute paths inside the container, and set the HCCL / Gloo interface. The unvalidated A3 preset uses 3,500 input and 1,500 output tokens as its reference workload.
### 3.2 Configuration Tips
-**Key Parameters:**
+**GPU parameters:** The following recommendations apply to the NVIDIA and AMD recipes. The Ascend presets use the separate settings described below.
@@ -117,13 +174,53 @@ This section provides deployment configurations optimized for different hardware
- **4-GPU deployment**: Requires 4× GPUs (e.g., MI300X, MI325X, MI355X) with TP=4, EP=4
- **8-GPU deployment**: Requires 8× GPUs (e.g., MI300X, MI325X, MI355X) with TP=8, EP=8
+### 3.3 Ascend A3 configuration (unvalidated reference)
+
+The generated A3 command follows the [8-card throughput reference](/docs/hardware-platforms/ascend-npus/model-deployment/best-practices/minimax_m2_5#minimax-m2-5-w8a8-8p-in3k5-out1k5-50ms). Keep the environment variables and server arguments together when reproducing that workload.
+
+| Setting | A3 preset | Purpose |
+| --- | --- | --- |
+| Device and parallelism | `--device npu --tp-size 16 --dp-size 16 --enable-dp-attention` | Use 16 dies on one node with DP attention. |
+| Quantization and communication | `--quantization modelslim --moe-a2a-backend ascend_fuseep --fuseep-mode 2` | Run the Ascend W8A8 weights with FuseEP. |
+| Speculative decoding | EAGLE3, 3 steps, top-k 1, 4 draft tokens, unquantized draft | Load the custom draft model from the supplied directory. |
+| Static memory and concurrency | `--mem-fraction-static 0.75 --max-running-requests 320` | Use the memory and request limits of the reference workload. |
+| Prefill | Chunk size 196608, maximum prefill tokens 8192, prefill delayer enabled | Match the reference scheduling configuration. |
+| NPU graph batches | `--cuda-graph-bs-decode 1 2 4 8 12 16 20` | Set capture sizes; the argument retains `cuda` in its name on NPU. |
+| API | `127.0.0.1:6688` | Send requests from the same host or container network. |
+
+The A3 preset fixes the reasoning parser to `minimax-append-think` and the tool parser to `minimax-m2`. The shared form displays a fixed 8-card count and enabled parsers; these controls are read-only for this preset. To use fewer A3 cards or a different workload, follow the corresponding [Ascend best practice](/docs/hardware-platforms/ascend-npus/model-deployment/best-practices/minimax_m2_5) with its complete configuration.
+
+### 3.4 Ascend A2 configuration (smoke tested)
+
+The A2 selector fixes TP=8, ModelSlim quantization, BF16, `--context-length 4096`, and `--max-running-requests 1`. It uses prefill chunks of 128, maximum prefill tokens of 4096, static memory fraction 0.96, and `--max-total-tokens 4096`, and disables graph capture and radix cache. No EAGLE3 draft model is required.
+
+The `minimax` reasoning parser separated reasoning from answer content in the tested runtime. The `minimax-m2` tool-call parser is configured, but tool calling was not tested. The shared form displays a fixed 8-card count and enabled parsers; these controls are read-only for this preset. The final run allocated a **4096-token KV pool on all eight ranks** and completed a synthetic 3500-input/128-output request. This validates that request size on this allocation, not every workload up to the configured limit. See section 5.4 for the earlier capacity failure and the final measurements. GPQA accuracy, standardized throughput, and sustained stability remain unmeasured for this single-request preset; [section 5.5](#5-5-ascend-a2-performance-benchmark) reports synthetic performance measurements using a separate server configuration with concurrency up to 8.
+
## 4. Model Invocation
+For the **Ascend A3** preset, use [the A3 service checks](#4-3-verify-the-ascend-a3-service), which target port 6688 and read the deployed model identifier from `/v1/models`. The GPU examples below use their own model identifier and endpoints.
+
+For **Ascend A2**, use `http://127.0.0.1:31216/v1` from the same host or container network. After startup completes, this short request uses the settings from the smoke test:
+
+```python Example
+from openai import OpenAI
+
+client = OpenAI(base_url="http://127.0.0.1:31216/v1", api_key="EMPTY")
+model = client.models.list().data[0].id
+response = client.chat.completions.create(
+ model=model,
+ messages=[{"role": "user", "content": "What is the capital of France? Answer with the city name only."}],
+ temperature=0,
+ max_tokens=512,
+)
+print(response.choices[0].message.content)
+```
+
### 4.1 Basic Usage
For basic API usage and request examples, please refer to:
-- [SGLang Basic Usage Guide](../../../docs/basic_usage/send_request)
+- [SGLang Basic Usage Guide](/docs/basic_usage/send_request)
**Testing Deployment:**
@@ -427,9 +524,50 @@ print(final_response.choices[0].message.content)
# Output: "The weather in Beijing is currently 22°C and sunny."
```
+### 4.3 Verify the Ascend A3 service
+
+Wait for the server to finish loading the weights and capturing graphs. Keep the generated command running, then open another terminal in the same container. The A3 preset listens on port **6688**.
+
+```bash Command
+curl --fail --silent --show-error --max-time 30 http://127.0.0.1:6688/health
+curl --fail --silent --show-error --max-time 30 http://127.0.0.1:6688/v1/models
+
+python3 - <<'PY'
+import json
+from urllib.request import Request, urlopen
+
+base = "http://127.0.0.1:6688"
+with urlopen(f"{base}/v1/models", timeout=30) as response:
+ model = json.load(response)["data"][0]["id"]
+payload = {
+ "model": model,
+ "messages": [{"role": "user", "content": "What is the capital of France?"}],
+ "temperature": 0,
+ "max_tokens": 2048,
+}
+request = Request(
+ f"{base}/v1/chat/completions",
+ data=json.dumps(payload).encode(),
+ headers={"Content-Type": "application/json"},
+)
+with urlopen(request, timeout=300) as response:
+ result = json.load(response)
+print(json.dumps(result, indent=2, ensure_ascii=False))
+choice = result["choices"][0]
+assert (choice["message"].get("content") or "").strip(), "No final answer returned"
+assert choice["finish_reason"] != "length", "Increase the token budget and retry"
+PY
+```
+
+Check that the final answer identifies Paris. Repeat requests and check the server logs for runtime errors. After any concurrent test, check `/health` again. A successful HTTP response alone does not establish model accuracy; use the evaluation below for that.
+
## 5. Benchmark
-This section uses **industry-standard configurations** for comparable benchmark results.
+For **Ascend A3**, use [the A3 throughput and accuracy benchmarks](#5-3-ascend-a3-benchmarks) with the command generated in section 3.
+
+For **Ascend A2**, [section 5.4](#5-4-ascend-a2-smoke-validation) records simple chat checks and serial synthetic generation probes. [Section 5.5](#5-5-ascend-a2-performance-benchmark) reports the official benchmark tool's results on a separate configuration with concurrency up to 8 and a synthetic workload. [Section 5.6](#5-6-ascend-a2-reference-length-performance) measures 3500-input/1500-output requests, and [section 5.7](#5-7-ascend-a2-fixed-50-question-gsm8k-regression) records a fixed 50-question GSM8K deployment regression. No GPQA accuracy result is reported here.
+
+The benchmark sections below use **industry-standard configurations** for comparable benchmark results; the A2 smoke record has a separate, limited scope.
### 5.1 Speed Benchmark
@@ -1046,3 +1184,333 @@ subject: world_religions, #q:171, acc: 0.895
Total latency: 81.468
Average accuracy: 0.825
```
+
+### 5.3 Ascend A3 benchmarks
+
+Use the A3 command generated in section 3 for both accuracy and performance evaluation. Record the SGLang commit, container image digest, CANN / TorchNPU / kernel versions, model revisions, visible dies, and host tuning with your logs. When testing `main`, record the installed source commit separately from the base container tag.
+
+#### 5.3.1 Throughput workload
+
+The [Ascend reference](/docs/hardware-platforms/ascend-npus/model-deployment/best-practices/minimax_m2_5#minimax-m2-5-w8a8-8p-in3k5-out1k5-50ms) lists **50 ms TPOT** for 8 A3 cards, W8A8, PD co-located, and 3.5k input / 1.5k output tokens. This is a reference value, not a measurement from a different allocation or software build. The reference table does not specify the TPOT percentile.
+
+After warm-up, run the reference workload from the same container:
+
+```bash Command
+set -o pipefail
+python3 -m sglang.bench_serving \
+ --dataset-name random \
+ --backend sglang \
+ --host 127.0.0.1 \
+ --port 6688 \
+ --max-concurrency 320 \
+ --num-prompts 1280 \
+ --random-input-len 3500 \
+ --random-output-len 1500 \
+ --random-range-ratio 1 \
+ --seed 1 2>&1 | tee minimax-a3-throughput.log
+```
+
+Keep the request success and failure counts, actual token counts, output and total token throughput, TTFT, TPOT, and end-to-end latency. Report mean, median, and P99 separately. Repeat the measurement with separate logs and check server health afterwards. Changes to concurrency, quantization, speculative decoding, or token lengths define a different benchmark configuration.
+
+#### 5.3.2 GPQA Diamond accuracy
+
+Install EvalScope in a separate environment following [Ascend accuracy evaluation](/docs/hardware-platforms/ascend-npus/evaluation/accuracy_evaluation). Keep the SGLang server running with the generated A3 command. The following request configuration follows the [upstream 8-card GPQA test](https://github.com/sgl-project/sglang/blob/c389dd086340878934bf96bc209534f1fbb8440c/test/registered/npu/accuracy/minimax_m2_5/test_npu_minimax_m2_5_w8a8_8p_in3k5_out1k5_50ms_gpqa.py):
+
+```python evaluate_minimax_a3.py
+import json
+from urllib.request import urlopen
+
+from evalscope import TaskConfig, run_task
+
+with urlopen("http://127.0.0.1:6688/v1/models", timeout=30) as response:
+ model = json.load(response)["data"][0]["id"]
+
+run_task(task_cfg=TaskConfig(
+ model=model,
+ api_url="http://127.0.0.1:6688/v1/chat/completions",
+ eval_type="openai_api",
+ datasets=["gpqa_diamond"],
+ eval_batch_size=64,
+ generation_config={"max_tokens": 65536, "temperature": 1.0},
+ timeout=60000,
+ stream=True,
+ work_dir="./minimax-a3-gpqa",
+))
+```
+
+Run the full **198-question** dataset and retain the EvalScope version, dataset revision, generation settings, reports, and all attempt scores. The upstream test declares an accuracy baseline of **0.852**; its [comparison utility](https://github.com/sgl-project/sglang/blob/c389dd086340878934bf96bc209534f1fbb8440c/python/sglang/test/ascend/e2e/test_npu_accuracy_utils.py) allows a fluctuation of 5 questions, giving a threshold of `0.852 - 5/198`, approximately **0.8267**, with up to 3 attempts. These are upstream validation criteria, not measured accuracy for this cookbook command.
+
+{/* TODO: Add A3 startup, accuracy, and performance evidence after validating the generated command on the target hardware. */}
+
+### 5.4 Ascend A2 smoke validation
+
+On September 22, 2026, the W8A8 main model completed loading, warm-up, and three short chat requests on **8 Ascend 910B4 devices with 32 GiB HBM each**. Eight-rank HCCL all-reduce and BF16 matrix operations also passed. The final September 23 run increased the static memory fraction to 0.96 and set `--max-total-tokens 4096`; these are the settings now exposed by the **Ascend A2** selector. These experiments used no EAGLE3 draft model and are separate from the A3 reference configuration.
+
+The runtime was SGLang `0.5.14.dev172+gf308abc05` (commit `f308abc05212c2f5f455de22a525e14afa63ee4f`), Python 3.11.15, Torch 2.10.0+cpu, TorchNPU 2.10.0, Transformers 5.8.1, and sgl-kernel-npu 2026.5.1. A follow-up inspection of the same environment on September 23 recorded CANN 9.0.0 and driver 25.5.2. The supplied environment had modified `python/pyproject.toml` and a deleted `python/pyproject_npu.toml`; this was not a clean source-build validation. An image digest and immutable model revision were not captured, so this record is not a fully pinned environment specification.
+
+After downloading the W8A8 main model listed above, the final tested service settings were:
+
+```bash Command
+export HCCL_BUFFSIZE=128
+export OMP_NUM_THREADS=1
+export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True
+python -m sglang.launch_server \
+ --model-path /workspace/issue1215/model \
+ --tokenizer-path /workspace/issue1215/metadata \
+ --device npu --tp-size 8 --quantization modelslim --dtype bfloat16 \
+ --reasoning-parser minimax --tool-call-parser minimax-m2 \
+ --host 127.0.0.1 --port 31216 \
+ --context-length 4096 --max-running-requests 1 \
+ --chunked-prefill-size 128 --max-prefill-tokens 4096 \
+ --mem-fraction-static 0.96 --max-total-tokens 4096 \
+ --disable-cuda-graph --disable-radix-cache
+```
+
+The paths above are the experiment paths: `model` contains the complete main weights, while `metadata` contains the tokenizer and configuration files from the same model repository, downloaded separately while weights were transferring. Replace both paths with your local directories. The selector defaults both paths to one complete repository directory; this path convenience differs from the original experiment's separate metadata directory. Keep tokenizer and weights from the same revision.
+
+The original September 22 requests used `/v1/chat/completions`, `temperature=0`, and `max_tokens=512`:
+
+| Prompt | Answer content | Finish reason |
+| --- | --- | --- |
+| What is the capital of France? Answer with the city name only. | `Paris` | `stop` |
+| 请用中文回答:2加2等于多少? | `2加2等于4。` | `stop` |
+| Return only a JSON object with one key named result whose value is 7 * 8. | `{"result":56}` (parsed JSON) | `stop` |
+
+The `minimax` reasoning parser separated reasoning from answer content in this runtime. The three answers contained no thinking tags, and the final health check returned HTTP 200. The tool-call parser was configured but tool calling was not tested.
+
+**Earlier baseline:** a September 23 serial follow-up retained `--mem-fraction-static 0.95` without an explicit total-token setting. After two warm-up requests, five of six simple answer checks passed; the JSON check exhausted its 128-token output budget and was inconclusive. Reasoning consumes that budget as well as final-answer tokens. Startup allocated 896 KV tokens; requests with 1024 and 2048 input tokens were rejected with `BadRequestError`, reporting a maximum input length of 890. The client stopped after those two failures, before attempting 3500 input tokens, and the subsequent health check returned HTTP 200. These limits describe the earlier baseline, not the final configuration above.
+
+**Final September 23 run:** with memory fraction 0.96 and `--max-total-tokens 4096`, startup reported 4096 allocated KV tokens on each of the eight ranks. After two warm-up requests, all six chat checks passed with `temperature=0`, `max_tokens=512`, and finish reason `stop`:
+
+| Check | Answer content | Completion tokens | End-to-end time (s) | First observed output (s) |
+| --- | --- | --- | --- | --- |
+| Capital of France | `Paris` | 87 | 21.172 | 0.474 |
+| 2 + 2 | `4` | 53 | 12.598 | 0.466 |
+| JSON object for 7 × 8 | `{"result":56}` | 185 | 43.747 | 0.477 |
+| 中国首都 | `北京` | 138 | 31.891 | 0.456 |
+| Copy a word | `orchard` | 73 | 16.724 | 0.459 |
+| Sort 3, 1, 2 as a JSON array | `[1,2,3]` | 142 | 33.590 | 0.465 |
+
+Six native `/generate` probes used exact input-token IDs from `/v1/tokenize`, requested 128 output tokens, and set `ignore_eos=true`. All completed with exactly 128 output tokens and the expected length limit. The inputs were synthetic repeated text: these are individual operational measurements, not a quality evaluation or standardized throughput benchmark.
+
+| Input tokens | Output tokens | End-to-end time (s) | First observed output (s) |
+| --- | --- | --- | --- |
+| 64 | 128 | 29.970 | 0.494 |
+| 128 | 128 | 29.313 | 0.478 |
+| 256 | 128 | 30.193 | 0.700 |
+| 1024 | 128 | 32.165 | 2.103 |
+| 2048 | 128 | 34.451 | 4.218 |
+| 3500 | 128 | 39.803 | 10.055 |
+
+Timing includes HTTP/SSE processing and ends at the stream's `DONE` event. First output means the first nonempty content or reasoning for chat, or the first text or positive output-token count for native generation; it is not the time to the final answer or a kernel timing. Completion counts include reasoning and special tokens as reported by the server. Parser buffering and chunks containing multiple tokens can affect observed timing. No cache flush was performed. Each input length was measured once at concurrency 1, so these results do not establish a latency percentile or saturation throughput.
+
+The final probe run completed and its post-run health check returned HTTP 200. Confirm startup KV allocation and actual request limits on your allocation; the successful 3500-input/128-output probe does not establish the full 4096-token boundary for every workload.
+
+These checks establish simple chat functionality and the listed synthetic request sizes. They do not establish GPQA accuracy, long-context answer quality, sustained stability, tool calling, or performance under the A3 reference workload. This observed 8×32 GiB low-concurrency run does not replace the upstream recommended hardware configuration.
+
+### 5.5 Ascend A2 performance benchmark
+
+On September 23, 2026, the official `python -m sglang.bench_serving` tool completed **10 runs and 76 measured requests** on the same 8 × 910B4 allocation and runtime recorded in section 5.4. All requests succeeded, with the planned input lengths and exactly 128 output tokens each. Warm-up requests are excluded from the 76 measured requests.
+
+This benchmark used a separate server configuration: `--mem-fraction-static 0.97`, `--max-total-tokens 16384`, `--context-length 8192`, and `--max-running-requests 8`. The interactive selector retains the single-request configuration from section 5.4. The benchmark launch command was:
+
+```bash Command
+export HCCL_BUFFSIZE=128
+export OMP_NUM_THREADS=1
+export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True
+python -m sglang.launch_server \
+ --model-path /workspace/issue1215/model \
+ --tokenizer-path /workspace/issue1215/metadata \
+ --device npu --tp-size 8 --quantization modelslim --dtype bfloat16 \
+ --reasoning-parser minimax --tool-call-parser minimax-m2 \
+ --host 127.0.0.1 --port 31216 \
+ --context-length 8192 --max-running-requests 8 \
+ --chunked-prefill-size 128 --max-prefill-tokens 4096 \
+ --mem-fraction-static 0.97 --max-total-tokens 16384 \
+ --disable-cuda-graph --disable-radix-cache
+```
+
+#### 5.5.1 Workload and measured results
+
+The source data contained **16 locally authored synthetic English prompts in ShareGPT-compatible JSON format**; it was not the ShareGPT dataset. The tool used `--dataset-name random`, `--random-range-ratio 1`, and `--tokenize-prompt` to produce exact input lengths of 128 or 1024 tokens. Each run used `--request-rate inf`, temperature 0, one warm-up request, and `--flush-cache`. The serial control used seed 17; each repeated group used seeds **17, 29, and 43**, once each.
+
+The repeated groups below report the arithmetic **mean and minimum–maximum across three runs**. TTFT, TPOT, and E2E use each run's mean latency before computing this summary. The serial control has one run and therefore no repeat range. Output throughput is aggregate output tokens per second from the benchmark, not per-request decoding speed. All latency values are in milliseconds.
+
+**128 input / 128 output tokens, concurrency 1** — one run of 4 requests; 4/4 succeeded.
+
+| Metric | Value |
+| --- | --- |
+| Output throughput (tok/s) | 4.266 |
+| Mean TTFT (ms) | 486.589 |
+| Mean TPOT (ms) | 232.385 |
+| Mean E2E (ms) | 29999.459 |
+
+**128 input / 128 output tokens, concurrency 4** — three runs of 8 requests; 24/24 succeeded.
+
+| Metric | Mean | Min–max |
+| --- | --- | --- |
+| Output throughput (tok/s) | 16.731 | 16.618–16.810 |
+| Mean TTFT (ms) | 807.497 | 801.011–814.685 |
+| Mean TPOT (ms) | 234.492 | 233.452–236.043 |
+| Mean E2E (ms) | 30588.025 | 30449.379–30792.190 |
+
+**128 input / 128 output tokens, concurrency 8** — three runs of 8 requests; 24/24 succeeded.
+
+| Metric | Mean | Min–max |
+| --- | --- | --- |
+| Output throughput (tok/s) | 32.457 | 32.373–32.532 |
+| Mean TTFT (ms) | 1312.882 | 1276.591–1359.876 |
+| Mean TPOT (ms) | 237.912 | 237.480–238.622 |
+| Mean E2E (ms) | 31527.731 | 31456.127–31607.222 |
+
+**1024 input / 128 output tokens, concurrency 8** — three runs of 8 requests; 24/24 succeeded.
+
+| Metric | Mean | Min–max |
+| --- | --- | --- |
+| Output throughput (tok/s) | 13.694 | 13.291–13.942 |
+| Mean TTFT (ms) | 12582.293 | 12329.436–13062.333 |
+| Mean TPOT (ms) | 272.578 | 268.251–280.396 |
+| Mean E2E (ms) | 47199.685 | 46422.970–48672.617 |
+
+Every run had a 100% request success rate; this measures successful request completion, not answer accuracy. Per-run percentiles remain in the detailed benchmark output and are not averaged into pooled percentiles. These short tests do not establish sustained stability or long-context answer quality. Their workload, runtime, and hardware differ from the A3 reference, so the table does not provide an A2-versus-A3 comparison.
+
+#### 5.5.2 Reproduce the benchmark
+
+Start the server above and wait for `/health` to return HTTP 200. Replace the two model paths below with your container paths, using the same tokenizer revision as the weights. Save the following as `benchmark_minimax_a2.py` and run it from the tested SGLang environment. It creates the synthetic data used here and runs the fixed suite, retaining each command, log, and detailed JSONL result. It refuses to overwrite an existing result and stops if a run fails validation.
+
+```python benchmark_minimax_a2.py
+import json
+from pathlib import Path
+import subprocess
+import sys
+
+model = "/workspace/issue1215/model"
+tokenizer = "/workspace/issue1215/metadata"
+out = Path("minimax-a2-perf")
+out.mkdir(exist_ok=True)
+topics = [
+ "gardening", "astronomy", "cooking", "geometry", "music", "weather",
+ "transport", "ecology", "history", "software", "literature", "chemistry",
+ "geography", "photography", "materials", "language",
+]
+data = [{"id": f"synthetic-{i}", "conversations": [
+ {"from": "human", "value": (
+ f"Explain the basic principles of {topic}. Give a clear example and discuss its practical limitations. "
+ ) * 8},
+ {"from": "gpt", "value": "This is a synthetic performance workload, not a quality evaluation."},
+]} for i, topic in enumerate(topics)]
+dataset = out / "synthetic-prompts.json"
+dataset.write_text(json.dumps(data), encoding="utf-8")
+cases = ([(128, 1, 4, 17)]
+ + [(128, c, 8, s) for c in (4, 8) for s in (17, 29, 43)]
+ + [(1024, 8, 8, s) for s in (17, 29, 43)])
+for length, concurrency, count, seed in cases:
+ name = f"in{length}-out128-c{concurrency}-n{count}-seed{seed}"
+ result = out / f"{name}.jsonl"
+ if result.exists():
+ raise RuntimeError(f"Refusing to overwrite {result}")
+ command = [
+ sys.executable, "-m", "sglang.bench_serving", "--backend", "sglang",
+ "--base-url", "http://127.0.0.1:31216", "--model", model, "--tokenizer", tokenizer,
+ "--dataset-name", "random", "--dataset-path", str(dataset),
+ "--random-input-len", str(length), "--random-output-len", "128",
+ "--random-range-ratio", "1", "--tokenize-prompt", "--num-prompts", str(count),
+ "--max-concurrency", str(concurrency), "--request-rate", "inf",
+ "--temperature", "0", "--seed", str(seed), "--warmup-requests", "1",
+ "--flush-cache", "--output-details", "--disable-tqdm", "--output-file", str(result),
+ ]
+ (out / f"{name}.command.json").write_text(json.dumps(command, indent=2), encoding="utf-8")
+ with (out / f"{name}.log").open("w", encoding="utf-8") as log:
+ subprocess.run(command, stdout=log, stderr=subprocess.STDOUT, check=True)
+ rows = [json.loads(line) for line in result.read_text().splitlines() if line.strip()]
+ if len(rows) != 1:
+ raise RuntimeError(f"Expected exactly one result in {result}")
+ row = rows[0]
+ if not (row["completed"] == count
+ and row["input_lens"] == [length] * count
+ and row["output_lens"] == [128] * count
+ and len(row["errors"]) == count and not any(row["errors"])):
+ raise RuntimeError(f"Incomplete or mismatched run: inspect {result}")
+ print(name, row["output_throughput"], flush=True)
+```
+
+Retain failed runs and missing-run records when reporting a reproduction; do not summarize only successful seeds. Keep each run's mean and percentile metrics separate. The synthetic data and performance measurements provide no GPQA or other model-quality score.
+
+### 5.6 Ascend A2 reference-length performance
+
+On September 24, 2026, the same A2 hardware and runtime completed three official `sglang.bench_serving` runs with **3500 input tokens and 1500 output tokens**, using seeds **1, 17, and 29**. Each run measured two requests at client concurrency 2. All **6/6 measured requests** completed with the exact planned token lengths and no request errors: 21000 input tokens and 9000 output tokens in total. Each run's one-request, 32-output-token warm-up is excluded from these counts.
+
+The server used the section 5.5 launch command with `--context-length 9216` instead of 8192. Its other settings remained TP8, ModelSlim, BF16, memory fraction 0.97, KV limit 16384, maximum running requests 8, prefill chunk 128, maximum prefill tokens 4096, and graph capture and radix cache disabled. **Client concurrency 2 did not change the server's maximum of 8.** No draft model was used; the interactive selector's conservative default is unchanged.
+
+The workload reused the 16 synthetic prompts from section 5.5. It matches only the A3 reference's token lengths, not its ShareGPT workload, concurrency 320, hardware, or EAGLE3 configuration. It does not validate A3 throughput or the reference's 50 ms TPOT target.
+
+| Metric | Mean | Min–max |
+| --- | --- | --- |
+| Output throughput (tok/s) | 8.004 | 7.896–8.065 |
+| Total throughput (tok/s) | 26.680 | 26.319–26.882 |
+| Mean TTFT (ms) | 10317.118 | 10267.749–10365.114 |
+| Mean TPOT (ms) | 243.163 | 241.222–246.570 |
+| Mean E2E (s) | 374.818 | 371.957–379.927 |
+
+These are arithmetic means and minimum–maximum values across the three run-level means or rates, not request extrema. Per-run percentiles are retained separately and are not averaged. Six measured requests provide a bounded performance observation, not a sustained-load or stability result.
+
+To reproduce, generate `minimax-a2-perf/synthetic-prompts.json` using the data-creation portion of section 5.5.2, launch the server with context 9216, and run the following from that working directory. Use the same model/tokenizer paths as the server, and retain each JSONL and log, including failures:
+
+```bash Command
+set -euo pipefail
+mkdir -p minimax-a2-reference-perf
+for SEED in 1 17 29; do
+ RESULT="minimax-a2-reference-perf/in3500-out1500-c2-n2-seed${SEED}.jsonl"
+ test ! -e "$RESULT"
+ python -m sglang.bench_serving \
+ --backend sglang --base-url http://127.0.0.1:31216 \
+ --model /workspace/issue1215/model \
+ --tokenizer /workspace/issue1215/metadata \
+ --dataset-name random --dataset-path minimax-a2-perf/synthetic-prompts.json \
+ --random-input-len 3500 --random-output-len 1500 --random-range-ratio 1 \
+ --tokenize-prompt --num-prompts 2 --max-concurrency 2 \
+ --request-rate inf --temperature 0 --seed "$SEED" \
+ --warmup-requests 1 --flush-cache --output-details --disable-tqdm \
+ --output-file "$RESULT" 2>&1 | tee "${RESULT%.jsonl}.log"
+done
+```
+
+Check every result for `completed=2`, input lengths `[3500,3500]`, output lengths `[1500,1500]`, and two empty error entries before aggregating it. Verify server health after each run. A successful process exit alone is insufficient to establish those conditions.
+
+### 5.7 Ascend A2 fixed 50-question GSM8K regression
+
+On September 24, 2026, a **preselected 50-question deployment regression completed with 49/50 correct (98%)**, zero capped outputs, zero transport/protocol errors, and no missing or duplicate questions. This is a bounded GSM8K test subset, not full GSM8K accuracy, an EvalScope result, GPQA accuracy, or evidence of A3 parity.
+
+The run used the same A2 hardware, W8A8 weights, and runtime as above. The server used context 9216, maximum running requests **2**, memory fraction 0.97, and KV limit 16384; all other launch settings matched section 5.6. The client submitted **one request at a time** to `/v1/chat/completions`, with `temperature=1.0`, `max_tokens=6144`, and generation seed `42 + original test row index`. The reasoning parser remained `minimax`; scoring used only final answer content.
+
+The protocol was frozen before an independent three-question pilot:
+
+1. Use the MIT-licensed [official GSM8K data at revision 3101c7d5072418e28b9008a6636bde82a006892c](https://github.com/openai/grade-school-math/tree/3101c7d5072418e28b9008a6636bde82a006892c/grade_school_math/data). Take training rows 0–4 as five worked demonstrations.
+2. Shuffle all 1319 test-row indices with Python `random.Random(20260924).shuffle`. Reserve the first three indices for the independent pilot and use the next 50, in order, for this regression. Do not include pilot answers in the result or select, replace, reorder, or drop questions based on observed answers.
+3. Build a single user message by joining the five `Question: ...\nAnswer: ...` demonstrations with two newlines, then append the test question as `Question: ...\nAnswer:`. Preserve the training answers, including their worked solutions. Apply the deployed MiniMax chat template through the chat endpoint.
+4. Before any generation, tokenize all 50 messages with the server and verify that each prompt plus the 6144-token output allowance fits context 9216. Submit each frozen question exactly once, with a per-request wall limit of 1800 seconds. Stop new submissions on a transport/protocol error; do not retry or resume a failed run.
+5. Parse only final content using the pinned runtime's last-number rule: remove commas, find matches of `-?\d+\.?\d*`, and parse the last match numerically. Compare it with the gold answer parsed the same way. A missing number, wrong answer, length cap, or transport/protocol error scores zero. Keep the denominator fixed at 50; report an incomplete run as incomplete rather than a full score.
+
+The selection can be reconstructed without model access:
+
+```python Example
+import random
+
+indices = list(range(1319))
+random.Random(20260924).shuffle(indices)
+pilot_ids = indices[:3] # Excluded from regression scoring.
+regression_ids = indices[3:53]
+assert len(set(regression_ids)) == 50
+assert not set(pilot_ids).intersection(regression_ids)
+```
+
+| Check | Observed result |
+| --- | --- |
+| Unique completed questions | 50/50 |
+| Correct final answers | 49/50 (98%) |
+| Wrong final answers | 1 |
+| Length-capped responses | 0 |
+| Transport/protocol errors | 0 |
+| Completion-token range | 292–2507 |
+
+All 50 prompts passed preflight. The client elapsed time, including preflight, was 9444.598 seconds; it is not a per-request latency benchmark. The downloaded run's 172 manifest entries passed hash verification, and rescoring the preserved final answers with the declared parser reproduced 49/50. Preserve the frozen IDs, prompt hashes, parameters, every response, and raw SSE when reproducing the regression. This result tests the declared subset and parsing protocol; it does not establish full-dataset accuracy, long-context quality, sustained stability, or the unvalidated A3 recipe's acceptance criteria.
diff --git a/docs/scripts/tests/diffusion-builders.test.mjs b/docs/scripts/tests/diffusion-builders.test.mjs
index 00ee1a7026..4c4967c53e 100644
--- a/docs/scripts/tests/diffusion-builders.test.mjs
+++ b/docs/scripts/tests/diffusion-builders.test.mjs
@@ -19,7 +19,7 @@ for (const page of pages) {
const source = readFileSync(join(docs, path), "utf8");
configs.set(page, (await import("data:text/javascript," + encodeURIComponent(source))).config);
}
-const get = (suffix) => [...configs].find(([path]) => path.endsWith(suffix))[1];
+const get = (suffix) => [...configs].find(([path]) => path.endsWith(join(...suffix.split("/"))))[1];
const selection = (config, overrides = {}) => ({
...config.commandBuilder.defaultSelection,
...Object.fromEntries(config.overlayDims.map((dim) => [dim.id, dim.default])),
diff --git a/docs/scripts/tests/minimax-m25-deployment.test.mjs b/docs/scripts/tests/minimax-m25-deployment.test.mjs
new file mode 100644
index 0000000000..e45b77bbfb
--- /dev/null
+++ b/docs/scripts/tests/minimax-m25-deployment.test.mjs
@@ -0,0 +1,156 @@
+// Run the selector's real command builder without a browser or JSX dependency.
+// Usage: node --test docs/scripts/tests/minimax-m25-deployment.test.mjs
+import assert from 'node:assert/strict';
+import { readFileSync } from 'node:fs';
+import { test } from 'node:test';
+
+const source = readFileSync(new URL('../../src/snippets/autoregressive/minimax-m25-deployment.jsx', import.meta.url), 'utf8');
+const hooksStart = source.indexOf(' const [values, setValues]');
+assert.ok(hooksStart > 0, 'The component must expose its pure setup before React hooks');
+const setup = source.slice(0, hooksStart).replace(/^export /, '');
+const { options, generateCommand, getInitialState, getUpdatedValues } = new Function(
+ `${setup}\nreturn { options, generateCommand, getInitialState, getUpdatedValues };\n};\nreturn MiniMaxM25Deployment();`
+)();
+const defaults = getInitialState();
+const build = (selection) => generateCommand({ ...defaults, ...selection });
+
+test('A2 exposes editable paths and hides A3-only fields', () => {
+ assert.ok(options.hardware.items.some((item) => item.id === 'a2'));
+ for (const name of ['ascendPreset', 'draftModelPath', 'networkInterface']) {
+ assert.equal(options[name].condition({ hardware: 'a2' }), false, name);
+ }
+ for (const name of ['modelPath', 'tokenizerPath']) {
+ assert.equal(options[name].condition({ hardware: 'a2' }), true, name);
+ }
+});
+
+test('A2 emits the tested eight-device low-concurrency command regardless of GPU selections', () => {
+ const command = build({ hardware: 'a2', gpuCount: '2gpu', thinking: 'disabled', toolcall: 'disabled' });
+ for (const expected of [
+ 'export HCCL_BUFFSIZE=128', 'export OMP_NUM_THREADS=1',
+ 'export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True',
+ '--device npu', '--tp-size 8', '--quantization modelslim', '--dtype bfloat16',
+ '--reasoning-parser minimax \\\n', '--tool-call-parser minimax-m2',
+ '--host 127.0.0.1 --port 31216', '--context-length 4096', '--max-running-requests 1',
+ '--chunked-prefill-size 128', '--max-prefill-tokens 4096', '--mem-fraction-static 0.96',
+ '--max-total-tokens 4096',
+ '--disable-cuda-graph', '--disable-radix-cache',
+ ]) assert.ok(command.includes(expected), `Missing ${expected}`);
+ assert.doesNotMatch(command, /EAGLE3|custom_eagle3|speculative|ascend_fuseep|--dp-size|flashinfer|fp8_e4m3/);
+ assert.ok(command.includes('MODEL_PATH=\'/models/MiniMax-M2.5-w8a8-QuaRot\''));
+ assert.ok(command.includes('TOKENIZER_PATH=\'/models/MiniMax-M2.5-w8a8-QuaRot\''));
+});
+
+test('A2 quotes both editable directories and refuses incomplete or multiline paths', () => {
+ const command = build({ hardware: 'a2', modelPath: "/models/owner's $(model)", tokenizerPath: '/models/tokenizer files' });
+ assert.ok(command.includes("MODEL_PATH='/models/owner'\\''s $(model)'"));
+ assert.ok(command.includes("TOKENIZER_PATH='/models/tokenizer files'"));
+ assert.ok(command.includes('--model-path "$MODEL_PATH"'));
+ assert.ok(command.includes('--tokenizer-path "$TOKENIZER_PATH"'));
+ for (const input of [
+ { modelPath: '' }, { modelPath: 'relative/model' }, { modelPath: '/models/main\nextra' },
+ { tokenizerPath: '' }, { tokenizerPath: 'relative/tokenizer' }, { tokenizerPath: '/models/tokenizer\nextra' },
+ ]) assert.doesNotMatch(build({ hardware: 'a2', ...input }), /python3? -m sglang\.launch_server/);
+});
+
+test('switching through A2 preserves A3 and restores a valid NVIDIA count', () => {
+ const a2 = getUpdatedValues({ ...defaults, hardware: 'mi300x', gpuCount: '2gpu' }, 'hardware', 'a2');
+ assert.match(generateCommand(a2), /--tp-size 8/);
+ assert.match(generateCommand(getUpdatedValues(a2, 'hardware', 'a3')), /--tp-size 16/);
+ const nvidia = getUpdatedValues(a2, 'hardware', 'h200');
+ assert.equal(nvidia.gpuCount, '4gpu');
+ assert.doesNotMatch(generateCommand(nvidia), /TOKENIZER_PATH|HCCL_BUFFSIZE|modelslim|--device npu/);
+});
+
+test('the existing default NVIDIA recipe retains its fusion and tensor parallel flags', () => {
+ const command = build({ hardware: 'h200', gpuCount: '4gpu' });
+ assert.match(command, /^SGLANG_USE_FUSED_PARALLEL_QKNORM=1/);
+ assert.match(command, /--model-path MiniMaxAI\/MiniMax-M2\.5/);
+ assert.match(command, /--tp 4(?:\s|$)/);
+ assert.match(command, /--enable-flashinfer-allreduce-fusion/);
+ assert.doesNotMatch(command, /--device npu|modelslim|ascend_fuseep/);
+});
+
+test('the AMD two-GPU recipe remains reachable and keeps its AMD backends', () => {
+ const command = build({ hardware: 'mi300x', gpuCount: '2gpu' });
+ assert.match(command, /--tp 2(?:\s|$)/);
+ assert.match(command, /--ep 2(?:\s|$)/);
+ assert.match(command, /--kv-cache-dtype fp8_e4m3/);
+ assert.match(command, /--attention-backend triton/);
+ assert.doesNotMatch(command, /ascend_fuseep|flashinfer/);
+});
+
+test('NPU keeps shared rows visible with fixed values matching its command', () => {
+ for (const hardware of ['a2', 'a3']) {
+ const state = { ...defaults, hardware };
+ for (const [name, expected] of [['gpuCount', '8gpu'], ['thinking', 'enabled'], ['toolcall', 'enabled']]) {
+ assert.notEqual(options[name].condition?.(state), false);
+ assert.equal(options[name].getValue(state), expected);
+ assert.ok(options[name].getDynamicItems(state).every((item) => item.disabled));
+ const changed = getUpdatedValues(state, name, name === 'gpuCount' ? '2gpu' : 'disabled');
+ assert.equal(options[name].getValue(changed), expected);
+ assert.equal(generateCommand(changed), generateCommand(state));
+ }
+ }
+});
+
+test('A3 produces the full 16-die W8A8 EAGLE3 recipe even after an AMD two-GPU selection', () => {
+ const command = build({ hardware: 'a3', gpuCount: '2gpu' });
+ for (const expected of [
+ '--device npu', '--tp-size 16', '--dp-size 16', '--enable-dp-attention',
+ '--quantization modelslim', '--moe-a2a-backend ascend_fuseep', '--fuseep-mode 2',
+ '--speculative-algorithm EAGLE3', '--speculative-num-steps 3',
+ '--speculative-eagle-topk 1', '--speculative-num-draft-tokens 4',
+ '--speculative-draft-model-quantization unquant', '--mem-fraction-static 0.75',
+ '--max-running-requests 320', '--chunked-prefill-size 196608',
+ '--max-prefill-tokens 8192', '--cuda-graph-bs-decode 1 2 4 8 12 16 20',
+ '--reasoning-parser minimax-append-think', '--tool-call-parser minimax-m2',
+ '--host 127.0.0.1 --port 6688', 'SGLANG_EXTERNAL_MODEL_PACKAGE=custom_eagle3',
+ 'SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=204800',
+ ]) assert.ok(command.includes(expected), `Missing ${expected}`);
+ assert.match(command, /export PYTHONPATH=.*DRAFT_MODEL_PATH/);
+ assert.doesNotMatch(command, /flashinfer|fp8_e4m3|SGLANG_USE_FUSED_PARALLEL_QKNORM/);
+});
+
+test('A3 uses quoted user-supplied model directories and network interface', () => {
+ const command = build({
+ hardware: 'a3', modelPath: '/models/main weights',
+ draftModelPath: '/models/draft weights', networkInterface: 'eth0',
+ });
+ assert.ok(command.includes("MODEL_PATH='/models/main weights'"));
+ assert.ok(command.includes("DRAFT_MODEL_PATH='/models/draft weights'"));
+ assert.ok(command.includes("HCCL_SOCKET_IFNAME='eth0'"));
+ assert.ok(command.includes("GLOO_SOCKET_IFNAME='eth0'"));
+ assert.ok(command.includes('--model-path "$MODEL_PATH"'));
+ assert.ok(command.includes('--speculative-draft-model-path "$DRAFT_MODEL_PATH"'));
+});
+
+test('A3 refuses incomplete paths and an invalid network interface instead of emitting a launch', () => {
+ for (const input of [
+ { modelPath: '' }, { draftModelPath: '' }, { modelPath: '/models/main\nextra' },
+ { modelPath: 'relative/model' }, { ascendPreset: '4p' },
+ { networkInterface: '' }, { networkInterface: 'eth0; echo unsafe' },
+ ]) assert.doesNotMatch(build({ hardware: 'a3', ...input }), /(?:python3? -m sglang\.launch_server|sglang serve)/);
+});
+
+test('A3 shell assignments preserve apostrophes and shell expansion characters literally', () => {
+ const command = build({ hardware: 'a3', modelPath: "/models/owner's $(model)" });
+ assert.ok(command.includes("MODEL_PATH='/models/owner'\\''s $(model)'"));
+});
+
+test('switching AMD two-GPU to A3 and then NVIDIA selects a valid NVIDIA count', () => {
+ const amd = { ...defaults, hardware: 'mi300x', gpuCount: '2gpu' };
+ const ascend = getUpdatedValues(amd, 'hardware', 'a3');
+ assert.match(generateCommand(ascend), /--tp-size 16/);
+ const nvidia = getUpdatedValues(ascend, 'hardware', 'h200');
+ assert.equal(nvidia.gpuCount, '4gpu');
+ assert.match(generateCommand(nvidia), /--tp 4/);
+ assert.doesNotMatch(generateCommand(nvidia), /--device npu|modelslim/);
+});
+
+test('returning from A3 to NVIDIA does not retain Ascend environment or paths', () => {
+ const command = build({ hardware: 'b200', gpuCount: '8gpu', modelPath: '/models/ascend-only' });
+ assert.match(command, /--tp 8(?:\s|$)/);
+ assert.match(command, /--moe-runner-backend flashinfer_trtllm_routed/);
+ assert.doesNotMatch(command, /ascend-only|modelslim|custom_eagle3|--device npu/);
+});
diff --git a/docs/src/snippets/autoregressive/minimax-m25-deployment.jsx b/docs/src/snippets/autoregressive/minimax-m25-deployment.jsx
index d1aa7d495f..c1cadf977d 100644
--- a/docs/src/snippets/autoregressive/minimax-m25-deployment.jsx
+++ b/docs/src/snippets/autoregressive/minimax-m25-deployment.jsx
@@ -1,5 +1,10 @@
export const MiniMaxM25Deployment = () => {
const modelFamily = 'MiniMaxAI';
+ const isAscend = (values) => values.hardware === 'a2' || values.hardware === 'a3';
+ const parserItems = (values) => [
+ { id: 'disabled', label: 'Disabled', default: true, disabled: isAscend(values) },
+ { id: 'enabled', label: 'Enabled', default: false, disabled: isAscend(values) }
+ ];
const options = {
hardware: {
@@ -12,13 +17,22 @@ export const MiniMaxM25Deployment = () => {
{ id: 'h100', label: 'H100', default: false },
{ id: 'mi300x', label: 'MI300X', default: false },
{ id: 'mi325x', label: 'MI325X', default: false },
- { id: 'mi355x', label: 'MI355X', default: false }
+ { id: 'mi355x', label: 'MI355X', default: false },
+ { id: 'a2', label: 'Ascend A2', description: '8 × 910B4 / 32 GiB per device. The fixed single-request configuration is smoke tested.', default: false },
+ { id: 'a3', label: 'Ascend A3', description: '8 cards / 16 dies · W8A8 + EAGLE3. Reference configuration only; A3 hardware validation is pending.', default: false }
]
},
gpuCount: {
name: 'gpuCount',
title: 'GPU Count',
+ getTitle: (values) => isAscend(values) ? 'NPU Card Count' : 'GPU Count',
+ getValue: (values) => isAscend(values) ? '8gpu' : values.gpuCount,
getDynamicItems: (values) => {
+ if (isAscend(values)) return [
+ { id: '2gpu', label: '2', disabled: true },
+ { id: '4gpu', label: '4', disabled: true },
+ { id: '8gpu', label: '8', disabled: true }
+ ];
const isAMD = values.hardware === 'mi300x' || values.hardware === 'mi325x' || values.hardware === 'mi355x';
return [
{
@@ -45,6 +59,8 @@ export const MiniMaxM25Deployment = () => {
thinking: {
name: 'thinking',
title: 'Thinking Capabilities',
+ getValue: (values) => isAscend(values) ? 'enabled' : values.thinking,
+ getDynamicItems: parserItems,
items: [
{ id: 'disabled', label: 'Disabled', default: true },
{ id: 'enabled', label: 'Enabled', default: false }
@@ -54,17 +70,164 @@ export const MiniMaxM25Deployment = () => {
toolcall: {
name: 'toolcall',
title: 'Tool Call Parser',
+ getValue: (values) => isAscend(values) ? 'enabled' : values.toolcall,
+ getDynamicItems: parserItems,
items: [
{ id: 'disabled', label: 'Disabled', default: true },
{ id: 'enabled', label: 'Enabled', default: false }
],
commandRule: (value) => value === 'enabled' ? '--tool-call-parser minimax-m2' : null
+ },
+ ascendPreset: {
+ name: 'ascendPreset',
+ title: 'A3 Configuration',
+ condition: () => false,
+ items: [
+ {
+ id: '8p-in3k5-out1k5',
+ label: '1 node / 8 cards / 16 dies',
+ subtitle: 'PD co-located · W8A8 + EAGLE3 · 3.5k input / 1.5k output',
+ default: true
+ }
+ ]
+ },
+ modelPath: {
+ name: 'modelPath',
+ title: 'W8A8 Model Directory',
+ type: 'text',
+ default: '/models/MiniMax-M2.5-w8a8-QuaRot',
+ condition: (values) => values.hardware === 'a3' || values.hardware === 'a2'
+ },
+ tokenizerPath: {
+ name: 'tokenizerPath',
+ title: 'Tokenizer Directory',
+ type: 'text',
+ default: '/models/MiniMax-M2.5-w8a8-QuaRot',
+ condition: (values) => values.hardware === 'a2'
+ },
+ draftModelPath: {
+ name: 'draftModelPath',
+ title: 'EAGLE3 Directory',
+ type: 'text',
+ default: '/models/MiniMax-M2.5-eagel-model-0318',
+ condition: (values) => values.hardware === 'a3'
+ },
+ networkInterface: {
+ name: 'networkInterface',
+ title: 'HCCL / Gloo Interface',
+ type: 'text',
+ default: 'lo',
+ condition: (values) => values.hardware === 'a3'
}
};
const generateCommand = (values) => {
const { hardware, gpuCount, thinking, toolcall } = values;
+ if (hardware === 'a2') {
+ const { modelPath, tokenizerPath } = values;
+ const isAbsolutePath = (path) => typeof path === 'string' && path.startsWith('/') && !/[\u0000-\u001f\u007f]/.test(path);
+ if (!isAbsolutePath(modelPath) || !isAbsolutePath(tokenizerPath)) {
+ return '# Enter absolute model and tokenizer directory paths inside the container.';
+ }
+ const shellQuote = (value) => "'" + value.replace(/'/g, "'\\''") + "'";
+ const environment = [
+ '# Ascend A2: 8 x 910B4, 32 GiB per device, single-request smoke configuration',
+ '# Complete the A2 prerequisites in this guide before launching.',
+ '# Observed KV pool: 4096 tokens; synthetic input up to 3500 + 128 output tested.',
+ '# Tool-call parser configured; tool calling has not been tested.',
+ `MODEL_PATH=${shellQuote(modelPath)}`,
+ `TOKENIZER_PATH=${shellQuote(tokenizerPath)}`,
+ 'export HCCL_BUFFSIZE=128',
+ 'export OMP_NUM_THREADS=1',
+ 'export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True',
+ ''
+ ];
+ const launch = [
+ 'python -m sglang.launch_server',
+ ' --model-path "$MODEL_PATH"',
+ ' --tokenizer-path "$TOKENIZER_PATH"',
+ ' --device npu --tp-size 8 --quantization modelslim --dtype bfloat16',
+ ' --reasoning-parser minimax',
+ ' --tool-call-parser minimax-m2',
+ ' --host 127.0.0.1 --port 31216',
+ ' --context-length 4096 --max-running-requests 1',
+ ' --chunked-prefill-size 128 --max-prefill-tokens 4096',
+ ' --mem-fraction-static 0.96 --max-total-tokens 4096',
+ ' --disable-cuda-graph --disable-radix-cache'
+ ];
+ return environment.join('\n') + '\n' + launch.join(' \\\n');
+ }
+
+ if (hardware === 'a3') {
+ const { ascendPreset, modelPath, draftModelPath, networkInterface } = values;
+ if (ascendPreset !== '8p-in3k5-out1k5') {
+ return '# Select the A3 configuration: 1 node / 8 cards / 16 dies.';
+ }
+ const isAbsolutePath = (path) => typeof path === 'string' && path.startsWith('/') && !/[\u0000-\u001f\u007f]/.test(path);
+ if (!isAbsolutePath(modelPath) || !isAbsolutePath(draftModelPath)) {
+ return '# Enter absolute paths to both model directories inside the container.';
+ }
+ if (typeof networkInterface !== 'string' || !/^[a-zA-Z0-9_.:-]+$/.test(networkInterface)) {
+ return '# Enter a valid network interface name from ip -brief address.';
+ }
+ const shellQuote = (value) => "'" + value.replace(/'/g, "'\\''") + "'";
+ const environment = [
+ '# Ascend A3: 1 node / 8 cards / 16 dies, PD co-located',
+ '# Complete the Ascend prerequisites in this guide before launching.',
+ `MODEL_PATH=${shellQuote(modelPath)}`,
+ `DRAFT_MODEL_PATH=${shellQuote(draftModelPath)}`,
+ 'export PYTHONPATH="${DRAFT_MODEL_PATH}${PYTHONPATH:+:${PYTHONPATH}}"',
+ '',
+ 'unset https_proxy http_proxy HTTPS_PROXY HTTP_PROXY ASCEND_LAUNCH_BLOCKING',
+ 'source /usr/local/Ascend/ascend-toolkit/set_env.sh',
+ 'source /usr/local/Ascend/nnal/atb/set_env.sh',
+ '',
+ 'export ASCEND_USE_FIA=1',
+ 'export DEEPEP_HCCL_BUFFSIZE=1024',
+ `export GLOO_SOCKET_IFNAME=${shellQuote(networkInterface)}`,
+ `export HCCL_SOCKET_IFNAME=${shellQuote(networkInterface)}`,
+ 'export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True',
+ 'export SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK=204800',
+ 'export SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1',
+ 'export SGLANG_EXTERNAL_MODEL_PACKAGE=custom_eagle3',
+ 'export SGLANG_SET_CPU_AFFINITY=1',
+ 'export STREAMS_PER_DEVICE=32',
+ 'export TASK_QUEUE_ENABLE=1',
+ ''
+ ];
+ const launch = [
+ 'python3 -m sglang.launch_server',
+ ' --model-path "$MODEL_PATH"',
+ ' --host 127.0.0.1 --port 6688',
+ ' --tp-size 16',
+ ' --enable-dp-attention',
+ ' --dp-size 16',
+ ' --mem-fraction-static 0.75',
+ ' --max-running-requests 320',
+ ' --disable-radix-cache',
+ ' --reasoning-parser minimax-append-think',
+ ' --tool-call-parser minimax-m2',
+ ' --prefill-delayer-max-delay-passes 500',
+ ' --enable-prefill-delayer',
+ ' --chunked-prefill-size 196608',
+ ' --max-prefill-tokens 8192',
+ ' --cuda-graph-bs-decode 1 2 4 8 12 16 20',
+ ' --moe-a2a-backend ascend_fuseep',
+ ' --fuseep-mode 2',
+ ' --quantization modelslim',
+ ' --speculative-algorithm EAGLE3',
+ ' --speculative-draft-model-path "$DRAFT_MODEL_PATH"',
+ ' --speculative-num-steps 3',
+ ' --speculative-eagle-topk 1',
+ ' --speculative-num-draft-tokens 4',
+ ' --speculative-draft-model-quantization unquant',
+ ' --dtype bfloat16',
+ ' --device npu'
+ ];
+ return environment.join('\n') + '\n' + launch.join(' \\\n');
+ }
+
const isAMD = hardware === 'mi300x' || hardware === 'mi325x' || hardware === 'mi355x';
if (gpuCount === '2gpu' && !isAMD) {
return '# Please select compatible hardware\n# 2-GPU requires AMD MI300X/MI325X/MI355X';
@@ -162,8 +325,21 @@ export const MiniMaxM25Deployment = () => {
return initialState;
};
+ const getUpdatedValues = (previous, optionName, value) => {
+ if (isAscend(previous) && ['gpuCount', 'thinking', 'toolcall'].includes(optionName)) return previous;
+ const next = { ...previous, [optionName]: value };
+ if (optionName === 'hardware' && value !== 'a3' && value !== 'a2') {
+ const items = options.gpuCount.getDynamicItems(next);
+ if (!items.some((item) => item.id === next.gpuCount && !item.disabled)) {
+ next.gpuCount = items.find((item) => item.default && !item.disabled).id;
+ }
+ }
+ return next;
+ };
+
const [values, setValues] = useState(getInitialState);
const [isDark, setIsDark] = useState(false);
+ const [copyState, setCopyState] = useState({ command: null, failed: false });
useEffect(() => {
const checkDarkMode = () => {
@@ -184,7 +360,7 @@ export const MiniMaxM25Deployment = () => {
}, []);
const handleRadioChange = (optionName, value) => {
- setValues((prev) => ({ ...prev, [optionName]: value }));
+ setValues((prev) => getUpdatedValues(prev, optionName, value));
};
const handleCheckboxChange = (optionName, itemId, isChecked) => {
@@ -205,6 +381,17 @@ export const MiniMaxM25Deployment = () => {
};
const command = generateCommand(values);
+ const handleCopyCommand = async () => {
+ try {
+ await navigator.clipboard.writeText(command);
+ setCopyState({ command, failed: false });
+ } catch {
+ setCopyState({ command, failed: true });
+ }
+ };
+ const copyLabel = copyState.command !== command
+ ? 'Copy command'
+ : copyState.failed ? 'Select text to copy' : 'Copied!';
const containerStyle = {
maxWidth: '900px',
@@ -220,6 +407,7 @@ export const MiniMaxM25Deployment = () => {
borderRadius: '4px',
display: 'flex',
alignItems: 'center',
+ flexWrap: 'wrap',
gap: '12px',
background: isDark ? '#1f2937' : '#fff',
};
@@ -232,11 +420,12 @@ export const MiniMaxM25Deployment = () => {
};
const itemsStyle = {
display: 'flex',
- rowGap: '2px',
+ rowGap: '6px',
columnGap: '6px',
flexWrap: 'wrap',
alignItems: 'center',
- flex: 1,
+ flex: '1 1 260px',
+ minWidth: 0,
};
const labelBaseStyle = {
padding: '4px 10px',
@@ -268,13 +457,16 @@ export const MiniMaxM25Deployment = () => {
};
const subtitleStyle = {
display: 'block',
- fontSize: '9px',
+ fontSize: '12px',
marginTop: '1px',
- lineHeight: '1.1',
+ lineHeight: '1.4',
opacity: 0.7,
};
const textInputStyle = {
flex: 1,
+ minWidth: 0,
+ width: '100%',
+ boxSizing: 'border-box',
padding: '8px 10px',
borderRadius: '4px',
border: `1px solid ${isDark ? '#4b5563' : '#d1d5db'}`,
@@ -284,6 +476,9 @@ export const MiniMaxM25Deployment = () => {
};
const commandDisplayStyle = {
flex: 1,
+ minWidth: 0,
+ width: '100%',
+ boxSizing: 'border-box',
padding: '12px 16px',
background: isDark ? '#111827' : '#f5f5f5',
borderRadius: '6px',
@@ -293,112 +488,152 @@ export const MiniMaxM25Deployment = () => {
color: isDark ? '#e5e7eb' : '#374151',
whiteSpace: 'pre-wrap',
overflowX: 'auto',
+ maxHeight: '420px',
+ overflowY: 'auto',
margin: 0,
border: `1px solid ${isDark ? '#374151' : '#e5e7eb'}`,
};
+ const renderOption = ([key, option]) => {
+ if (option.condition && !option.condition(values)) {
+ return null;
+ }
+ const items = option.getDynamicItems ? option.getDynamicItems(values) : option.items || [];
+ const selectedValue = option.getValue ? option.getValue(values) : values[option.name];
+ const title = option.getTitle ? option.getTitle(values) : option.title;
+ const description = items.find((item) => item.id === selectedValue)?.description;
+ return (
+