[Doc]Checking and modifying Markdown formatting issues and link validity (#28586)

Signed-off-by: a60124901 <anyuxin4@h-partners.com> Co-authored-by: sglang-npu-bot <sglangnpu@163.com> Co-authored-by: ronnie_zheng <zl19940307@163.com>
2026-06-30 20:27:38 +03:00
parent 1c2523b719
commit 97fc4dfd73
14 changed files with 7024 additions and 35 deletions
@@ -84,13 +84,16 @@ If you need to use model which is not in `python/sglang/test/ascend/test_ascend_

 1. Register account and upload your model to [modelscope](https://modelscope.cn/models).
 2. Make sure your model is pre-cached on the CI server and is on the way "/data/ascend-ci-share-pkking-sglang/modelscope/hub/models/{your_model_repo}/{your_model}".
-If this is not the case, use following command on CI server:
-  ```bash
-  modelscope download \
-    --model {your_model_repo}/{your_model} \
-    --local_dir /data/ascend-ci-share-pkking-sglang/modelscope/hub/models/{your_model_repo}/{your_model}
-  ```
+  If this is not the case, use following command on CI server:
+
+    ```bash
+    modelscope download \
+      --model {your_model_repo}/{your_model} \
+      --local_dir /data/ascend-ci-share-pkking-sglang/modelscope/hub/models/{your_model_repo}/{your_model}
+    ```
+
  > Note: If you don’t have access to CI server, please ask maintainers (zl19940307@163.com) to download your model.
+
 3. Add model to `python/sglang/test/ascend/test_ascend_utils.py` (use docker `"/root/.cache/modelscope/hub/models/{your_model_repo}/{your_model}"` path).

 ## Write documentation
@@ -99,6 +102,7 @@ We recommend new contributors start by writing documentation, which helps you qu
 For more details, please refer to [docs/README.md](https://github.com/sgl-project/sglang/tree/main/docs/README.md).

 ## Test the accuracy
+
 If your code changes the model output, please run the accuracy tests. A quick sanity check is the few-shot GSM8K.

 ```bash
@@ -121,9 +125,11 @@ You can find additional accuracy eval examples in:
 - [test_gpt_oss_1gpu.py](https://github.com/sgl-project/sglang/blob/main/test/manual/core/test_gpt_oss_1gpu.py)

 ## Benchmark the speed
+
 Refer to [Benchmark and Profiling](../../developer_guide/benchmark_and_profiling).

 ## Requesting a review for merge
+
 You can follow the pull request merge process described in [MAINTAINER.md](https://github.com/sgl-project/sglang/blob/main/.github/MAINTAINER.md).
 You will need to work with the Merge Oncall, Codeowner, and other reviewers to get their approvals.
 Then your PR can be merged.
@@ -168,6 +174,7 @@ cool-down-minutes:
 Users listed in [CI_PERMISSIONS.json](https://github.com/sgl-project/sglang/blob/main/.github/CI_PERMISSIONS.json) may have a per-user cooldown interval. In practice, we use the minimum of the workflow’s default window and the user-specific interval.

 ## Code style guidance
+
 - Avoid code duplication. If the same code snippet (more than five lines) appears multiple times, extract it into a shared function.
 - Minimize device synchronization. Reduce expensive CPU-NPU synchronization operations, such as `tensor.item()` or `tensor.cpu()`, whenever possible. Use vectorized code.
 - Prioritize extreme efficiency. SGLang is a runtime, and most of your code runs on the critical path for every request. Optimize all minor overheads as much as possible, especially in the model forward code.
@@ -5,6 +5,7 @@ description: "Complete installation guide for SGLang on Ascend NPUs, including c
 You can install SGLang using any of the methods below. Please go through `System Settings` section to ensure the clusters are operating at optimal performance. Feel free to leave an issue [here at sglang](https://github.com/sgl-project/sglang/issues) if you encounter any issues or have any problems.

 ## Component Version Mapping For SGLang
+
 <table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
  <colgroup>
    <col style={{width: "34%"}} />
@@ -97,6 +98,7 @@ conda activate sglang_npu

 Note on Anaconda repository restrictions
 If you encounter an error like “Terms of Service have not been accepted” during the conda create step, the default Anaconda repository is blocking package downloads. To resolve this, configure a mirror (e.g., Tsinghua Open Source Mirror):
+
 ```bash Command
 # Add Tsinghua mirrors
 conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
@@ -104,8 +106,10 @@ conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/
 conda config --set show_channel_urls yes
 conda config --remove channels defaults
 ```
+
 Edit the system-level conda config to remove any hardcoded defaults, e.g. vi ~/miniconda3/.condarc
 Then remove the failed environment and recreate it:
+
 ```bash Command
 conda clean -i
 conda env remove -n sglang_npu
@@ -157,12 +161,15 @@ pip install triton-ascend==3.2.1.dev20260530 \
  --extra-index-url=https://mirrors.huaweicloud.com/ascend/repos/pypi/nightly \
  --trusted-host mirrors.huaweicloud.com
 ```
+
 For installation of Triton on Ascend nightly builds or from sources, follow [installation guide](https://gitcode.com/Ascend/triton-ascend/blob/master/docs/sources/getting-started/installation.md)

 #### SGLang Kernels NPU
+
 We provide SGL kernels for Ascend NPU, check [installation guide](https://github.com/sgl-project/sgl-kernel-npu/blob/main/python/sgl_kernel_npu/README.md).

 #### DeepEP-compatible Library
+
 We provide a DeepEP-compatible Library as a drop-in replacement of deepseek-ai's DeepEP library, check the [installation guide](https://github.com/sgl-project/sgl-kernel-npu/blob/main/python/deep_ep/README.md).

 #### Some other dependencies
@@ -187,7 +194,9 @@ pip install -e python[all_npu]
 ```

 ### Method 2: Using Docker Image
+
 #### Obtain Image
+
 You can download the SGLang image or build an image based on Dockerfile to obtain the Ascend NPU image.

 <Warning>
@@ -226,6 +235,7 @@ docker pull quay.io/ascend/sglang:main-cann9.0.0-910b
 </Tabs>

 2. Build an image based on Dockerfile
+
 ```bash Command
 # Clone the SGLang repository
 git clone https://github.com/sgl-project/sglang.git
@@ -241,6 +251,7 @@ docker build --build-arg TARGETARCH=<arch_tag> -t <image_name> -f npu.Dockerfile
 ```

 #### Create Docker
+
 __Notice:__ `--privileged` and `--network=host` are required by RDMA, which is typically needed by Ascend NPU clusters.

 <Tabs>
@@ -325,8 +336,11 @@ cat /proc/sys/vm/swappiness # shows 10
 ```

 ## Running SGLang Service
+
 ### Running Service For Large Language Models
+
 #### PD Mixed Scene
+
 ```bash Command
 # Enabling CPU Affinity
 export SGLANG_SET_CPU_AFFINITY=1
@@ -338,6 +352,7 @@ python3 -m sglang.launch_server \
 ```

 #### PD Disaggregation Scene
+
 1. Launch Prefill Server

 <Tabs>
@@ -439,6 +454,7 @@ python3 -m sglang.launch_server \
 </Tabs>

 3. Launch Router
+
 ```bash Command
 python3 -m sglang_router.launch_router \
    --pd-disaggregation \
@@ -450,7 +466,9 @@ python3 -m sglang_router.launch_router \
 ```

 ### Running Service For Multimodal Language Models
+
 #### PD Mixed Scene
+
 ```bash Command
 python3 -m sglang.launch_server \
    --model-path Qwen/Qwen3-VL-30B-A3B-Instruct \
@@ -519,6 +537,7 @@ curl http://127.0.0.1:8000/v1/chat/completions \
 ```

 Some models return responses accompanied with thinking process content. To disable this output, configure parameters as follows:
+
 ```bash Command
 curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
@@ -100,6 +100,7 @@ export HF_ENDPOINT=https://hf-mirror.com
 # Start text model server
 sglang serve --model-path /home/weights/Qwen2.5-7B-Instruct --attention-backend ascend --host 0.0.0.0 --port 30000 &
 ```
+
 For more details of SGLang server, refer to the [Ascend NPU Quick Start](/docs/hardware-platforms/ascend-npus/ascend_npu_quick_start)

 #### Execute Accuracy Evaluation
@@ -118,7 +119,7 @@ evalscope eval \

 Upon completion, results similar to the following will be displayed:

-```
+```text
 +---------------------+-----------+----------+----------+-------+---------+---------+
 | Model               | Dataset   | Metric   | Subset   |   Num |   Score | Cat.0   |
 +=====================+===========+==========+==========+=======+=========+=========+
@@ -225,8 +226,10 @@ For more details, refer to the [EvalScope documentation](https://evalscope.readt
 3. Ensure URL ends with `/v1`, e.g., `http://localhost:30000/v1`

 ### EvalScope SSL certificate verification failed
+
 When using EvalScope commands without specifying a dataset or model path, it will attempt to download automatically, which may encounter an SSL certificate verification error:
-```
+
+```text
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/requests/sessions.py", line 605, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -280,8 +283,10 @@ unset HTTPS_PROXY
 ```

 ### Download Dataset Error
+
 For this error
-```
+
+```text
 root@localhost:/home/# wget https://www.modelscope.cn/datasets/evalscope/MMStar/resolve/master/MMStar.tsv
 --2026-05-12 12:08:01--  https://www.modelscope.cn/datasets/evalscope/MMStar/resolve/master/MMStar.tsv
 Connecting to 141.5.152.215:6688... connected.
@@ -289,8 +294,10 @@ ERROR: cannot verify www.modelscope.cn's certificate, issued by ‘CN=Huawei Web
  Self-signed certificate encountered.
 To connect to www.modelscope.cn insecurely, use `--no-check-certificate`.
 ```
-You can add `--no-check-certificate`
-```
+
+You can add `--no-check-certificate'
+
+```bash
 wget https://www.modelscope.cn/datasets/evalscope/MMStar/resolve/master/MMStar.tsv --no-check-certificate
 ```

@@ -43,16 +43,16 @@ Thread 0x0000fff873f6f120 (most recent call first):
 (SGLangEngine pid=3872176) [Error]: An internal error occurs in the runtime module on the host.
 (SGLangEngine pid=3872176) Rectify the fault based on the error information in the ascend log.
 (SGLangEngine pid=3872176) [PID: 3874122] 2026-04-14-12:14:41.897.548 AclNN_Runtime_Error(EZ9903): aclrtLaunchKerneWithHostArgs failed, return: 507000
-(SGLangEngine pid=3872176)	Solution: In this scenario, collect the plog when the fault occurs and locate the fault based on the plog.
-(SGLangEngine pid=3872176)	TraceBack (most recent call last):
-(SGLangEngine pid=3872176)	Check kernel task failed, stream_id=2028, task_id=48, retCode=0x7080005.[FUNC:LaunchKernel][FILE:context.cc][LINE:1585]
-(SGLangEngine pid=3872176)	rtsLaunchKernelWithHostArgs execution failed, reason=kernel type error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:61]
-(SGLangEngine pid=3872176)	rtsLaunchKernelWithHostArgs failed, runtime result = 507000.[FUNC:ReportCallError][FILE:Log_inner.cpp][LINE:148]
-(SGLangEngine pid=3872176)	aclrtLaunchKerneWWithHostArgs failed, return: 507000
-(SGLangEngine pid=3872176) 	Launch kernel failed.
-(SGLangEngine pid=3872176)	#### KernelLaunch failed: /home/850b160/cann-8.5.8/opp/built-in/op_impl/ai_core/tbe//kernel/ascend910_93/ops_legacy/fill/Fill_41dadce325bOf810d03359af2a38990b_high_performance.o
-(SGLangEngine pid=3872176)	Kernel Run failed. opType: 18, Fill
-(SGLangEngine pid=3872176)	launch failed for Fill, errno:361001.
+(SGLangEngine pid=3872176)  Solution: In this scenario, collect the plog when the fault occurs and locate the fault based on the plog.
+(SGLangEngine pid=3872176)  TraceBack (most recent call last):
+(SGLangEngine pid=3872176)  Check kernel task failed, stream_id=2028, task_id=48, retCode=0x7080005.[FUNC:LaunchKernel][FILE:context.cc][LINE:1585]
+(SGLangEngine pid=3872176)  rtsLaunchKernelWithHostArgs execution failed, reason=kernel type error[FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:61]
+(SGLangEngine pid=3872176)  rtsLaunchKernelWithHostArgs failed, runtime result = 507000.[FUNC:ReportCallError][FILE:Log_inner.cpp][LINE:148]
+(SGLangEngine pid=3872176)  aclrtLaunchKerneWWithHostArgs failed, return: 507000
+(SGLangEngine pid=3872176)  Launch kernel failed.
+(SGLangEngine pid=3872176)  #### KernelLaunch failed: /home/850b160/cann-8.5.8/opp/built-in/op_impl/ai_core/tbe//kernel/ascend910_93/ops_legacy/fill/Fill_41dadce325bOf810d03359af2a38990b_high_performance.o
+(SGLangEngine pid=3872176)  Kernel Run failed. opType: 18, Fill
+(SGLangEngine pid=3872176)  launch failed for Fill, errno:361001.
 (SGLangEngine pid=3872176)
 (SGLangEngine pid=3872176) Exception raised from operator() at build/CMakeFiles/torch_npu.dir/compiler depend.ts:26 (most recent call first):
 (SGLangEngine pid=3872176) frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)+ 0xb0 (0xffff806848c0 in /root/anaconda3/envs/slime_re/Lib/python3.11/site-packages/torch/lib/libc10.so)
@@ -76,7 +76,7 @@ Too many captured graphs cause a conflict in the graph mode update stream. Each

 ### Error message

-```
+```text
 [root@os-node-created-6z9tp pd_7p1d_tp2_20260414_030537]# grep -nR -E "aivec error|ACL" *,log
 prefill 1.log:5739:EZ9999[PID: 164243] 2026-04-14-06:40:35.033.745 (EZ9999): The error from device(chipId:0, dieId:1), serial number is 1, there is an exception of aivec error, core id is 23, error code = 0, dump info: pc start: 0x12420156a000, current: 0x12420156ad74, vec error info: 0xdb1751f50e, mte error info: 0x98f6388707, ifu error info: 0x212c93fc00000, ccu error info: 0x998e981c00000000, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100dccc00.[FUNC:PrintCoreInfo][FILE:device error core proc.cc][LINE:347]
 prefill_1.log:5747:EZ9999[PID: 164242] 2026-04-14-06:40:35.034.456 (EZ9999):  The error from device(chipId:0, dieId:0), serial number is 1, there is an exception of aivec error, core id is 45, error code = O, dump info: pc start: 0x12400156a000, current: 0x12400156ad74, vec error info: 0x7304583407, mte error info: 0x27e2b1c765, ifu error info: 0x212c93fc00000, ccu error info: 0x6e0836f300000000, cube error info: 0, biu error info: 0O, aic error mask: 0x6500020bd00028c, para base: 0x12c100dccc0O.[FUNC:PrintCoreInfo][FILE:device_error_core_proc.cc][LINE:347]
@@ -206,7 +206,8 @@ If the NPUs are not occupied, you can use `--tp` to deploy across multiple devic
 ## 5. How to update sgl-kernel-npu

 ### Solution
-```shell
+
+```bash
 git clone https://github.com/sgl-project/sgl-kernel-npu.git

 source /usr/local/Ascend/ascend-toolkit/set_env.sh
@@ -281,12 +282,15 @@ The machine cannot directly access the HuggingFace server due to network restric
    --dataset-path /path/to/ShareGPT_V3_unfiltered_cleaned_split.json \
    ...
  ```
+
 ## 7. Unexpected type fp8

 ### Cause
+
 FP8 model is not supported

 ### Solution
+
 Replace model weights, e.g., switch Qwen/Qwen3.5-27B-FP8 to Eco-Tech/Qwen3.5-27B-w8a8-mtp.

 ## 8. Docker image versions: stable release vs. daily build
@@ -148,6 +148,7 @@ extern "C" __global__ __aicore__ void helloworld(
 ```

 **Key points:**
+
 - Class methods must be marked with `__aicore__`, indicating they run on AICore.
 - Use `AscendC::TPipe` + `AscendC::TQue` to build a pipeline that overlaps data
  movement and computation.
@@ -201,6 +202,7 @@ HOST_API at::Tensor helloworld(const at::Tensor &x, const at::Tensor &y)
 ```

 **Key points:**
+
 - The namespace must be `sglang::npu_kernel`.
 - Function signatures follow the pattern
  `at::Tensor <op_name>(const at::Tensor &input, ...)`.
@@ -249,6 +251,7 @@ TORCH_LIBRARY_IMPL(npu, PrivateUse1, m)
 ```

 **Schema conventions:**
+
 - The namespace is fixed to `npu`. In SGLang, operators are called via
  `torch.ops.npu.<op_name>()`.
 - Output tensor parameters use the `Tensor(a!)` mutating annotation.
@@ -258,6 +261,7 @@ TORCH_LIBRARY_IMPL(npu, PrivateUse1, m)
  [PyTorch Schema Reference](https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/README.md#func).

 **Implementation binding rules:**
+
 - The device name is fixed to `PrivateUse1` (PyTorch NPU backend identifier).
 - Use the `TORCH_FN` macro to bind to the implementation function.
 - For complex operators with optional parameters, use lambda expressions to
@@ -436,6 +440,7 @@ python tests/python/sgl_kernel_npu/test_helloworld.py
 ```

 **Testing checklist:**
+
 - Cover typical input shapes (power-of-2 sizes and non-standard sizes).
 - Cover different data types (bf16 / fp16, etc.).
 - For operators with in-place behavior, verify correctness of output tensors.
@@ -493,11 +498,13 @@ submitting a PR.
   steps above. Ensure all tests pass.
 4. **Run pre-commit**: Ensure code formatting compliance.
 5. **Commit and push**:
+
   ```bash
   git add .
   git commit -m "feat: add <op_name> operator"
   git push origin feature/add-my-op
   ```
+
 6. **Create a PR**: Open a Pull Request on GitHub from your branch to
   `sgl-project/sgl-kernel-npu:main`.
 7. **Wait for CI and review**: CI checks include linting, compilation, and
@@ -17,6 +17,7 @@ The msProf tool is used to collect and analyze key performance indicators of ope
 ```bash
 msprof op python3 test_xxxxx.py
 ```
+
 The following is a case of using msprof for data collection.

 | Attribute | Value |
@@ -65,7 +66,9 @@ The following is a case of using msprof for data collection.
 | cube_utilization(%) | 0 |

 Below is the field-by-field breakdown of the operator performance record, aligned with the official specification.
+
 ### 1. Basic Identification Fields
+
 | Field | Value | Definition (per official docs) |
 |-------|-------|--------------------------------|
 | Name | DequantSwigluQuant_int32_high_performance_100000000 | **Op Name**: Name of the fused operator (dequantization + SwiGLU activation + quantization), with an int32 high-performance implementation suffix. |
@@ -74,6 +77,7 @@ Below is the field-by-field breakdown of the operator performance record, aligne
 | Accelerator Core | AI_VECTOR_CORE | **Task Type**: The operator runs on the AI Vector Core; other common types include AI_CORE (matrix computation core) and AI_CPU. |

 ### 2. Timing & Scheduling Fields
+
 | Field | Value | Definition (per official docs) |
 |-------|-------|--------------------------------|
 | Start Time(us) | 1774489226717521.715 | **Task Start Time**: Absolute start timestamp of the operator task on the device side, in microseconds. |
@@ -81,6 +85,7 @@ Below is the field-by-field breakdown of the operator performance record, aligne
 | Wait Time(us) | 0 | **Task Wait Time**: Time interval between the end of the previous task and the start of the current task. A value of 0 means no idle wait between task dispatches. |

 ### 3. Core Configuration & Precision Fields
+
 | Field | Value | Definition (per official docs) |
 |-------|-------|--------------------------------|
 | Block Dim | 36 | **Block Num**: Number of parallel thread blocks for the operator task, corresponding to Block Dim in the SIMT programming model. One AI Vector Core executes only one thread block at a time, so this value reflects the scale of occupied parallel compute resources. |
@@ -88,6 +93,7 @@ Below is the field-by-field breakdown of the operator performance record, aligne
 | HF32 Eligible | NO | **HF32 Eligible**: Indicates whether the HF32 high-precision floating-point format is enabled; `NO` means it is not used. This field is reported only at the `--task-time=l1` collection level. |

 ### 4. Input & Output Information
+
 | Field | Value | Definition (per official docs) |
 |-------|-------|--------------------------------|
 | Input Shapes | 163840,1024;128,1024;16384;;;;128 | **Input Shapes**: Dimensions of each input tensor, separated by semicolons; empty values represent scalar inputs. Breakdown: 7 inputs with shapes `[163840,1024]`, `[128,1024]`, `[163840]`, 3 scalars, and `[128]`. |
@@ -99,6 +105,7 @@ Below is the field-by-field breakdown of the operator performance record, aligne
 | Context ID | N/A | **Context ID**: Identifier for sub-tasks at Sub Task granularity; N/A means no sub-task splitting for this operator. |

 ### 5. AI Core Performance Metrics (aic_* series)
+
 | Field | Value | Definition (per official docs) |
 |-------|-------|--------------------------------|
 | aicore_time(us) | 0 | Theoretical execution time on AI Core, in microseconds. |
@@ -111,6 +118,7 @@ Below is the field-by-field breakdown of the operator performance record, aligne
 | aic_icache_miss_rate | 0 | Instruction cache miss rate of AI Core. |

 ### 6. AI Vector Core Performance Metrics (aiv_* series)
+
 | Field | Value | Definition & Interpretation |
 |-------|-------|------------------------------|
 | aiv_time(us) | 59.128 | **aiv_time**: Theoretical execution time on Vector Core under ideal conditions (all blocks scheduled simultaneously with equal duration). In practice, this value is slightly smaller than real execution time due to staggered block startup. |
@@ -122,11 +130,11 @@ Below is the field-by-field breakdown of the operator performance record, aligne
 | aiv_icache_miss_rate | 0.005 | Vector Core instruction cache miss rate of 0.5%, extremely low and indicates efficient instruction fetch. |

 ### 7. Utilization Metrics
+
 | Field | Value | Definition (per official docs) |
 |-------|-------|--------------------------------|
 | cube_utilization(%) | 0 | **cube_utilization**: Utilization rate of the matrix multiplication unit. The value is 0 because the operator is purely vector-based. |

-
 ## Optimization

 ### Specification
@@ -390,6 +390,7 @@ The following environment variables are used in other best practice configuratio
 The table shows feature-by-feature compatibility of SGLang capabilities on Ascend NPU. Features that have performance impact on Ascend are included. Each cell at the intersection of two features indicates whether they can be used together.

 The symbols used are defined as follows:
+
 - 🟢 = Full compatibility
 - 🟠 = Partial compatibility
 - ❌ = No compatibility
@@ -791,7 +792,6 @@ Enables Expert Parallelism Load Balancing to dynamically redistribute experts ac

 Casts model weight tensors to the Ascend NPU ACL FRACTAL_NZ format (format 29) for improved memory access efficiency on Da Vinci AI cores. Enabled by default globally; disable with `SGLANG_NPU_DISABLE_ACL_FORMAT_WEIGHT=1` when the format conflicts with other features (e.g., EPLB). Applied to linear weights, MoE weights, quantized weights, MLA QKV projections, and KV Cache (via `PA_NZ` / `nzcache` modes for FIA). Related env var: `SGLANG_USE_FIA_NZ=1` (requires `SGLANG_NPU_USE_MLAPO=1`).

-
 ## See Also

 - [Ascend NPU Best Practice](/docs/hardware-platforms/ascend-npus/best_practice) — Complete deployment configurations and benchmark results for all supported models
@@ -436,6 +436,7 @@ The output format is **hardcoded in `bench_serving.py`**. All formatting decisio
 </table>

 ##### Time per Output Token (TPOT) – Excluding First Token
+
 Formula: <code>(E2E Latency - TTFT) / (Number of output tokens - 1)</code>

 <table>
@@ -460,7 +461,6 @@ Formula: <code>(E2E Latency - TTFT) / (Number of output tokens - 1)</code>
  </tbody>
 </table>

-
 ## 3. Online Service: Multimodal Model

 Test `Qwen/Qwen2.5-VL-7B-Instruct` for vision-language tasks.
@@ -259,6 +259,7 @@ python3 -m sglang.profiler \
  --num-steps 3 \
  --cpu --gpu &
 ```
+
 ```bash Command
 # Terminal 2: Wait for "Waiting for N steps" output from Terminal 1, then send requests.
 # The profiler starts recording once /start_profile is received by the server.
@@ -407,15 +408,18 @@ them via:

 - **When profiling starts**: server log outputs
  `Profiling starts. Traces will be saved to: <path> (with profile id: <id>)`
-```
+
+```text
 [2026-05-19 13:23:15] Profiling starts. Traces will be saved to: /tmp/1779196995.6948605 (with profile id: 1779196995.6979997)
 [2026-05-19 13:23:15] [WARNING] [350443] profiler.py: Invalid parameter export_type: None, reset it to text.
 [2026-05-19 13:23:15] [WARNING] [350443] profiler.py: Invalid parameter export_type: None, reset it to text.
 [2026-05-19 13:23:15] INFO:     127.0.0.1:40714 - "POST /start_profile HTTP/1.1" 200 OK
 ```
+
 - **When profiling stops**: server log outputs
  `Profiling done. Traces are saved to: <path>`
-```
+
+```text
 [2026-05-19 13:23:17] Stop profiling...
 [2026-05-19 13:23:17] [WARNING] [350443] profiler.py: Incorrect schedule: Stop profiler while current state is RECORD which may result in incomplete parsed data.
 [rank0]:[W519 13:23:17.084812760 compiler_depend.ts:3136] Warning: The indexFromRank 0is not equal indexFromCurDevice 4 , which might be normal if the number of devices on your collective communication server is inconsistent.Otherwise, you need to check if the current device is correct when calling the interface.If it's incorrect, it might have introduced an error. (function operator())
@@ -424,8 +428,10 @@ them via:
 [2026-05-19 13:23:32] [INFO] [352725] profiler.py: All profiling data parsed in a total time of 0:00:14.305669
 [2026-05-19 13:23:32] Profiling done. Traces are saved to: /tmp/1779196995.6948605
 ```
+
 - **CLI output**: `sglang.profiler` outputs `Dump profiling traces to <path>`
-```
+
+```text
 Dump profiling traces to /tmp/1779243331.3219
 Waiting for 10 steps and the trace to be flushed.... (profile_by_stage=False)
 ```
@@ -312,6 +312,7 @@ python3 -m sglang.launch_server \
 ```

 > **Implementation Notes:**
+>
 > - GGUF weights are pre-dequantized to FP16/BF16 during model loading on CPU, then transferred to NPU for inference. This trades higher memory usage for faster runtime performance (no per-forward-pass dequantization overhead).
 > - MoE layers use `npu_grouped_matmul` and `npu_moe_init_routing` / `npu_moe_finalize_routing` for high-performance expert computation.
 > - TP (tensor parallelism) sharding is supported for both dense and MoE GGUF models.
@@ -64,7 +64,7 @@ Please note all your testing and benchmarking results in PR description.

 ### Benchmark

- **(Required) MMMU**: follow MMMU benchmark [README.md](https://github.com/sgl-project/sglang/blob/main/benchmark/mmmu/README.md) to get SGLang vs. HF Transformer accuracy comparison. The accuracy score from SGLang run should not be much lower than that from HF Transformer run. Similarly, follow https://docs.sglang.io/developer_guide/benchmark_and_profiling.html to get performance comparison: TTFT and throughput must meet or exceed baselines (e.g., HF Transformer).
+- **(Required) MMMU**: follow MMMU benchmark [README.md](https://github.com/sgl-project/sglang/blob/main/benchmark/mmmu/README.md) to get SGLang vs. HF Transformer accuracy comparison. The accuracy score from SGLang run should not be much lower than that from HF Transformer run. Similarly, follow [https://docs.sglang.io/developer_guide/benchmark_and_profiling.html](https://docs.sglang.io/developer_guide/benchmark_and_profiling.html) to get performance comparison: TTFT and throughput must meet or exceed baselines (e.g., HF Transformer).
 - **(Optional) Other evals**: If you ran other evals, please note the results in PR description.

 <Tip>
@@ -332,7 +332,7 @@ Using the same Llama wrapper from the previous section, here is how to package a

 1. Create your project

-```
+```text
 sglang_custom_project/
 |----setup.py
 |----custom_llm/
@@ -404,8 +404,8 @@ Update the `config.json` under your HuggingFace model checkpoint directory so th

 ```json Config
 {
-  "architectures": ["LlamaWrapper"],
-  ...
+    "architectures": ["LlamaWrapper"],
+    ...
 }
 ```

@@ -435,7 +435,7 @@ For example, let's build a custom model based on Qwen2-VL-Instruct that takes th

 Create the project:

-```
+```text
 sglang_custom_project_vl/
 |----setup.py
 |----custom_vlm/
@@ -24,6 +24,7 @@ pip install .
 ```

 Before run servers ensure that *.so files of mooncake in LD_LIBRARY_PATH
+
 ```bash
 export LD_LIBRARY_PATH=/usr/local/python3.11.14/lib/python3.11/site-packages/mooncake:$LD_LIBRARY_PATH
 ```
@@ -29,7 +29,6 @@ pip install -e .
 ```
 </CodeGroup>

-
 ## Run Model

 Current SGLang-MindSpore supports Qwen3 and DeepSeek V3/R1 models. This doc uses Qwen3-8B as an example.
@@ -164,6 +163,7 @@ export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python  # to avoid protobuf binary
 </CodeGroup>

 ## Support
+
 For MindSpore-specific issues:

 - Refer to the [MindSpore documentation](https://www.mindspore.cn/)