[diffusion] docs: consolidate diffusion documentation into docs (#18095)

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: JiaxinD <djx2048@gmail.com>
2026-02-11 16:55:07 -08:00
parent 7eaf866846
commit f06ab17a73
30 changed files with 512 additions and 1481 deletions
@@ -9,3 +9,4 @@ Adding new models and alternative backends.
   support_new_models.md
   transformers_fallback.md
   modelscope.md
+   mindspore_models.md
@@ -0,0 +1,151 @@
+# MindSpore Models
+
+## Introduction
+
+MindSpore is a high-performance AI framework optimized for Ascend NPUs. This doc guides users to run MindSpore models in SGLang.
+
+## Requirements
+
+MindSpore currently only supports Ascend NPU devices. Users need to first install CANN 8.5.
+The CANN software packages can be downloaded from the [Ascend Official Website](https://www.hiascend.com).
+
+## Supported Models
+
+Currently, the following models are supported:
+
+- **Qwen3**: Dense and MoE models
+- **DeepSeek V3/R1**
+- *More models coming soon...*
+
+## Installation
+
+> **Note**: Currently, MindSpore models are provided by an independent package `sgl-mindspore`. Support for MindSpore is built upon current SGLang support for Ascend NPU platform. Please first [install SGLang for Ascend NPU](../../platforms/ascend_npu.md) and then install `sgl-mindspore`:
+
+```shell
+git clone https://github.com/mindspore-lab/sgl-mindspore.git
+cd sgl-mindspore
+pip install -e .
+```
+
+
+## Run Model
+
+Current SGLang-MindSpore supports Qwen3 and DeepSeek V3/R1 models. This doc uses Qwen3-8B as an example.
+
+### Offline inference
+
+Use the following script for offline inference:
+
+```python
+import sglang as sgl
+
+# Initialize the engine with MindSpore backend
+llm = sgl.Engine(
+    model_path="/path/to/your/model",  # Local model path
+    device="npu",                      # Use NPU device
+    model_impl="mindspore",            # MindSpore implementation
+    attention_backend="ascend",        # Attention backend
+    tp_size=1,                         # Tensor parallelism size
+    dp_size=1                          # Data parallelism size
+)
+
+# Generate text
+prompts = [
+    "Hello, my name is",
+    "The capital of France is",
+    "The future of AI is"
+]
+
+sampling_params = {"temperature": 0, "top_p": 0.9}
+outputs = llm.generate(prompts, sampling_params)
+
+for prompt, output in zip(prompts, outputs):
+    print(f"Prompt: {prompt}")
+    print(f"Generated: {output['text']}")
+    print("---")
+```
+
+### Start server
+
+Launch a server with MindSpore backend:
+
+```bash
+# Basic server startup
+python3 -m sglang.launch_server \
+    --model-path /path/to/your/model \
+    --host 0.0.0.0 \
+    --device npu \
+    --model-impl mindspore \
+    --attention-backend ascend \
+    --tp-size 1 \
+    --dp-size 1
+```
+
+For distributed server with multiple nodes:
+
+```bash
+# Multi-node distributed server
+python3 -m sglang.launch_server \
+    --model-path /path/to/your/model \
+    --host 0.0.0.0 \
+    --device npu \
+    --model-impl mindspore \
+    --attention-backend ascend \
+    --dist-init-addr 127.0.0.1:29500 \
+    --nnodes 2 \
+    --node-rank 0 \
+    --tp-size 4 \
+    --dp-size 2
+```
+
+## Troubleshooting
+
+#### Debug Mode
+
+Enable sglang debug logging by log-level argument.
+
+```bash
+python3 -m sglang.launch_server \
+    --model-path /path/to/your/model \
+    --host 0.0.0.0 \
+    --device npu \
+    --model-impl mindspore \
+    --attention-backend ascend \
+    --log-level DEBUG
+```
+
+Enable mindspore info and debug logging by setting environments.
+
+```bash
+export GLOG_v=1  # INFO
+export GLOG_v=0  # DEBUG
+```
+
+#### Explicitly select devices
+
+Use the following environment variable to explicitly select the devices to use.
+
+```shell
+export ASCEND_RT_VISIBLE_DEVICES=4,5,6,7  # to set device
+```
+
+#### Some communication environment issues
+
+In case of some environment with special communication environment, users need set some environment variables.
+
+```shell
+export MS_ENABLE_LCCL=off # current not support LCCL communication mode in SGLang-MindSpore
+```
+
+#### Some dependencies of protobuf
+
+In case of some environment with special protobuf version, users need set some environment variables to avoid binary version mismatch.
+
+```shell
+export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python  # to avoid protobuf binary version mismatch
+```
+
+## Support
+For MindSpore-specific issues:
+
+- Refer to the [MindSpore documentation](https://www.mindspore.cn/)
@@ -1,9 +0,0 @@
-Image Generation
-================
-
-Models for generating images and videos using diffusion.
-
-.. toctree::
-   :maxdepth: 1
-
-   diffusion_models.md
@@ -8,7 +8,6 @@ Browse by category below to find models suited for your needs.
   :maxdepth: 2

   text_generation/index
-   image_generation/index
   retrieval_ranking/index
   specialized/index
   extending/index
@@ -1,28 +1,28 @@
-# Reward Models
-
-These models output a scalar reward score or classification result, often used in reinforcement learning or content moderation tasks.
-
-```{important}
-They are executed with `--is-embedding` and some may require `--trust-remote-code`.
-```
-
-## Example launch Command
-
-```shell
-python3 -m sglang.launch_server \
-  --model-path Qwen/Qwen2.5-Math-RM-72B \  # example HF/local path
-  --is-embedding \
-  --host 0.0.0.0 \
-  --tp-size=4 \                          # set for tensor parallelism
-  --port 30000 \
-```
-
-## Supported models
-
-| Model Family (Reward)                                                     | Example HuggingFace Identifier                              | Description                                                                     |
-|---------------------------------------------------------------------------|-----------------------------------------------------|---------------------------------------------------------------------------------|
-| **Llama (3.1 Reward / `LlamaForSequenceClassification`)**                   | `Skywork/Skywork-Reward-Llama-3.1-8B-v0.2`            | Reward model (preference classifier) based on Llama 3.1 (8B) for scoring and ranking responses for RLHF.  |
-| **Gemma 2 (27B Reward / `Gemma2ForSequenceClassification`)**                | `Skywork/Skywork-Reward-Gemma-2-27B-v0.2`             | Derived from Gemma‑2 (27B), this model provides human preference scoring for RLHF and multilingual tasks.  |
-| **InternLM 2 (Reward / `InternLM2ForRewardMode`)**                         | `internlm/internlm2-7b-reward`                       | InternLM 2 (7B)–based reward model used in alignment pipelines to guide outputs toward preferred behavior.  |
-| **Qwen2.5 (Reward - Math / `Qwen2ForRewardModel`)**                         | `Qwen/Qwen2.5-Math-RM-72B`                           | A 72B math-specialized RLHF reward model from the Qwen2.5 series, tuned for evaluating and refining responses.  |
-| **Qwen2.5 (Reward - Sequence / `Qwen2ForSequenceClassification`)**          | `jason9693/Qwen2.5-1.5B-apeach`                      | A smaller Qwen2.5 variant used for sequence classification, offering an alternative RLHF scoring mechanism.  |
+# Reward Models
+
+These models output a scalar reward score or classification result, often used in reinforcement learning or content moderation tasks.
+
+```{important}
+They are executed with `--is-embedding` and some may require `--trust-remote-code`.
+```
+
+## Example launch Command
+
+```shell
+python3 -m sglang.launch_server \
+  --model-path Qwen/Qwen2.5-Math-RM-72B \  # example HF/local path
+  --is-embedding \
+  --host 0.0.0.0 \
+  --tp-size=4 \                          # set for tensor parallelism
+  --port 30000 \
+```
+
+## Supported models
+
+| Model Family (Reward)                                                     | Example HuggingFace Identifier                              | Description                                                                     |
+|---------------------------------------------------------------------------|-----------------------------------------------------|---------------------------------------------------------------------------------|
+| **Llama (3.1 Reward / `LlamaForSequenceClassification`)**                   | `Skywork/Skywork-Reward-Llama-3.1-8B-v0.2`            | Reward model (preference classifier) based on Llama 3.1 (8B) for scoring and ranking responses for RLHF.  |
+| **Gemma 2 (27B Reward / `Gemma2ForSequenceClassification`)**                | `Skywork/Skywork-Reward-Gemma-2-27B-v0.2`             | Derived from Gemma‑2 (27B), this model provides human preference scoring for RLHF and multilingual tasks.  |
+| **InternLM 2 (Reward / `InternLM2ForRewardMode`)**                         | `internlm/internlm2-7b-reward`                       | InternLM 2 (7B)–based reward model used in alignment pipelines to guide outputs toward preferred behavior.  |
+| **Qwen2.5 (Reward - Math / `Qwen2ForRewardModel`)**                         | `Qwen/Qwen2.5-Math-RM-72B`                           | A 72B math-specialized RLHF reward model from the Qwen2.5 series, tuned for evaluating and refining responses.  |
+| **Qwen2.5 (Reward - Sequence / `Qwen2ForSequenceClassification`)**          | `jason9693/Qwen2.5-1.5B-apeach`                      | A smaller Qwen2.5 variant used for sequence classification, offering an alternative RLHF scoring mechanism.  |
@@ -1,7 +1,5 @@
 # Diffusion Language Models

-> This page covers **text generation** using diffusion-based LLMs. For **image and video generation**, see [Diffusion Models](../image_generation/diffusion_models.md).
-
 Diffusion language models have shown promise for non-autoregressive text generation with parallel decoding capabilities. Unlike auto-regressive language models, different diffusion language models require different decoding strategies.

 ## Example Launch Command