sglang/docs_new/docs/supported-models/mindspore-models.mdx

---
title: "MindSpore Models"
---

MindSpore is a high-performance AI framework optimized for [Ascend NPUs](../hardware-platforms/ascend-npus/SGLang-installation-with-NPUs-support). This doc guides users to run MindSpore models in SGLang.

## Requirements

MindSpore currently only supports Ascend NPU devices. Users need to first install Ascend CANN software packages. The CANN software packages can be downloaded from the [Ascend Official Website](https://www.hiascend.com). The recommended version is 8.3.RC2.

## Supported Models

Currently, the following models are supported:

<CardGroup cols={3}>
  <Card title="Qwen3" icon="cube">
    Dense and MoE models
  </Card>
  <Card title="DeepSeek V3/R1" icon="cube">
    DeepSeek V3 and R1 models
  </Card>
  <Card title="More Coming Soon" icon="clock">
    Additional models are on the way
  </Card>
</CardGroup>

## Installation

<Note>Currently, MindSpore models are provided by an independent package `sgl-mindspore`. Support for MindSpore is built upon current SGLang support for Ascend NPU platform. Please first [install SGLang for Ascend NPU](../hardware-platforms/ascend-npus/SGLang-installation-with-NPUs-support) and then install `sgl-mindspore`.</Note>

<CodeGroup>
```bash Install
git clone https://github.com/mindspore-lab/sgl-mindspore.git
cd sgl-mindspore
pip install -e .
```
</CodeGroup>

## Run Model

Current SGLang-MindSpore supports Qwen3 and DeepSeek V3/R1 models. This doc uses Qwen3-8B as an example.

### Offline Infer

Use the following script for offline infer:

<CodeGroup>
```python Offline Infer
import sglang as sgl

# Initialize the engine with MindSpore backend
llm = sgl.Engine(
    model_path="/path/to/your/model",  # Local model path
    device="npu",                      # Use NPU device
    model_impl="mindspore",            # MindSpore implementation
    attention_backend="ascend",        # Attention backend
    tp_size=1,                         # Tensor parallelism size
    dp_size=1                          # Data parallelism size
)

# Generate text
prompts = [
    "Hello, my name is",
    "The capital of France is",
    "The future of AI is"
]

sampling_params = {"temperature": 0, "top_p": 0.9}
outputs = llm.generate(prompts, sampling_params)

for prompt, output in zip(prompts, outputs):
    print(f"Prompt: {prompt}")
    print(f"Generated: {output['text']}")
    print("---")
```
</CodeGroup>

### Start Server

<CodeGroup>
```bash Single Node
python3 -m sglang.launch_server \
    --model-path /path/to/your/model \
    --host 0.0.0.0 \
    --device npu \
    --model-impl mindspore \
    --attention-backend ascend \
    --tp-size 1 \
    --dp-size 1
```
```bash Multi-Node Distributed
python3 -m sglang.launch_server \
    --model-path /path/to/your/model \
    --host 0.0.0.0 \
    --device npu \
    --model-impl mindspore \
    --attention-backend ascend \
    --dist-init-addr 127.0.0.1:29500 \
    --nnodes 2 \
    --node-rank 0 \
    --tp-size 4 \
    --dp-size 2
```
</CodeGroup>

## Troubleshooting

### Debug Mode

Enable sglang debug logging by log-level argument:

<CodeGroup>
```bash Debug Mode
python3 -m sglang.launch_server \
    --model-path /path/to/your/model \
    --host 0.0.0.0 \
    --device npu \
    --model-impl mindspore \
    --attention-backend ascend \
    --log-level DEBUG
```
</CodeGroup>

Enable MindSpore info and debug logging by setting environments:

<CodeGroup>
```bash INFO
export GLOG_v=1
```
```bash DEBUG
export GLOG_v=0
```
</CodeGroup>

### Explicitly Select Devices

Use the following environment variable to explicitly select the devices to use:

<CodeGroup>
```bash Select Devices
export ASCEND_RT_VISIBLE_DEVICES=4,5,6,7
```
</CodeGroup>

### Some Communication Environment Issues

In case of some environment with special communication environment, users need to set some environment variables:

<CodeGroup>
```bash Disable LCCL
export MS_ENABLE_LCCL=off # current not support LCCL communication mode in SGLang-MindSpore
```
</CodeGroup>

### Some Dependencies of Protobuf

In case of some environment with special protobuf version, users need to set some environment variables to avoid binary version mismatch:

<CodeGroup>
```bash Fix Protobuf
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
```
</CodeGroup>

## Support

For MindSpore-specific issues, refer to the [MindSpore documentation](https://www.mindspore.cn/).