minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[NPU] Update quantization&CI documentation (#21100)

Browse Source 
Co-authored-by: Tamir Baydasov <41994229+TamirBaydasov@users.noreply.github.com>
This commit is contained in:
Артем Савкин
2026-03-28 21:42:21 +03:00
committed by GitHub
co-authored by Tamir Baydasov
parent ced69c9f84
commit 27071e0a43
22 changed files with 255 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/basic_usage/deepseek_v3.md
+1 -1
View File
@@ -74,7 +74,7 @@ Detailed commands for reference:
- [16 x A100 (INT8)](https://github.com/sgl-project/sglang/tree/main/benchmark/deepseek_v3#example-serving-with-16-a100a800-with-int8-quantization)
- [32 x L40S (INT8)](https://github.com/sgl-project/sglang/tree/main/benchmark/deepseek_v3#example-serving-with-32-l40s-with-int8-quantization)
- [Xeon 6980P CPU](../platforms/cpu_server.md#example-running-deepseek-r1)
- [4 x Atlas 800I A3 (int8)](../platforms/ascend_npu_deepseek_example.md#running-deepseek-with-pd-disaggregation-on-4-x-atlas-800i-a3)
- [4 x Atlas 800I A3 (int8)](../platforms/ascend/ascend_npu_deepseek_example.md#running-deepseek-with-pd-disaggregation-on-4-x-atlas-800i-a3)
### Download Weights
If you encounter errors when starting the server, ensure the weights have finished downloading. It's recommended to download them beforehand or restart multiple times until all weights are downloaded. Please refer to [DeepSeek V3](https://huggingface.co/deepseek-ai/DeepSeek-V3-Base#61-inference-with-deepseek-infer-demo-example-only) official guide to download the weights.