minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[Doc] Default lora backend: csgmv (#16444)

Browse Source
This commit is contained in:
Huapeng Zhou
2026-01-05 12:45:49 +08:00
committed by GitHub
parent c63e9cb29e
commit 078270473a
2 changed files with 4 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/advanced_features/lora.ipynb
+3 -4
View File
@@ -534,11 +534,10 @@
"\n",
"SGLang supports two LoRA backends that you can choose from using the `--lora-backend` argument:\n",
"\n",
"- `triton`: Default basic Triton-based backend.\n",
"- `csgmv`: Chunked SGMV backend optimized for high concurrency scenarios.\n",
"- `triton`: Basic Triton-based backend.\n",
"- `csgmv`: Default chunked SGMV backend optimized for high concurrency scenarios.\n",
"\n",
"The `csgmv` backend was recently introduced to improve performance especially at high-concurrency scenarios. Our benchmark shows that it achieves 20% to 80% latency improvements over the basic triton backend.\n",
"Currently it is at preview phase, we expect to make it our the default LoRA backend in future release. Before that, you can adopt it by manually setting the `--lora-backend` server config."
"The `csgmv` backend was recently introduced to improve performance especially at high-concurrency scenarios. Our benchmark shows that it achieves 20% to 80% latency improvements over the basic triton backend."
]
},
{