minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[misc] Use --cuda-graph-max-bs-decode in tests, examples, and docs (#29591)

Browse Source
This commit is contained in:
Liangsheng Yin
2026-06-28 18:38:28 -07:00
committed by GitHub
parent 3217410cf6
commit 909123ddb8
140 changed files with 304 additions and 290 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/docs/references/faq.mdx
+1 -1
View File
@@ -23,7 +23,7 @@ This error may result from kernel errors or out-of-memory issues:
### The server hangs
- If the server hangs during initialization or running, it can be memory issues (out of memory), network issues (nccl errors), or other bugs in sglang.
- If it is out of memory, you might see that `avail mem` is very low during the initialization or right after initialization. In this case,
you can try to decrease `--mem-fraction-static`, decrease `--cuda-graph-max-bs`, or decrease `--chunked-prefill-size`.
you can try to decrease `--mem-fraction-static`, decrease `--cuda-graph-max-bs-decode`, or decrease `--chunked-prefill-size`.
- Other bugs, please file an issue on GitHub.
docs_new/docs/references/multi_node_deployment/lws_pd/lws_pd_deploy.mdx
+2 -2
View File
@@ -384,7 +384,7 @@ spec:
- "32768"
- --disaggregation-ib-device
- "mlx5_bond_0,mlx5_bond_1,mlx5_bond_2,mlx5_bond_3"
- --cuda-graph-max-bs
- --cuda-graph-max-bs-decode
- "64"
- --max-running-requests
- "2048"
@@ -519,7 +519,7 @@ spec:
- --disaggregation-ib-device
# should modify according your rdma env
- "mlx5_bond_0,mlx5_bond_1,mlx5_bond_2,mlx5_bond_3"
- --cuda-graph-max-bs
- --cuda-graph-max-bs-decode
- "64"
- --max-running-requests
- "2048"
docs_new/docs/references/multi_node_deployment/rbg_pd/deepseekv32_pd.mdx
+2 -2
View File
@@ -285,7 +285,7 @@ spec:
- "2048"
- --tp-size
- "8" # Size of Tensor Parallelism
- --cuda-graph-max-bs
- --cuda-graph-max-bs-decode
- "16"
- --dist-init-addr
- $(LWS_LEADER_ADDRESS):20102
@@ -356,7 +356,7 @@ spec:
- mlx5_0,mlx5_1,mlx5_2,mlx5_3,mlx5_4,mlx5_5,mlx5_6,mlx5_7
- --max-running-requests
- "4096"
- --cuda-graph-max-bs
- --cuda-graph-max-bs-decode
- "16"
- --tp-size
- "8" # Size of Tensor Parallelism