minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Fix stale CUDA graph benchmark and docs refs (#28041)

Browse Source
This commit is contained in:
Mohammad Miadh Angkad
2026-06-13 21:51:42 -07:00
committed by GitHub
parent 5da3b37a9d
commit 91c63aeb4d
4 changed files with 11 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/docs/advanced_features/breakable_cuda_graph.mdx
+6 -6
View File
@@ -33,7 +33,7 @@ This mode is intended for debugging only — it eliminates the performance benef
For production use, you can mark specific functions as "non-graphable" using the `@eager_on_graph` decorator. During CUDA graph capture, these functions run eagerly between captured graph segments. Outside of capture, they behave normally.
```python
from sglang.srt.model_executor.breakable_cuda_graph.breakable_cuda_graph import eager_on_graph
from sglang.srt.model_executor.runner_backend_utils.breakable_cuda_graph import eager_on_graph
@eager_on_graph(enable=True)
def my_dynamic_op(x):
@@ -44,7 +44,7 @@ def my_dynamic_op(x):
You can also insert a bare graph break (no computation) using the `break_graph()` helper:
```python
from sglang.srt.model_executor.breakable_cuda_graph.breakable_cuda_graph import break_graph
from sglang.srt.model_executor.runner_backend_utils.breakable_cuda_graph import break_graph
def forward(self, x):
x = self.layer1(x)
@@ -169,16 +169,16 @@ For typical use cases with a small number of graph breaks, the overhead is negli
</thead>
<tbody>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>python/sglang/srt/model_executor/breakable_cuda_graph/breakable_cuda_graph.py</code></td>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>python/sglang/srt/model_executor/runner_backend_utils/breakable_cuda_graph/breakable_cuda_graph.py</code></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Core implementation: <code>eager_on_graph</code>, <code>BreakableCUDAGraph</code>, <code>BreakableCUDAGraphCapture</code></td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>python/sglang/srt/model_executor/breakable_cuda_graph/cuda_utils.py</code></td>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>python/sglang/srt/model_executor/runner_backend_utils/breakable_cuda_graph/cuda_utils.py</code></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>CUDA runtime binding utilities</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>python/sglang/srt/model_executor/cuda_graph_runner.py</code></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Integration with the main CUDA graph runner</td>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>python/sglang/srt/model_executor/runner_backend/breakable_cuda_graph_backend.py</code></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Integration with CUDA graph runner backends</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>python/sglang/srt/server_args.py</code></td>
docs_new/docs/advanced_features/piecewise_cuda_graph.mdx
+2 -2
View File
@@ -264,8 +264,8 @@ Use `--enforce-piecewise-cuda-graph` to skip all auto-disable checks (for testin
</thead>
<tbody>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>python/sglang/srt/model_executor/piecewise_cuda_graph_runner.py</code></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Main runner: init, capture, replay</td>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>python/sglang/srt/model_executor/runner_backend/tc_piecewise_cuda_graph_backend.py</code></td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Backend implementation: compile, capture, replay</td>
</tr>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}><code>python/sglang/srt/compilation/compile.py</code></td>