minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

fix(router): make HTTP pool idle timeout configurable (#24329)

Browse Source 
Signed-off-by: Revanth Reddy Airre <revanthreddy@hippocraticai.com>
This commit is contained in:
Revanth Reddy Airre
2026-05-06 22:11:11 -07:00
committed by GitHub
parent 8bb0ce192b
commit d363315de9
7 changed files with 82 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs_new/docs/advanced_features/sgl_model_gateway.mdx
+26
View File
@@ -944,6 +944,32 @@ Response:
***
## Reliability and Flow Control
### HTTP Client Pool
Configure the idle timeout for pooled upstream HTTP connections:
<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
<colgroup>
<col style={{width: "34%"}} />
<col style={{width: "33%"}} />
<col style={{width: "33%"}} />
</colgroup>
<thead>
<tr style={{borderBottom: "2px solid #d55816"}}>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Default</th>
<th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>`--pool-idle-timeout-secs`</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>50</td>
<td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>Idle timeout in seconds for pooled upstream HTTP connections. Can also be set with `SMG_POOL_IDLE_TIMEOUT_SECS`.</td>
</tr>
</tbody>
</table>
### Retries
Configure exponential backoff retries: