minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity

fix(router): make HTTP pool idle timeout configurable (#24329)

Browse Source 
Signed-off-by: Revanth Reddy Airre <revanthreddy@hippocraticai.com>
This commit is contained in:
Revanth Reddy Airre
2026-05-06 22:11:11 -07:00
committed by GitHub
parent 8bb0ce192b
commit d363315de9
7 changed files with 82 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/advanced_features/sgl_model_gateway.md
+8
View File
@@ -593,6 +593,14 @@ Response:
## Reliability and Flow Control
### HTTP Client Pool
Configure the idle timeout for pooled upstream HTTP connections:
| Parameter | Default | Description |
|-----------|---------|-------------|
| `--pool-idle-timeout-secs` | 50 | Idle timeout in seconds for pooled upstream HTTP connections. Can also be set with `SMG_POOL_IDLE_TIMEOUT_SECS`. |
### Retries
Configure exponential backoff retries: