minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

fix(router): configure HTTP client connection settings (#24330)

Browse Source 
Signed-off-by: Revanth Reddy Airre <revanthreddy@hippocraticai.com>
This commit is contained in:
Revanth Reddy Airre
2026-05-07 11:42:45 -07:00
committed by GitHub
parent 90a618e37b
commit be088f8076
8 changed files with 137 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/advanced_features/sgl_model_gateway.md
+5 -2
View File
@@ -593,13 +593,16 @@ Response:
## Reliability and Flow Control
### HTTP Client Pool
### HTTP Client
Configure the idle timeout for pooled upstream HTTP connections:
Configure upstream HTTP client connection settings:
| Parameter | Default | Description |
|-----------|---------|-------------|
| `--pool-idle-timeout-secs` | 50 | Idle timeout in seconds for pooled upstream HTTP connections. Can also be set with `SMG_POOL_IDLE_TIMEOUT_SECS`. |
| `--connect-timeout-secs` | 10 | Timeout in seconds for new upstream HTTP connections. Can also be set with `SMG_CONNECT_TIMEOUT_SECS`. |
| `--pool-max-idle-per-host` | 500 | Maximum idle upstream HTTP connections to keep per host. Can also be set with `SMG_POOL_MAX_IDLE_PER_HOST`. |
| `--tcp-keepalive-secs` | 30 | TCP keepalive idle time in seconds for upstream HTTP connections. Can also be set with `SMG_TCP_KEEPALIVE_SECS`. |
### Retries