Migrate all callers from /get_server_info to /server_info (#21463)
This commit is contained in:
@@ -212,7 +212,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
|
||||
| Argument | Description | Defaults | Options |
|
||||
| --- | --- | --- | --- |
|
||||
| `--api-key` | Set API key of the server. It is also used in the OpenAI API compatible server. | `None` | Type: str |
|
||||
| `--admin-api-key` | Set **admin API key** for administrative/control endpoints (e.g., weights update, cache flush, `/get_server_info`). Endpoints marked as admin-only require `Authorization: Bearer <admin_api_key>` when this is set. | `None` | Type: str |
|
||||
| `--admin-api-key` | Set **admin API key** for administrative/control endpoints (e.g., weights update, cache flush, `/server_info`). Endpoints marked as admin-only require `Authorization: Bearer <admin_api_key>` when this is set. | `None` | Type: str |
|
||||
| `--served-model-name` | Override the model name returned by the v1/models endpoint in OpenAI API server. | `None` | Type: str |
|
||||
| `--weight-version` | Version identifier for the model weights. Defaults to 'default' if not specified. | `default` | Type: str |
|
||||
| `--chat-template` | The builtin chat template name or the path of the chat template file. This is only used for OpenAI-compatible API server. | `None` | Type: str |
|
||||
|
||||
@@ -77,7 +77,7 @@ SGLang Model Gateway is a high-performance model-routing gateway for large-scale
|
||||
|
||||
### Control Plane
|
||||
|
||||
- **Worker Manager** discovers capabilities (`/get_server_info`, `/get_model_info`), tracks load, and registers/removes workers in the shared registry.
|
||||
- **Worker Manager** discovers capabilities (`/server_info`, `/get_model_info`), tracks load, and registers/removes workers in the shared registry.
|
||||
- **Job Queue** serializes add/remove requests and exposes status (`/workers/{worker_id}`) so clients can track onboarding progress.
|
||||
- **Load Monitor** feeds cache-aware and power-of-two policies with live worker load statistics.
|
||||
- **Health Checker** continuously probes workers and updates readiness, circuit breaker state, and router metrics.
|
||||
@@ -552,7 +552,7 @@ Response:
|
||||
| `GET` | `/engine_metrics` | Engine-level metrics from workers |
|
||||
| `GET` | `/v1/models` | List available models |
|
||||
| `GET` | `/get_model_info` | Get model information |
|
||||
| `GET` | `/get_server_info` | Get server information |
|
||||
| `GET` | `/server_info` | Get server information |
|
||||
| `POST` | `/flush_cache` | Clear all caches |
|
||||
| `GET` | `/get_loads` | Get all worker loads |
|
||||
| `POST` | `/wasm` | Upload WASM module |
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
"\n",
|
||||
"- `/generate` (text generation model)\n",
|
||||
"- `/get_model_info`\n",
|
||||
"- `/get_server_info`\n",
|
||||
"- `/server_info`\n",
|
||||
"- `/health`\n",
|
||||
"- `/health_generate`\n",
|
||||
"- `/flush_cache`\n",
|
||||
@@ -140,7 +140,7 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"url = f\"http://localhost:{port}/get_server_info\"\n",
|
||||
"url = f\"http://localhost:{port}/server_info\"\n",
|
||||
"\n",
|
||||
"response = requests.get(url)\n",
|
||||
"print_highlight(response.text)"
|
||||
|
||||
@@ -352,4 +352,4 @@ python3 -m sglang.bench_serving \
|
||||
### Notes
|
||||
|
||||
- The script raises the file descriptor soft limit (`RLIMIT_NOFILE`) to help with many concurrent connections.
|
||||
- For sglang, `/get_server_info` is queried post-run to report speculative decoding accept length when available.
|
||||
- For sglang, `/server_info` is queried post-run to report speculative decoding accept length when available.
|
||||
|
||||
Reference in New Issue
Block a user