[Docs] Sync docs_new with legacy docs and update migration redirects (#23337)

Co-authored-by: Mingyi <wisclmy0611@gmail.com>
This commit is contained in:
zijiexia
2026-04-21 00:15:17 -07:00
committed by GitHub
co-authored by Mingyi
parent f63def8510
commit 900aad5f72
179 changed files with 16014 additions and 8162 deletions
+27 -29
View File
@@ -7,7 +7,7 @@ Apart from the OpenAI compatible APIs, the SGLang Runtime also provides its nati
- `/generate` (text generation model)
- `/get_model_info`
- `/get_server_info`
- `/server_info`
- `/health`
- `/health_generate`
- `/flush_cache`
@@ -35,7 +35,7 @@ server_process, port = launch_server_cmd(
"python3 -m sglang.launch_server --model-path qwen/qwen2.5-0.5b-instruct --host 0.0.0.0 --log-level warning"
)
wait_for_server(f"http://localhost:{port}")
wait_for_server(f"http://localhost:{port}", process=server_process)
```
## Generate (text generation model)
@@ -96,7 +96,7 @@ Gets the server information including CLI arguments, token limits, and memory po
- `get_max_total_num_tokens`
```python Example
url = f"http://localhost:{port}/get_server_info"
url = f"http://localhost:{port}/server_info"
response = requests.get(url)
print_highlight(response.text)
@@ -124,6 +124,14 @@ print_highlight(response.text)
Flush the radix cache. It will be automatically triggered when the model weights are updated by the `/update_weights` API.
Parameters:
- `timeout` (query, float, default `0`, unit: seconds): Wait time for idle state before flushing. `0` means fail fast if not idle. When HiCache async operations are in-flight, a non-zero timeout allows the server to wait until idle before flushing, avoiding unnecessary 400 errors.
```bash Command
# With timeout (wait up to 30s for idle state)
curl -s -X POST "http://127.0.0.1:30000/flush_cache?timeout=30"
```
```python Example
url = f"http://localhost:{port}/flush_cache"
@@ -176,14 +184,12 @@ Encode text into embeddings. Note that this API is only available for [embedding
Therefore, we launch a new server to server an embedding model.
```python Example
embedding_process, port = launch_server_cmd(
"""
embedding_process, port = launch_server_cmd("""
python3 -m sglang.launch_server --model-path Alibaba-NLP/gte-Qwen2-1.5B-instruct \
--host 0.0.0.0 --is-embedding --log-level warning
"""
)
""")
wait_for_server(f"http://localhost:{port}")
wait_for_server(f"http://localhost:{port}", process=embedding_process)
```
```python Example
@@ -205,14 +211,12 @@ terminate_process(embedding_process)
Rerank a list of documents given a query using a cross-encoder model. Note that this API is only available for cross encoder model like [BAAI/bge-reranker-v2-m3](https://huggingface.co/BAAI/bge-reranker-v2-m3) with `attention-backend` `triton` and `torch_native`.
```python Example
reranker_process, port = launch_server_cmd(
"""
reranker_process, port = launch_server_cmd("""
python3 -m sglang.launch_server --model-path BAAI/bge-reranker-v2-m3 \
--host 0.0.0.0 --disable-radix-cache --chunked-prefill-size -1 --attention-backend triton --is-embedding --log-level warning
"""
)
""")
wait_for_server(f"http://localhost:{port}")
wait_for_server(f"http://localhost:{port}", process=reranker_process)
```
```python Example
@@ -253,14 +257,12 @@ Parameters:
The response contains `scores` - a list of probability lists, one per item, each in the order of `label_token_ids`.
```python Example
score_process, port = launch_server_cmd(
"""
score_process, port = launch_server_cmd("""
python3 -m sglang.launch_server --model-path qwen/qwen2.5-0.5b-instruct \
--host 0.0.0.0 --log-level warning
"""
)
""")
wait_for_server(f"http://localhost:{port}")
wait_for_server(f"http://localhost:{port}", process=score_process)
```
```python Example
@@ -297,13 +299,11 @@ SGLang Runtime also supports reward models. Here we use a reward model to classi
# Note that SGLang now treats embedding models and reward models as the same type of models.
# This will be updated in the future.
reward_process, port = launch_server_cmd(
"""
reward_process, port = launch_server_cmd("""
python3 -m sglang.launch_server --model-path Skywork/Skywork-Reward-Llama-3.1-8B-v0.2 --host 0.0.0.0 --is-embedding --log-level warning
"""
)
""")
wait_for_server(f"http://localhost:{port}")
wait_for_server(f"http://localhost:{port}", process=reward_process)
```
```python Example
@@ -347,7 +347,7 @@ expert_record_server_process, port = launch_server_cmd(
"python3 -m sglang.launch_server --model-path Qwen/Qwen1.5-MoE-A2.7B --host 0.0.0.0 --expert-distribution-recorder-mode stat --log-level warning"
)
wait_for_server(f"http://localhost:{port}")
wait_for_server(f"http://localhost:{port}", process=expert_record_server_process)
```
```python Example
@@ -376,13 +376,11 @@ terminate_process(expert_record_server_process)
This example demonstrates how to use the /tokenize and /detokenize endpoints together. We first tokenize a string, then detokenize the resulting IDs to reconstruct the original text. This workflow is useful when you need to handle tokenization externally but still leverage the server for detokenization.
```python Example
tokenizer_free_server_process, port = launch_server_cmd(
"""
tokenizer_free_server_process, port = launch_server_cmd("""
python3 -m sglang.launch_server --model-path qwen/qwen2.5-0.5b-instruct
"""
)
""")
wait_for_server(f"http://localhost:{port}")
wait_for_server(f"http://localhost:{port}", process=tokenizer_free_server_process)
```
```python Example