[Config] Retire get_global_server_args, and clear the deprecated flags that have a replacement (#38375)

This commit is contained in:
Cheng Wan
2026-09-08 16:42:12 -07:00
committed by GitHub
parent 5177a3ec08
commit db272201a2
213 changed files with 625 additions and 892 deletions
@@ -392,7 +392,7 @@ python3 -m sglang.launch_server \
The default configuration is `--speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4`. Find the best values for your workload with [bench_speculative.py](https://github.com/sgl-project/sglang/blob/main/scripts/playground/bench_speculative.py). The minimum viable config is `--speculative-num-steps 1 --speculative-eagle-topk 1 --speculative-num-draft-tokens 2`.
<Note>
For large batch sizes (>48), increase `--max-running-requests` beyond the default of 48 for MTP. Also set `--cuda-graph-bs` to include your target batch sizes (default captured sizes for speculative decoding: 48).
For large batch sizes (>48), increase `--max-running-requests` beyond the default of 48 for MTP. Also set `--cuda-graph-bs-decode` to include your target batch sizes (default captured sizes for speculative decoding: 48).
</Note>
<Tip>
@@ -102,7 +102,7 @@ import { Playground } from "/src/snippets/_playground.jsx";
- **Remote code.** Use `--trust-remote-code` for the Hugging Face checkpoint.
- **Topology.** The 8x B300 recipe uses TP=8 and EP=8. H200, B200, and H20 use a 2-node 16 GPU layout with TP=16 and EP=16; the command panel injects the multi-node rank flags for you.
- **LongCat sparse attention.** Keep `--nsa-prefill-backend fa3` with `--chunked-prefill-size 2048` for the model-card-aligned prefill path.
- **LongCat sparse attention.** Keep `--dsa-prefill-backend fa3` with `--chunked-prefill-size 2048` for the model-card-aligned prefill path.
- **Memory.** The recipe uses `--kv-cache-dtype bfloat16` and starts at `--mem-fraction-static 0.92`. Tune memory only after the generated command launches cleanly on your cluster.
- **Weight loading.** `--model-loader-extra-config '{"enable_multithread_load":true,"num_threads":12}'` loads checkpoint shards in parallel and reduces startup time.
- **FP8 backend selection.** Do not pass `--fp8-gemm-runner-backend` manually. SGLang selects the correct backend for the LongCat FP8 scale layout.
@@ -260,7 +260,7 @@ checkpoint's calibration scales automatically.
is NVFP4-packed; the BF16 and FP8 checkpoints keep a dense head).
The Ascend comparison in #35629 used a 910C with BF16 target weights,
`--tp-size 2 --attention-backend ascend --mamba-ssm-dtype bfloat16
--mamba-scheduler-strategy extra_buffer`, and disabled RadixCache for both
--mamba-radix-cache-strategy extra_buffer`, and disabled RadixCache for both
baseline and DFlash2 to exclude cache warm-up and prefix reuse. The DFlash2
run added the three flags shown above.
Accuracy used zero-shot GSM8K with greedy sampling, `max_new_tokens=2048`,