[Config] Retire get_global_server_args, and clear the deprecated flags that have a replacement (#38375)
This commit is contained in:
@@ -392,7 +392,7 @@ python3 -m sglang.launch_server \
|
||||
The default configuration is `--speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4`. Find the best values for your workload with [bench_speculative.py](https://github.com/sgl-project/sglang/blob/main/scripts/playground/bench_speculative.py). The minimum viable config is `--speculative-num-steps 1 --speculative-eagle-topk 1 --speculative-num-draft-tokens 2`.
|
||||
|
||||
<Note>
|
||||
For large batch sizes (>48), increase `--max-running-requests` beyond the default of 48 for MTP. Also set `--cuda-graph-bs` to include your target batch sizes (default captured sizes for speculative decoding: 48).
|
||||
For large batch sizes (>48), increase `--max-running-requests` beyond the default of 48 for MTP. Also set `--cuda-graph-bs-decode` to include your target batch sizes (default captured sizes for speculative decoding: 48).
|
||||
</Note>
|
||||
|
||||
<Tip>
|
||||
|
||||
@@ -102,7 +102,7 @@ import { Playground } from "/src/snippets/_playground.jsx";
|
||||
|
||||
- **Remote code.** Use `--trust-remote-code` for the Hugging Face checkpoint.
|
||||
- **Topology.** The 8x B300 recipe uses TP=8 and EP=8. H200, B200, and H20 use a 2-node 16 GPU layout with TP=16 and EP=16; the command panel injects the multi-node rank flags for you.
|
||||
- **LongCat sparse attention.** Keep `--nsa-prefill-backend fa3` with `--chunked-prefill-size 2048` for the model-card-aligned prefill path.
|
||||
- **LongCat sparse attention.** Keep `--dsa-prefill-backend fa3` with `--chunked-prefill-size 2048` for the model-card-aligned prefill path.
|
||||
- **Memory.** The recipe uses `--kv-cache-dtype bfloat16` and starts at `--mem-fraction-static 0.92`. Tune memory only after the generated command launches cleanly on your cluster.
|
||||
- **Weight loading.** `--model-loader-extra-config '{"enable_multithread_load":true,"num_threads":12}'` loads checkpoint shards in parallel and reduces startup time.
|
||||
- **FP8 backend selection.** Do not pass `--fp8-gemm-runner-backend` manually. SGLang selects the correct backend for the LongCat FP8 scale layout.
|
||||
|
||||
@@ -260,7 +260,7 @@ checkpoint's calibration scales automatically.
|
||||
is NVFP4-packed; the BF16 and FP8 checkpoints keep a dense head).
|
||||
The Ascend comparison in #35629 used a 910C with BF16 target weights,
|
||||
`--tp-size 2 --attention-backend ascend --mamba-ssm-dtype bfloat16
|
||||
--mamba-scheduler-strategy extra_buffer`, and disabled RadixCache for both
|
||||
--mamba-radix-cache-strategy extra_buffer`, and disabled RadixCache for both
|
||||
baseline and DFlash2 to exclude cache warm-up and prefix reuse. The DFlash2
|
||||
run added the three flags shown above.
|
||||
Accuracy used zero-shot GSM8K with greedy sampling, `max_new_tokens=2048`,
|
||||
|
||||
Reference in New Issue
Block a user