minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[MLX] bench_one_batch: thread --quantization through to MlxModelRunner (#25221)

Browse Source 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Le Zhang
2026-05-14 09:56:42 -07:00
committed by GitHub
co-authored by Claude Opus 4.7
parent 90afd680f3
commit 3fc60e5748
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/bench_one_batch.py
+1
View File
@@ -531,6 +531,7 @@ class _MlxBenchRunner:
trust_remote_code=server_args.trust_remote_code, trust_remote_code=server_args.trust_remote_code,
disable_radix_cache=True, disable_radix_cache=True,
mem_fraction_static=server_args.mem_fraction_static, mem_fraction_static=server_args.mem_fraction_static,
quantization=server_args.quantization,
) )
if server_args.max_total_tokens is not None: if server_args.max_total_tokens is not None:
init_kwargs["pool_size"] = server_args.max_total_tokens init_kwargs["pool_size"] = server_args.max_total_tokens