minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

Reduce gemma4 moe deterministic test runtime (#24754)

Browse Source
This commit is contained in:
Khoa Pham
2026-05-08 20:46:56 -07:00
committed by GitHub
parent 8e534e8f15
commit 1610aa77ab
1 changed files with 2 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/registered/core/test_gemma4_moe_deterministic.py
+2 -6
View File
@@ -7,10 +7,6 @@ unified extend kernel read the new tokens at `out_cache_loc` (full-pool
index space) while `SWAKVPool.set_kv_buffer` had written them at the index space) while `SWAKVPool.set_kv_buffer` had written them at the
SWA-translated indices. With diverse prompts the OOB never materialises; SWA-translated indices. With diverse prompts the OOB never materialises;
the repro is same-prompt × high-concurrency, which is what this test fires. the repro is same-prompt × high-concurrency, which is what this test fires.
Adapted from the repro script in the bug report (200 identical completions
at concurrency 128, `--max-running-requests 16`). Pre-fix this loses
~40-50% of requests within ~30-40s; post-fix all 200 succeed.
""" """
import concurrent.futures import concurrent.futures
@@ -27,7 +23,7 @@ from sglang.test.test_utils import (
popen_launch_server, popen_launch_server,
) )
register_cuda_ci(est_time=420, suite="stage-b-test-2-gpu-large") register_cuda_ci(est_time=107, suite="stage-b-test-2-gpu-large")
PROMPT = ( PROMPT = (
@@ -37,7 +33,7 @@ PROMPT = (
"egg. How much in dollars does she make every day at the farmers' market?\n" "egg. How much in dollars does she make every day at the farmers' market?\n"
"Answer:" "Answer:"
) )
NUM_REQUESTS = 200 NUM_REQUESTS = 180
CONCURRENCY = 128 CONCURRENCY = 128
MAX_TOKENS = 256 MAX_TOKENS = 256