tier realstore: 开 HC_TIMING 打点 + prefetch policy 改 wait_complete

best_effort 下低并发请求 compute 赢比赛、L3 命中不生效(mc1/2/4 实为重算);
wait_complete 后真实 L3 命中:mc1 TTFT 2989→661ms,mc8 聚合 203k→402k tok/s。
This commit is contained in:
2026-10-10 15:54:11 +08:00
parent 5f2ffa70b8
commit 8d9bce30a7
@@ -32,6 +32,7 @@ services:
SGLANG_ENABLE_DSV41_ENGRAM_HOST_TABLE: 1
SGLANG_RAGGED_VERIFY_MODE: static
SGLANG_HICACHE_MOONCAKE_CONFIG_PATH: /data/ymk/ds41/mooncake-store.json
SGLANG_HICACHE_TIMING: 1
CUDA_VISIBLE_DEVICES: 0,1
command: >
sglang serve
@@ -57,6 +58,6 @@ services:
--hicache-io-backend direct
--hicache-write-policy write_through
--hicache-storage-backend mooncake
--hicache-storage-prefetch-policy best_effort
--hicache-storage-prefetch-policy wait_complete
--hicache-size 0
--max-total-tokens 1600000