tier realstore: 开 HC_TIMING 打点 + prefetch policy 改 wait_complete
best_effort 下低并发请求 compute 赢比赛、L3 命中不生效(mc1/2/4 实为重算); wait_complete 后真实 L3 命中:mc1 TTFT 2989→661ms,mc8 聚合 203k→402k tok/s。
This commit is contained in:
@@ -32,6 +32,7 @@ services:
|
||||
SGLANG_ENABLE_DSV41_ENGRAM_HOST_TABLE: 1
|
||||
SGLANG_RAGGED_VERIFY_MODE: static
|
||||
SGLANG_HICACHE_MOONCAKE_CONFIG_PATH: /data/ymk/ds41/mooncake-store.json
|
||||
SGLANG_HICACHE_TIMING: 1
|
||||
CUDA_VISIBLE_DEVICES: 0,1
|
||||
command: >
|
||||
sglang serve
|
||||
@@ -57,6 +58,6 @@ services:
|
||||
--hicache-io-backend direct
|
||||
--hicache-write-policy write_through
|
||||
--hicache-storage-backend mooncake
|
||||
--hicache-storage-prefetch-policy best_effort
|
||||
--hicache-storage-prefetch-policy wait_complete
|
||||
--hicache-size 0
|
||||
--max-total-tokens 1600000
|
||||
|
||||
Reference in New Issue
Block a user