hicache: STORAGE_BATCH_SIZE 可env化(SGLANG_HICACHE_STORAGE_BATCH_SIZE,默认128不变)
build-sglang-image / build (push) Failing after 18m40s

打点显示每批存储 IO 有 ~30-70ms 固定开销(与传输介质无关),
调大批次可摊薄。env 化便于不动代码做 128/512/1024 对照。
This commit is contained in:
2026-10-10 17:55:14 +08:00
parent ac11e9162d
commit 7d5759b2d9
2 changed files with 3 additions and 1 deletions
+2
View File
@@ -761,6 +761,8 @@ class Envs:
# Per-stage timing logs for the L3 storage prefetch path (query / transfer /
# ack reduce / drain / graft / load-back). Emits "HC_TIMING ..." lines.
SGLANG_HICACHE_TIMING = EnvBool(False)
# Pages per batched storage IO call (query / get / put chunking).
SGLANG_HICACHE_STORAGE_BATCH_SIZE = EnvInt(128)
SGLANG_HICACHE_FILE_BACKEND_STORAGE_DIR = EnvStr(None)
# File-backend LRU eviction (opt-in; sizes accept SI/IEC suffixes, "0" disables).
SGLANG_HICACHE_FILE_BACKEND_MAX_SIZE = EnvStr(None)
@@ -20,7 +20,7 @@ if TYPE_CHECKING:
logger = logging.getLogger(__name__)
# Max pages per batched storage IO call.
STORAGE_BATCH_SIZE = 128
STORAGE_BATCH_SIZE = envs.SGLANG_HICACHE_STORAGE_BATCH_SIZE.get()
@dataclass