Files
sglang/deploy/b300-k8s/snapshots/20261117-cache-nolimit/NOTES.md
T

931 B
Raw Blame History

2026-11-17 cache + gateway no-limit 实验参数(失败记录)

  • P: hicache_host_memory_mode=cache, optimistic_prefill_attempts=0, tokenizer-worker-num=16, hicache ratio 4 / write_through / L3 mooncake, L3 SSD 4T.
  • D: tokenizer-worker-num=16.
  • Gateway: g9i-td-admit off, p-queue-reqs-cap 0, p-queue-token-cap 0, p-qps 0, decode-max-running 1000000, disable-circuit-breaker, disable-health-check.
  • Image: sealos.hub:5000/prj-d7te67bj0sq2fgdrmbig/sglang:recipe-ns-l3ssd-433257c7b-baked-20261115
  • c580 run c580-cache-nolimit-20261117: warmup 4,720/6,385 (74%), 02:33:32 失败 (ServerDisconnected/warmup_failure;P/D/gateway 0 restart,无 503)。
  • 观察:P 单批 32k/23k chunk 正常,但每 rank 同时仅 1 个请求;gateway smg_worker_requests_active 峰值 1,840,P queue 峰值 503,gateway 约 1 核。
  • 结论:该 no-limit/no-cb 网关配置不适合作为 c580 基线;仅作异常记录。