Files
sglang/deploy/b300-k8s/feishu-baseline-20261117.xml

63 lines
5.5 KiB
XML
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<h1>当前基线:ds41-gist-exp(4T L3 SSD,2026-11-17)</h1>
<callout emoji="📌" background-color="light-blue"><p>本节是 2026-11-17 定稿的 c580/c620 基线,后续 Dynamo 叠加以此为底座。旧的 ds41-gist-1p1d 章节已归档,不再作为当前部署。</p></callout>
<h2>1. 基线身份与镜像</h2>
<ul>
<li><b>工作负载</b>:<code>ds41-gist-exp</code> / <code>cw-dblft6sgrhagmpdt8ef0</code>,namespace <code>prj-d7te67bj0sq2fgdrmbig</code>,b300-01。</li>
<li><b>镜像</b>:<code>sealos.hub:5000/prj-d7te67bj0sq2fgdrmbig/sglang:recipe-ns-l3ssd-433257c7b-baked-20261115</code>(40.1GB,含 Mooncake 0.3.14 + L3 SSD 支持)。</li>
<li><b>manifest</b>:<code>deploy/b300-k8s/ds41-gist-exp-20261117-dtok16-fastokens.yaml</code>;基线副本 <code>deploy/b300-k8s/baselines/20261117-dtok16-fastokens/manifest.yaml</code>;b300 副本在 <code>/data/ymk/deploy/b300-k8s/</code>。</li>
<li><b>形态</b>:RBG 三角色 <code>p</code> / <code>d</code> / <code>gateway</code> + <code>mooncake-master</code>;全 hostNetwork / hostPID / privileged;P=GPU0-3,D=GPU4-7。</li>
</ul>
<h2>2. 部署方式与端口</h2>
<table>
<thead><tr><th><p>角色</p></th><th><p>端口</p></th><th><p>说明</p></th></tr></thead>
<tbody>
<tr><td><p>P</p></td><td><p>30000 / 8998 / 9400-9403</p></td><td><p>HTTP / bootstrap / Mooncake client metrics</p></td></tr>
<tr><td><p>D</p></td><td><p>30300</p></td><td><p>HTTP</p></td></tr>
<tr><td><p>gateway</p></td><td><p>30400 / 29000</p></td><td><p>业务 / metrics</p></td></tr>
<tr><td><p>mooncake-master</p></td><td><p>50051 / 18081 / 19003</p></td><td><p>store / metadata / metrics</p></td></tr>
</tbody>
</table>
<p>默认入口:<code>http://127.0.0.1:30400</code>;当前 new-api/AgentX 直接使用该入口。</p>
<h2>3. 关键参数</h2>
<p><b>P(prefill)</b>:tp4 / dp4 / ep4,megamoe + flashmla_mega,ep-redundant 32,DSpark block 3,decoder SWA bounded replay;chunked-prefill 131072 / max-prefill 131072,max-running 512;fastokens + tokenizer-worker 16;HiCache ratio 4、page_first_direct、direct、write_through、Mooncake L3;<b>cache 模式,无 buffer_only、无 optimistic</b>;mem-fraction 0.80。</p>
<p><b>D(decode)</b>:tp4 / dp4 / ep4,ep-redundant 64,DSpark block 3,stream-interval 32,swa-prefix-tails 2048;max-running 512,cuda-graph-max-bs-decode 512;chunked 32768 / max-prefill 8192;fastokens + tokenizer-worker 16;mem-fraction 0.80。</p>
<p><b>gateway</b>:镜像内 <code>sgl-model-gateway</code>;<b>无 P admission cap</b>(无 p-qps / p-queue-reqs-cap / p-queue-token-cap,td-admit 默认 off);保留 health check + circuit breaker;<code>--g9i-decode-max-running 512</code>。这是 c580 能跑通的关键。</p>
<p><b>L3</b>:Mooncake master 4TB SSD offload;<code>offload_on_evict=false</code>;L3 配置见 <code>/data/ymk/ds41/mooncake-store-l3ssd.json</code>。</p>
<h2>4. 启动 / 重启流程</h2>
<pre lang="bash"><code># 1) 停完整工作负载并确认 GPU 0
MSYS_NO_PATHCONV=1 wsl -e /home/ubuntu/.local/bin/kmctl stop cw ds41-gist-exp --wait --timeout 10m --request-timeout 180s
ssh 24.125.93.200.207 "nvidia-smi --query-gpu=index,memory.used --format=csv,noheader"
# 2) 清 L3 SSD(实验/基线前必须)
ssh 24.125.93.200.207 'bash /data/ymk/deploy/b300-k8s/clean_l3ssd_cache.sh'
# 3) 用完整平台 manifest 启动(不要只 kubectl patch gateway)
MSYS_NO_PATHCONV=1 wsl -e /home/ubuntu/.local/bin/kmctl update cw ds41-gist-exp \
-f /mnt/d/B300/deploy/b300-k8s/ds41-gist-exp-20261117-dtok16-fastokens.yaml
# 4) 等 4 Pod 1/1,检查 P/D/gateway health 200,再 smoke
curl -sS http://127.0.0.1:30400/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"model":"/data/models/DeepSeek-V4.1-Flash","messages":[{"role":"user","content":"1+1=?"}],"max_tokens":8}'
</code></pre>
<p>启动基线:P <code>max_total_num_tokens≈16,158,208</code> / 128 per-rank / avail ≈23GB;D <code>58,859,520</code> / 128 per-rank / avail ≈79GB。正常 warmup 6,385 条约 6.5 分钟。</p>
<h2>5. 基线结果</h2>
<table>
<thead><tr><th><p>场景</p></th><th><p>RPS</p></th><th><p>TTFT p50/p95/p99</p></th><th><p>ITL p95</p></th><th><p>结论</p></th></tr></thead>
<tbody>
<tr><td><p>c580 基线</p></td><td><p>24.045</p></td><td><p>1.743 / 3.772 / 6.580 s</p></td><td><p>10.74 ms</p></td><td><p>推荐工作点</p></td></tr>
<tr><td><p>c620 首轮(清 L3)</p></td><td><p>25.998</p></td><td><p>1.814 / 4.041 / 8.257 s</p></td><td><p>11.69 ms</p></td><td><p>吞吐 +8%,尾部变差</p></td></tr>
<tr><td><p>c620 不重启 r2/r3</p></td><td><p>25.85 / 25.61</p></td><td><p>p95 5.39 / 6.44 s,p99 9.56 / 10.79 s</p></td><td><p>10.96 / 10.68 ms</p></td><td><p>D KV 接近/打满,TTFT 尾部持续劣化,不建议连测</p></td></tr>
<tr><td><p>buffer_only</p></td><td><p>23.749</p></td><td><p>2.153 / 4.904 / 10.483 s</p></td><td><p>10.32 ms</p></td><td><p>吞吐 -1.2%、TTFT 更差,不采用</p></td></tr>
</tbody>
</table>
<p>结论:<b>c580 是稳定基线</b>;c620 只适合短时冲吞吐,且每轮之间必须重启/清池。buffer_only 不作为基线。</p>
<h2>6. 作为 Dynamo 底座</h2>
<p>当前镜像是 <b>SGLang + Mooncake 0.3.14 + L3 SSD</b> 基线,<b>不包含 Dynamo/nixl</b>。后续套 Dynamo 需要以本镜像为 base 另装 Dynamo 1.6.x / nixl,旧 <code>gist-nvlink-7f6fe2244-dynamo-local-20261026</code> 是更早的 10-23 代码,不是本基线。</p>
<p>Dynamo 叠加时保留本页的 P/D 参数、L3 SSD 和 gateway 入口;先做单机 4P4D 冒烟,再评估 Dynamo 的 P/D 角色、discovery 和 KV 传输配置。</p>