[diffusion] feat: gate /health and /health_generate on warmup completion and add liveness endpoint (#33787)

Co-authored-by: Mick <mickjagger19@icloud.com>
2026-08-07 17:59:23 +08:00
parent a42683eb62
commit 7af3d000f2
4 changed files with 162 additions and 13 deletions
@@ -218,6 +218,24 @@ sglang serve \
  --port 30010
 ```

+### Health endpoints
+
+SGLang Diffusion separates process liveness from inference readiness:
+
+| Endpoint | Success condition | Recommended use |
+| --- | --- | --- |
+| `GET /liveness` | The HTTP server is accepting requests. It remains `200` during server warmup. | Kubernetes liveness probe |
+| `GET /health` | The server is ready for normal inference traffic. It returns `503` while server-based synthetic warmup is running and `200` after it completes. | Startup and readiness probes |
+| `GET /health_generate` | Compatibility alias for `/health`. It does not currently issue a generation request in SGLang Diffusion. | Existing integrations only |
+
+`/health` gates only server-based warmup. With `--warmup-mode off` or
+`--warmup-mode request`, it returns `200` once the HTTP server starts; those modes
+do not promise that compilation or other first-request work has completed. If
+server-based warmup fails, the server terminates instead of reporting ready.
+
+Do not use `/health` as a liveness probe: a long server warmup can legitimately
+keep it at `503` for several minutes.
+
 ### Cloud Storage

 SGLang Diffusion can upload generated images and videos to S3-compatible object storage after generation.
@@ -50,6 +50,33 @@ Base the decision on available memory on the selected GPU(s).
 - For multi-GPU deployment: the least-free selected GPU is the bottleneck. A busy 80GiB GPU can behave like a much smaller GPU.
 - For single-GPU deployment: FSDP shards DiT weights across multiple GPUs. It is not useful for keeping a single-GPU deployment on one GPU; for that case use CPU offload.

+## Health Probes
+
+Use `/liveness` to check that the HTTP process is alive and `/health` to check
+that the server is ready for inference. During server-based warmup, `/liveness`
+returns `200` while `/health` returns `503`. Configure the startup probe with a
+failure budget large enough for model loading and compilation:
+
+```yaml
+startupProbe:
+  httpGet:
+    path: /health
+    port: 30010
+  periodSeconds: 10
+  failureThreshold: 180
+readinessProbe:
+  httpGet:
+    path: /health
+    port: 30010
+livenessProbe:
+  httpGet:
+    path: /liveness
+    port: 30010
+```
+
+See [Health endpoints](/docs/sglang-diffusion/api/cli#health-endpoints) for the
+status-code contract and warmup-mode behavior.
+
 ## Performance Modes

 `--performance-mode` applies safe presets without overriding explicit offload, FSDP, or parallelism flags. `auto` is the default. Use `manual` when you need to keep performance-related server args under explicit user control. `--mode` is a short alias.