[diffusion] feat: support quant-videogen prq kv-cache quantization (memory-saving) for causal-dit (#32581)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Mick <mickjagger19@icloud.com>
2026-08-08 12:56:54 +08:00
parent 24c84dfa68
commit f64328c7f6
16 changed files with 1174 additions and 54 deletions
@@ -32,6 +32,16 @@ sglang serve \
  --enable-torch-compile false
 ```

+### Optional KV-Cache Compression
+
+Long-running sessions can enable lossy int4 PRQ compression for completed
+causal KV-cache chunks by installing `quant-videogen` as described in the
+quantization guide and adding `--kv-cache-quant int4` to the server command.
+The current and most recent completed chunks remain in BF16. See
+[Causal KV-Cache Quantization](/docs/sglang-diffusion/quantization#causal-kv-cache-quantization)
+for the algorithm, tuning options, measured memory-latency tradeoff, and
+support limits.
+
 ## 3. Realtime WebUI

 The lightweight local WebUI is useful for validating latency, frame transport, and camera control behavior.