minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity

[router] Raise chat body cap to 32 MB for multimodal payloads (#38735)

Browse Source 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Kan Wu
2026-09-09 22:14:49 -07:00
committed by GitHub
co-authored by Claude Fable 5
parent 01ad27ecd9
commit 6f481ad0e3
1 changed files with 3 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
experimental/sgl-router/src/server/routes/chat.rs
+3 -8
View File
@@ -107,14 +107,9 @@ fn prefill_policy_reason(
} }
} }
/// Per-route body-size cap on `/v1/chat/completions`. 5 MiB accommodates a /// Maximum buffered chat-completions body (32MiB). Sized for base64 multimodal inputs;
/// long context — a ~1 M-token context tokenized as JSON fits under this — /// enforced by the `DefaultBodyLimit`, and returns 413 PAYLOAD_TOO_LARGE.
/// while preventing a hostile client from forcing the router to pub const MAX_CHAT_BODY_BYTES: usize = 32 << 20;
/// heap-allocate hundreds of MiB before forwarding. The cap is wired in
/// `crate::server::app::build_router` as a route-level `DefaultBodyLimit`
/// layer; axum's `Bytes` extractor enforces it and returns 413
/// PAYLOAD_TOO_LARGE before this handler runs.
pub const MAX_CHAT_BODY_BYTES: usize = 5 << 20;
/// Minimal probe over the request body — we only need the `stream` field /// Minimal probe over the request body — we only need the `stream` field
/// and the `model` field to decide between buffered vs SSE forwarding and /// and the `model` field to decide between buffered vs SSE forwarding and