sgl-router: experimental Rust HTTP router for SGLang worker pools (#25851)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 15:34:05 +08:00
parent aae04b1241
commit 6e8fe176be
131 changed files with 28623 additions and 55 deletions
@@ -0,0 +1,20 @@
+# sgl-router
+
+Slim, KV-aware, OpenAI-compatible router for SGLang workers.
+
+**Status:** functional single-worker HTTP proxy. Exposes `/v1/tokenize`,
+`/v1/detokenize`, `/v1/models`, `/v1/chat/completions` (buffered and SSE),
+plus `/healthz` / `/readyz`. Forwards to one configured worker via reqwest;
+parity-tested against `transformers.AutoTokenizer`. Multi-worker routing,
+service discovery, and observability still pending.
+
+## Building
+
+```bash
+cd experimental/sgl-router
+cargo build --release
+```
+
+## License
+
+Apache-2.0.