minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity

[ci][smg] fix docker release ci and add it to pr test (#14683)

Browse Source
This commit is contained in:
Simo Lin
2025-12-08 20:56:38 -08:00
committed by GitHub
parent ce4e836be5
commit af60cad05d
3 changed files with 26 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sgl-model-gateway/README.md
+1 -1
View File
@@ -27,7 +27,7 @@ High-performance model routing control and data plane for large-scale LLM deploy
- Advanced load balancing with cache-aware request reuse, load-aware (power-of-two) selection, and per-model policy overrides.
## Feature Highlights
- Multiple load balancing strategies (`random`, `round_robin`, `cache_aware`, `power_of_two`) with DP-aware scheduling.
- Multiple load balancing strategies (`random`, `round_robin`, `cache_aware`, `power_of_two`, `bucket`) with DP-aware scheduling.
- Multi-model HTTP serving and inference gateway routing with model-specific policies.
- Prefill/decode disaggregation, including bootstrap port handling and cache-aware merging.
- gRPC routing with fully Rust tokenizer loading, reasoning parser selection, and tool parser integration for OpenAI-compatible endpoints—supporting streaming and non-streaming modes across DeepSeek, Llama, Kimi K2, Qwen, GPT-OSS, Mistral, Step-3, GLM4, and other reasoning-capable models.