Commit Graph
21 Commits
Author SHA1 Message Date
AMD-yanfeiwang a3abea3560 [Fix] sgl-router: Resend on a closed pooled connection; add --disable-circuit-breaker (#44075) 2026-10-18 18:23:18 +08:00
AMD-yanfeiwang 08f15e76bb [Feat] sgl-router: Add --dp-rank-policy round_robin for --dp-aware (#43511) 2026-10-15 12:03:01 +08:00
2167cf90c3 [sgl-router] Bound streaming time-to-headers by --request-timeout-secs (retry 5/5) (#42632)
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 18:35:11 +08:00
08fa37e48d [sgl-router] Add reorg bucket config with per-group admission (#42429)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local>
2026-10-05 18:17:32 +08:00
Kan WuandKan Wu d7565215ec [sgl-router] Route to a specific DP rank inside multi-rank workers (--dp-aware) (#41974)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
2026-10-01 14:50:59 -07:00
41cbe65de0 [sgl-router] Enforce compatible PD version groups (#41613)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-01 20:24:20 +08:00
46637965ab [sgl-router] Allow load-only routing without a tokenizer (#41611)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-01 18:30:35 +08:00
c5a38d3998 [sgl-router] Add --tokenizer-backend fast and --tokenizer-l1-cache-mb (#41268)
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 18:55:58 +08:00
b4584ab838 [sgl-router] Take SGLang's render defaults: --default-chat-template-kwargs and thinking/effort envs (#41221)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 11:01:17 +08:00
Kan WuandClaude Fable 5.1 4a9dc5c4af [sgl-router] refactor - move policy-required states under src/state (#40272)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-20 16:49:23 -07:00
aed3fb1cdd [Router] Log every request at one site; derive its outcome from the final status (3/3) (#39465)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-09-19 03:11:00 +08:00
6de4666e43 [Router] Derive error status from a failure class; preserve the worker's status (1/3) (#39463)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 16:32:45 +08:00
Kan WuandClaude Fable 5.1 3ce7e2a29f [sgl-router] Render chat prompts with dynamo-render (#38983)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-17 15:57:20 +08:00
7835f1de9a [Router] Add the shutdown-drain configuration surface (#39015)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 00:26:50 -07:00
5f17e3a75f [Router] Fleet-wide sampling contract 1/3: the config surface (#39000)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 21:39:06 -07:00
5bebe7a033 [Router] Add bucket-aware policy domains and native cache indexing (#38108)
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
Co-authored-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: yangbodong22011 <13137470+yangbodong22011@users.noreply.github.com>
2026-09-06 19:47:51 +08:00
ecd97de1fc [Router] Add load-aware prefill admission and bounded policy proposals (#37843)
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-09-05 11:53:26 +08:00
Vincent Gaoandinkcherry 54cadad151 [Router] Add composable scoring and eligibility policies (#37731)
Co-authored-by: inkcherry <mingzhi.liu@amd.com>
2026-09-04 00:01:41 +08:00
Kangyan-ZhouandClaude Opus 4.8 6bc3953b48 feat(agentic router): add sticky-session routing policy (#27394)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 20:05:27 -07:00
Kangyan-ZhouandClaude Opus 4.8 bcf89928b4 [router] Configure experimental sgl-router via CLI flags instead of a config file (#27073)
Signed-off-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 10:02:10 +08:00
Kangyan-ZhouandClaude Opus 4.7 6e8fe176be sgl-router: experimental Rust HTTP router for SGLang worker pools (#25851)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 15:34:05 +08:00