   
|
315080a0de
|
[sgl-router] Exclude portless prefills and retry bootstrap discovery (#41610)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-10-01 14:32:14 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
c78fa09290
|
[Router] Drop the snapshot client's redirect-following fallback (#42010)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-10-01 14:31:53 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
047be72a8c
|
[Router] Harden peer-bootstrap edges: refuse redirects and metric docs (#41999)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-10-01 14:15:42 +08:00 |
|
  
|
3b6d37a56e
|
[Router] Gate readiness on peer bootstrap, and make it observable (13/13) (#40699)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-10-01 11:55:46 +08:00 |
|
  
|
36330b8c08
|
[Router] k8s e2e for cache-aware peer bootstrap (12/13) (#40698)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-10-01 09:42:59 +08:00 |
|
 
|
4b3f63ec70
|
[Router] Prove a bootstrapped replica answers like the one it copied (11/13) (#40697)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-30 19:19:26 +08:00 |
|
 
|
4c930125c7
|
[Router] Ask the fleet when a graft's splice goes unwitnessed (10/13) (#40696)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-30 18:59:21 +08:00 |
|
 
|
f8500474c9
|
[Router] Share one fleet-wide fetch across a discovery burst (9/13) (#40695)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-30 18:07:51 +08:00 |
|
 
|
4d4d3f2e5c
|
[Router] Sweep the fleet for a peer snapshot and hand it to the pump (8/13) (#40694)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-30 16:34:12 +08:00 |
|
 
|
4a4e1ce014
|
[Router] Hold a booting rank's batches and graft a snapshot on the pump (7/13) (#40693)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-29 20:07:37 +08:00 |
|
 
|
8d1643b21e
|
[Router] Vet a peer's snapshot before it may touch the tree (6/13) (#40692)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-29 19:58:11 +08:00 |
|
 
|
8b3a4cad8b
|
[Router] Track a booting rank's bootstrap and fetch a peer's snapshot (5/13) (#40691)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-29 19:06:13 +08:00 |
|
 
|
7bcfcf5aa0
|
[Router] Watch EndpointSlices for sibling router replicas (4/13) (#40690)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-29 18:56:32 +08:00 |
|
 
|
c5a38d3998
|
[sgl-router] Add --tokenizer-backend fast and --tokenizer-l1-cache-mb (#41268)
Co-authored-by: Kan Wu <kan.wu@MacBook-Pro.local>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-29 18:55:58 +08:00 |
|
 
|
6aacca2d91
|
[Router] Name a replica's siblings with --kv-peer-selector (3/13) (#40689)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-28 18:49:09 +08:00 |
|
 
|
2f88c65289
|
[Router] Serve the cache-aware tree at /internal/kv_snapshot (2/13) (#40688)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-28 18:38:52 +08:00 |
|
  
|
47ad1928a2
|
[sgl-router] Scope input_ids forwarding by renderer: all text chats for DeepSeek-V4 (#41226)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-28 18:37:40 +08:00 |
|
 
|
df378bca42
|
[Router] Give the cache-aware tree a snapshot surface (1/13) (#40687)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-28 16:59:59 +08:00 |
|
 
|
b4584ab838
|
[sgl-router] Take SGLang's render defaults: --default-chat-template-kwargs and thinking/effort envs (#41221)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-27 11:01:17 +08:00 |
|
 Shangming CaiandClaude Fable 5
|
710a41ba57
|
[sgl-router] Book the input_ids forwarding outcome only for built bodies (#41342)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-26 19:39:01 +08:00 |
|
 
|
7152aabdcb
|
[sgl-router] Track input_ids forwarding outcomes per chat request (#41185)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
2026-09-26 13:09:46 +08:00 |
|
 Kan WuandClaude Fable 5.1
|
2e3568567b
|
[sgl-router] Launch reorg routing with existing policy options (#40766)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-23 10:18:11 +08:00 |
|
 
|
5c18780c31
|
[Router] Keep e2e workers inside the job's CUDA_VISIBLE_DEVICES allotment (#40686)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-22 11:32:12 -07:00 |
|
   
|
2032f3a071
|
[Router] Abort the engine when a client disconnects mid-request (#39461)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Kan Wu <wukanustc@gmail.com>
|
2026-09-22 12:35:56 +08:00 |
|
Kan Wu
|
59a723ef1e
|
[sgl-router] refactor - SLO ordering for bucket selection (#40292)
|
2026-09-22 11:16:10 +08:00 |
|
Kan Wu
|
27f796ca6c
|
[sgl-router] Fix readiness, IPv6 discovery, logging, and model validation (#40604)
|
2026-09-22 11:13:06 +08:00 |
|
Kan Wu
|
d47b8c454c
|
[sgl-router] Release cancelled circuit-breaker probes (#40603)
|
2026-09-21 15:39:50 -07:00 |
|
 Kan WuandShangming Cai
|
008470abd8
|
[sgl-router] Bound streaming lifetimes and release guards on idle disconnect (#40391)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-21 10:07:07 -07:00 |
|
Kan Wu
|
0f6761b54f
|
[sgl-router] Add SGLang-compatible DeepSeek V4.1 Flash rendering (#40532)
|
2026-09-21 18:35:09 +08:00 |
|
 
|
0abb251a20
|
[sgl-router] Match DeepSeek V4 rendering to SGLang (#40530)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-21 17:56:41 +08:00 |
|
 Kan WuandClaude Fable 5.1
|
2016f5e7a1
|
[sgl-router] Add Kimi-K3 rendering with SGLang parity (#40390)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-21 17:38:49 +08:00 |
|
 
|
630b1ef322
|
[sgl-router] Fix reorg admission proxy test build after BucketResolver::new (#40537)
Co-authored-by: Kan Wu <wukanustc@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-21 16:42:29 +08:00 |
|
 Kan WuandClaude Fable 5.1
|
a9871012ac
|
[sgl-router] refactor - generalized admission policy definitions (#40271)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-21 16:26:30 +08:00 |
|
 Kan WuandClaude Fable 5.1
|
fcb080bd40
|
[sgl-router] refactor - cache-aware policy (#40366)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-21 13:53:57 +08:00 |
|
 Kan WuandClaude Fable 5.1
|
4027740569
|
[sgl-router] refactor - session-aware policy (#40379)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-20 17:22:36 -07:00 |
|
 Kan WuandClaude Fable 5.1
|
aedda8377e
|
[sgl-router] refactor - layout BucketResolver, Bucket, EngineGroup and implement PowerOfTwo (#40241)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-20 16:57:07 -07:00 |
|
 Kan WuandClaude Fable 5.1
|
4a9dc5c4af
|
[sgl-router] refactor - move policy-required states under src/state (#40272)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-20 16:49:23 -07:00 |
|
Kan Wu
|
c610c40399
|
[sgl-router] refactor - config and organize CLI options (#39867)
|
2026-09-20 19:40:04 +08:00 |
|
 Kan WuandCursor
|
671630abf1
|
[sgl-router] refactor - main startup logic (#39861)
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-09-20 16:55:34 +08:00 |
|
Kan Wu
|
99d53fe0c2
|
[sgl-router] refactor - chat_completions() into modules (#39848)
|
2026-09-20 13:51:25 +08:00 |
|
  
|
aed3fb1cdd
|
[Router] Log every request at one site; derive its outcome from the final status (3/3) (#39465)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-19 03:11:00 +08:00 |
|
 
|
4e0b56c811
|
[Router] Treat an upstream 503/429 as backpressure, not a breaker fault (2/3) (#39464)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-19 00:46:29 +08:00 |
|
 
|
6de4666e43
|
[Router] Derive error status from a failure class; preserve the worker's status (1/3) (#39463)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-18 16:32:45 +08:00 |
|
  
|
3bf243d6a9
|
[Router] Shard the cache-aware KV tree by chain root (#39167)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-18 15:00:17 +08:00 |
|
  
|
72d9419bef
|
[Router] Sample k random candidates for the min-load fallback (--min-load-choices) (#39170)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-18 03:09:09 +08:00 |
|
 
|
e4cbb28ea1
|
[router] Improve SGLang chat render parity (#39133)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-17 23:53:40 +08:00 |
|
 Kan WuandClaude Fable 5.1
|
3ce7e2a29f
|
[sgl-router] Render chat prompts with dynamo-render (#38983)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-17 15:57:20 +08:00 |
|
 
|
b02e16a895
|
[Router] Fleet-wide sampling contract 3/3: splice injection without re-serializing (#39002)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 09:59:48 -07:00 |
|
 
|
3e03879f68
|
[Router] Drain readiness before SIGTERM shutdown so k8s deregisters the pod first (#39016)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 09:59:26 -07:00 |
|
Kan Wu
|
279339f113
|
[sgl-router] Share model-file discovery for chat formatters (#39485)
|
2026-09-16 09:36:11 -07:00 |
|