|
2032f3a071
|
[Router] Abort the engine when a client disconnects mid-request (#39461)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Kan Wu <wukanustc@gmail.com>
|
2026-09-22 12:35:56 +08:00 |
|
|
Kan Wu
|
59a723ef1e
|
[sgl-router] refactor - SLO ordering for bucket selection (#40292)
|
2026-09-22 11:16:10 +08:00 |
|
|
Kan Wu
|
27f796ca6c
|
[sgl-router] Fix readiness, IPv6 discovery, logging, and model validation (#40604)
|
2026-09-22 11:13:06 +08:00 |
|
|
Kan Wu
|
d47b8c454c
|
[sgl-router] Release cancelled circuit-breaker probes (#40603)
|
2026-09-21 15:39:50 -07:00 |
|
|
Kan WuandShangming Cai
|
008470abd8
|
[sgl-router] Bound streaming lifetimes and release guards on idle disconnect (#40391)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-21 10:07:07 -07:00 |
|
|
Kan Wu
|
0f6761b54f
|
[sgl-router] Add SGLang-compatible DeepSeek V4.1 Flash rendering (#40532)
|
2026-09-21 18:35:09 +08:00 |
|
|
|
0abb251a20
|
[sgl-router] Match DeepSeek V4 rendering to SGLang (#40530)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-21 17:56:41 +08:00 |
|
|
Kan WuandClaude Fable 5.1
|
2016f5e7a1
|
[sgl-router] Add Kimi-K3 rendering with SGLang parity (#40390)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-21 17:38:49 +08:00 |
|
|
|
630b1ef322
|
[sgl-router] Fix reorg admission proxy test build after BucketResolver::new (#40537)
Co-authored-by: Kan Wu <wukanustc@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-21 16:42:29 +08:00 |
|
|
Kan WuandClaude Fable 5.1
|
a9871012ac
|
[sgl-router] refactor - generalized admission policy definitions (#40271)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-21 16:26:30 +08:00 |
|
|
Kan WuandClaude Fable 5.1
|
fcb080bd40
|
[sgl-router] refactor - cache-aware policy (#40366)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-21 13:53:57 +08:00 |
|
|
Kan WuandClaude Fable 5.1
|
4027740569
|
[sgl-router] refactor - session-aware policy (#40379)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-20 17:22:36 -07:00 |
|
|
Kan WuandClaude Fable 5.1
|
aedda8377e
|
[sgl-router] refactor - layout BucketResolver, Bucket, EngineGroup and implement PowerOfTwo (#40241)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-20 16:57:07 -07:00 |
|
|
Kan WuandClaude Fable 5.1
|
4a9dc5c4af
|
[sgl-router] refactor - move policy-required states under src/state (#40272)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-20 16:49:23 -07:00 |
|
|
Kan Wu
|
c610c40399
|
[sgl-router] refactor - config and organize CLI options (#39867)
|
2026-09-20 19:40:04 +08:00 |
|
|
Kan WuandCursor
|
671630abf1
|
[sgl-router] refactor - main startup logic (#39861)
Co-authored-by: Cursor <cursoragent@cursor.com>
|
2026-09-20 16:55:34 +08:00 |
|
|
Kan Wu
|
99d53fe0c2
|
[sgl-router] refactor - chat_completions() into modules (#39848)
|
2026-09-20 13:51:25 +08:00 |
|
|
|
aed3fb1cdd
|
[Router] Log every request at one site; derive its outcome from the final status (3/3) (#39465)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-19 03:11:00 +08:00 |
|
|
|
4e0b56c811
|
[Router] Treat an upstream 503/429 as backpressure, not a breaker fault (2/3) (#39464)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-19 00:46:29 +08:00 |
|
|
|
6de4666e43
|
[Router] Derive error status from a failure class; preserve the worker's status (1/3) (#39463)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-18 16:32:45 +08:00 |
|
|
|
3bf243d6a9
|
[Router] Shard the cache-aware KV tree by chain root (#39167)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-18 15:00:17 +08:00 |
|
|
|
72d9419bef
|
[Router] Sample k random candidates for the min-load fallback (--min-load-choices) (#39170)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-18 03:09:09 +08:00 |
|
|
|
e4cbb28ea1
|
[router] Improve SGLang chat render parity (#39133)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-17 23:53:40 +08:00 |
|
|
Kan WuandClaude Fable 5.1
|
3ce7e2a29f
|
[sgl-router] Render chat prompts with dynamo-render (#38983)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-17 15:57:20 +08:00 |
|
|
|
b02e16a895
|
[Router] Fleet-wide sampling contract 3/3: splice injection without re-serializing (#39002)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 09:59:48 -07:00 |
|
|
|
3e03879f68
|
[Router] Drain readiness before SIGTERM shutdown so k8s deregisters the pod first (#39016)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 09:59:26 -07:00 |
|
|
Kan Wu
|
279339f113
|
[sgl-router] Share model-file discovery for chat formatters (#39485)
|
2026-09-16 09:36:11 -07:00 |
|
|
Kan WuandClaude Fable 5.1
|
ad94978adf
|
[sgl-router] Rename chat encoder to chat formatter (#39459)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-16 08:46:11 -07:00 |
|
|
Kan WuandClaude Fable 5.1
|
e41026f434
|
[sgl-router] Forward input_ids only for string content; count tokenize errors only when forwardable (#39458)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-16 22:49:41 +08:00 |
|
|
|
8baeded6f3
|
[Router] Pin to the prefix owner when the whole fleet is queueing (--saturation-queue-floor) (#39169)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 16:16:24 +08:00 |
|
|
|
54f0d72adf
|
[Router] Fleet-wide sampling contract 2/3: enforce and inject per request (#39001)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 00:29:33 -07:00 |
|
|
|
7835f1de9a
|
[Router] Add the shutdown-drain configuration surface (#39015)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 00:26:50 -07:00 |
|
|
|
e2d56bbbfc
|
[Router] Bound the e2e worker memory budget so prefill graph capture stops OOMing (#39713)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 00:13:09 -07:00 |
|
|
|
d634320e48
|
[Router] Count open HTTP exchanges until their response body finishes (#39014)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 00:04:38 -07:00 |
|
|
|
afde31a2f5
|
[router] Speak cleartext h2c on both edges: serve it inbound, forward it outbound (#39006)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 22:30:31 -07:00 |
|
|
|
c1f5b4736a
|
[Router] Add --worker-queue-limit: stop sending cache-affinity traffic to a queueing worker (#39168)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 13:24:48 +08:00 |
|
|
|
5f17e3a75f
|
[Router] Fleet-wide sampling contract 1/3: the config surface (#39000)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 21:39:06 -07:00 |
|
|
|
bbef93e22a
|
[Router] Add a KV storage-tier coverage row to the Grafana dashboard (4/4) (#39111)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 17:09:11 -07:00 |
|
|
|
63845a1bb2
|
[router] Resolve a wire protocol per worker at registration (#39004)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 16:54:37 -07:00 |
|
|
|
fb91baedab
|
[Router] Real-GPU e2e coverage for storage-tier-aware cache routing (3/4) (#39110)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 15:48:51 -07:00 |
|
|
|
3c48c1e967
|
[Router] Expose the KV storage-tier stream and tree occupancy on /metrics (2/4) (#39109)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 13:30:05 -07:00 |
|
|
|
0e9b6bf8d8
|
[Router] Extract worker selection into policies::selection (no behavior change) (#39322)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 15:49:31 +08:00 |
|
|
ishandhanani
|
3f871a246c
|
feat(agent sessions): attribute stored KV cache blocks to sessions (#37482)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-09-14 21:34:49 -07:00 |
|
|
|
45b511b2ef
|
[Router] Honor KV-event storage tiers in the cache-aware tree (1/4) (#39108)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-14 21:18:15 -07:00 |
|
|
Kan WuandClaude Fable 5.1
|
e89d8facab
|
[sgl-router] Prepare dynamo-render dependencies (#39457)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-14 19:08:16 -07:00 |
|
|
Kan Wu
|
f539c1fc65
|
[sgl-router] Stream outcome observability for 2xx SSE streams (#38737)
|
2026-09-13 19:46:44 -07:00 |
|
|
ishandhanani
|
ddc1df1203
|
fix(router): simplify bucket context limit check (#39259)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-09-13 12:27:59 -07:00 |
|
|
Vincent Gao
|
8e7deb329e
|
[Router] Preserve global cache affinity with bucket routing (#38814)
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
|
2026-09-11 17:41:27 +08:00 |
|
|
Kan WuandClaude Fable 5
|
6f481ad0e3
|
[router] Raise chat body cap to 32 MB for multimodal payloads (#38735)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-09 22:14:49 -07:00 |
|
|
|
5bebe7a033
|
[Router] Add bucket-aware policy domains and native cache indexing (#38108)
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
Co-authored-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: yangbodong22011 <13137470+yangbodong22011@users.noreply.github.com>
|
2026-09-06 19:47:51 +08:00 |
|