|
afde31a2f5
|
[router] Speak cleartext h2c on both edges: serve it inbound, forward it outbound (#39006)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 22:30:31 -07:00 |
|
|
|
c1f5b4736a
|
[Router] Add --worker-queue-limit: stop sending cache-affinity traffic to a queueing worker (#39168)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-16 13:24:48 +08:00 |
|
|
|
5f17e3a75f
|
[Router] Fleet-wide sampling contract 1/3: the config surface (#39000)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 21:39:06 -07:00 |
|
|
|
bbef93e22a
|
[Router] Add a KV storage-tier coverage row to the Grafana dashboard (4/4) (#39111)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 17:09:11 -07:00 |
|
|
|
63845a1bb2
|
[router] Resolve a wire protocol per worker at registration (#39004)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 16:54:37 -07:00 |
|
|
|
fb91baedab
|
[Router] Real-GPU e2e coverage for storage-tier-aware cache routing (3/4) (#39110)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 15:48:51 -07:00 |
|
|
|
3c48c1e967
|
[Router] Expose the KV storage-tier stream and tree occupancy on /metrics (2/4) (#39109)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 13:30:05 -07:00 |
|
|
|
0e9b6bf8d8
|
[Router] Extract worker selection into policies::selection (no behavior change) (#39322)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-15 15:49:31 +08:00 |
|
|
ishandhanani
|
3f871a246c
|
feat(agent sessions): attribute stored KV cache blocks to sessions (#37482)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-09-14 21:34:49 -07:00 |
|
|
|
45b511b2ef
|
[Router] Honor KV-event storage tiers in the cache-aware tree (1/4) (#39108)
Co-authored-by: Kangyan Zhou <kangyan.zhou@radixark.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
2026-09-14 21:18:15 -07:00 |
|
|
Kan WuandClaude Fable 5.1
|
e89d8facab
|
[sgl-router] Prepare dynamo-render dependencies (#39457)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-14 19:08:16 -07:00 |
|
|
Kan Wu
|
f539c1fc65
|
[sgl-router] Stream outcome observability for 2xx SSE streams (#38737)
|
2026-09-13 19:46:44 -07:00 |
|
|
ishandhanani
|
ddc1df1203
|
fix(router): simplify bucket context limit check (#39259)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-09-13 12:27:59 -07:00 |
|
|
Vincent Gao
|
8e7deb329e
|
[Router] Preserve global cache affinity with bucket routing (#38814)
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
|
2026-09-11 17:41:27 +08:00 |
|
|
Kan WuandClaude Fable 5
|
6f481ad0e3
|
[router] Raise chat body cap to 32 MB for multimodal payloads (#38735)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-09-09 22:14:49 -07:00 |
|
|
|
5bebe7a033
|
[Router] Add bucket-aware policy domains and native cache indexing (#38108)
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
Co-authored-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: yangbodong22011 <13137470+yangbodong22011@users.noreply.github.com>
|
2026-09-06 19:47:51 +08:00 |
|
|
|
ecd97de1fc
|
[Router] Add load-aware prefill admission and bounded policy proposals (#37843)
Signed-off-by: Vincent Gao <vincentbo@linux.alibaba.com>
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
|
2026-09-05 11:53:26 +08:00 |
|
|
Vincent Gaoandinkcherry
|
54cadad151
|
[Router] Add composable scoring and eligibility policies (#37731)
Co-authored-by: inkcherry <mingzhi.liu@amd.com>
|
2026-09-04 00:01:41 +08:00 |
|
|
Alex NailsandAlison Shao
|
28262c20df
|
[CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
|
2026-09-02 19:46:08 -07:00 |
|
|
|
360d10d6bc
|
[Feature] Add process-local in-memory KV indexer and Router integration (#33370)
Co-authored-by: Wu, Yutong <yutong.wu@amd.com>
Co-authored-by: TianDi101 <ditian12@amd.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
|
2026-08-20 10:45:35 +08:00 |
|
|
Cheng Wan
|
c70c7d72a8
|
config: the readback and the resolving view say what they are (#35027)
|
2026-08-17 16:18:19 -07:00 |
|
|
Lianmin Zheng
|
9ca4023b13
|
[Core] Clean up array-like msgspec structs (#32688)
|
2026-07-28 16:25:22 -07:00 |
|
|
Xiaoyu Zhang
|
b276a9acee
|
chore: cleanup garbage code (#29770)
|
2026-07-02 16:14:01 +08:00 |
|
|
Liangsheng Yin
|
cd6dedf972
|
[router] Count requests/responses at the HTTP edge for true intake (#29467)
|
2026-06-26 21:21:41 -07:00 |
|
|
Kangyan-ZhouandClaude Opus 4.8
|
364bf976be
|
[router] Tokenize prompt once at ingress; forward input_ids to the engine (all policies) (#28744)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-19 16:07:17 -07:00 |
|
|
Kangyan-ZhouandClaude Opus 4.8
|
abf7011cdd
|
[router] Raise chat body cap to 5 MiB for long contexts (#28742)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-19 15:50:34 -07:00 |
|
|
Kangyan-ZhouandClaude Opus 4.8
|
cab62855f5
|
[router] Align sgl_router_ttft_seconds buckets with engine TTFT grid (#28717)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-19 08:53:46 -07:00 |
|
|
Jia-Wei Jiang
|
ca84d52b78
|
[Router] [Docs] Refresh policy-selection notes (#28321)
Signed-off-by: JiangJiaWei1103 <waynechuang97@gmail.com>
|
2026-06-16 17:07:19 -07:00 |
|
|
Jia-Wei Jiang
|
a88ba6cc0b
|
[Fix] Reduce power of two to constant time (#28228)
Signed-off-by: JiangJiaWei1103 <waynechuang97@gmail.com>
|
2026-06-14 23:25:22 -07:00 |
|
|
Kangyan-ZhouandClaude Opus 4.8
|
c0dfe4c8ec
|
[router] Reconcile workers that registered without resolving model_ids (#27980)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-14 18:47:50 -07:00 |
|
|
Kangyan-ZhouandClaude Opus 4.8
|
21647f1f5d
|
[router] Apply chat template before cache-aware hashing (fix overlap=0 on chat traffic) (#27386)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-06-10 10:27:54 -07:00 |
|
|
Kangyan-ZhouandClaude Opus 4.8
|
0c7faf01fb
|
test(sgl-router): cover sticky scale-up no-redistribution e2e (#27742)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
2026-06-10 08:54:31 -07:00 |
|
|
Kangyan-ZhouandClaude Opus 4.8
|
badab6b136
|
[router] Add request/TTFT/worker metrics + Grafana dashboard to experimental sgl-router (#27591)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-06-09 07:25:56 -07:00 |
|
|
Kangyan-ZhouandClaude Opus 4.8
|
95090b837e
|
[router] Add /flush_cache endpoint to experimental sgl-router (#27612)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-06-08 18:53:29 -07:00 |
|
|
Matt Van HornandMatt Van Horn
|
317fc6a9dd
|
refactor: replace oversized 1.3MB tiny_tokenizer.json fixture with a genuinely tiny byte-level BPE fixture (#27235)
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
|
2026-06-08 17:40:32 -07:00 |
|
|
Kangyan-ZhouandClaude Opus 4.8
|
6bc3953b48
|
feat(agentic router): add sticky-session routing policy (#27394)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-06-07 20:05:27 -07:00 |
|
|
Zhangheng
|
e4a7388187
|
feat(agentic router, 1/N): Add LoadBasedPolicy (#26480)
|
2026-06-06 00:13:27 +08:00 |
|
|
Kangyan-ZhouandClaude Opus 4.8
|
bcf89928b4
|
[router] Configure experimental sgl-router via CLI flags instead of a config file (#27073)
Signed-off-by: Kangyan Zhou <zky314343421@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
2026-06-05 10:02:10 +08:00 |
|
|
Kangyan-ZhouandClaude Opus 4.7
|
6e8fe176be
|
sgl-router: experimental Rust HTTP router for SGLang worker pools (#25851)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-05-25 15:34:05 +08:00 |
|