77a15cf4e2
[MemCache] Add shared component size and path APIs to the tree core interface ( #43722 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: Jialin <Jialin@users.noreply.github.com >
2026-10-11 20:14:07 -07:00
95ce9caaf2
[Rust] Lock hyper 1.12.0 and want 0.3.2 in the rust workspace ( #43724 )
...
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com >
Co-authored-by: shodoco <shodoco@users.noreply.github.com >
2026-10-11 20:12:55 -07:00
Banruo Liu and Shangming Cai
d62b9c2480
[KV Events] Publish LoRA adapter names and namespaced block hashes ( #42455 )
...
Signed-off-by: Banruo Liu <lichangxinlivingshade@gmail.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-11 23:23:30 +08:00
Shuwen Wang
99d6f4e158
[MemCache] refactor: require an anchored lock receipt for non-root releases ( #41452 )
2026-10-10 22:43:49 +08:00
49e629d239
[Rust] Upgrade sglang-radix-tree to PyO3 0.29 ( #42807 )
...
Co-authored-by: colleen-tml <296620403+colleen-tml@users.noreply.github.com >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 20:56:50 +08:00
Sage
6fc8d9da32
[rust-server] preserve PD routing in chat and completions ( #42989 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-10-10 17:29:46 +08:00
f2e9a2bed1
[unified memory pool] Allocate without transfer waits, pin floating SWA reservations, drop the HiCache startup asserts ( #43206 )
...
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com >
Co-authored-by: ZYHowell <ZYHowell@users.noreply.github.com >
2026-10-09 20:19:30 -07:00
16423cb6d2
[sgl-router] Serve DeepSeek-V4.1 Flash through sglang-processor ( #43000 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-10 09:23:09 +08:00
d511e6bd1d
[sgl-router] Parse tool calls when serving chat through /generate ( #42999 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:07:30 +08:00
3805d78796
[sgl-router] Serve /v1/chat/completions through /generate, with reasoning parsing ( #42998 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-09 19:17:52 +08:00
b130f8af02
[sgl-router] Serve /v1/completions through /generate, with the shared OpenAI layer ( #42997 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 16:04:04 +08:00
f620d733d2
[rust-processor] Share the reasoning splitter and tool constraint across hosts ( #42996 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-09 15:19:41 +08:00
3ba90f1b52
[rust] Update Dynamo frontend crates to latest compatible releases ( #42666 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:46:01 +08:00
shodoco
86624f14eb
[Rust] Pass the media item fps hint to multimodal processors ( #43143 )
2026-10-08 17:51:06 -07:00
Sage
5a5ea81036
[rust-server] warm up each rust listener in-process ( #43004 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-10-08 14:09:46 -07:00
943621c336
[sgl-router] Render DeepSeek-V4 through sglang-processor ( #42665 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-08 20:16:20 +08:00
45abea0269
[rust-processor] DeepSeek-V4 parity with SGLang, plus a reusable parity harness and skill ( #42664 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 18:35:30 +08:00
Liangsheng Yin
5a4d01a3ae
[MemCache] Return only the matched length from match_prefix; read KV indices off the node path ( #42923 )
2026-10-08 01:10:14 -07:00
Rain Jiang
c52dcc5a01
The sgalng api specs ( #41585 )
2026-10-07 01:55:01 -07:00
jain-ria and ishandhanani
fd62ee84f7
feat(grpc): expose follower metadata and node-local KV sources ( #39659 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-10-06 16:39:01 -07:00
0f57a39152
[rust-processor] Gate render, tokenizer and parser behind cargo features ( #42663 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 10:58:38 -07:00
Shangming Cai
1291f3175d
[rust-processor] Fix chat-template precedence wording in the processor README ( #42756 )
2026-10-06 19:10:26 +08:00
cf19ae6d04
[rust-processor] Split sglang-processor into tokenizer, render and parser ( #42662 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 18:59:02 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
0bdd0cc8dc
[Deps] Upgrade the CUDA PyTorch stack to 2.14 ( #38641 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-04 00:48:02 -07:00
937c0a6cfe
Fix unified HiCache physical transfers ( #39479 )
...
Co-authored-by: yhzhuang <yhzhuang@fb.com >
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com >
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
2026-10-04 00:38:56 -07:00
4611ea365e
[HiCache]: Fix host lock ownership across radix splits ( #38480 )
...
Signed-off-by: Lukas Humbel <lhumbel@nvidia.com >
Co-authored-by: Jialin Ouyang <jialino@meta.com >
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com >
2026-10-04 11:16:04 +08:00
2bae12b9b3
[HiCache] Fix write-back SWA insert backups tripping the write-through pending-ack assert ( #42264 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: 842974287 <842974287@users.noreply.github.com >
2026-10-03 09:45:56 -07:00
Liangsheng Yin
50be533d09
[mem_cache] Rename cache_unfinished_req to checkpoint_req and count cache hits only when a request finishes ( #41520 )
2026-10-01 21:41:45 -07:00
jain-ria
fb92ba1127
[Rust] Wire the gRPC server into the frontend lifecycle ( #41964 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-10-01 10:44:56 -07:00
jain-ria
ebccddce19
[Rust] Implement the Rust gRPC adapter for native generation ( #41766 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-09-30 18:17:42 -07:00
Sage
8e4bcc5ca9
[rust-renderer] sglang-processor lib ( #41896 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-30 15:09:35 -07:00
jain-ria
21ed9fbf44
[Rust] Extract shared runtime.v1 protobuf bindings ( #41726 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-09-30 10:13:08 -07:00
jain-ria
46beacf13d
[grpc rust] Define semantic frontend contract ( #41709 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-09-29 13:40:04 -07:00
jain-ria and ishandhanani
069523f999
[Rust] Extract a transport-neutral frontend core ( #39385 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-09-28 21:18:50 -07:00
77091cea68
[Radix Cache] Sync Rust TreeCore and make it the default ( #39627 )
...
Co-authored-by: Ke Bao <ispobaoke@gmail.com >
Co-authored-by: Zhangheng <hzh0425@apache.org >
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com >
2026-09-29 10:18:57 +08:00
Rain Jiang
6c4f36e47a
Rust server unify datapath for mm and generate requests ( #39679 )
2026-09-27 23:36:07 -07:00
Lianmin Zheng and Lu Fang
3ecb6cd729
[MemCache] Unify component eviction cursors and lock receipts ( #41276 )
...
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com >
2026-09-25 20:53:35 -07:00
ac62ddf164
[Rust frontend] Decode input_ids without untagged buffering ( #41246 )
...
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com >
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com >
2026-09-25 12:47:38 -07:00
7fef014d51
feat(kv-hints): add kv hint envelope to request transport ( #38891 )
...
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com >
2026-09-23 10:31:13 -07:00
Sage
b4701d672d
[rust-renderer] decouple renderer sampling from protocols ( #40747 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-22 10:48:28 -07:00
Sage
14e9c40a72
[Observability] Expose python/rust frontend identity in /server_info ( #39993 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-21 23:32:52 +08:00
7b1c2ed0a4
[rust-renderer] Standalone preprocessing ( #36718 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
Co-authored-by: Rain Jiang <96632942+rainj-me@users.noreply.github.com >
2026-09-20 22:03:12 +08:00
Lianmin Zheng and Jialin Ouyang
f5a1434700
[HiCache] Document transfer arguments ( #40239 )
...
Clarify the legacy host_indices argument and label Mamba test arguments,
including the current staging_tokens parameter. Executable code is unchanged.
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com >
2026-09-18 14:08:19 -07:00
Shuwen Wang
191172fa74
[Unified Tree] fix: exempt host-locked aux nodes from the sanity_check host-LRU check ( #39980 )
2026-09-18 15:29:29 +00:00
William Arnold and ishandhanani
0214954f26
[gRPC] Stream engine state changes ( #39915 )
...
Signed-off-by: William Arnold <7565007+Aphoh@users.noreply.github.com >
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-09-17 19:17:25 -07:00
Thomas Wang
6c73368c32
[AMD][DSV4] Enable hicache on deepseek-v4 fp8 unified attn ( #37778 )
2026-09-17 11:29:23 -07:00
1f60ddef5d
[PD] Introduce runtime role switching between prefill and decode ( #28403 )
...
Signed-off-by: huanglong <huanglong@linux.alibaba.com >
Signed-off-by: inkcherry <mingzhi.liu@amd.com >
Co-authored-by: huanglong <huanglong@linux.alibaba.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com >
2026-09-18 01:45:12 +08:00
William Arnold and ishandhanani
f45aad44bd
[gRPC] Expose native pause status ( #37488 )
...
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-09-16 15:16:52 -07:00
Liangsheng Yin
2f7d04da5d
dsv4.1: Rust extension modules for image preprocessing, KV pool names, and PD bootstrap ( #39677 )
2026-09-15 18:11:48 -07:00
Lianmin Zheng
b803cfa0c4
Add external multimodal processors to the Rust frontend ( #39329 )
2026-09-15 15:38:47 -07:00