d511e6bd1d
[sgl-router] Parse tool calls when serving chat through /generate ( #42999 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 21:07:30 +08:00
3805d78796
[sgl-router] Serve /v1/chat/completions through /generate, with reasoning parsing ( #42998 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-09 19:17:52 +08:00
b130f8af02
[sgl-router] Serve /v1/completions through /generate, with the shared OpenAI layer ( #42997 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 16:04:04 +08:00
f620d733d2
[rust-processor] Share the reasoning splitter and tool constraint across hosts ( #42996 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-09 15:19:41 +08:00
3ba90f1b52
[rust] Update Dynamo frontend crates to latest compatible releases ( #42666 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-09 11:46:01 +08:00
shodoco
86624f14eb
[Rust] Pass the media item fps hint to multimodal processors ( #43143 )
2026-10-08 17:51:06 -07:00
Sage
5a5ea81036
[rust-server] warm up each rust listener in-process ( #43004 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-10-08 14:09:46 -07:00
943621c336
[sgl-router] Render DeepSeek-V4 through sglang-processor ( #42665 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
2026-10-08 20:16:20 +08:00
45abea0269
[rust-processor] DeepSeek-V4 parity with SGLang, plus a reusable parity harness and skill ( #42664 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-08 18:35:30 +08:00
Liangsheng Yin
5a4d01a3ae
[MemCache] Return only the matched length from match_prefix; read KV indices off the node path ( #42923 )
2026-10-08 01:10:14 -07:00
Rain Jiang
c52dcc5a01
The sgalng api specs ( #41585 )
2026-10-07 01:55:01 -07:00
jain-ria and ishandhanani
fd62ee84f7
feat(grpc): expose follower metadata and node-local KV sources ( #39659 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-10-06 16:39:01 -07:00
0f57a39152
[rust-processor] Gate render, tokenizer and parser behind cargo features ( #42663 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 10:58:38 -07:00
Shangming Cai
1291f3175d
[rust-processor] Fix chat-template precedence wording in the processor README ( #42756 )
2026-10-06 19:10:26 +08:00
cf19ae6d04
[rust-processor] Split sglang-processor into tokenizer, render and parser ( #42662 )
...
Co-authored-by: Kan Wu <kan.wu@radixark.ai >
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com >
2026-10-06 18:59:02 +08:00
Mohammad Miadh Angkad and Mohammad Angkad
0bdd0cc8dc
[Deps] Upgrade the CUDA PyTorch stack to 2.14 ( #38641 )
...
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai >
2026-10-04 00:48:02 -07:00
937c0a6cfe
Fix unified HiCache physical transfers ( #39479 )
...
Co-authored-by: yhzhuang <yhzhuang@fb.com >
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com >
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com >
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai >
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com >
2026-10-04 00:38:56 -07:00
4611ea365e
[HiCache]: Fix host lock ownership across radix splits ( #38480 )
...
Signed-off-by: Lukas Humbel <lhumbel@nvidia.com >
Co-authored-by: Jialin Ouyang <jialino@meta.com >
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com >
2026-10-04 11:16:04 +08:00
2bae12b9b3
[HiCache] Fix write-back SWA insert backups tripping the write-through pending-ack assert ( #42264 )
...
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com >
Co-authored-by: 842974287 <842974287@users.noreply.github.com >
2026-10-03 09:45:56 -07:00
Liangsheng Yin
50be533d09
[mem_cache] Rename cache_unfinished_req to checkpoint_req and count cache hits only when a request finishes ( #41520 )
2026-10-01 21:41:45 -07:00
jain-ria
fb92ba1127
[Rust] Wire the gRPC server into the frontend lifecycle ( #41964 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-10-01 10:44:56 -07:00
jain-ria
ebccddce19
[Rust] Implement the Rust gRPC adapter for native generation ( #41766 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-09-30 18:17:42 -07:00
Sage
8e4bcc5ca9
[rust-renderer] sglang-processor lib ( #41896 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-30 15:09:35 -07:00
jain-ria
21ed9fbf44
[Rust] Extract shared runtime.v1 protobuf bindings ( #41726 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-09-30 10:13:08 -07:00
jain-ria
46beacf13d
[grpc rust] Define semantic frontend contract ( #41709 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
2026-09-29 13:40:04 -07:00
jain-ria and ishandhanani
069523f999
[Rust] Extract a transport-neutral frontend core ( #39385 )
...
Signed-off-by: jain-ria <riajain@NVIDIA.com >
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-09-28 21:18:50 -07:00
77091cea68
[Radix Cache] Sync Rust TreeCore and make it the default ( #39627 )
...
Co-authored-by: Ke Bao <ispobaoke@gmail.com >
Co-authored-by: Zhangheng <hzh0425@apache.org >
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com >
2026-09-29 10:18:57 +08:00
Rain Jiang
6c4f36e47a
Rust server unify datapath for mm and generate requests ( #39679 )
2026-09-27 23:36:07 -07:00
Lianmin Zheng and Lu Fang
3ecb6cd729
[MemCache] Unify component eviction cursors and lock receipts ( #41276 )
...
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com >
2026-09-25 20:53:35 -07:00
ac62ddf164
[Rust frontend] Decode input_ids without untagged buffering ( #41246 )
...
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com >
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com >
2026-09-25 12:47:38 -07:00
7fef014d51
feat(kv-hints): add kv hint envelope to request transport ( #38891 )
...
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com >
2026-09-23 10:31:13 -07:00
Sage
b4701d672d
[rust-renderer] decouple renderer sampling from protocols ( #40747 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-22 10:48:28 -07:00
Sage
14e9c40a72
[Observability] Expose python/rust frontend identity in /server_info ( #39993 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
2026-09-21 23:32:52 +08:00
7b1c2ed0a4
[rust-renderer] Standalone preprocessing ( #36718 )
...
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com >
Co-authored-by: Rain Jiang <96632942+rainj-me@users.noreply.github.com >
2026-09-20 22:03:12 +08:00
Lianmin Zheng and Jialin Ouyang
f5a1434700
[HiCache] Document transfer arguments ( #40239 )
...
Clarify the legacy host_indices argument and label Mamba test arguments,
including the current staging_tokens parameter. Executable code is unchanged.
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com >
2026-09-18 14:08:19 -07:00
Shuwen Wang
191172fa74
[Unified Tree] fix: exempt host-locked aux nodes from the sanity_check host-LRU check ( #39980 )
2026-09-18 15:29:29 +00:00
William Arnold and ishandhanani
0214954f26
[gRPC] Stream engine state changes ( #39915 )
...
Signed-off-by: William Arnold <7565007+Aphoh@users.noreply.github.com >
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-09-17 19:17:25 -07:00
Thomas Wang
6c73368c32
[AMD][DSV4] Enable hicache on deepseek-v4 fp8 unified attn ( #37778 )
2026-09-17 11:29:23 -07:00
1f60ddef5d
[PD] Introduce runtime role switching between prefill and decode ( #28403 )
...
Signed-off-by: huanglong <huanglong@linux.alibaba.com >
Signed-off-by: inkcherry <mingzhi.liu@amd.com >
Co-authored-by: huanglong <huanglong@linux.alibaba.com >
Co-authored-by: Shangming Cai <csmthu@gmail.com >
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com >
2026-09-18 01:45:12 +08:00
William Arnold and ishandhanani
f45aad44bd
[gRPC] Expose native pause status ( #37488 )
...
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com >
2026-09-16 15:16:52 -07:00
Liangsheng Yin
2f7d04da5d
dsv4.1: Rust extension modules for image preprocessing, KV pool names, and PD bootstrap ( #39677 )
2026-09-15 18:11:48 -07:00
Lianmin Zheng
b803cfa0c4
Add external multimodal processors to the Rust frontend ( #39329 )
2026-09-15 15:38:47 -07:00
Zhiqiang Xie
7f5dd19256
[HiCache] Rework the buffer-mode storage prefetch pipeline and retry bookkeeping ( #39283 )
2026-09-15 10:48:49 -07:00
Yuxuan Zhang and Xinyuan Tong
17ba2c2e7c
Support non-strict GLM47 tool calls with EBNF constraints ( #38890 )
...
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com >
2026-09-15 16:41:48 +08:00
ishandhanani
3f871a246c
feat(agent sessions): attribute stored KV cache blocks to sessions ( #37482 )
...
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com >
2026-09-14 21:34:49 -07:00
Kan Wu
07e1918924
[Rust] Use Dynamo native renderers when chat templates are missing ( #38939 )
2026-09-14 17:29:10 -07:00
Shuwen Wang and Claude Opus 5
7f1f8c706a
[mem_cache][10/N] refactor: drop the redundant _component suffix in unified_cache/components ( #35644 )
...
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com >
2026-09-13 21:38:20 +08:00
Jundong Liu and Shuwen Wang
7078e5ffbc
[HiCache] Publish a host store event for storage-prefetch refills ( #38486 )
...
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com >
2026-09-13 10:29:14 +00:00
Shuwen Wang
bd45cd50ca
[Unified Tree] Port SWA Branching-Point Caching to the Rust TreeCore ( #37584 )
2026-09-12 18:37:57 +08:00
Yanbin Jiang and Shuwen Wang
0b415fa573
[HiCache][LoRA] Isolate storage pages by extra key ( #38577 )
...
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com >
2026-09-12 18:18:14 +08:00