Commit Graph
152 Commits
Author SHA1 Message Date
Shangming Cai 1291f3175d [rust-processor] Fix chat-template precedence wording in the processor README (#42756) 2026-10-06 19:10:26 +08:00
cf19ae6d04 [rust-processor] Split sglang-processor into tokenizer, render and parser (#42662)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 18:59:02 +08:00
Mohammad Miadh AngkadandMohammad Angkad 0bdd0cc8dc [Deps] Upgrade the CUDA PyTorch stack to 2.14 (#38641)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
2026-10-04 00:48:02 -07:00
937c0a6cfe Fix unified HiCache physical transfers (#39479)
Co-authored-by: yhzhuang <yhzhuang@fb.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Yonghao Zhuang <yhzhuang@users.noreply.github.com>
Co-authored-by: Cheng Wan <cheng.wan@radixark.ai>
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
2026-10-04 00:38:56 -07:00
4611ea365e [HiCache]: Fix host lock ownership across radix splits (#38480)
Signed-off-by: Lukas Humbel <lhumbel@nvidia.com>
Co-authored-by: Jialin Ouyang <jialino@meta.com>
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-10-04 11:16:04 +08:00
2bae12b9b3 [HiCache] Fix write-back SWA insert backups tripping the write-through pending-ack assert (#42264)
Co-authored-by: merrymercy <merrymercy@users.noreply.github.com>
Co-authored-by: 842974287 <842974287@users.noreply.github.com>
2026-10-03 09:45:56 -07:00
Liangsheng Yin 50be533d09 [mem_cache] Rename cache_unfinished_req to checkpoint_req and count cache hits only when a request finishes (#41520) 2026-10-01 21:41:45 -07:00
jain-ria fb92ba1127 [Rust] Wire the gRPC server into the frontend lifecycle (#41964)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
2026-10-01 10:44:56 -07:00
jain-ria ebccddce19 [Rust] Implement the Rust gRPC adapter for native generation (#41766)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
2026-09-30 18:17:42 -07:00
Sage 8e4bcc5ca9 [rust-renderer] sglang-processor lib (#41896)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
2026-09-30 15:09:35 -07:00
jain-ria 21ed9fbf44 [Rust] Extract shared runtime.v1 protobuf bindings (#41726)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
2026-09-30 10:13:08 -07:00
jain-ria 46beacf13d [grpc rust] Define semantic frontend contract (#41709)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
2026-09-29 13:40:04 -07:00
jain-riaandishandhanani 069523f999 [Rust] Extract a transport-neutral frontend core (#39385)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-09-28 21:18:50 -07:00
77091cea68 [Radix Cache] Sync Rust TreeCore and make it the default (#39627)
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Zhangheng <hzh0425@apache.org>
Co-authored-by: alphabetc1 <alphabetc1@users.noreply.github.com>
2026-09-29 10:18:57 +08:00
Rain Jiang 6c4f36e47a Rust server unify datapath for mm and generate requests (#39679) 2026-09-27 23:36:07 -07:00
Lianmin ZhengandLu Fang 3ecb6cd729 [MemCache] Unify component eviction cursors and lock receipts (#41276)
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
2026-09-25 20:53:35 -07:00
ac62ddf164 [Rust frontend] Decode input_ids without untagged buffering (#41246)
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
2026-09-25 12:47:38 -07:00
7fef014d51 feat(kv-hints): add kv hint envelope to request transport (#38891)
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
2026-09-23 10:31:13 -07:00
Sage b4701d672d [rust-renderer] decouple renderer sampling from protocols (#40747)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
2026-09-22 10:48:28 -07:00
Sage 14e9c40a72 [Observability] Expose python/rust frontend identity in /server_info (#39993)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
2026-09-21 23:32:52 +08:00
7b1c2ed0a4 [rust-renderer] Standalone preprocessing (#36718)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: Rain Jiang <96632942+rainj-me@users.noreply.github.com>
2026-09-20 22:03:12 +08:00
Lianmin ZhengandJialin Ouyang f5a1434700 [HiCache] Document transfer arguments (#40239)
Clarify the legacy host_indices argument and label Mamba test arguments,
including the current staging_tokens parameter. Executable code is unchanged.

Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
2026-09-18 14:08:19 -07:00
Shuwen Wang 191172fa74 [Unified Tree] fix: exempt host-locked aux nodes from the sanity_check host-LRU check (#39980) 2026-09-18 15:29:29 +00:00
William Arnoldandishandhanani 0214954f26 [gRPC] Stream engine state changes (#39915)
Signed-off-by: William Arnold <7565007+Aphoh@users.noreply.github.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-09-17 19:17:25 -07:00
Thomas Wang 6c73368c32 [AMD][DSV4] Enable hicache on deepseek-v4 fp8 unified attn (#37778) 2026-09-17 11:29:23 -07:00
1f60ddef5d [PD] Introduce runtime role switching between prefill and decode (#28403)
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
2026-09-18 01:45:12 +08:00
William Arnoldandishandhanani f45aad44bd [gRPC] Expose native pause status (#37488)
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-09-16 15:16:52 -07:00
Liangsheng Yin 2f7d04da5d dsv4.1: Rust extension modules for image preprocessing, KV pool names, and PD bootstrap (#39677) 2026-09-15 18:11:48 -07:00
Lianmin Zheng b803cfa0c4 Add external multimodal processors to the Rust frontend (#39329) 2026-09-15 15:38:47 -07:00
Zhiqiang Xie 7f5dd19256 [HiCache] Rework the buffer-mode storage prefetch pipeline and retry bookkeeping (#39283) 2026-09-15 10:48:49 -07:00
Yuxuan ZhangandXinyuan Tong 17ba2c2e7c Support non-strict GLM47 tool calls with EBNF constraints (#38890)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-15 16:41:48 +08:00
ishandhanani 3f871a246c feat(agent sessions): attribute stored KV cache blocks to sessions (#37482)
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
2026-09-14 21:34:49 -07:00
Kan Wu 07e1918924 [Rust] Use Dynamo native renderers when chat templates are missing (#38939) 2026-09-14 17:29:10 -07:00
Shuwen WangandClaude Opus 5 7f1f8c706a [mem_cache][10/N] refactor: drop the redundant _component suffix in unified_cache/components (#35644)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-13 21:38:20 +08:00
Jundong LiuandShuwen Wang 7078e5ffbc [HiCache] Publish a host store event for storage-prefetch refills (#38486)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-09-13 10:29:14 +00:00
Shuwen Wang bd45cd50ca [Unified Tree] Port SWA Branching-Point Caching to the Rust TreeCore (#37584) 2026-09-12 18:37:57 +08:00
Yanbin JiangandShuwen Wang 0b415fa573 [HiCache][LoRA] Isolate storage pages by extra key (#38577)
Co-authored-by: Shuwen Wang <47200617+alphabetc1@users.noreply.github.com>
2026-09-12 18:18:14 +08:00
Shuwen Wang b9cb96496d [Unified Tree] Preserve aux LRU recency when splitting nodes (#38482) 2026-09-12 18:17:38 +08:00
Jialin Ouyang 908226fea2 [Rust TreeCore] Support external cache linker (#37306) 2026-09-10 19:22:24 +08:00
Po-Han Huang (NVIDIA) 51c8581a26 [Rust] Gate health on startup warmup completion (#37994) 2026-09-09 15:55:13 -07:00
Zhiqiang Xie beaf3d9252 [HiCache] Replace skip_lock_node_ids with a segment lock protocol (#36848) 2026-09-09 14:55:51 -07:00
Jialin Ouyang 7a464a7014 [Rust TreeCore] Harden runtime and CI parity (#37303) 2026-09-09 10:40:22 +08:00
Rain Jiang b83a59835d sglang-server remove opaque type (#38095) 2026-09-08 00:41:31 -07:00
Shuwen Wang 19b46863f3 fix: align write-through pending across tree cores (#37278) 2026-09-04 22:08:25 +08:00
Lianmin Zheng 978cc228ca [Rust] Bound multimodal media ingress (#37967) 2026-09-04 05:22:44 -07:00
12735c2d76 [mm] refactor mm code for rust tokenizer manager (#34660)
Co-authored-by: Rain Jiang <rain-jiang@outlook.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-04 03:03:25 -07:00
3bac084d4e [Model] Add native IFM K2 Horizon serving support (#37654)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-03 16:39:43 +08:00
Alex NailsandAlison Shao 28262c20df [CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
2026-09-02 19:46:08 -07:00
f8cbf000f4 [AMD] Enable FP4 indexer for Deepseek V4 (#37353)
Co-authored-by: 1am9trash <1am9trash@gmail.com>
Co-authored-by: AMD-yanfeiwang <256076023+AMD-yanfeiwang@users.noreply.github.com>
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-09-02 09:45:08 -07:00
Lianmin Zheng f8f04bafa8 Rust server: align launcher and request validation behavior (#37327) 2026-09-01 23:42:36 -07:00