Commit Graph
84 Commits
Author SHA1 Message Date
Liangsheng Yin 82c9a8b037 [Perf] deepseek-v4.1: Default PD to MegaAttention, with the KV layout in the PD handshake (#45075) 2026-10-18 21:16:45 -07:00
fea48754de [Feat] rust/grpc: Add ListModels, Detokenize, and Abort to api.v1 (#42118)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
2026-10-19 09:53:56 +08:00
0d13efb2b8 [Feat] rust-server: Serve /v1/tokenize and /v1/detokenize from the Rust frontend (#44767)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-17 01:30:50 -07:00
5cbe2371a5 [Feat] rust-server: Report Python's launch-time /server_info fields from the Rust frontend (#44764)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 18:14:50 -07:00
b62a48ee15 [Feat] sglang-server: Serve /flush_cache from the Rust frontend (#44766)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 17:24:01 -07:00
metamergebotandmetamergebot f39c2693aa [CI] rust-server: Raise the ChunkEvent frame-size cap to 152 bytes (#44773)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
2026-10-16 13:47:52 -07:00
553068af25 [Fix] rust-server: Honour cache_salt and require_reasoning on /generate (#44533)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 10:38:09 -07:00
26a2f18c02 [Feat] sglang-server: serve /v1/loads from the Rust frontend (#44532)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-16 10:35:37 -07:00
shodocoandmetamergebot 2957ff3165 [Fix] sglang-server: Keep the matched stop token in output_ids (#42637)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
2026-10-15 18:51:47 -07:00
f110b2f801 [Feat] rust-server: Report cached_tokens and reasoning_tokens in meta_info (#44531)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-15 17:50:27 -07:00
0b1a73599a [Feat] rust-server: Apply the x-override-* request headers (#44312)
Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-15 15:50:04 -07:00
02a43eeef6 [Feat] rust-server: Attach configured metadata headers to a healthy /health_generate (#44315)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-15 12:04:23 -07:00
209f8220d6 [Fix] sglang-server: Honour extra_key on the Rust frontend's /generate (#44207)
Co-authored-by: fei-xx <fei-xx@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-14 16:44:31 -07:00
aa8793f33b [Rust] Serve scheduler metrics on the Rust frontend's /metrics (#43942)
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-14 16:43:06 -07:00
9732188a58 [Rust] Serve /weights_checker from the Rust frontend (#43926)
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-14 16:41:10 -07:00
817781b828 [Feat] sgl-router: Render Qwen3.8 (Max, 27B, Flash-Next) through sglang-processor (#43689)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-14 17:09:35 +08:00
Kan WuandClaude Opus 5.5 4560da58ee [sgl-router] Render GLM-5.3 and GLM-5.3-Flash through sglang-processor (#43625)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:19:44 +08:00
Xinyuan TongandNihar Patel 7dc2930585 [Feature] Add Aaronson-Gumbel text watermarking (#37577)
Co-authored-by: Nihar Patel <nihpat95@gmail.com>
2026-10-13 12:47:50 -07:00
ce5352a814 [Rust] Serve any model name on the OpenAI chat and completions routes (#43923)
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-13 11:58:57 -07:00
149ef7d1b7 [Rust] Let an embedding package wrap the frontend HTTP app (#43924)
Co-authored-by: hanminglu <hanminglu@users.noreply.github.com>
Co-authored-by: shodoco <shodoco@users.noreply.github.com>
2026-10-13 11:44:43 -07:00
e775fa7253 [Build][CI] Drop Python 3.10: require Python 3.11+ and run CUDA CI on Python 3.12 (#43021)
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
2026-10-12 18:40:16 -07:00
Sage 6fc8d9da32 [rust-server] preserve PD routing in chat and completions (#42989)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
2026-10-10 17:29:46 +08:00
d511e6bd1d [sgl-router] Parse tool calls when serving chat through /generate (#42999)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 21:07:30 +08:00
3805d78796 [sgl-router] Serve /v1/chat/completions through /generate, with reasoning parsing (#42998)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-09 19:17:52 +08:00
f620d733d2 [rust-processor] Share the reasoning splitter and tool constraint across hosts (#42996)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
2026-10-09 15:19:41 +08:00
3ba90f1b52 [rust] Update Dynamo frontend crates to latest compatible releases (#42666)
Co-authored-by: Kan Wu <kan.wu@radixark.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 11:46:01 +08:00
shodoco 86624f14eb [Rust] Pass the media item fps hint to multimodal processors (#43143) 2026-10-08 17:51:06 -07:00
Sage 5a5ea81036 [rust-server] warm up each rust listener in-process (#43004)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
2026-10-08 14:09:46 -07:00
Rain Jiang c52dcc5a01 The sgalng api specs (#41585) 2026-10-07 01:55:01 -07:00
jain-riaandishandhanani fd62ee84f7 feat(grpc): expose follower metadata and node-local KV sources (#39659)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-10-06 16:39:01 -07:00
jain-ria fb92ba1127 [Rust] Wire the gRPC server into the frontend lifecycle (#41964)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
2026-10-01 10:44:56 -07:00
jain-ria ebccddce19 [Rust] Implement the Rust gRPC adapter for native generation (#41766)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
2026-09-30 18:17:42 -07:00
jain-ria 46beacf13d [grpc rust] Define semantic frontend contract (#41709)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
2026-09-29 13:40:04 -07:00
jain-riaandishandhanani 069523f999 [Rust] Extract a transport-neutral frontend core (#39385)
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
2026-09-28 21:18:50 -07:00
Rain Jiang 6c4f36e47a Rust server unify datapath for mm and generate requests (#39679) 2026-09-27 23:36:07 -07:00
ac62ddf164 [Rust frontend] Decode input_ids without untagged buffering (#41246)
Co-authored-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
2026-09-25 12:47:38 -07:00
Sage 14e9c40a72 [Observability] Expose python/rust frontend identity in /server_info (#39993)
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
2026-09-21 23:32:52 +08:00
1f60ddef5d [PD] Introduce runtime role switching between prefill and decode (#28403)
Signed-off-by: huanglong <huanglong@linux.alibaba.com>
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
Co-authored-by: huanglong <huanglong@linux.alibaba.com>
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Huang Long <121648372+LLLL114@users.noreply.github.com>
2026-09-18 01:45:12 +08:00
Liangsheng Yin 2f7d04da5d dsv4.1: Rust extension modules for image preprocessing, KV pool names, and PD bootstrap (#39677) 2026-09-15 18:11:48 -07:00
Lianmin Zheng b803cfa0c4 Add external multimodal processors to the Rust frontend (#39329) 2026-09-15 15:38:47 -07:00
Yuxuan ZhangandXinyuan Tong 17ba2c2e7c Support non-strict GLM47 tool calls with EBNF constraints (#38890)
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-15 16:41:48 +08:00
Kan Wu 07e1918924 [Rust] Use Dynamo native renderers when chat templates are missing (#38939) 2026-09-14 17:29:10 -07:00
Po-Han Huang (NVIDIA) 51c8581a26 [Rust] Gate health on startup warmup completion (#37994) 2026-09-09 15:55:13 -07:00
Rain Jiang b83a59835d sglang-server remove opaque type (#38095) 2026-09-08 00:41:31 -07:00
Lianmin Zheng 978cc228ca [Rust] Bound multimodal media ingress (#37967) 2026-09-04 05:22:44 -07:00
12735c2d76 [mm] refactor mm code for rust tokenizer manager (#34660)
Co-authored-by: Rain Jiang <rain-jiang@outlook.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-09-04 03:03:25 -07:00
3bac084d4e [Model] Add native IFM K2 Horizon serving support (#37654)
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
2026-09-03 16:39:43 +08:00
Lianmin Zheng f8f04bafa8 Rust server: align launcher and request validation behavior (#37327) 2026-09-01 23:42:36 -07:00
Lianmin Zheng 579270d459 [Rust] Simplify request defaults and document batch header ABI (#37226) 2026-08-31 12:31:35 -07:00
Lianmin Zheng 2138494272 [Rust] Keep sampling and scheduler wire schemas in sync (#37222) 2026-08-31 12:31:07 -07:00