|
a0b04dbe4c
|
feat(grpc): add generation request semantics (#32588)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
|
2026-08-04 18:53:45 -07:00 |
|
|
Kan WuandClaude Fable 5
|
17d19081d9
|
[mm] sglang-mm: server vision pipeline core (fetch/driver/pipeline) + Qwen VL (#32364)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-08-04 00:47:00 -07:00 |
|
|
Liangsheng Yin
|
157401f050
|
[CI] Build the Rust extensions on the 5090 pool and seed the cache from main (#33460)
|
2026-08-03 23:33:20 -07:00 |
|
|
Rain Jiang
|
4494fb96b2
|
refactor the tcp listener binding logic (#33420)
|
2026-08-03 22:10:52 -07:00 |
|
|
Chengyu LinandRain Jiang
|
e00f32ed4f
|
feat: rust sglang server openai apis (#33103)
Co-authored-by: Rain Jiang <rain-jiang@outlook.com>
|
2026-08-02 23:13:44 -07:00 |
|
|
Kan WuandClaude Fable 5
|
8186eeb939
|
[rust-server] Reland: fix TCP-layer TTFT stalls (#33026) (#33269)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
2026-08-02 20:52:29 -07:00 |
|
|
|
5d2dbb35a6
|
[rust-server] PD disaggregation support (#33125)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Rain Jiang <rain-jiang@outlook.com>
|
2026-08-02 18:32:59 -07:00 |
|
|
Kan Wu
|
06554515f4
|
Revert "[rust-server] fix TCP-layer TTFT stalls" (#33257)
|
2026-08-02 01:20:11 -07:00 |
|
|
|
574ead753a
|
[rust-server] fix TCP-layer TTFT stalls (#33026)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2026-08-01 12:13:13 -07:00 |
|
|
Rain Jiang
|
1d640aaea2
|
bump dynamo-tokenizers to 1.7.0 (#32981)
|
2026-07-31 15:02:28 -07:00 |
|
|
Rain JiangandAlex Nails
|
9dcaf6bfdf
|
rust server build release artifacts (#33096)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
|
2026-07-31 12:36:25 -07:00 |
|
|
Rain Jiang
|
4af8ddb576
|
support rust sglang server (#29799)
|
2026-07-31 11:56:31 -07:00 |
|
|
Rain Jiang
|
3312645a30
|
wire the rust server modules into lib, runtime, and tokenizer manager (#32877)
|
2026-07-30 12:46:19 -07:00 |
|
|
Rain Jiang
|
047635ee35
|
add the rust server native api handlers and runtime threads (#32876)
|
2026-07-30 12:46:19 -07:00 |
|
|
Rain Jiang
|
30643f88bc
|
add the rust server api frame codec and http server entry (#32875)
|
2026-07-30 12:46:19 -07:00 |
|
|
Rain Jiang
|
4facc0e18a
|
add the rust server ingress tests, guard, and submit modules (#32874)
|
2026-07-30 12:46:18 -07:00 |
|
|
Rain Jiang
|
e2c65af229
|
add the rust server ingress request validation and api server common types (#32873)
|
2026-07-30 12:46:17 -07:00 |
|
|
Rain Jiang
|
922d6e5542
|
add the rust server tokenizer, detokenizer, and egress modules (#32872)
|
2026-07-30 12:46:17 -07:00 |
|
|
Rain Jiang
|
35f2e6ab58
|
update Cargo.lock for the rust sglang-server dependencies (#32871)
|
2026-07-30 12:32:27 -07:00 |
|
|
Rain Jiang
|
d7a4c830e5
|
sglang rust server tokenizer manager, ring and runtime (#32358)
|
2026-07-29 18:42:12 -07:00 |
|
|
Rain Jiang
|
6e48c13497
|
sglang rust server egress message (#32342)
|
2026-07-29 17:16:06 -07:00 |
|
|
Rain Jiang
|
d24de56995
|
sglang rust server sampling message (#32343)
|
2026-07-29 14:55:06 -07:00 |
|
|
Rain Jiang
|
2ca2ca753a
|
sglang rust server request message (#32242)
|
2026-07-29 10:54:56 -07:00 |
|
|
Rain Jiang
|
d943636a48
|
support regex that compatible with python re lib however apply more l… (#32676)
|
2026-07-28 14:25:31 -07:00 |
|
|
Rain Jiang
|
be7cc17307
|
sglang rust server environ fsm error id gen (#32240)
|
2026-07-24 20:55:10 +00:00 |
|
|
Rain Jiang
|
20eb37a2a1
|
init sglang rust server project (#32256)
|
2026-07-23 21:47:15 +00:00 |
|
|
Rain Jiang
|
7fe82dd02e
|
create rust workspace (#32014)
|
2026-07-23 12:02:41 -07:00 |
|
|
|
02236fa38c
|
Add Inkling model support (#31681)
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Qiaolin Yu <qiaolin.yu@radixark.ai>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Aurick Qiao <aurick@thinkingmachines.ai>
Co-authored-by: Joseph <jk@thinkingmachines.ai>
|
2026-07-19 22:57:37 -07:00 |
|
|
Connor CarpenterandIshan Dhanani
|
cc7d6659fd
|
feat(grpc): support disaggregated generation requests (#30440)
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
|
2026-07-08 14:53:56 -07:00 |
|
|
ishandhanani
|
aaa31eb0a1
|
feat: first-class session identity in SGLang (#29436)
|
2026-06-28 07:16:38 -07:00 |
|
|
Alex Nails
|
1d19721394
|
[gRPC] Native server: Rust crate (1/N) (#23506)
|
2026-05-18 22:31:38 -07:00 |
|
|
Alex NailsandClaude Opus 4.7
|
c3ab5bec7d
|
ci: consolidate rust + protoc install across workflows (#23700)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-29 13:39:02 -07:00 |
|
|
Alex NailsandClaude Opus 4.7
|
bea4c895c1
|
[gRPC] Pass --experimental_allow_proto3_optional to protoc in build.rs (#23226)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-20 17:20:14 +08:00 |
|
|
Alex Nails
|
10e17cc55e
|
[gRPC] Native gRPC server: proto + Rust crate scaffold + server args (#22736)
|
2026-04-20 12:39:35 +08:00 |
|
|
Ata Fatahi
|
e3b3acfa6f
|
Rename rust folder to sgl-router (#2464)
Signed-off-by: Ata Fatahi <immrata@gmail.com>
|
2024-12-12 09:40:41 -08:00 |
|
|
Ata Fatahi
|
6efa9e4a6d
|
Bump sglang-router to 0.1.1 (#2459)
Signed-off-by: Ata Fatahi <immrata@gmail.com>
|
2024-12-11 17:40:03 -08:00 |
|
|
Ata FatahiandByron Hsu
|
7791fd9948
|
Include version info into the router package (#2456)
Signed-off-by: Ata Fatahi <immrata@gmail.com>
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2024-12-11 17:31:20 -08:00 |
|
|
Ata Fatahi
|
2ac36b9a7b
|
Make request payload size configurable (#2444)
Signed-off-by: Ata Fatahi <immrata@gmail.com>
|
2024-12-11 16:55:21 -08:00 |
|
|
Byron Hsu
|
2d60a5ee75
|
Update v0.1.0.md
|
2024-12-11 13:48:18 -08:00 |
|
|
Byron Hsu
|
2e4a5907c9
|
[router] Release router 0.1.0 with dynamic scaling and fault tolerance (#2455)
|
2024-12-11 13:42:35 -08:00 |
|
|
Byron Hsu
|
c0ee46fe10
|
[router] Update doc for dynamic scaling and fault tolerance (#2454)
|
2024-12-11 13:11:42 -08:00 |
|
|
Byron Hsu
|
864bf2ba00
|
[router] remove main.rs because only lib.rs is used for py binding (#2453)
|
2024-12-11 12:13:19 -08:00 |
|
|
Byron Hsu
|
a4cca7fc53
|
[router] Add retries based fault tolerance (#2452)
|
2024-12-11 12:13:08 -08:00 |
|
|
Byron Hsu
|
0fb88aaa77
|
[router] Use borrow if possible to save cost (#2441)
|
2024-12-11 01:38:50 -08:00 |
|
|
Byron Hsu
|
d4de9a6235
|
[router] Refactor: decouple select and send stage (#2440)
|
2024-12-11 00:51:21 -08:00 |
|
|
Byron Hsu
|
6387098f5f
|
[router] add health checking in router init (#2393)
|
2024-12-08 17:17:37 -08:00 |
|
|
Byron Hsu
|
2a717c5078
|
[Router] fix interrupt from terminal (#2413)
|
2024-12-08 16:58:41 -08:00 |
|
|
Byron Hsu
|
a1e697b25b
|
[router] Improve cleanup logic (#2411)
|
2024-12-08 15:24:02 -08:00 |
|
|
Byron Hsu
|
ef995dae1e
|
[router] Health check on worker before adding to the router (#2392)
|
2024-12-07 15:39:54 -08:00 |
|
|
Byron Hsu
|
c36736c841
|
[router] Add remove worker api (#2380)
|
2024-12-06 17:16:03 -08:00 |
|