minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
16,448 Commits 2 Branches 0 Tags
fde9ad253161c3d866727dfb4be2772623d57ca4
Commit Graph
207 Commits
Author SHA1 Message Date
sglang-botsglang-botBrayden ZhongJimmy Shonghnyls2002Alex NailsLiangsheng Yin
 		fde9ad2531 [Feature] Add Muse Glimmer model support (#34262) 
Co-authored-by: sglang-bot <232288953+sglang-bot@users.noreply.github.com>
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
 2026-08-11 15:41:52 -07:00
Dustin LuongandJimmy Shong 2c3ecf32f1 [Fix] Reformat /vertex_generate successful predictions (#33446) 
Co-authored-by: Jimmy Shong <69131491+Jiminator@users.noreply.github.com>
 2026-08-07 00:00:48 +00:00
Connor CarpenterishandhananiAlex Nails
 		a0b04dbe4c feat(grpc): add generation request semantics (#32588) 
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
 2026-08-04 18:53:45 -07:00
Lianmin Zheng 4794b401d5 [Observability] Add startup, memory, and hybrid SWA diagnostics (#33375) 2026-08-04 12:50:09 -07:00
Liangsheng Yin 45c00daa1b [misc] Deep-merge nested config overrides and parse request bodies with orjson (#33351) 2026-08-02 23:55:06 -07:00
Cheng Wan 0b3e8bedd1 config: keep runtime hicache and weight-version updates off ServerArgs (#33336) 
The scheduler's runtime HiCache attach/detach wrote its own ServerArgs so the
internal-state readback would show the change; that readback already reports
the resolved config, so the writes become get_context().override(...) and the
namespace readers see them too.

The tokenizer side is per-engine — several Engines can share one process — so
its control-plane updates (weight version, model path + load format, HiCache
attach/detach) stay with the manager instead of moving to the process-global
bags. TokenizerManager gains record_config_updates / config_value /
resolved_config_dict, and the readbacks that used to observe the instance write
(/server_info, /model_info, the HiCache status endpoint, the gRPC bridge) now
overlay those updates onto the startup config.

test_server_info's stub grew the real manager instead of a SimpleNamespace, so
the overlay it now exercises cannot drift from production.

Writer ratchet 26 -> 19.
 2026-08-02 21:23:38 -07:00
Kan WuClaude Fable 5CursorRain Jiang
 		5d2dbb35a6 [rust-server] PD disaggregation support (#33125) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Rain Jiang <rain-jiang@outlook.com>
 2026-08-02 18:32:59 -07:00
Rain Jiang 4af8ddb576 support rust sglang server (#29799) 2026-07-31 11:56:31 -07:00
Mick 22faf9fef8 embedding: centralize capabilities and complete OpenAI compatibility (#32481) 2026-07-30 10:28:52 +08:00
Mike Qiuliusy58Alex Nails
 		f9c14e6bd4 [FEAT] Support fast engine recovery through weight cache (#27139) 
Signed-off-by: Michael Qiu <qiudayu.qdy@antgroup.com>
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
 2026-07-25 14:31:21 +08:00
Cheng Wan f5dcbe8f14 Revert RuntimeContext config-namespace reads/roles (#31813–#31817) (#32100) 2026-07-22 11:52:41 -07:00
ishandhananiandConnor Carpenter 21065bc862 feat: add native gRPC sidecar module launcher (#31076) 
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Connor Carpenter <connorc@nvidia.com>
Co-authored-by: Connor Carpenter <connorc@nvidia.com>
 2026-07-22 06:39:08 -07:00
Cheng Wan 11a4c2d057 config: read resolved config via namespace accessors (#31814) 2026-07-22 01:18:05 -07:00
Mick 6a25dd7b5f fix: warm up Kimi VLM vision encoder at startup (#31298) 2026-07-20 08:50:57 +08:00
Baizhou Zhang 7a03d30149 Support MiMo V2.5 with zigzag context parallelism (#29972) 2026-07-19 02:36:40 -07:00
Yoray Zack 77d23a796e [1/N] elastic-ep: Add runtime EP scale-up (#30164) 2026-07-16 15:53:44 -07:00
Jorge António 26cb0fcdda Empty _REQ_TYPES_WITH_OPAQUE_FIELDS on the msgpack IPC path (#29465 Task 4) (#30182) 2026-07-15 14:55:06 -07:00
weireweireandweireweire f2c875d1c8 [PD] Route PD server warmup to every DP rank (#30748) 
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
 2026-07-15 15:59:41 +08:00
Lianmin Zhengandtanujtiwari1998 cfc66e05c5 [tokenizer] Support pluggable tokenizer worker class in multi-tokenizer mode (#30630) 
Co-authored-by: tanujtiwari1998 <168470992+tanujtiwari1998@users.noreply.github.com>
 2026-07-09 17:31:54 -07:00
cctryandcctry 1959335997 refactor(load-snapshot): build LoadSnapshot directly, drop legacy get_loads IPC (#30525) 
Co-authored-by: cctry <cctry@fb.com>
 2026-07-09 05:58:10 -07:00
Xinyuan Tong 074bb928f0 Move template manager files under parser; update CODEOWNERS (#26052) 2026-07-08 16:47:36 -07:00
Cheng Wan b14f7b4f75 [refactor] Move model-capability adjustments into the resolution pipeline (#30299) 2026-07-07 21:26:55 -07:00
Alex NailsandClaude Opus 4.7 3d2e7cc601 [gRPC] Native server: launcher + HTTP + server args wiring (3/4) (#23508) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-07-07 14:57:25 -07:00
Xinyuan Tong 854b46be99 feat(parser): resolve special-token suffix at runtime for compatibility (#29920) 2026-07-05 00:13:46 +08:00
Jialin Ouyang 40594bd381 [passthrough] engine: zstd request-body decompression + header overrides (#29684) 2026-06-30 23:48:56 -07:00
jonah-bermanandXinyuan Tong 2f34dbe372 Add native Exa-backed web_search support (#29342) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-06-27 14:41:09 +01:00
Rain JiangandLianmin Zheng be1930133a Convert IPC dataclasses to msgspec.Struct with opt-in msgpack transport (#28688) 
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
 2026-06-26 12:04:03 -07:00
Lianmin Zheng 118d6b2e5e [Cleanup] Style and type annotation improvements extracted from #28688 (#29224) 2026-06-25 12:51:15 -07:00
Lianmin Zheng 6c839368e0 [Cleanup] IPC struct renames, better typing, and SenderWrapper removal (#29214) 2026-06-24 17:25:24 -07:00
Lianmin Zheng d5c566e59b Extract profile request cleanups (#29098) 2026-06-24 11:22:58 -07:00
Lianmin Zheng 54b9b9d0c9 Remove threading atexit monkey patch (#28812) 2026-06-20 18:30:50 -07:00
Rain Jiang ef01618dfb support MPServer and embedded server for granian to enable muti tokenizer worker (#28573) 2026-06-18 17:59:12 -07:00
Liangsheng Yin 14f6348524 [Fix] Demote OpenAIServingResponses init failure log to one-line WARNING (#28349) 2026-06-15 16:58:33 -07:00
ybyang 50993554d8 fix(health): make health-check rid unique across tokenizer workers (#28143) 2026-06-14 00:01:52 -07:00
Xinyuan TongKai-Hsun ChenKristin CowalcijkaerostaglaziermagBlake LeddenPanJasonLeoyzenkennyu
 		85712fa5b0 Fix Responses API request handling (#25881) 
Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org>
Co-authored-by: Kristin Cowalcijk <kristincowalcijk@gmail.com>
Co-authored-by: aerosta <63026763+aerosta@users.noreply.github.com>
Co-authored-by: glaziermag <glaziermag@users.noreply.github.com>
Co-authored-by: Blake Ledden <blake.ledden@gmail.com>
Co-authored-by: PanJason <pyyjason@gmail.com>
Co-authored-by: Leoyzen <leoyzen@gmail.com>
Co-authored-by: kennyu <966806+kennyu@users.noreply.github.com>
 2026-06-12 14:47:55 -07:00
+3
Xinyuan TongJairo David Campaña RoseroKaran BansaleasonYufeng HeqingchanghanClaude Opus 4.7Ajay AnuboluRavitez DondetiRatish PXiaoshuai ZhangRicardo-M-LXinyuan Tong
 		b3270264e4 Fix Anthropic Messages API compatibility (#25876) 
Co-authored-by: Jairo David Campaña Rosero <jairocampana10001@gmail.com>
Co-authored-by: Karan Bansal <3264937+karanb192@users.noreply.github.com>
Co-authored-by: eason <85663565+mango766@users.noreply.github.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: qingchanghan <17794466+qingchanghan@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ajay Anubolu <124525760+AjAnubolu@users.noreply.github.com>
Co-authored-by: Ravitez Dondeti <13931987+dondetir@users.noreply.github.com>
Co-authored-by: Ratish P <114130421+Ratish1@users.noreply.github.com>
Co-authored-by: Xiaoshuai Zhang <15795935+jetd1@users.noreply.github.com>
Co-authored-by: Ricardo-M-L <69202550+Ricardo-M-L@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuan.tong@radixark.ai>
 2026-06-12 14:46:57 -07:00
Lianmin Zhengcctrycctry
 		88a7b0fd30 Classify malformed-multimodal rejects as invalid_request (#27451) 
Co-authored-by: cctry <cctry@meta.com>
Co-authored-by: cctry <cctry@fb.com>
 2026-06-06 10:19:22 -07:00
EduardDurech e9dbbd19e9 [model] Apertus Tool/Function and Reasoning parser (#25100) 2026-06-06 00:04:31 -07:00
Liangsheng Yin 578f232e5e Fix trace_modules gate disabling default trace contexts (#27173) 2026-06-03 14:03:42 -04:00
Lianmin Zheng 53b8378307 Fix weights_checker checksum for 0-dim tensors and multi-GPU (#26863) 2026-05-31 21:27:03 -07:00
R0CKSTARAlex NailsClaude Opus 4.6
 		a952e9174f [MLX] Support Qwen3.5 (dense) Model (#25754) 
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-05-30 17:05:02 +08:00
Sam HandYihao Wang a95b4e2e09 [Feature] WebSocket streaming audio input for ASR (#22848) 
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com>
 2026-05-27 22:44:55 +08:00
Kangyan-ZhouandClaude Opus 4.7 085777210c feat(kv-events): expose structured KV-event publisher block on /server_info (#25844) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-23 01:59:45 +08:00
Randall LinandCursor 791a2f057f Add overridable hooks for custom chat serving implementations (#25807) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-21 11:21:25 +08:00
Lianmin Zheng f21fe6ad4d Fix PD disaggregation warmup: set request_name and improve error logging (#25542) 2026-05-18 09:49:56 -07:00
lw9527 8d27ce7371 Optimize uvicorn startup command (#25041) 2026-05-12 00:28:13 -07:00
huangtingwei 27445f9836 Add ChatCompletionRequest-style support to /v1/tokenize (#23981) 2026-05-06 18:35:20 -07:00
fzyzcjy c4c5541618 Support getting checksums in weight checker (#24537) 2026-05-06 22:59:28 +08:00
ishandhanani 5b7ce417d0 [P/D disagg] - support decode side radix cache (#19746) 2026-05-01 21:55:34 +08:00
Liangsheng Yin 573e12a7fc Merge /get_load into /v1/loads (#23010) 2026-04-17 13:36:51 -07:00