minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
18,577 Commits 2 Branches 0 Tags
ab03a8e7eb82d35907bfbeb645bf0af8b4ce290e
Commit Graph
54 Commits
Author SHA1 Message Date
Xueshen Liu ab03a8e7eb [Perf] Fork-safe import: no CUDA context at import time, lighter argument parsing (#40201) 2026-09-21 10:48:35 +08:00
Xinyuan Tong f0bf652534 Add Ling-3.0-flash-VL model support (#38526) 2026-09-17 00:19:26 +08:00
Mohammad Miadh AngkadBrayden ZhongMohammad AngkadXinyuan TongXinyuan Tong
 		6c514ab025 Force reasoning mode for GLM-5.3 chat templates (#39227) 
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Mohammad Angkad <mohammad.angkad@radixark.ai>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-09-15 13:16:26 +08:00
YousafClaudeXinyuan TongXinyuan Tong
 		593c7a900d Add granite_thinking_parser reasoning parser for Granite 4.2 (#38693) 
Signed-off-by: Yousaf Shah <yousaf.shah@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-09-11 07:18:35 -07:00
Xinyuan Tong e8a36d339c Auto-detect GLM-5.3 chat templates as glm45/glm47 parsers (#38297) 2026-09-11 09:02:44 -04:00
Cheng Wan f1a512c51c [Config] msgspec.Struct for the config tier (#38753) 2026-09-09 19:41:19 -07:00
zijiexiaandXinyuan Tong b5c9b68f03 [Kimi-K3] Recover the reply when the model skips the think channel (#37743) 
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-09-07 14:08:42 +00:00
sglang-botandsglang-bot 6252993afe chore: update CI test est_time values (#38238) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2026-09-06 17:49:41 -07:00
+2
Xinyuan TongzRzRzRzRzRzRzRShijin Zhangzanes-opsBaizhou ZhangJian Chenzijiexiaandyluo7Ehsan Akhgarikpham-sglBBufRaiden Makoto
 		97c6978369 GLM-5.3-Flash support (#36507) 
Co-authored-by: zRzRzRzRzRzRzR <Yuxuan.Zhang2@liverpool.ac.uk>
Co-authored-by: Shijin Zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: zanes-ops <zanes@nvidia.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Jian Chen <jianchen0311@gmail.com>
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
Co-authored-by: andyluo7 <43718156+andyluo7@users.noreply.github.com>
Co-authored-by: Ehsan Akhgari <ehsan.akhgari@gmail.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Raiden Makoto <81530826+Raiden-Makoto@users.noreply.github.com>
 2026-09-06 02:27:59 -07:00
Xinyuan TongBBufalphabetc1
 		55bf3380e0 Support Hy4-preview (#36805) 
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: alphabetc1 <2508695655@qq.com>
 2026-09-04 18:03:49 -07:00
junduandMa Mingfei 8770c1db1f [CPU][CI]: rename Xeon CPU CI suites to stage-*-intel (#37395) 
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-09-04 10:08:41 +08:00
Xinyuan Tongyhyang201Yuhao Yang
 		27b7a2dc3b [Kimi K3] Rework skipped-think fix as opt-in force_nonempty_content with streaming coverage (#34187) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
 2026-09-03 17:37:28 +08:00
Yash AkhauriXiaoyu ZhangXinyuan Tong
 		3bac084d4e [Model] Add native IFM K2 Horizon serving support (#37654) 
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-09-03 16:39:43 +08:00
Alex NailsandAlison Shao 28262c20df [CI][RFC] Replace black-jupyter with ruff-format (#37210) 
Co-authored-by: Alison Shao <a.shao@wustl.edu>
 2026-09-02 19:46:08 -07:00
Khoa PhamClaude Opus 5Xinyuan TongXinyuan Tong
 		fbf909b460 [Fix] Alpha-channel images and tool-result media ordering (port of #36507) (#37320) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-09-02 18:23:06 -07:00
Xinyuan Tongluoyuan.luo得泽翎悦羽癫tiwei.btwLiangsheng Yin文赋JustinTong
 		20621aa14b [Model] Support Ling-3.0-flash (BailingMoeV3) (#33561) 
Signed-off-by: JustinTong <justintong0323@gmail.com>
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
Co-authored-by: 得泽 <zhangkaihong.zkh@antgroup.com>
Co-authored-by: 翎悦 <vito.yy@antgroup.com>
Co-authored-by: 羽癫 <yudian.zy@antgroup.com>
Co-authored-by: tiwei.btw <tiwei.btw@antgroup.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: 文赋 <zibin.zb@antgroup.com>
Co-authored-by: JustinTong <justintong0323@gmail.com>
 2026-08-26 17:27:23 -07:00
Cheng Wan 413df1f8db config: ServerArgs holds the raw input (#36255) 2026-08-26 05:14:05 -07:00
junduZhang, MingxuMingxuZhCopilot Autofix powered by AIClaude
 		514b997e6c Register CPU CI for 17 e2e tests and partition xeon base-c suite (#35227) 
Co-authored-by: Zhang, Mingxu <mingxu.zhang@intel.com>
Co-authored-by: MingxuZh <109504044+MingxuZh@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-08-24 10:49:38 +08:00
Jason Wiemels defb2a3100 feat(openai): Accept the input_audio content part in chat completions (#33606) 2026-08-19 13:37:50 -07:00
Khoa PhamClaude Opus 5Xinyuan TongXinyuan Tong
 		307a90f6d3 Stop losing Kimi-K3 tool calls to reasoning, constraint conflicts, and truncation (#34881) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-08-18 12:42:56 -07:00
Mick e8c7dddfa0 [VLM] add content-addressed preprocessing cache infrastructure (#34398) 2026-08-13 14:09:07 +08:00
+3
Liangsheng Yinhao-cyberEnrico FalcoSvyatoslavAndreas HassellofLeoyzenChenglun Hurobellliu-devGavin.ZhuXinyuan Tongtancheng33dineshx29Kangyan Zhou
 		5899674504 [Fix] Make DeepSeek-V4 reasoning and tool-call streaming parsing chunk-invariant (#34458) 
Co-authored-by: hao-cyber <89575785+hao-cyber@users.noreply.github.com>
Co-authored-by: Enrico Falco <enrico9034@gmail.com>
Co-authored-by: Svyatoslav <85786374+slivanovich@users.noreply.github.com>
Co-authored-by: Andreas Hassellof <andreas@ombori.com>
Co-authored-by: Leoyzen <leoyzen@gmail.com>
Co-authored-by: Chenglun Hu <chenglunhu@gmail.com>
Co-authored-by: robellliu-dev <robell.liu@huawei.com>
Co-authored-by: Gavin.Zhu <gavin.z@gmicloud.ai>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: tancheng33 <garrytancheng@gmail.com>
Co-authored-by: dineshx29 <dinesh.b.offl@gmail.com>
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
 2026-08-11 20:03:28 -07:00
Eric ZhangandClaude Opus 5 c69d59395b [inkling] Render tool-result media instead of coercing content to str (#33898) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-08 14:39:19 +08:00
Cheng Wan 99cfc90658 config: retire ServerArgs.override in favour of derive() 
`ServerArgs.override(source, **fields)` was the last way to change a resolved
`ServerArgs` in place. Every remaining call-site was one of two things, and
neither wanted an in-place write:

- **A config for someone else.** A draft worker's context length, an encode
  worker's device, the compile script's watchdog, the client's port pick, a test
  fixture's backends. These already deepcopied first — the write was on the copy.
- **A launcher-stage resolution.** `resolve_auto_parsers` detected the chat
  template's parsers and wrote them back, to be inherited by the schedulers it
  spawns.

Both are "one config becomes another", so `derive(source, **fields)` returns the
variant and leaves the receiver — and any bags projected from it — untouched. It
deliberately is not `dataclasses.replace`: resolution does not re-run, because
the values being set are decided after it, from inputs it never had. Provenance
and the resolvable-field stash work as before, on the copy.

`resolve_auto_parsers` now computes the parsers and returns the config to launch
with; the detection helpers stop taking a config to mutate. `HiMambaRadixCache`
re-applied a HiCache layout normalization `__post_init__` already performs (the
same duplicate removed from `UnifiedRadixCache` in ebb1c88d23) and just goes.

With no in-place mutation left, `ServerArgs.__setattr__` raising after
resolution *is* the guarantee, so the textual writer ratchet retires and
`test_server_args_derive.py` pins the contract instead: the receiver survives
deriving, the published instance still refuses assignment, and deriving does not
publish. `SGLANG_STRICT_CONFIG_MUTATION` was already unused — the guard has been
unconditional since the mutation sweep — and goes with it.

The detection tests drop their `SimpleNamespace` stand-in for a real
`ServerArgs`; the test kit and the MLA chunk-metadata fixture publish a derived
variant instead of writing the runner's published config.
 2026-08-05 19:30:53 -07:00
Xinyuan Tonghnyls2002Liangsheng YinA-transformer
 		e2cf21b9e5 [Kimi K3] Add reasoning, tool-call, and OpenAI serving support (#33025) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: A-transformer <cl5743590921@gmail.com>
 2026-08-01 14:57:23 -07:00
Xinyuan Tong 68d442945f Flush dropped reasoning at stream end when stream_reasoning=False (#32225) 2026-07-31 08:25:54 +08:00
Ke Bao 07a087bf45 Fix Inkling tool-call parsing recovery, content handling, and streaming (#32861) 2026-07-30 14:11:09 +08:00
Willow LopezandJiminator ffd4705baa fix(reasoning): honor Poolside template thinking defaults (#32540) 
Co-authored-by: Jiminator <69131491+Jiminator@users.noreply.github.com>
 2026-07-29 21:02:19 +00:00
Shangming CaiClaude Fable 5Xinyuan Tong
 		1d350aaad3 fix(reasoning): let --enable-strict-thinking works for DeepSeek-V4 (#32400) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-07-27 18:15:47 +08:00
Ke Bao 5ab3d90b81 Fix dropped Inkling reasoning at stream end (#31787) 2026-07-20 23:23:15 +08:00
Cheng WanChunan ZengKe BaoYanbin JiangYuhao YangQiaolin YuZhichen ZengAurick QiaoJoseph
 		02236fa38c Add Inkling model support (#31681) 
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Yanbin Jiang <jybsuper@gmail.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Qiaolin Yu <qiaolin.yu@radixark.ai>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Aurick Qiao <aurick@thinkingmachines.ai>
Co-authored-by: Joseph <jk@thinkingmachines.ai>
 2026-07-19 22:57:37 -07:00
Brayden Zhong 0ee236ebdf more fixes for Nemotron 3 parser for tool call and force nonempty content (#30533) 2026-07-14 06:50:27 +08:00
Xinyuan TongXinyuan Tongzijiexia
 		0663ebc783 [minimax-m3] Split 4/4: model + VL + glue + function-call + fp8 quant + generic infra (#28715) 
Co-authored-by: Xinyuan Tong <xinyuan-tong@users.noreply.github.com>
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
 2026-07-11 11:11:06 +08:00
Liangsheng Yin 23390589f7 [misc] Remove unit test cases that fail the admission criteria (round 3) (#30713) 2026-07-09 19:42:51 -07:00
Liangsheng Yin a36c873147 [misc] Remove unit test cases that fail the admission criteria (round 2) (#30703) 2026-07-09 16:35:59 -07:00
Liangsheng Yin c53559ba10 [misc] Remove unit test cases that fail the admission criteria (#30690) 2026-07-09 15:31:28 -07:00
Cheng Wan 1f15308dca [refactor] Retire the legacy config accessor and the remaining process singletons (#30493) 2026-07-09 02:10:47 -07:00
Xinyuan Tong 074bb928f0 Move template manager files under parser; update CODEOWNERS (#26052) 2026-07-08 16:47:36 -07:00
jundu e04ed05193 [CI] reduce CPU CI scope with base-c suite (#28623) 2026-06-24 16:18:06 +08:00
Xinyuan TongKai-Hsun ChenKristin CowalcijkaerostaglaziermagBlake LeddenPanJasonLeoyzenkennyu
 		85712fa5b0 Fix Responses API request handling (#25881) 
Co-authored-by: Kai-Hsun Chen <kaihsun@apache.org>
Co-authored-by: Kristin Cowalcijk <kristincowalcijk@gmail.com>
Co-authored-by: aerosta <63026763+aerosta@users.noreply.github.com>
Co-authored-by: glaziermag <glaziermag@users.noreply.github.com>
Co-authored-by: Blake Ledden <blake.ledden@gmail.com>
Co-authored-by: PanJason <pyyjason@gmail.com>
Co-authored-by: Leoyzen <leoyzen@gmail.com>
Co-authored-by: kennyu <966806+kennyu@users.noreply.github.com>
 2026-06-12 14:47:55 -07:00
MingxuZh 21ba329dac [Xeon] CPU CI enhancement for Intel Xeon platforms (#24649) 2026-05-28 10:49:04 +08:00
Yuzhen Zhou dac78768f0 [RL][TITO] Preserve whitespace in reasoning parser outputs (#24251) 2026-05-20 19:45:09 +00:00
Liangsheng Yin b7d62bd724 [CI] Rename basic CI stage-a/b/c -> base-a/b/c for symmetry with extra CI (#25420) 2026-05-15 18:26:55 -07:00
Jimmy Shong 096ad02b06 [Model] Laguna-XS.2 Model Support (#24204) 2026-05-09 05:43:13 +08:00
sglang-botandsglang-bot da175b964d chore: update CI test est_time values (#23785) 
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
 2026-04-26 20:17:50 -07:00
Xinyuan TongpengmengQiaolin-Yuchengvjiangrussellfeng
 		6d03861476 support Hy3 preview (#23533) 
Co-authored-by: pengmeng <pengmeng@tencent.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: russellfeng <russellfeng@tencent.com>
 2026-04-24 12:03:24 -07:00
sglang-botCheng Wansglang-botCheng Wan
 		265696b176 chore: update CI test est_time values (#22565) 
Co-authored-by: Cheng Wan <chwan@rice.edu>
Co-authored-by: sglang-bot <sglang-bot@users.noreply.github.com>
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
 2026-04-10 18:15:02 -07:00
Xinyuan TongPengyu Chenkpham-sglClaude Opus 4.6Andy Luogemini-code-assist[bot]adarshxs
 		2813cb6d9a [New Model] Gemma 4 (#21952) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Pengyu Chen <pychen96@gmail.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Andy Luo <andy.luo@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: adarshxs <adarsh.shirawalmath@gmail.com>
 2026-04-06 20:24:44 -07:00
Liangsheng Yin bd6a585605 Consolidate reasoning tests into test/registered/reasoning/ (#22139) 2026-04-05 01:09:11 -07:00
Ke Bao 47f4fd275a [CI] Fix test suite names and add suite validation (#21937) 2026-04-03 23:47:17 +08:00