minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity
13,941 Commits 2 Branches 0 Tags
b3270264e4d2417bf051f5660015071aeda40120
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
+3
Xinyuan TongJairo David Campaña RoseroKaran BansaleasonYufeng HeqingchanghanClaude Opus 4.7Ajay AnuboluRavitez DondetiRatish PXiaoshuai ZhangRicardo-M-LXinyuan Tong
 		b3270264e4 Fix Anthropic Messages API compatibility (#25876) 
Co-authored-by: Jairo David Campaña Rosero <jairocampana10001@gmail.com>
Co-authored-by: Karan Bansal <3264937+karanb192@users.noreply.github.com>
Co-authored-by: eason <85663565+mango766@users.noreply.github.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: qingchanghan <17794466+qingchanghan@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ajay Anubolu <124525760+AjAnubolu@users.noreply.github.com>
Co-authored-by: Ravitez Dondeti <13931987+dondetir@users.noreply.github.com>
Co-authored-by: Ratish P <114130421+Ratish1@users.noreply.github.com>
Co-authored-by: Xiaoshuai Zhang <15795935+jetd1@users.noreply.github.com>
Co-authored-by: Ricardo-M-L <69202550+Ricardo-M-L@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuan.tong@radixark.ai>
 2026-06-12 14:46:57 -07:00
Xinyuan Tong 9f6b2339f9 docs(minimax-m3): warm-steady-state benchmark numbers (#28062) 2026-06-12 08:42:23 -07:00
Xinyuan Tong dba617f2ec doc: update docs for new model (#28060) 2026-06-12 21:14:35 +08:00
Xinyuan Tong 66ab5c9c7a fix(gateway): make sgl-model-gateway a cargo workspace so maturin 1.14 accepts the parent README (#27997) 2026-06-11 23:37:29 -07:00
Xinyuan Tong 7425bebb6c docs(cookbook): restore Gemma 4 transformers commit pin (#27321) 2026-06-04 17:43:36 -07:00
Xinyuan Tong 45a66f4088 [Docs] Update unified Text/Vision/Audio model cookbook: install + sgl-eval accuracy (#27171) 2026-06-03 09:32:08 -07:00
Xinyuan Tong fa5c8a3101 [model] support encoder-free unified Text/Vision/Audio model (#27167) 2026-06-03 23:58:06 +08:00
Xinyuan Tong 79c844527c Upgrade xgrammar to 0.2.1 (#25676) 2026-05-29 11:40:07 +08:00
Xinyuan Tong bed20249f1 fix(tool_call): reland schema type normalization (#26433) 2026-05-28 14:31:18 +08:00
Xinyuan Tong 64c7c6851b fix(tool_call): normalize non-standard JSON Schema types in tool params (#23476) 2026-05-26 15:23:59 +08:00
Xinyuan Tong 40faf44f7a [auto-detect] match Ring-2.6/Ling XML kv tool-call format via vocab signature (#25366) 2026-05-20 23:34:52 -07:00
Xinyuan TongandXinyuan Tong 52eebc82ae [Docs] MiMo-V2.5 cookbook: B200 benchmarks + multi-layer EAGLE acceptance profile + long-context reference (#25359) 
Co-authored-by: Xinyuan Tong <xinyuan.tong@radixark.ai>
 2026-05-19 23:15:21 -07:00
Xinyuan Tong 0aedc5678b loader: yield filtered MTP weights lazily to avoid OOM hang on multi-layer EAGLE (#25748) 2026-05-20 12:33:54 +08:00
Xinyuan Tong aad00b0ed8 Upgrade transformers to 5.8.1 (#25451) 2026-05-19 22:20:30 +08:00
Xinyuan Tong 7cb4669a04 [Fix] DeepSeek-V3.2: build structural tag locally to encode both wrapper and invoke layers (#25233) 2026-05-15 14:32:50 -07:00
Xinyuan Tong 33f1d3915f [NEW MODEL] Add H200 validation for Ring-2.6-1T cookbook (#25370) 2026-05-15 11:47:15 -07:00
Xinyuan Tong c3daa77e9a [NEW MODEL] Add Ring-2.6-1T cookbook (#25360) 2026-05-14 23:32:22 -07:00
Xinyuan Tong 5b589ed2e7 feat(constrained): two-phase reasoning grammar + --enable-strict-thinking (#23953) 2026-05-07 14:21:51 -07:00
Xinyuan Tong af2a2ac618 fix(function_call): handle Kimi-K2.5 bare numeric tool call IDs (#23950) 2026-05-07 14:20:02 -07:00
Xinyuan Tong d8f9d32a05 feat(reasoning): auto-detect reasoning/tool-call parser from chat template (#23952) 2026-05-07 14:19:16 -07:00
Xinyuan Tong f1395af543 fix(openai): map reasoning.enabled to thinking AND enable_thinking (#23951) 2026-05-07 14:01:35 -07:00
Xinyuan Tong 1e404afec2 fix(req_pool): bump pool.size to match actual tensor row count after #24243 (#24439) 2026-05-05 16:58:26 -07:00
Xinyuan Tong 8d1b6f0c00 Add zRzRzRzRzRzRzR to CI permissions (#24432) 2026-05-05 12:37:50 -07:00
Xinyuan Tong 989a16187d [Bench] Fix bench_serving missing reasoning_content stream chunks (#23954) 2026-04-30 15:00:27 -07:00
Xinyuan Tong 1376761841 fix(moe): repair dead import in fused_moe_native after MoE refactor (#24069) 2026-04-29 11:14:52 -07:00
Xinyuan Tong 4cf109bbd1 debug followup (#24058) 2026-04-29 23:03:27 +08:00
Xinyuan Tong 1279ae0787 Bugfix (#24027) 2026-04-29 21:13:51 +08:00
Xinyuan Tong 832b4f59ed [Bench] fix MMMU answer-extraction regex dropping multi-line responses (#23864) 2026-04-29 14:48:49 +08:00
Xinyuan Tong e458a9248f docs: enable MiMo V2.5 MTP cookbook path (#23945) 2026-04-28 10:22:19 -07:00
Xinyuan Tong 3fce8f2009 [Docs] add cookbook for Ling-2.6 family (#23947) 2026-04-29 00:42:04 +08:00
Xinyuan Tong f34222da1b [Docs] add cookbook for MiMo-V2.5 family (#23851) 2026-04-28 01:38:41 +08:00
Xinyuan Tong 96b0c64c88 Add docs_new code owner (#23855) 2026-04-27 10:31:35 -07:00
Xinyuan Tong e5198386bd Upgrade transformers from 5.5.4 to 5.6.0 (#23525) 2026-04-26 22:33:54 -07:00
Xinyuan TongpengmengQiaolin-Yuchengvjiangrussellfeng
 		6d03861476 support Hy3 preview (#23533) 
Co-authored-by: pengmeng <pengmeng@tencent.com>
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
Co-authored-by: chengvjiang <chengvjiang@tencent.com>
Co-authored-by: russellfeng <russellfeng@tencent.com>
 2026-04-24 12:03:24 -07:00
Xinyuan Tong 4868e367f8 docs: add Hunyuan 3 Preview cookbook (#23532) 2026-04-23 02:44:47 -07:00
Xinyuan Tong de962f3274 docs(cookbook): add Qwen3.6-27B dense variant (#23486) 2026-04-23 01:22:46 +08:00
Xinyuan TongClaude Opus 4.7Ma Mingfei
 		f63def8510 [XPU] Fix DeepSeek-OCR tests under transformers 5.x (#23044) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
 2026-04-21 14:57:56 +08:00
Xinyuan Tong 082eaed0a4 test: fix flaky required function calling assertion (#22890) 2026-04-16 09:44:26 -07:00
Xinyuan Tong 34fef07a15 Upgrade transformers to 5.5.3 and refactor hf_transformers_utils into subpackage (#21569) 2026-04-15 20:03:44 -07:00
Xinyuan Tong 3f60df8012 [Test] Fix flaky test_function_call_specific by adding strict=True (#22624) 2026-04-12 01:22:57 -07:00
Xinyuan Tong 9a4e8089ff [Whisper] Batch encoder forward for concurrent prefill requests (#22361) 2026-04-12 14:15:14 +08:00
Xinyuan Tong 7c6db40540 Fix tool call constrained decoding and parsing for models with native formats (#21593) 2026-04-10 20:37:23 -07:00
Xinyuan TongPengyu Chenkpham-sglClaude Opus 4.6Andy Luogemini-code-assist[bot]adarshxs
 		2813cb6d9a [New Model] Gemma 4 (#21952) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Pengyu Chen <pychen96@gmail.com>
Co-authored-by: kpham-sgl <khoa.pham@radixark.ai>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Andy Luo <andy.luo@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: adarshxs <adarsh.shirawalmath@gmail.com>
 2026-04-06 20:24:44 -07:00
Xinyuan Tong 87a2768269 VLM: change default mm-attention backend from triton_attn to fa4 (on blackwell) (#21595) 2026-04-01 14:29:59 +08:00
Xinyuan Tong ced69c9f84 feat: enable CUDA graph and timestamp for the whisper model(#21190) 2026-03-29 01:46:03 +08:00
Xinyuan Tong a02cff7f2b [Fix] Patch is_flash_attn_2_available for flash-attn-4 in VLM input format test (#20946) 2026-03-19 13:00:51 -07:00
Xinyuan TongAlex NailsDimitrios Bariamisdbari
 		6b8a6545b2 Add Mistral Small 4 (Pixtral) support (#20708) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Alex Nails <alexnails@radixark.ai>
Co-authored-by: Dimitrios Bariamis <12195802+dbari@users.noreply.github.com>
Co-authored-by: dbari <dbari@users.noreply.github.com>
 2026-03-18 14:15:32 -07:00
Xinyuan TongKangyan-ZhouAlison ShaoMick
 		d1e95af282 Upgrade transformers==5.3.0 (#17784) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
Co-authored-by: Alison Shao <alisonshao@mac.lan>
Co-authored-by: Mick <mickjagger19@icloud.com>
 2026-03-18 13:50:43 -07:00
Xinyuan Tong c330b687a1 [Bugfix] Fix GLM-4.6V vision regression in glm4v_moe and glm_ocr (#20463) 2026-03-13 21:48:28 -07:00
Xinyuan Tong 11d9c36c2f Replace soundfile+torchaudio with torchcodec AudioDecoder in load_audio (#20190) 2026-03-10 17:26:29 +00:00
Xinyuan Tong d116a8cd94 [Bugfix] Fix load_audio: mono before resample + use torchaudio (#20054) 2026-03-09 19:24:20 +08:00
Xinyuan TongClaude Opus 4.6BakerBunker
 		4a757990a1 [VLM] Replace decord with torchcodec for video decoding (#20055) 
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: BakerBunker <17872844+BakerBunker@users.noreply.github.com>
 2026-03-09 19:23:49 +08:00
Xinyuan Tong 9ebffef1ef [FIX] NSA backend page_table overflow in speculative decoding target_verify (#19016) 2026-03-05 16:04:58 -08:00
Xinyuan Tong dbf1247fe0 Add KimiK2Detector with tool interruption support (#19696) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-03-03 14:04:49 +08:00
Xinyuan TongMahmoudAshraf97gemini-code-assist[bot]
 		581bf53e03 Whisper model support & /v1/audio/transcriptions endpoint & benchmark (#16983) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: MahmoudAshraf97 <hassouna97.ma@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-23 17:28:37 -08:00
Xinyuan Tong 677b66af80 fix KimiK2Detector regex patterns with re.DOTALL (#19120) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-02-21 22:13:08 +08:00
Xinyuan Tong 4a362a0e04 fix tool handling in OpenAIServingChat (#18996) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-02-21 22:07:09 +08:00
Xinyuan Tong cc451671b5 [FEAT] Add Anthropic compatible API endpoint (#18630) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-02-21 19:37:38 +08:00
Xinyuan TongXiaoyu ZhangBBuf
 		398b81f78c Support GlmMoeDsaForCausalLM (#18521) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Signed-off-by: BBuf <1182563586@qq.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: BBuf <1182563586@qq.com>
 2026-02-10 15:20:10 +08:00
Xinyuan TongXiaoyu ZhangBaizhou Zhang
 		e8a2c13380 Deepseekv32 compatibility with transformers v5 (#18297) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
 2026-02-10 14:50:40 +08:00
Xinyuan Tong 0b4d4f2838 Fix MMLU benchmark to auto-download data and resolve path issue (#18486) 2026-02-09 10:40:40 -05:00
Xinyuan Tong 9409c43593 Fix flaky tool calls in the Kimi K2.5 model (#17914) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-01-28 20:58:16 -08:00
Xinyuan Tong 3c34d2c3eb [FIX] kimi_k2 reasoning parser (#17901) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-01-28 19:47:09 -08:00
Xinyuan Tong 37c04c2245 fix: Refactor register_image_processor to use kwarg instead of positional arg (#17685) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-01-24 15:31:01 -08:00
Xinyuan Tong 2069050d3f fix: Handle multiple named chat templates in HuggingFace tokenizers (#17236) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2026-01-18 17:20:04 +08:00
Xinyuan Tong ae434f7821 Move limit-mm-data-per-request to make code clean (#15775) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-24 10:59:39 -08:00
Xinyuan Tong 47cdb65a45 fix: update argument extraction in R1 chat template (#15547) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-21 09:18:49 +08:00
Xinyuan Tong 0a346d3bd9 feat: Add limit-mm-data-per-request argument to server arguments (#15418) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-20 10:44:47 -08:00
Xinyuan TongmomaekMuqi Li
 		41683536d3 feat: DeepSeek-V3.2 Streaming tool call output (#15278) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: momaek <momaek17@gmail.com>
Co-authored-by: Muqi Li <muqi1029@gmail.com>
 2025-12-17 17:43:58 -08:00
Xinyuan Tong 96705514bd fix: dpskv32 chat history processing, default drop_thinking to true (#15064) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-13 12:16:04 -08:00
Xinyuan Tong 8bf10e71cd remove dpsk3.2 sys prompt (#14923) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 20:26:13 -08:00
Xinyuan Tong 4885f8b9c1 fix: handle Jinja2 template errors as client errors in OpenAIServingChat (#14748) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 17:06:26 -08:00
Xinyuan Tong 9975acf50f [refactor] Update reasoning parameter to require_reasoning (#14922) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:55:59 -08:00
Xinyuan Tong fd1ebbb0d6 update mistral detector (#14921) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-12-11 16:43:36 -08:00
Xinyuan TongandYueming Yuan 6d37e70883 ministral3 (#14251) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Yueming Yuan <yy28@illinois.edu>
 2025-12-04 14:31:26 -08:00
Xinyuan Tong a355794905 Expend compatibility check for all quantized MoE models (#13465) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-11-20 01:24:27 +08:00
Xinyuan TongalisonshaoMickBaizhou ZhangKangyan-Zhou
 		b1c688fba2 refactor: cleanup vision attention related codes (#13228) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: alisonshao <54658187+alisonshao@users.noreply.github.com>
Co-authored-by: Mick <mickjagger19@icloud.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
 2025-11-16 22:44:40 +08:00
Xinyuan Tong 4983b7e7aa Fix strict level setting for Kimi K2 tool calls when not explicitly set (#13077) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-11-12 00:01:42 -08:00
Xinyuan Tong d2a8f71c2f [feat] Add SGLANG_TOOL_STRICT_LEVEL for tool-call behavior control (#12423) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-11-01 13:15:02 -07:00
Xinyuan Tong 0bfe1d145c fa3 & trtllm_mha spec overlap (#11874) 2025-10-31 17:38:13 -07:00
Xinyuan Tongandxinyuant 684864814b Feat: deepseek-ocr logits processor (#12415) 
Co-authored-by: xinyuant <xinyuant@usc.edu>
 2025-10-31 23:35:22 +08:00
Xinyuan TongandCursor Agent 82cfcd3bb8 [Refactor] tuning_fused_moe for MLLM and small refactor (#11224) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2025-10-31 08:54:14 +08:00
Xinyuan Tong 729f612dc6 Update openai package version to 2.6.1 (#12222) 2025-10-28 11:23:40 +08:00
Xinyuan Tong 3f4cc0aff0 Remove description for --enable-beta-spec argument (#12177) 2025-10-26 16:49:39 -07:00
Xinyuan Tong b9fb74f3bc fix: bench_serving ITL calculation when using spec-decoding (#12064) 2025-10-24 17:02:44 +08:00
Xinyuan Tong 1f106ee365 [grammar] Avoid server crash when grammar backend is None (#11401) 2025-10-09 18:38:10 -07:00
Xinyuan Tong c4d77774e1 update sampling_params documentation with defaults (#11315) 2025-10-07 18:36:26 -07:00
Xinyuan Tong e3c7f09146 Update tool parser and related documentation (#11223) 2025-10-07 11:03:40 -07:00
Xinyuan Tong fd8a0b29c0 fix: correct scale parameter remapping logic in Llama4ForConditionalGeneration (#11282) 2025-10-06 17:28:23 -07:00
Xinyuan Tong 0cd1996eae feat: add shortcut detection for multimodal templates in Jinja format (#11209) 2025-10-06 04:13:17 +00:00
Xinyuan Tongandyhyang201 652c24a653 Update transformers package version to 4.57.0 (#11222) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
 2025-10-05 23:45:14 +00:00
Xinyuan TongandMick a9ce2bcb3c [Doc] Update multimodal language models documentation (#11111) 
Co-authored-by: Mick <mickjagger19@icloud.com>
 2025-09-30 22:10:31 -07:00
Xinyuan Tong 1193f13181 fix: KimiK2Detector Improve tool call ID parsing with regex (#10972) 2025-09-30 17:44:44 -07:00
Xinyuan Tongyhyang201Yangmin LiYuan Luogemini-code-assist[bot]
 		62e2e99db6 fix: make inference deterministic for large TP (#10930) 
Co-authored-by: yhyang201 <yhyang201@gmail.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Yuan Luo <yuan.luo@hotmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-27 02:46:45 -07:00
Xinyuan Tong 6088548216 Update CODEOWNERS to include JustinTong0323 in FC (#10939) 2025-09-25 22:55:56 -07:00
Xinyuan Tong 71f24ef8f6 feat: add cache_salt support to request (#10718) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-09-23 23:30:25 -07:00
Xinyuan TongandQiaolin Yu aab35bccb4 fix: draft model IMA by overide max_positional_embeddings (#10787) 
Co-authored-by: Qiaolin Yu <qy254@cornell.edu>
 2025-09-23 12:56:16 -07:00
Xinyuan Tong 12d6cf18f0 Refactors radix cache for extra key support (#10317) 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
 2025-09-22 02:16:16 +08:00
Xinyuan Tongandgemini-code-assist[bot] 6f4676ef85 fix: tool parse in large streaming chunk beginning with normal content (#10397) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-12 22:29:35 -07:00
Xinyuan Tong f3b5db6ee8 Feat: support disable tool parser (#10184) 2025-09-10 14:03:55 -07:00