minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity
13,736 Commits 2 Branches 0 Tags
c95179bc85e8e8814f3e28253252270569b20f3a
Commit Graph
46 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Khoa Pham c95179bc85 [Spec] Fuse small kenrels under gather_spec_extras (#27233) 2026-06-08 15:02:04 -07:00
Khoa PhamandClaude Opus 4.8 bf172c492a Cookbook for QAT (#27396) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-05 11:18:11 -07:00
Khoa Pham b5560ffc36 Fix flashinfer autotune oom glm51 (#24195) 2026-06-02 23:28:57 -07:00
Khoa PhamandClaude Opus 4.8 6d53615699 [Gemma4] Use hard GSM8K accuracy floor for 31B MTP test (#27101) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-03 01:57:12 -04:00
Khoa Pham a711c57a32 [Spec] Fix Gemma 4 MTP with trtllm_mha crash issue (#26966) 2026-06-02 14:37:51 -07:00
Khoa Pham 22bb9a6421 test: disable test_gemma4_mtp_26b_a4b_extra from CI (#27082) 2026-06-02 14:00:48 -07:00
Khoa PhamandCursor 08526c7fca [Spec] FrozenKVMTP fold assistant seed into captured draft graph (#25539) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-01 22:27:25 -07:00
Khoa Pham da01f2974e [Log] include max_token_num and hidden_dim in FlashInfer workspace init log (#26605) 2026-06-01 13:26:52 -07:00
Khoa PhamandClaude Opus 4.8 cb8a103b81 chore: add @pyc96 as codeowner for FrozenKVMTP module (#26953) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-01 13:06:58 -07:00
Khoa PhamandCursor 621a79728c test: stabilize Gemma4 26B-A4B MTP GSM8K test with deterministic inference + tuned threshold (#26653) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-28 20:58:48 -07:00
Khoa PhamandCursor 93445e6359 [spec decoding] support kimi-k2.6-eagle3.1-mla draft (#26506) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-28 14:26:39 -07:00
Khoa Pham a38620b92c chore: add @pyc96 as codeowner for gemma4 files (#26601) 2026-05-28 12:37:50 -07:00
Khoa PhamandClaude Opus 4.7 9040feebd8 [Gemma4] Add test for MTP models (#24552) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-27 21:36:28 -07:00
Khoa PhamandClaude Opus 4.7 b0ce16d0c5 [CP] 1/N: Support MLA Prefill Context Parallel (#23292) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-23 03:07:09 -07:00
Khoa PhamandClaude Opus 4.7 f829cafa3e [perf] skip add_special_tokens=False kwarg on chat-template tokenize for slow tokenizers (#25953) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-21 22:45:54 -07:00
Khoa PhamandClaude Opus 4.7 b29e41e8b3 Respect user override for Gemma4 attention backend (#25547) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-18 10:46:25 -07:00
Khoa PhamandClaude Opus 4.7 856f13a916 chore(codeowners): add kpham-sgl to frozen_kv_mtp files (#25211) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 20:01:05 -07:00
Khoa PhamandClaude Opus 4.7 371cb2ade2 [Bench] Add MEM profile activity to bench_serving (#25026) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 16:22:11 -07:00
Khoa Pham c665edec6e [env] Make max KV chunk capacity configurable via SGLANG_MAX_KV_CHUNK_CAPACITY (#25120) 2026-05-12 22:37:45 -07:00
Khoa Pham 49ac447c94 [bench] Agentic support for bench_serving.py (#25016) 2026-05-12 16:00:45 -07:00
Khoa Pham 1610aa77ab Reduce gemma4 moe deterministic test runtime (#24754) 2026-05-08 20:46:56 -07:00
Khoa PhamandPengyu Chen d2c1034163 [Gemma 4] Adding MTP support (#24436) 
Co-authored-by: Pengyu Chen <pychen96@gmail.com>
 2026-05-07 14:08:41 -07:00
Khoa Pham 932d89690a Gemma4-mtp cookbook (#24433) 2026-05-05 23:04:41 +08:00
Khoa Pham d22853480d Fix deterministic inference on models with SWAKVPool (#24395) 2026-05-05 20:20:46 +08:00
Khoa Pham ef2b1b6d89 Fix flashinfer workspace OOM (#24172) 2026-05-04 01:26:35 -07:00
Khoa PhamandClaude Opus 4.7 c2d60c3667 chore(codeowners): add @kpham-sgl as codeowner for ngram files (#24349) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-03 23:27:33 -07:00
Khoa Pham ddcacaf1bd Fix failing test_nvidia_nemotron_3_nano by fixing test_grouped_topk (#23874) 2026-04-28 15:03:58 -07:00
Khoa PhamandClaude Opus 4.7 826f2d0620 chore(codeowners): add @kpham-sgl as owner for gemma4 files (#23916) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 11:43:32 -07:00
Khoa Pham 5a0eea8ac5 [CI] Adding Gemma 4 to Nightly CI (#22408) 2026-04-16 19:30:16 -07:00
Khoa PhamandClaude Opus 4.6 1f8df97054 Fix broken streaming response with --incremental-streaming-output (#22549) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-12 15:05:58 -07:00
Khoa Pham aeeff58cd4 [Spec][Ngram] Clean up unused stateless batchMatch (#22487) 2026-04-10 21:52:56 -07:00
Khoa PhamandClaude Opus 4.6 04bd8e1218 [Spec][Ngram] Return token counts in list_external_corpora API (#22471) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-10 21:50:02 -07:00
Khoa Pham f127d67823 [Spec][Ngram] Misc enhance support for multiple SAMs (#22294) 2026-04-08 19:56:23 -07:00
Khoa Pham 12272b6791 [Spec][Ngram] 6/N: Load an external corpus and construct a Suffix Automaton (#21425) 2026-04-06 00:11:14 -07:00
Khoa Pham b2008bf9e0 [Spec][Ngram] 5/N: Store and advance anchor match state across decode steps (#21243) 2026-04-05 22:21:05 -07:00
Khoa PhamandClaude Opus 4.6 cd75d54fc5 [Bugfix] Fix CUDA graph replay issues in trtllm_mla draft_extend (#21987) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-03 01:45:13 -07:00
Khoa Pham 2b5aed94f5 Remove maxItems=1 restriction when tool_choice is specified (#20208) 2026-04-03 02:35:24 +00:00
Khoa Pham f836658077 [Spec][Ngram] 4/N: Remove max_match_window_size and min_match_window_size, matching all suffixes of the Trie (#21225) 2026-04-01 22:09:46 -07:00
Khoa Pham 153359b4dd Multi tool streaming fix (#20004) 2026-04-01 21:53:05 -07:00
Khoa Pham 8d4fca5908 [Security] 1/N: Bind ZMQ sockets to localhost to prevent unauthenticated remote access (#21435) 2026-03-26 23:33:49 -07:00
kpham-sglandClaude Opus 4.6 59cb9a9da6 [Spec][Ngram] 3/N: Fix synchronization issues in Ngram.cpp (#21186) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-23 00:37:07 -07:00
kpham-sglandClaude Opus 4.6 bc4aaab6a1 [Spec][Ngram] 2/N: Rename branch length to max trie depth (#21181) 
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-22 23:35:25 -07:00
kpham-sgl 6d160b42bb [Spec][Ngram] 1/N: Reference based Speculative Decoding refactor (#20393) 2026-03-22 00:55:10 -07:00
kpham-sglandhnyls2002 346a4131cf [Spec] Refactor NaN/OOB checks to async maybe_detect_* with env-var control (#19899) 
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
 2026-03-05 13:51:05 -08:00
kpham-sgl 376dfb03f7 Fix issue 19717 by making qo_indptr uniform strided instead of packed (#19807) 2026-03-04 15:27:10 -08:00
kpham-sgl 88290690f0 Add kpham-sgl into CI Permission list (#19819) 2026-03-03 18:55:10 -08:00