minke.yu/sglang
1
0
Fork 0
Code Issues Pull Requests Actions 11 Packages Projects Releases Wiki Activity
18,687 Commits 2 Branches 0 Tags
b081dd3d234b30029fabba7dc358d76834a40e3a
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Khoa Pham bc22e1de9e [DSpark] Fix draft CUDA graph stream explosion (#40658) 2026-09-21 23:05:45 -07:00
Khoa PhamandQiaolin Yu 018b73c7a0 [PD] Pack draft KV head slices for DCP transfers (#40500) 
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
 2026-09-21 21:11:27 -07:00
Khoa Pham c4d3770a68 [Kimi K3] Fix CUDA graph stream explosion (#40640) 2026-09-21 17:49:50 -07:00
Khoa Pham b3e4d198af [PD] Bound cached-prefix DCP transfers by pack capacity (#40376) 2026-09-21 01:17:15 +08:00
Khoa PhamandCursor 10b0bcfd18 [PD] Allow decode radix cache and HiCache L1/L2 with DCP (#40263) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-18 17:54:35 -07:00
Khoa Pham b98a2d1096 [Docs] GLM-5.3-Flash cookbook: temporarily remove the DCP option (#40036) 2026-09-17 16:17:25 -07:00
Khoa Pham e4a6b090f4 [CI] Run test_unified_radix_cache_kl_dcp on cutedsl_mla with bf16 KV cache (#39553) 2026-09-15 00:35:01 -07:00
Khoa PhamandClaude Fable 5.1 2c37b90ad6 [Fix][DCP] Localize widened KV ids in MLA retraction CPU backup/restore (#39487) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-15 13:33:41 +08:00
Khoa PhamandClaude Fable 5.1 6ba96d329f [DCP] Resolve --dcp-comm-backend to fi_a2a/a2a by default for every model (#39165) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-11 23:48:42 -07:00
Khoa PhamandClaude Fable 5.1 0d08668821 [Cookbook] Kimi-K3: keep DCP under HiCache L1+L2 with DSPARK (#39190) 
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-11 23:14:24 -07:00
Khoa PhamClaude Fable 5Cursor
 		a207786205 [PD] Transfer the DCP-replicated DSPARK draft KV in DCP1->DCP-N relayouts (#37709) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-11 17:15:18 -07:00
Khoa PhamandClaude Opus 5 cf3173aeb9 [Perf] Walk the radix tree by offset instead of re-slicing token storage (ported from #36507) (#37324) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-03 11:13:03 +08:00
Khoa PhamClaude Opus 5Xinyuan TongXinyuan Tong
 		fbf909b460 [Fix] Alpha-channel images and tool-result media ordering (port of #36507) (#37320) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-09-02 18:23:06 -07:00
Khoa PhamandClaude Opus 5 c66a285c94 [Kernel] GLM 5.3 Flash related kernels (ported from #36507) (#37477) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-01 22:17:09 -07:00
Khoa Pham 97744189b8 [Kernel] Raise shape limits in shared FLA and MoE kernels (ported from #36507) (#37317) 2026-08-31 18:53:46 -07:00
Khoa PhamCursorClaude Opus 5
 		3760296be8 [PD] Pack DCP1→DCP-N PD KV transfers into dest-contiguous RDMA blocks (#35762) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-28 21:41:30 -07:00
Khoa Pham 78d36f5f62 fix: kill_process_tree waits for the reap by default (#36589) 2026-08-27 02:34:05 -07:00
Khoa PhamandClaude Opus 5 0c84eaba7f [CI] Grant no-cooldown and tag permissions to seven contributors (#36445) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-25 23:23:38 -07:00
Khoa PhamandClaude Opus 5 0ee3749e07 [Fix] Read the granite sinks dtype from the exec bag, not the legacy global shim (#35921) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-21 18:13:18 -07:00
Khoa PhamClaude Opus 5Ke Bao
 		eac91ac362 [Fix] Land the decode mamba checkpoint depth on the tree page under DCP (#35412) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
 2026-08-20 12:15:42 -07:00
Khoa PhamandCursor 0e4a09480c [HiCache] Support DCP with DSpark (#35221) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-19 01:42:19 -07:00
Khoa PhamClaude Opus 5Xinyuan TongXinyuan Tong
 		307a90f6d3 Stop losing Kimi-K3 tool calls to reasoning, constraint conflicts, and truncation (#34881) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-08-18 12:42:56 -07:00
Khoa PhamandClaude Opus 5 f44a130c5e [DCP] Drop the prefill index-selection syncs by taking each rank's rows by stride (#35084) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-17 19:55:51 -07:00
Khoa PhamandClaude Opus 5 0fb040cbeb [DCP]Localize HiCache DCP indices once per transfer, not per layer (#34889) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-16 21:54:51 -07:00
Khoa PhamandClaude Opus 5 3adbbec2fd [MoE] Route every trtllm-gen MoE call site through one PDL guard (#34789) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-14 22:09:03 -07:00
Khoa PhamandClaude Opus 5 f3beb2c529 [CI] Disable the prefill CUDA graph on the P worker of test_kimi_linear_pd_dcp4 (#34779) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-13 15:26:21 -07:00
Khoa PhamandClaude Opus 5 81fe452810 [DCP] Share one pack kernel between both a2a backends (#34651) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-13 15:06:02 -07:00
Khoa PhamandCursor 652a2709d1 [Docs] Add decode context parallelism to advanced features (#34654) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-13 13:51:57 -07:00
Khoa PhamandClaude Opus 5 cfdfd31826 Add thanhhao98 to CI_PERMISSIONS.json (#34649) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-12 19:02:03 -07:00
Khoa PhamandClaude Opus 5 6c6294b7be [DCP] Fuse the a2a pack/unpack copies in the MLA LSE reduce (#34614) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-12 16:04:46 -07:00
Khoa PhamandClaude Opus 5 0967885121 [DCP] Drop two per-layer launches from the MLA target-verify path (#34240) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-10 17:11:03 -07:00
Khoa Pham ee3ee8393c fix: preserve GQA head mapping in Triton DCP prefill (#34161) 2026-08-09 18:52:18 -07:00
Khoa PhamandClaude Opus 5 ce1b9f88b6 config: derive the runner's DCP topology from its ParallelState (#34133) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-09 01:18:24 -07:00
Khoa PhamandClaude Opus 5 07297049e9 config: route DCP topology reads through get_parallel() (#33925) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-07 14:53:54 -07:00
Khoa PhamandClaude Opus 5 beabc5949b Enable MoE deferred finalize by default and drop its expert_weights dtype workaround (#33618) 
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-05 17:56:47 -07:00
Khoa Pham 5f79cf3511 [DCP] Match the replicated draft KV pool's page granularity to its allocator (#33348) 2026-08-05 11:40:28 -07:00
Khoa Pham 91fae8a72c [DCP] Bound a request by the aggregate KV pool, not one rank's share (#33448) 2026-08-03 19:36:25 -07:00
Khoa PhamCursorClaude Opus 5
 		131bd51b01 [Spec] Add trtllm_mha support for Gemma 4 MTP draft attention backend (#25545) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-08-01 16:16:04 -07:00
Khoa Pham 1496bfee93 [Kimi] Support DCP + DSpark (ported from kimi-k3 branch) (#32828) 2026-07-31 17:39:00 -07:00
Khoa PhamandYangmin Li 2573190b93 feat: support Kimi Linear PD disaggregation with DCP (#32837) 
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
 2026-07-31 02:14:09 -07:00
Khoa PhamandClaude Opus 4.8 7a896215e7 [CP] Migrate MLA prefill CP (DeepSeek V3) to CP-v2 zigzag strategy (#31619) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-17 22:37:53 -07:00
Khoa PhamandCursor 96dd96c02b [DCP] Auto-disable tc_piecewise and breakable prefill CUDA graphs under DCP (#31532) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-16 22:17:50 -07:00
Khoa Phamandkpham-sgl d310fce85f [Fix] Update PR #25015 revert for fused topk=1 draft postprocess (#31519) 
Co-authored-by: kpham-sgl <kpham-sgl@users.noreply.github.com>
 2026-07-16 21:51:16 -07:00
Khoa Phamandkpham-sgl ead703815e [DCP] Enable decode context parallel for Kimi K2.5 NVFP4 (#31514) 
Co-authored-by: kpham-sgl <kpham-sgl@users.noreply.github.com>
 2026-07-16 21:50:42 -07:00
Khoa PhamandClaude Fable 5 dc60f65661 chore: bump tokenspeed_mla to 0.1.8 (#31385) 
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-15 22:37:54 -07:00
Khoa Pham f49cbbd67d Fix GLM/DeepSeek NVFP4 + flashinfer_trtllm long-context "!!!!" collapse (NaN routing) (#31001) 2026-07-13 12:54:38 -07:00
Khoa PhamCursorJiminatorClaude Opus 4.8
 		87992eeec4 [DeepSeek V2] Reorder dual-stream MoE to main-first to avoid CUDA graph stream explosion (#30460) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Jiminator <jimmysh341@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-09 16:59:39 -07:00
Khoa Phamkpham-sglXinyuan TongXinyuan TongClaude Fable 5
 		16372b4c5f [Spec] Anchor GLM-5.2 MTP IndexShare topk on the draft-extend step (#29787) 
Co-authored-by: kpham-sgl <264503018+kpham-sgl@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-06 20:36:48 -07:00
Khoa PhamCursorClaude Fable 5
 		3a679459e5 [bench] Add agentic-trace multi-turn dataset to bench_serving (#29215) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-07-06 19:45:44 -07:00
Khoa PhamandKhoa Pham 6ce02b95ad fix: reorder DSA indexer dual-stream ops to avoid CUDA graph stream explosion (#30025) 
Co-authored-by: Khoa Pham <264503018+kpham-sgl@users.noreply.github.com>
 2026-07-03 13:53:16 -07:00
Khoa PhamClaude Opus 4.8Mohammad Miadh Angkad
 		dbb51c46ac [CI] Migrate JIT tests missed by #29066 to runner_config registration (#29715) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-06-29 23:21:14 -07:00
Khoa PhamCursorClaude Opus 4.8
 		3e16be2122 [CI] Migrate JIT tests to runner config registration (#29066) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-29 19:12:14 -07:00
Khoa PhamandClaude Opus 4.7 6bdecb8206 [DeepSeek V3] Reland: run routed experts on main stream in dual-stream MoE (#29463) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-29 15:05:01 -07:00
Khoa PhamClaude Opus 4.8bef0rewindAlejandroParedesLTCursor
 		f480c5f1f9 [Spec] Frozen-KV MTP: delay target KV binding to pool init + reset stale draft out_cache_loc (#29616) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: bef0rewind <490285+ronhuafeng@users.noreply.github.com>
Co-authored-by: AlejandroParedesLT <99832715+AlejandroParedesLT@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-29 11:57:10 -07:00
Khoa PhamandCursor da802ddcaf (perf): Shard Kimi-K2.5 Eagle3 draft fc + symm-mem AG (#29223) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-27 17:40:32 -07:00
Khoa Phamkpham-sglJackey Hua
 		9214b9338f [DSA] Enable draft-extend CUDA graph for DeepSeek Sparse Attention (#29413) 
Co-authored-by: kpham-sgl <kpham-sgl@users.noreply.github.com>
Co-authored-by: Jackey Hua <107608053+zhendonghua@users.noreply.github.com>
 2026-06-26 23:53:46 -07:00
Khoa PhamCursorClaude Opus 4.8
 		999199f9ff [DeepSeek V3] Run routed experts on main stream in dual-stream MoE (#29142) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-26 00:48:59 -07:00
Khoa PhamandCursor eeee3abbbf [CI] Fix false spec accept length failure after profiling phase (#29329) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-25 22:19:30 -07:00
Khoa PhamandClaude Opus 4.7 6c92f9f328 fix(bench): pass DCP_RANK/DCP_WORLD_SIZE to set_mla_kv_buffer_kernel (#29356) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-25 18:13:57 -07:00
Khoa PhamandClaude Opus 4.7 c8c6757bed test(dp-attn): drop --enable-torch-compile from TestDPAttentionDP2TP2 (#29345) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-25 16:41:35 -07:00
Khoa PhamClaude Opus 4.8Xinyuan Tong
 		ed26a109ee [Fix] Return streaming logprobs when reasoning/tool parser is active (#28601) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
 2026-06-23 09:56:09 -07:00
Khoa PhamandCursor 0642cd5020 (chore): bump tokenspeed_mla to 0.1.7 (#28759) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-21 21:25:37 -07:00
Khoa PhamandClaude Opus 4.7 e4fd613def [Spec] Fix return_hidden_states under spec V2 (issue #26163) (#28496) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-06-17 16:21:36 -07:00
Khoa Pham 3c4130c741 [Kimi K2.5] Fix eagle3 aux capture for tp>1 when AR fusion is enabled (#28343) 2026-06-17 12:59:11 -07:00
Khoa PhamCursorQiaolin Yu
 		b54f8432ad Batch EAGLE draft/draft-extend replay memcpys via grouped foreach copy (#28465) 
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
 2026-06-17 00:34:21 -07:00
Khoa Pham a0c6e0b3a4 chore: bump tokenspeed_mla 0.1.1 -> 0.1.6 (#28116) 2026-06-12 19:57:36 -07:00
Khoa PhamandQiaolin Yu d1a39b0c74 [DeepSeek V3] Defer moe finalize and fused it with main stream add (#27720) 
Co-authored-by: Qiaolin Yu <liin1211@outlook.com>
 2026-06-12 19:00:51 -07:00
Khoa PhamandMohammad Miadh Angkad 82eedd5bd0 [DeepEP] Enable fabric handles automatically when supported (#27107) 
Co-authored-by: Mohammad Miadh Angkad <176301910+mmangkad@users.noreply.github.com>
 2026-06-12 17:17:44 -07:00
Khoa PhamRichard WangClaude Opus 4.8
 		1cf8efdd08 docs: Diffusion Gemma cookbook (#27824) 
Co-authored-by: Richard Wang <wangrichard08@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-10 09:33:01 -07:00
Khoa PhamandCursor 9a3e845fc1 [Spec] Add nvtx to spec regions (#27615) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-08 22:58:03 -07:00
Khoa Pham 0f8673851c test: fix gemma GSM8K thresholds in nightly text eval (#27342) 2026-06-08 19:58:42 -07:00
Khoa Pham c95179bc85 [Spec] Fuse small kenrels under gather_spec_extras (#27233) 2026-06-08 15:02:04 -07:00
Khoa PhamandClaude Opus 4.8 bf172c492a Cookbook for QAT (#27396) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-05 11:18:11 -07:00
Khoa Pham b5560ffc36 Fix flashinfer autotune oom glm51 (#24195) 2026-06-02 23:28:57 -07:00
Khoa PhamandClaude Opus 4.8 6d53615699 [Gemma4] Use hard GSM8K accuracy floor for 31B MTP test (#27101) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-03 01:57:12 -04:00
Khoa Pham a711c57a32 [Spec] Fix Gemma 4 MTP with trtllm_mha crash issue (#26966) 2026-06-02 14:37:51 -07:00
Khoa Pham 22bb9a6421 test: disable test_gemma4_mtp_26b_a4b_extra from CI (#27082) 2026-06-02 14:00:48 -07:00
Khoa PhamandCursor 08526c7fca [Spec] FrozenKVMTP fold assistant seed into captured draft graph (#25539) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-01 22:27:25 -07:00
Khoa Pham da01f2974e [Log] include max_token_num and hidden_dim in FlashInfer workspace init log (#26605) 2026-06-01 13:26:52 -07:00
Khoa PhamandClaude Opus 4.8 cb8a103b81 chore: add @pyc96 as codeowner for FrozenKVMTP module (#26953) 
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-01 13:06:58 -07:00
Khoa PhamandCursor 621a79728c test: stabilize Gemma4 26B-A4B MTP GSM8K test with deterministic inference + tuned threshold (#26653) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-28 20:58:48 -07:00
Khoa PhamandCursor 93445e6359 [spec decoding] support kimi-k2.6-eagle3.1-mla draft (#26506) 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-05-28 14:26:39 -07:00
Khoa Pham a38620b92c chore: add @pyc96 as codeowner for gemma4 files (#26601) 2026-05-28 12:37:50 -07:00
Khoa PhamandClaude Opus 4.7 9040feebd8 [Gemma4] Add test for MTP models (#24552) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-27 21:36:28 -07:00
Khoa PhamandClaude Opus 4.7 b0ce16d0c5 [CP] 1/N: Support MLA Prefill Context Parallel (#23292) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-23 03:07:09 -07:00
Khoa PhamandClaude Opus 4.7 f829cafa3e [perf] skip add_special_tokens=False kwarg on chat-template tokenize for slow tokenizers (#25953) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-21 22:45:54 -07:00
Khoa PhamandClaude Opus 4.7 b29e41e8b3 Respect user override for Gemma4 attention backend (#25547) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-18 10:46:25 -07:00
Khoa PhamandClaude Opus 4.7 856f13a916 chore(codeowners): add kpham-sgl to frozen_kv_mtp files (#25211) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 20:01:05 -07:00
Khoa PhamandClaude Opus 4.7 371cb2ade2 [Bench] Add MEM profile activity to bench_serving (#25026) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-13 16:22:11 -07:00
Khoa Pham c665edec6e [env] Make max KV chunk capacity configurable via SGLANG_MAX_KV_CHUNK_CAPACITY (#25120) 2026-05-12 22:37:45 -07:00
Khoa Pham 49ac447c94 [bench] Agentic support for bench_serving.py (#25016) 2026-05-12 16:00:45 -07:00
Khoa Pham 1610aa77ab Reduce gemma4 moe deterministic test runtime (#24754) 2026-05-08 20:46:56 -07:00
Khoa PhamandPengyu Chen d2c1034163 [Gemma 4] Adding MTP support (#24436) 
Co-authored-by: Pengyu Chen <pychen96@gmail.com>
 2026-05-07 14:08:41 -07:00
Khoa Pham 932d89690a Gemma4-mtp cookbook (#24433) 2026-05-05 23:04:41 +08:00
Khoa Pham d22853480d Fix deterministic inference on models with SWAKVPool (#24395) 2026-05-05 20:20:46 +08:00
Khoa Pham ef2b1b6d89 Fix flashinfer workspace OOM (#24172) 2026-05-04 01:26:35 -07:00
Khoa PhamandClaude Opus 4.7 c2d60c3667 chore(codeowners): add @kpham-sgl as codeowner for ngram files (#24349) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-03 23:27:33 -07:00
Khoa Pham ddcacaf1bd Fix failing test_nvidia_nemotron_3_nano by fixing test_grouped_topk (#23874) 2026-04-28 15:03:58 -07:00
Khoa PhamandClaude Opus 4.7 826f2d0620 chore(codeowners): add @kpham-sgl as owner for gemma4 files (#23916) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-28 11:43:32 -07:00
Khoa Pham 5a0eea8ac5 [CI] Adding Gemma 4 to Nightly CI (#22408) 2026-04-16 19:30:16 -07:00