 Trevor Morris
|
7f540274a7
|
Use Flashinfer 0.6.18 release for CUDA 13.4 package (#37073)
|
2026-08-29 21:06:00 -07:00 |
|
|
Trevor Morris
|
a16872767f
|
Update CUDA 13.4 image to flashinfer 0.6.18rc10, cutedsl 4.8. Fix sgl- wheel unpinning (#36929)
|
2026-08-28 17:52:15 -07:00 |
|
|
Trevor Morris
|
ca179b761b
|
Use kernel build node for cu134 image (#36912)
|
2026-08-28 14:56:23 -07:00 |
|
|
Trevor Morris
|
ce6e1f46b4
|
Limit concurrent build jobs for cu134 container (#36756)
|
2026-08-27 20:24:35 -07:00 |
|
|
Trevor Morris
|
1eb629ab4f
|
[NVIDIA] Tune custom all reduce v2 for sm_107 (#36397)
|
2026-08-26 15:19:31 -07:00 |
|
|
Trevor Morris
|
3ce243da3f
|
[NVIDIA] Add CUDA 13.4 container for initial Rubin support (#36233)
|
2026-08-26 15:02:16 -07:00 |
|
|
Trevor Morris
|
3ed2a0adf3
|
feat: Add flashinfer mHC fusion for DSV4 (#33616)
|
2026-08-07 08:01:33 +00:00 |
|
|
Trevor Morris
|
81c7a54ecd
|
[NVIDIA] Use sm_100f instead of sm_100a for sgl-kernel and FlashMLA (#33433)
|
2026-08-05 01:36:46 -07:00 |
|
|
Trevor Morris
|
a6221d776f
|
feat: Support nvidia/MiniMax-M3-NVFP4 (#31989)
|
2026-07-30 14:32:03 -07:00 |
|
|
Trevor Morris
|
b8c25bfaa7
|
[NVIDIA] Support flashinfer a2a with flashinfer_trtllm_routed moe (#22394)
|
2026-06-29 16:23:58 -07:00 |
|
|
Trevor Morris
|
f74a1722e6
|
[NVIDIA] Support TF32 matmul to improve MiniMax gate gemm performance (#22744)
|
2026-06-23 14:54:54 -07:00 |
|
|
Trevor Morris
|
c0bb04b67f
|
[NVIDIA] Support NVFP4 MoE for DeepSeek-V4 (#25820)
|
2026-06-21 19:35:14 -07:00 |
|
|
Trevor MorrisandClaude Opus 4.7
|
9c53853ea3
|
Use pack topk ids triton kernel for flashinfer_trtllm_routed (#25702)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-06-16 19:27:14 -07:00 |
|
|
Trevor Morris
|
20f4272109
|
fix: Fix DSR1 perf regression due to unnecessarily falling back to triton gemm (#28073)
|
2026-06-15 09:45:09 -04:00 |
|
|
Trevor Morris
|
0bac184425
|
[NVIDIA] Update Minimax-M2.5,M2.7 docs with flags for performance (#24465)
|
2026-06-11 14:58:44 -07:00 |
|
|
Trevor Morris
|
5da265de30
|
[NVIDIA] Fix FP8 gemm performance with fp16 models (MInimax-M2.5) (#22300)
|
2026-06-07 02:45:00 +00:00 |
|
|
Trevor Morris
|
7dbd0dd9f0
|
MiniMax-M2.5 - Support dp attention, dp reduce scatter, FP4 all gather, AR fusion in prepare_attn (#20067)
|
2026-04-10 12:41:27 -07:00 |
|
|
Trevor Morris
|
7546d04c81
|
[NVIDIA] Enable FP4 flashinfer trtllm routed moe (#21240)
|
2026-04-07 16:16:29 -07:00 |
|
|
Trevor Morris
|
5cc246e095
|
Fix extra calls to get_numa_node_if_available to clean up logs (#21781)
|
2026-04-06 16:18:40 -07:00 |
|
|
Trevor Morris
|
56266de624
|
[CI] Add basic unit test for Minimax-M2.5 (#21792)
|
2026-04-06 15:48:33 -07:00 |
|
|
Trevor Morris
|
d24ea24e18
|
[NVIDIA] Enable fp8 flashinfer_trtllm_routed MoE for MiniMax-M2.5 (#20394)
|
2026-04-01 23:02:06 -07:00 |
|
|
Trevor Morris
|
8950d129bd
|
[refactor] Clean up duplicate flashinfer trtllm moe code (#21233)
|
2026-04-01 13:52:22 -07:00 |
|
|
Trevor Morris
|
b91f78d255
|
[bugfix] Fix rope theta config for MiniMax after transformers v5 update (#21241)
|
2026-03-31 11:37:03 -07:00 |
|
|
Trevor Morris
|
7160b6cb76
|
[NVIDIA] Enable automatic NUMA configuration (#19452)
|
2026-03-27 18:44:13 -07:00 |
|
|
Trevor Morris
|
df1d046de2
|
Add packed_modules_mapping for MiniMax-M2 (#19995)
|
2026-03-18 14:10:01 -07:00 |
|
|
Trevor Morris
|
2c2c4e446b
|
[NVIDIA] Add flashinfer all-to-all MOE dispatcher (#14668)
|
2026-01-24 22:59:55 +08:00 |
|
|
Trevor Morris
|
c8c64876a7
|
[NVIDIA] Fixes for NVFP4 all-gather with spec decoding (#15280)
|
2025-12-16 16:58:39 -08:00 |
|
|
Trevor Morris
|
0eea17e3b5
|
Use dual stream for DS MoE whenever cuda graph is used (instead of with token threshold) (#9405)
|
2025-11-22 00:37:41 -08:00 |
|
|
Trevor Morris
|
151e13687a
|
Don't fuse wk+weight_proj for nextn (#12863)
|
2025-11-11 20:02:52 -08:00 |
|
|
Trevor Morris
|
211f4070e5
|
fix: Lazy import mooncake-ep to fix extra gpu contexts being created (#12641)
|
2025-11-04 12:28:36 -08:00 |
|
|
Trevor Morris
|
dbcf85b7f0
|
Add --speculative-moe-runner-backend server arg (#10183)
|
2025-11-04 00:20:56 -08:00 |
|
|
Trevor Morris
|
9ff9fa7f95
|
Fuse wk and weight_proj in Indexer for DeepSeekV3.2-FP4 (#12094)
|
2025-10-29 18:42:56 -07:00 |
|
|
Trevor Morris
|
fdd00295b5
|
Fix 'BypassedTopKOutput' object has no attribute 'topk_weights' for DeepEP (#12231)
|
2025-10-28 09:28:25 -07:00 |
|
|
Trevor Morris
|
384733639a
|
[DSv32] Use torch.compile for _get_logits_head_gate (#11565)
|
2025-10-13 18:38:39 -07:00 |
|
|
Trevor Morris
|
c9cff2b984
|
Fix DeepSeek-v3.2 default config (ValueError: not enough values to unpack (expected 4, got 3)) (#11557)
|
2025-10-13 11:27:40 -07:00 |
|
|
Trevor Morris
|
a4b424c632
|
[DeepSeek-V3.2] Include indexer kv cache when estimating kv cache size (#11309)
|
2025-10-08 23:59:46 -07:00 |
|
|
Trevor Morris
|
a6cc86df9d
|
Fix DSR1 accuracy for flashinfer_trtllm MoE with FP8 quantization (#11081)
|
2025-09-30 10:33:12 -07:00 |
|
|
Trevor Morris
|
c7e85f5378
|
fix: flashinfer_cutlass_moe: Use max of global expert scales instead of local for input scale (#10296)
|
2025-09-11 20:19:17 -07:00 |
|
|
Trevor Morris
|
a91e90d9a3
|
[2/2] Fuse routed scaling factor into select_experts (#8690)
|
2025-08-20 15:10:16 -07:00 |
|
|
Trevor Morris
|
eff4eb3fdd
|
Add fp4 quantize before all-gather for Flashinfer cutlass MoE DP (max throughput) (#7667)
|
2025-08-15 22:08:11 -07:00 |
|
|
Trevor Morris
|
13c48dcf88
|
[1/2][resubmit again] sgl-kernel: Fuse routed scaling factor into moe_fused_gate (#9088)
|
2025-08-12 20:12:38 -07:00 |
|
|
Trevor Morris
|
52e1f52f32
|
[bugfix] Fix missing args in bench one batch (#8877)
|
2025-08-09 01:34:03 -07:00 |
|
|
Trevor MorrisandYineng Zhang
|
a60f88b5a4
|
Add unit test for flashinfer fp4 moe (#8330)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-08-08 17:55:37 -07:00 |
|
|
Trevor Morris
|
591c232f7c
|
[1/2][resubmit] sgl-kernel: Fuse routed scaling factor into moe_fused_gate (select_experts) (#8770)
|
2025-08-08 17:55:06 -07:00 |
|
|
Trevor Morris
|
c0e84297c2
|
Use reduce scatter for DP (#8539)
|
2025-08-06 16:21:26 -07:00 |
|
|
Trevor Morris
|
9bd4872a34
|
[bugfix] Fix typo in modelopt quant: 'FusedMoE' object has no attribute 'local_num_experts' (#8768)
|
2025-08-04 11:08:08 -07:00 |
|
|
Trevor Morris
|
32f2815451
|
Do layernorm before allgather for DP attention (#8631)
|
2025-08-03 00:53:08 -07:00 |
|
|
Trevor Morris
|
89caf7a3c6
|
[bugfix] Apply routed scaling factor to cutlass_fused_experts_fp8 (#8688)
|
2025-08-01 19:00:24 -07:00 |
|
|
Trevor Morris
|
f642524fd9
|
[1/2] sgl-kernel: Fuse routed scaling factor into select_experts (#8364)
|
2025-08-01 18:14:24 -07:00 |
|
|
Trevor Morris
|
6a7528e623
|
[bugfix] Fix page size for create_flashmla_kv_indices_triton() for cutlass mla (#8685)
|
2025-08-01 14:28:04 -07:00 |
|
|
Trevor Morris
|
b7170cc820
|
[bugfix] Fix flashinfer cutlass EP moe after MoE refactor (#8630)
|
2025-07-31 13:57:08 -07:00 |
|
|
Trevor Morris
|
58c468f404
|
Fix FP4 MoE accuracy from missing routed_scaling_factor (#8333)
|
2025-07-25 16:40:23 -07:00 |
|
|
Trevor Morris
|
bb9b608c86
|
[PD][NIXL] Set is_sorted=False to fix NIXL_ERR_NOT_FOUND (#7330)
|
2025-06-26 10:39:39 -07:00 |
|
|
Trevor Morris
|
5f527834a8
|
[PD] NIXL: Register kv args in advance and cleanup finished requests (#6717)
|
2025-06-24 11:26:09 -07:00 |
|
|
|
5962e70d8d
|
FlashInfer NVFP4 MoE with EP & 2-stream shared expert (#7327)
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
Co-authored-by: alcanderian <alcanderian@gmail.com>
|
2025-06-22 13:38:47 -07:00 |
|
|
Trevor Morris
|
e806f708c9
|
[PD] Make bootstrap code common between NIXL and Mooncake (#6473)
|
2025-05-27 12:47:38 -07:00 |
|
|
Trevor Morris
|
7adf245ba2
|
[Metrics] Add KV events publishing (#6098)
|
2025-05-19 14:19:54 -07:00 |
|
|
Trevor Morris
|
0ab3f437ab
|
Cutlass MLA: Disable split kv due to https://github.com/NVIDIA/cutlass/issues/2274 (#6101)
|
2025-05-08 18:44:30 -07:00 |
|
|
Trevor Morris
|
8d463fe351
|
Cutlass MLA decode - fix dtype error (#5868)
|
2025-04-28 21:12:58 -07:00 |
|
|
Trevor Morris
|
84810da4ae
|
Add Cutlass MLA attention backend (#5390)
|
2025-04-27 20:58:53 -07:00 |
|
|
Trevor Morris
|
4dce1cc608
|
[PD] Add NIXL transfer backend (#5477)
|
2025-04-22 01:36:12 +08:00 |
|
|
Trevor Morris
|
e8f62b20ca
|
BLackwell cutlass mla: Add check for bad page size/block num combinations (#5431)
|
2025-04-15 14:07:42 -07:00 |
|
|
Trevor Morris
|
f65b8d5c89
|
Blackwell Cutlass MLA kernel (#5142)
|
2025-04-11 22:16:51 -07:00 |
|
|
Trevor Morris
|
11d760d56a
|
FP4 weight loading and inference (2/2) (#3972)
|
2025-04-08 17:26:21 -07:00 |
|
|
Trevor Morris
|
e9f8e42318
|
Support FP4 gemm (1/2) (#3899)
|
2025-03-24 19:50:23 -07:00 |
|