 Cheng Wan
|
84b006b278
|
Cleanup MoE Refactor (#9223)
|
2025-08-15 02:28:33 -07:00 |
|
|
Cheng Wan
|
e3e75a786a
|
Fix the deprecation warning for enable_flashinfer_mxfp4_moe (#9214)
|
2025-08-14 23:59:35 -07:00 |
|
|
Cheng Wan
|
295895120d
|
[6/N] MoE Refactor: Cleanup MoE-related configs (#8849)
|
2025-08-14 21:14:53 -07:00 |
|
|
Cheng Wan
|
b87aacb5c5
|
[DP Attention] Refactor: adding some utility functions (#9136)
|
2025-08-13 21:08:06 -07:00 |
|
|
Cheng Wan
|
5f5b3b2449
|
[5/n] DP Enhancement: Correct num_token_non_padded (#9107)
|
2025-08-12 12:23:46 -07:00 |
|
|
Cheng Wanandgemini-code-assist[bot]
|
f003cd3548
|
[CI] Fix CI tests (#9050)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-08-10 23:52:05 -07:00 |
|
|
Cheng Wan
|
5018809222
|
[DP] fix: engine crash when decode batch is padded (#8995)
|
2025-08-09 01:29:29 -07:00 |
|
|
Cheng Wan
|
a47baff12c
|
[hotfix] use the original implementation in 8785 (#8994)
|
2025-08-08 21:47:25 -07:00 |
|
|
Cheng Wan
|
fd7e15b76d
|
Revert "[bug fix] Ensure local token and global token buffers are pointing to different storage " (#8993)
|
2025-08-08 21:34:17 -07:00 |
|
|
Cheng Wan
|
1d24db8348
|
Expert Parallelism for GPT-OSS (#8944)
|
2025-08-08 00:46:42 -07:00 |
|
|
Cheng Wan
|
5b6acc1495
|
fix glm4 moe (#8883)
|
2025-08-06 18:02:31 -07:00 |
|
|
Cheng Wan
|
78aad91037
|
[CI] fix pip upgrade (#8881)
|
2025-08-06 15:02:32 -07:00 |
|
|
Cheng Wan
|
b102353f8f
|
[MoE] Enable renormalize=False in Triton kernels (#8735)
|
2025-08-03 17:03:04 -07:00 |
|
|
Cheng Wan
|
0e0eef00ce
|
[DP] fix the compatibility issue between DP attention and --attention-backend triton (#8723)
|
2025-08-03 03:06:57 -07:00 |
|
|
Cheng Wan
|
cb099d2095
|
[CUDA Graph] save cuda graph memory by using next_token_logits_buffer (#8579)
|
2025-08-03 03:06:47 -07:00 |
|
|
Cheng Wan
|
7a91330149
|
Save cuda graph memory for fa3 (#8567)
|
2025-08-03 03:06:31 -07:00 |
|
|
Cheng Wan
|
a437aa9987
|
[hotfix] fix mixtral with tensor-level compressed-tensor quantization (#8721)
|
2025-08-02 22:59:25 -07:00 |
|
|
Cheng Wan
|
6c88f6c8d9
|
[5/N] MoE Refactor: Update MoE parallelism arguments (#8658)
|
2025-08-01 01:20:03 -07:00 |
|
|
Cheng Wan
|
7a1f7fc504
|
[Feature] Hybrid EP and TP (#8590)
|
2025-07-31 02:53:25 -07:00 |
|
|
Cheng Wan
|
32fa1e9cc2
|
[4/N] MoE Refactor: Unified Triton Kernel for FusedMoE and EPMoE (#8515)
|
2025-07-31 02:34:02 -07:00 |
|
|
Cheng Wan
|
e179e0b797
|
update sgl-kernel for EP: python part (#8550)
|
2025-07-31 00:14:39 -07:00 |
|
|
|
a5f5ab4030
|
update sgl-kernel for EP: kernel part (#8514)
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
|
2025-07-30 22:19:55 -07:00 |
|
|
Cheng Wan
|
9effeb5bdd
|
Support EPLB in FusedMoE (#8448)
|
2025-07-29 16:02:41 -07:00 |
|
|
Cheng Wan
|
9c138a0445
|
[3/N] MoE Refactor: Simplify DeepEP Output (#8421)
|
2025-07-28 11:37:17 -07:00 |
|
|
Cheng Wan
|
4d921f2b79
|
[hotfix] fix merge conflicts in FlashInferEPMoE (#8405)
|
2025-07-27 01:24:10 -07:00 |
|
|
Cheng Wan
|
bf0f448fe5
|
[2/N] MoE Refactor: Unify weight loader and quant methods (#8397)
|
2025-07-27 01:00:21 -07:00 |
|
|
Cheng Wan
|
e236d8fee8
|
Save peak memory in logits processor (#8343)
|
2025-07-26 01:46:42 -07:00 |
|
|
Cheng Wan
|
c0fb25e949
|
DP Enhancement (#8280)
|
2025-07-24 21:36:21 -07:00 |
|
|
Cheng Wan
|
abda2542d5
|
Fix tuning_fused_moe_triton.py (#8175)
|
2025-07-19 17:33:50 -07:00 |
|
|
Cheng Wan
|
15ad6c9086
|
[1/N] MoE Refactor: refactor select_experts (#7966)
|
2025-07-19 00:51:15 -07:00 |
|
|
Cheng Wan
|
49b8777460
|
Refactor: move all quantization-related code to srt/layer/quantization (#7989)
|
2025-07-17 00:47:07 -07:00 |
|
|
Cheng Wan
|
02404a1e35
|
[ci] recover 8-gpu deepep test (#8105)
|
2025-07-17 00:46:40 -07:00 |
|
|
Cheng Wan
|
9069884b51
|
[ci] disable memory imbalance check for draft worker (#8108)
|
2025-07-16 20:41:47 -07:00 |
|
|
Cheng Wan
|
98aa836bbf
|
Overlap the gating function with shared experts in DeepSeek (#7978)
|
2025-07-12 13:41:50 -07:00 |
|
|
Cheng Wan
|
475a249bb8
|
temporarily disable deepep-8-gpu and activate two small tests (#7961)
|
2025-07-11 14:22:05 -07:00 |
|
|
Cheng Wan
|
d487555f84
|
[CI] Add deepep tests to CI (#7872)
|
2025-07-09 01:49:47 -07:00 |
|
|
Cheng Wan
|
6c903611ca
|
Fix incorrect spec_num_draft_tokens in draft_extend (#7757)
|
2025-07-05 02:18:16 -07:00 |
|
|
Cheng Wan
|
8fc910db03
|
DP Attention with Auto DeepEP Dispatch (#7222)
|
2025-07-05 01:54:24 -07:00 |
|
|
Cheng Wan
|
cb432f1770
|
saving hidden_states.clone() (#7705)
|
2025-07-04 20:07:42 -07:00 |
|
|
Cheng Wan
|
1b8cf77b01
|
[Fix] incorrect assert in EPLB (#7575)
|
2025-06-26 14:59:20 -07:00 |
|
|
Cheng Wan
|
f8d48fd311
|
Fix dtype for idle input in spec decoding (#7456)
|
2025-06-23 11:23:25 -07:00 |
|
|
Cheng Wan
|
ac5010e0ba
|
Fix CUDA Graph Check under Deepep with DP FFN (#7451)
|
2025-06-22 20:35:58 -07:00 |
|
|
Cheng Wan
|
5041df2d01
|
Fix 7285 Merge Conflicts (#7403)
|
2025-06-20 16:02:50 -07:00 |
|
|
Cheng Wan
|
256801e973
|
Update usage_processor.py (#7402)
|
2025-06-20 15:55:38 -07:00 |
|
|
Cheng Wan
|
73b13e69b4
|
Optimize DP attn scheduling for speculative decoding (#7285)
|
2025-06-20 15:06:41 -07:00 |
|
|
Cheng Wan
|
8609e637a9
|
Fix All-Gather under world size one (#7219)
|
2025-06-20 14:57:34 -07:00 |
|
|
Cheng Wan
|
22bfed7509
|
[DeepSeekNextN] fix: residual of head norm can be None (#7398)
|
2025-06-20 14:45:16 -07:00 |
|
|
Cheng Wan
|
e879d8b7a8
|
[Feature] Comprehensive Hybrid Parallelism Support (#6389)
|
2025-06-20 14:43:11 -07:00 |
|
|
Cheng Wan
|
3c2274fbee
|
Implement gather before attn (#6378)
|
2025-06-15 21:08:56 -07:00 |
|
|
Cheng Wan
|
499f5e620c
|
Fix one missing arg in DeepEP (#6878)
|
2025-06-04 19:14:47 -07:00 |
|
|
Cheng Wan
|
81964328b7
|
Set num_fused_shared_experts as num_shared_experts when shared_experts fusion is not disabled (#6736)
|
2025-06-04 15:53:22 -07:00 |
|
|
Cheng Wan
|
8a5480528d
|
[Refactor] Rename n_share_experts_fusion as num_fused_shared_experts (#6735)
|
2025-06-03 17:48:24 -07:00 |
|
|
Cheng Wan
|
ced3c07afe
|
Support token-level quantization for EP MoE (#6782)
|
2025-05-30 17:26:30 -07:00 |
|
|
Cheng Wan
|
b581b22504
|
Fix one bug in the grouped-gemm triton kernel (#6772)
|
2025-05-30 01:42:08 -07:00 |
|
|
Cheng Wan
|
a3d7f4b673
|
fix communicator for non-dp lm head (#6662)
|
2025-05-27 02:31:12 -07:00 |
|
|
|
b2e95f62b4
|
Fix two issues related to --moe-dense-tp-size=1 (#5657)
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: 颉沆 <xiehang.lsy@alibaba-inc.com>
|
2025-05-12 23:51:39 -07:00 |
|
|
Cheng Wanandliusy58
|
25c83fff6a
|
Performing Vocabulary Parallelism for LM Head across Attention TP Groups (#5558)
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
|
2025-05-11 23:36:29 -07:00 |
|
|
Cheng Wan
|
9bddf1c82d
|
Deferring 8 GPU test (#6102)
|
2025-05-07 18:49:58 -07:00 |
|
|
Cheng WanandByron Hsu
|
711efe7814
|
Integrating PD disaggregation with DP attention and DeepEP (#5435)
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2025-04-23 01:46:01 -07:00 |
|
|
Cheng Wan
|
6aca583420
|
Fix several minor issues in PD disaggregation (#5444)
|
2025-04-15 23:04:41 -07:00 |
|
|
Cheng Wan
|
038bc5d521
|
Support --enable-llama4-multimodal (#5254)
|
2025-04-11 01:24:14 -07:00 |
|
|
Cheng Wan
|
76c48a0913
|
[DeepEP] fix: import buffer error (#5179)
|
2025-04-08 22:12:14 -07:00 |
|
|
Cheng Wan
|
4c54f44202
|
[deepep] fix: shared experts are not initialized when shared experts fusion is enabled (#5072)
|
2025-04-04 15:08:30 -07:00 |
|
|
Cheng Wan
|
0d658ac3df
|
Support recording experts workload in QWen2-MoE (#4775)
|
2025-03-26 00:29:57 -07:00 |
|
|
Cheng Wan
|
7b5fc71972
|
fix SUPPORT_CUTLASS_BLOCK_FP8 flag (#4640)
|
2025-03-20 21:45:07 -07:00 |
|
|
Cheng Wan
|
3196999f63
|
Reduce computation and communication in DP attention (#4521)
|
2025-03-18 13:41:36 -07:00 |
|
|
Cheng WanandAbatom
|
2f6bacee03
|
[moe] fix: correct the cache size in the last chunk (#3679)
Co-authored-by: Abatom <abzhonghua@gmail.com>
|
2025-03-12 22:22:13 -07:00 |
|
|
Cheng Wan
|
6b0aeb58fd
|
[moe] optim: reduce memory consumption in fused_moe (#3692)
|
2025-02-20 02:25:05 +08:00 |
|