 Cheng Wan
|
96cc10834a
|
[CI] update estimated elapsed time of some unittests (#14347)
|
2025-12-03 01:21:40 -08:00 |
|
|
Cheng WanandKangyan Zhou
|
0fe74af563
|
Remove incorrect deep_gemm assertions from server_args.py (#14113)
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
|
2025-11-28 20:25:39 -08:00 |
|
|
Cheng Wan
|
b12c9e5c0a
|
Fix installation for nvidia-nvshmem-cu12 (#14033)
|
2025-11-26 18:27:12 -08:00 |
|
|
Cheng Wan
|
bf10869203
|
[Doc] Add an Introduction to Expert Parallelism (#13783)
|
2025-11-24 14:46:51 -08:00 |
|
|
Cheng WanandBaizhou Zhang
|
6d0e0b9bfc
|
[11/N] MoE Refactor: Simplifying SBO Implementation with Dispatcher Hooks (#13327)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-11-21 01:11:37 -08:00 |
|
|
Cheng WanandMick
|
a5ad0069b2
|
fix: change performance log directory to cache path (#13482)
Co-authored-by: Mick <mickjagger19@icloud.com>
|
2025-11-18 15:18:43 +08:00 |
|
|
Cheng Wan
|
2b8b9d8496
|
[CI] use cached deepep installation in gb200 CI (#13388)
|
2025-11-16 12:49:01 -08:00 |
|
|
Cheng Wan
|
13366843a4
|
[CI] check unit-test-backend-8-gpu-h20 in workflow (#13355)
|
2025-11-15 16:44:10 -08:00 |
|
|
Cheng Wan
|
83087247d1
|
[hotfix] missing w13_weight_fp8 and w2_weight_fp8 in UE8M0 requantization (#12259)
|
2025-10-28 19:10:38 -07:00 |
|
|
Cheng Wan
|
b1e13e7cea
|
[hotfix] Incorrect CombineOverlapArgs in SBO (#12230)
|
2025-10-28 01:23:06 -07:00 |
|
|
Cheng Wan
|
649949807f
|
[10/N] MoE Refactor: reorganize deepgemm runner in DeepEPMoE (#12054)
|
2025-10-24 19:16:17 -07:00 |
|
|
Cheng Wan
|
bfc3b3f786
|
[9/N] MoE Refactor: cleanup dispatcher interfaces (#11847)
|
2025-10-20 10:11:46 -07:00 |
|
|
Cheng Wan
|
5b214b50b6
|
[Refactor] move deep_gemm_wrapper out of quantization (#11784)
|
2025-10-17 18:57:54 -07:00 |
|
|
Cheng Wan
|
1bdd010291
|
Revert "Deprecate global_server_args_dict" (#11520)
|
2025-10-12 17:40:40 -07:00 |
|
|
Cheng Wan
|
6cd296940a
|
[lint] Fix the lint issue (#11516)
|
2025-10-12 16:22:46 -07:00 |
|
|
Cheng Wan
|
52fcbbb8bd
|
Revert "perf: optimize qwen-vl with symm mem allreduce" (#11436)
|
2025-10-10 12:30:05 -07:00 |
|
|
Cheng Wan
|
97cd38e58d
|
Skip weight loading in deepgemm compilation (#11312)
|
2025-10-07 21:52:46 -07:00 |
|
|
Cheng Wan
|
3c06b673af
|
[8/N] MoE Refactor: deprecate EPMoE (#11211)
|
2025-10-07 21:51:41 -07:00 |
|
|
Cheng Wan
|
41763ba079
|
Remove gdrcopy check in ci_install_deepep.sh (#11237)
|
2025-10-05 17:35:22 -07:00 |
|
|
Cheng Wan
|
6535fda127
|
[Profile] dump memory trace when cuda graph profile is enabled (#11083)
|
2025-09-29 17:36:48 -07:00 |
|
|
Cheng Wan
|
4844fac91d
|
Refactor TopK to ensure readability and extensibility (#9338)
|
2025-09-14 19:16:25 -07:00 |
|
|
Cheng Wan
|
2f8ba6fe82
|
[Fix] MoE: fix w8a8_fp8 MoE and add tests to cover this code path (#10429)
|
2025-09-14 17:34:28 -07:00 |
|
|
Cheng Wan
|
5a7e10fe4c
|
[MoE] fix: incorrect weight initialization for cutlass_fused_experts_fp8 (#10144)
|
2025-09-07 19:43:59 -07:00 |
|
|
Cheng Wan
|
a5a03209e9
|
Fix circular import (#10107)
|
2025-09-06 01:34:17 -07:00 |
|
|
Cheng Wanandgemini-code-assist[bot]
|
21af5c0404
|
[Fix] Compatibility between DP attention and pipeline parallelism (#10100)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-09-06 01:34:10 -07:00 |
|
|
Cheng Wan
|
3fa62da78c
|
[7/N] MoE Refactor: the implementation of new framework (#9269)
|
2025-09-05 21:09:09 -07:00 |
|
|
Cheng Wan
|
453511acc7
|
Save memory for expert model parallel (#9957)
|
2025-09-04 13:31:47 -07:00 |
|
|
Cheng Wan
|
84b006b278
|
Cleanup MoE Refactor (#9223)
|
2025-08-15 02:28:33 -07:00 |
|
|
Cheng Wan
|
e3e75a786a
|
Fix the deprecation warning for enable_flashinfer_mxfp4_moe (#9214)
|
2025-08-14 23:59:35 -07:00 |
|
|
Cheng Wan
|
295895120d
|
[6/N] MoE Refactor: Cleanup MoE-related configs (#8849)
|
2025-08-14 21:14:53 -07:00 |
|
|
Cheng Wan
|
b87aacb5c5
|
[DP Attention] Refactor: adding some utility functions (#9136)
|
2025-08-13 21:08:06 -07:00 |
|
|
Cheng Wan
|
5f5b3b2449
|
[5/n] DP Enhancement: Correct num_token_non_padded (#9107)
|
2025-08-12 12:23:46 -07:00 |
|
|
Cheng Wanandgemini-code-assist[bot]
|
f003cd3548
|
[CI] Fix CI tests (#9050)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-08-10 23:52:05 -07:00 |
|
|
Cheng Wan
|
5018809222
|
[DP] fix: engine crash when decode batch is padded (#8995)
|
2025-08-09 01:29:29 -07:00 |
|
|
Cheng Wan
|
a47baff12c
|
[hotfix] use the original implementation in 8785 (#8994)
|
2025-08-08 21:47:25 -07:00 |
|
|
Cheng Wan
|
fd7e15b76d
|
Revert "[bug fix] Ensure local token and global token buffers are pointing to different storage " (#8993)
|
2025-08-08 21:34:17 -07:00 |
|
|
Cheng Wan
|
1d24db8348
|
Expert Parallelism for GPT-OSS (#8944)
|
2025-08-08 00:46:42 -07:00 |
|
|
Cheng Wan
|
5b6acc1495
|
fix glm4 moe (#8883)
|
2025-08-06 18:02:31 -07:00 |
|
|
Cheng Wan
|
78aad91037
|
[CI] fix pip upgrade (#8881)
|
2025-08-06 15:02:32 -07:00 |
|
|
Cheng Wan
|
b102353f8f
|
[MoE] Enable renormalize=False in Triton kernels (#8735)
|
2025-08-03 17:03:04 -07:00 |
|
|
Cheng Wan
|
0e0eef00ce
|
[DP] fix the compatibility issue between DP attention and --attention-backend triton (#8723)
|
2025-08-03 03:06:57 -07:00 |
|
|
Cheng Wan
|
cb099d2095
|
[CUDA Graph] save cuda graph memory by using next_token_logits_buffer (#8579)
|
2025-08-03 03:06:47 -07:00 |
|
|
Cheng Wan
|
7a91330149
|
Save cuda graph memory for fa3 (#8567)
|
2025-08-03 03:06:31 -07:00 |
|
|
Cheng Wan
|
a437aa9987
|
[hotfix] fix mixtral with tensor-level compressed-tensor quantization (#8721)
|
2025-08-02 22:59:25 -07:00 |
|
|
Cheng Wan
|
6c88f6c8d9
|
[5/N] MoE Refactor: Update MoE parallelism arguments (#8658)
|
2025-08-01 01:20:03 -07:00 |
|
|
Cheng Wan
|
7a1f7fc504
|
[Feature] Hybrid EP and TP (#8590)
|
2025-07-31 02:53:25 -07:00 |
|
|
Cheng Wan
|
32fa1e9cc2
|
[4/N] MoE Refactor: Unified Triton Kernel for FusedMoE and EPMoE (#8515)
|
2025-07-31 02:34:02 -07:00 |
|
|
Cheng Wan
|
e179e0b797
|
update sgl-kernel for EP: python part (#8550)
|
2025-07-31 00:14:39 -07:00 |
|
|
|
a5f5ab4030
|
update sgl-kernel for EP: kernel part (#8514)
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
|
2025-07-30 22:19:55 -07:00 |
|
|
Cheng Wan
|
9effeb5bdd
|
Support EPLB in FusedMoE (#8448)
|
2025-07-29 16:02:41 -07:00 |
|
|
Cheng Wan
|
9c138a0445
|
[3/N] MoE Refactor: Simplify DeepEP Output (#8421)
|
2025-07-28 11:37:17 -07:00 |
|
|
Cheng Wan
|
4d921f2b79
|
[hotfix] fix merge conflicts in FlashInferEPMoE (#8405)
|
2025-07-27 01:24:10 -07:00 |
|
|
Cheng Wan
|
bf0f448fe5
|
[2/N] MoE Refactor: Unify weight loader and quant methods (#8397)
|
2025-07-27 01:00:21 -07:00 |
|
|
Cheng Wan
|
e236d8fee8
|
Save peak memory in logits processor (#8343)
|
2025-07-26 01:46:42 -07:00 |
|
|
Cheng Wan
|
c0fb25e949
|
DP Enhancement (#8280)
|
2025-07-24 21:36:21 -07:00 |
|
|
Cheng Wan
|
abda2542d5
|
Fix tuning_fused_moe_triton.py (#8175)
|
2025-07-19 17:33:50 -07:00 |
|
|
Cheng Wan
|
15ad6c9086
|
[1/N] MoE Refactor: refactor select_experts (#7966)
|
2025-07-19 00:51:15 -07:00 |
|
|
Cheng Wan
|
49b8777460
|
Refactor: move all quantization-related code to srt/layer/quantization (#7989)
|
2025-07-17 00:47:07 -07:00 |
|
|
Cheng Wan
|
02404a1e35
|
[ci] recover 8-gpu deepep test (#8105)
|
2025-07-17 00:46:40 -07:00 |
|
|
Cheng Wan
|
9069884b51
|
[ci] disable memory imbalance check for draft worker (#8108)
|
2025-07-16 20:41:47 -07:00 |
|
|
Cheng Wan
|
98aa836bbf
|
Overlap the gating function with shared experts in DeepSeek (#7978)
|
2025-07-12 13:41:50 -07:00 |
|
|
Cheng Wan
|
475a249bb8
|
temporarily disable deepep-8-gpu and activate two small tests (#7961)
|
2025-07-11 14:22:05 -07:00 |
|
|
Cheng Wan
|
d487555f84
|
[CI] Add deepep tests to CI (#7872)
|
2025-07-09 01:49:47 -07:00 |
|
|
Cheng Wan
|
6c903611ca
|
Fix incorrect spec_num_draft_tokens in draft_extend (#7757)
|
2025-07-05 02:18:16 -07:00 |
|
|
Cheng Wan
|
8fc910db03
|
DP Attention with Auto DeepEP Dispatch (#7222)
|
2025-07-05 01:54:24 -07:00 |
|
|
Cheng Wan
|
cb432f1770
|
saving hidden_states.clone() (#7705)
|
2025-07-04 20:07:42 -07:00 |
|
|
Cheng Wan
|
1b8cf77b01
|
[Fix] incorrect assert in EPLB (#7575)
|
2025-06-26 14:59:20 -07:00 |
|
|
Cheng Wan
|
f8d48fd311
|
Fix dtype for idle input in spec decoding (#7456)
|
2025-06-23 11:23:25 -07:00 |
|
|
Cheng Wan
|
ac5010e0ba
|
Fix CUDA Graph Check under Deepep with DP FFN (#7451)
|
2025-06-22 20:35:58 -07:00 |
|
|
Cheng Wan
|
5041df2d01
|
Fix 7285 Merge Conflicts (#7403)
|
2025-06-20 16:02:50 -07:00 |
|
|
Cheng Wan
|
256801e973
|
Update usage_processor.py (#7402)
|
2025-06-20 15:55:38 -07:00 |
|
|
Cheng Wan
|
73b13e69b4
|
Optimize DP attn scheduling for speculative decoding (#7285)
|
2025-06-20 15:06:41 -07:00 |
|
|
Cheng Wan
|
8609e637a9
|
Fix All-Gather under world size one (#7219)
|
2025-06-20 14:57:34 -07:00 |
|
|
Cheng Wan
|
22bfed7509
|
[DeepSeekNextN] fix: residual of head norm can be None (#7398)
|
2025-06-20 14:45:16 -07:00 |
|
|
Cheng Wan
|
e879d8b7a8
|
[Feature] Comprehensive Hybrid Parallelism Support (#6389)
|
2025-06-20 14:43:11 -07:00 |
|
|
Cheng Wan
|
3c2274fbee
|
Implement gather before attn (#6378)
|
2025-06-15 21:08:56 -07:00 |
|
|
Cheng Wan
|
499f5e620c
|
Fix one missing arg in DeepEP (#6878)
|
2025-06-04 19:14:47 -07:00 |
|
|
Cheng Wan
|
81964328b7
|
Set num_fused_shared_experts as num_shared_experts when shared_experts fusion is not disabled (#6736)
|
2025-06-04 15:53:22 -07:00 |
|
|
Cheng Wan
|
8a5480528d
|
[Refactor] Rename n_share_experts_fusion as num_fused_shared_experts (#6735)
|
2025-06-03 17:48:24 -07:00 |
|
|
Cheng Wan
|
ced3c07afe
|
Support token-level quantization for EP MoE (#6782)
|
2025-05-30 17:26:30 -07:00 |
|
|
Cheng Wan
|
b581b22504
|
Fix one bug in the grouped-gemm triton kernel (#6772)
|
2025-05-30 01:42:08 -07:00 |
|
|
Cheng Wan
|
a3d7f4b673
|
fix communicator for non-dp lm head (#6662)
|
2025-05-27 02:31:12 -07:00 |
|
|
|
b2e95f62b4
|
Fix two issues related to --moe-dense-tp-size=1 (#5657)
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
Co-authored-by: 颉沆 <xiehang.lsy@alibaba-inc.com>
|
2025-05-12 23:51:39 -07:00 |
|
|
Cheng Wanandliusy58
|
25c83fff6a
|
Performing Vocabulary Parallelism for LM Head across Attention TP Groups (#5558)
Co-authored-by: liusy58 <liusy58@linux.alibaba.com>
|
2025-05-11 23:36:29 -07:00 |
|
|
Cheng Wan
|
9bddf1c82d
|
Deferring 8 GPU test (#6102)
|
2025-05-07 18:49:58 -07:00 |
|
|
Cheng WanandByron Hsu
|
711efe7814
|
Integrating PD disaggregation with DP attention and DeepEP (#5435)
Co-authored-by: Byron Hsu <byronhsu1230@gmail.com>
|
2025-04-23 01:46:01 -07:00 |
|
|
Cheng Wan
|
6aca583420
|
Fix several minor issues in PD disaggregation (#5444)
|
2025-04-15 23:04:41 -07:00 |
|
|
Cheng Wan
|
038bc5d521
|
Support --enable-llama4-multimodal (#5254)
|
2025-04-11 01:24:14 -07:00 |
|
|
Cheng Wan
|
76c48a0913
|
[DeepEP] fix: import buffer error (#5179)
|
2025-04-08 22:12:14 -07:00 |
|
|
Cheng Wan
|
4c54f44202
|
[deepep] fix: shared experts are not initialized when shared experts fusion is enabled (#5072)
|
2025-04-04 15:08:30 -07:00 |
|
|
Cheng Wan
|
0d658ac3df
|
Support recording experts workload in QWen2-MoE (#4775)
|
2025-03-26 00:29:57 -07:00 |
|
|
Cheng Wan
|
7b5fc71972
|
fix SUPPORT_CUTLASS_BLOCK_FP8 flag (#4640)
|
2025-03-20 21:45:07 -07:00 |
|
|
Cheng Wan
|
3196999f63
|
Reduce computation and communication in DP attention (#4521)
|
2025-03-18 13:41:36 -07:00 |
|
|
Cheng WanandAbatom
|
2f6bacee03
|
[moe] fix: correct the cache size in the last chunk (#3679)
Co-authored-by: Abatom <abzhonghua@gmail.com>
|
2025-03-12 22:22:13 -07:00 |
|
|
Cheng Wan
|
6b0aeb58fd
|
[moe] optim: reduce memory consumption in fused_moe (#3692)
|
2025-02-20 02:25:05 +08:00 |
|