 Cheng WanandClaude Opus 4.7
|
c7878dbb6d
|
[MoE] Deprecate act_and_mul_triton; fold filter_expert into JIT silu/gelu_and_mul (#23707)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-26 01:41:35 -07:00 |
|
|
Cheng Wan
|
d9c72bdd2b
|
Skip unselected experts in flashinfer_trtllm (#23493)
|
2026-04-23 17:30:19 -07:00 |
|
|
Cheng WanandClaude Opus 4.7
|
000a2525e1
|
Move expert_mask_gpu from FusedMoE layer to StandardDispatcher (#23585)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-23 17:17:27 -07:00 |
|
|
Cheng WanandClaude Opus 4.7
|
ebcc2b3eec
|
ci: run weekly est_time update on Monday using p90 of last 15 runs (#23120)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-19 14:39:27 -07:00 |
|
|
Cheng WanandClaude Opus 4.7
|
5f7aee726a
|
refactor(moe): de-duplicate triton MoE runner path into shared helpers (#23019)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
2026-04-17 17:05:13 -07:00 |
|
|
Cheng Wan
|
b5e4ae7b1a
|
fix: match est_time updates by backend, not just suite (#22563)
|
2026-04-10 17:54:50 -07:00 |
|
|
Cheng Wan
|
0011d2aec0
|
fix: track est_time per suite instead of per backend (#22557)
|
2026-04-10 16:58:40 -07:00 |
|
|
Cheng WanandClaude Opus 4.6
|
3f39b3d811
|
feat: add weekly workflow to update CI test est_time values (#22545)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-04-10 15:03:37 -07:00 |
|
|
Cheng WanandClaude Opus 4.6
|
6d95602ea3
|
Reduce GPU memory for MoE parallel groups (#22515)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-04-10 13:23:23 -07:00 |
|
|
Cheng WanandClaude Opus 4.6
|
37107bee6f
|
[Observability] Add pending token count to prefill log and get_load (#22480)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
2026-04-10 02:05:21 -07:00 |
|
|
Cheng Wan
|
c01ee848b0
|
Revert "fix: use consistent time denominator for throughput metrics in bench_one_batch_server" (#21276)
|
2026-03-23 22:14:54 -07:00 |
|
|
Cheng Wan
|
d6b12c401c
|
Revert "[bugfix] Fix PPMissingLayer AttributeError when Using PP" (#21189)
|
2026-03-22 23:28:36 -07:00 |
|
|
Cheng Wan
|
6e54361608
|
Refactor CUDA graph input buffers with shared buffer pool (#19180)
|
2026-02-24 02:24:40 -08:00 |
|
|
Cheng Wan
|
84c67c8be0
|
Refactor graph input buffers (#18991)
|
2026-02-20 18:09:31 -08:00 |
|
|
Cheng Wan
|
38ee749dd9
|
Fix adjust_num_token_non_padded_for_attn_tp returning CPU tensor (#19051)
|
2026-02-20 23:23:38 +08:00 |
|
|
Cheng Wan
|
b59a22f781
|
fix lint on main (#19052)
|
2026-02-20 15:30:57 +08:00 |
|
|
Cheng Wan
|
13a4a0406e
|
Fix flashinfer autotune to only wrap run_once() (#19004)
|
2026-02-19 20:02:21 -08:00 |
|
|
Cheng Wan
|
64bca5315f
|
Fix long prompt KV allocation by falling back to torch native APIs when exceeding Triton tensor limit (#18250)
|
2026-02-19 19:15:05 -08:00 |
|
|
Cheng Wan
|
73a7f0d049
|
Revert "Add SDAR model support" (#19032)
|
2026-02-19 16:03:56 -08:00 |
|
|
Cheng Wan
|
84c09913eb
|
Moving _alloc_extend_naive out of npu allocator (#18200)
|
2026-02-04 02:09:55 -08:00 |
|
|
Cheng Wan
|
ab8b99eb23
|
Refine logprob logic for request handling (#17986)
|
2026-02-01 19:11:52 -08:00 |
|
|
Cheng Wan
|
6a6b36367e
|
Fix logprob_start_len handling for prefill-only requests (#17395)
|
2026-01-29 15:14:43 -08:00 |
|
|
Cheng Wan
|
a416af4be7
|
Fix capture_sizes range for pcg (#17956)
|
2026-01-29 12:46:35 -08:00 |
|
|
Cheng Wan
|
2b461c15b4
|
Update logprob_start_len handling in scheduler (#16240)
|
2025-12-31 14:11:24 -08:00 |
|
|
Cheng Wan
|
5fb734f1a5
|
Enhance comments in set_extend_input_len method (#16130)
|
2025-12-29 20:47:33 -08:00 |
|
|
Cheng Wan
|
60f1ca6925
|
Refactor: Moving extend_logprob_start_len calculation out of prepare_for_extend (#16105)
|
2025-12-30 12:38:33 +08:00 |
|
|
Cheng Wan
|
c2e0913e17
|
Fix extend_input_len calculation in decode.py (#16103)
|
2025-12-29 13:13:24 -08:00 |
|
|
Cheng Wan
|
6f9d0a89a0
|
[scheduler] fix: correcting extend_logprob_start_len calculation (#15922)
|
2025-12-28 14:57:04 -08:00 |
|
|
Cheng Wan
|
c457aad54a
|
Update test parameters for deepep_large test (#16001)
|
2025-12-28 00:58:19 -08:00 |
|
|
Cheng Wan
|
2ec57cefd9
|
hotfix: add type hints to scheduler mixins (#15916)
|
2025-12-26 17:08:11 -08:00 |
|
|
Cheng Wan
|
988b14ca0e
|
refactor: add type hints to scheduler mixins (#15913)
|
2025-12-26 16:50:07 -08:00 |
|
|
Cheng Wan
|
5290cef97c
|
Fix condition check for require_gathered_buffer (#15328)
|
2025-12-17 12:42:47 -08:00 |
|
|
Cheng Wan
|
96cc10834a
|
[CI] update estimated elapsed time of some unittests (#14347)
|
2025-12-03 01:21:40 -08:00 |
|
|
Cheng WanandKangyan Zhou
|
0fe74af563
|
Remove incorrect deep_gemm assertions from server_args.py (#14113)
Co-authored-by: Kangyan Zhou <zky314343421@gmail.com>
|
2025-11-28 20:25:39 -08:00 |
|
|
Cheng Wan
|
b12c9e5c0a
|
Fix installation for nvidia-nvshmem-cu12 (#14033)
|
2025-11-26 18:27:12 -08:00 |
|
|
Cheng Wan
|
bf10869203
|
[Doc] Add an Introduction to Expert Parallelism (#13783)
|
2025-11-24 14:46:51 -08:00 |
|
|
Cheng WanandBaizhou Zhang
|
6d0e0b9bfc
|
[11/N] MoE Refactor: Simplifying SBO Implementation with Dispatcher Hooks (#13327)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-11-21 01:11:37 -08:00 |
|
|
Cheng WanandMick
|
a5ad0069b2
|
fix: change performance log directory to cache path (#13482)
Co-authored-by: Mick <mickjagger19@icloud.com>
|
2025-11-18 15:18:43 +08:00 |
|
|
Cheng Wan
|
2b8b9d8496
|
[CI] use cached deepep installation in gb200 CI (#13388)
|
2025-11-16 12:49:01 -08:00 |
|
|
Cheng Wan
|
13366843a4
|
[CI] check unit-test-backend-8-gpu-h20 in workflow (#13355)
|
2025-11-15 16:44:10 -08:00 |
|
|
Cheng Wan
|
83087247d1
|
[hotfix] missing w13_weight_fp8 and w2_weight_fp8 in UE8M0 requantization (#12259)
|
2025-10-28 19:10:38 -07:00 |
|
|
Cheng Wan
|
b1e13e7cea
|
[hotfix] Incorrect CombineOverlapArgs in SBO (#12230)
|
2025-10-28 01:23:06 -07:00 |
|
|
Cheng Wan
|
649949807f
|
[10/N] MoE Refactor: reorganize deepgemm runner in DeepEPMoE (#12054)
|
2025-10-24 19:16:17 -07:00 |
|
|
Cheng Wan
|
bfc3b3f786
|
[9/N] MoE Refactor: cleanup dispatcher interfaces (#11847)
|
2025-10-20 10:11:46 -07:00 |
|
|
Cheng Wan
|
5b214b50b6
|
[Refactor] move deep_gemm_wrapper out of quantization (#11784)
|
2025-10-17 18:57:54 -07:00 |
|
|
Cheng Wan
|
1bdd010291
|
Revert "Deprecate global_server_args_dict" (#11520)
|
2025-10-12 17:40:40 -07:00 |
|
|
Cheng Wan
|
6cd296940a
|
[lint] Fix the lint issue (#11516)
|
2025-10-12 16:22:46 -07:00 |
|
|
Cheng Wan
|
52fcbbb8bd
|
Revert "perf: optimize qwen-vl with symm mem allreduce" (#11436)
|
2025-10-10 12:30:05 -07:00 |
|
|
Cheng Wan
|
97cd38e58d
|
Skip weight loading in deepgemm compilation (#11312)
|
2025-10-07 21:52:46 -07:00 |
|
|
Cheng Wan
|
3c06b673af
|
[8/N] MoE Refactor: deprecate EPMoE (#11211)
|
2025-10-07 21:51:41 -07:00 |
|
|
Cheng Wan
|
41763ba079
|
Remove gdrcopy check in ci_install_deepep.sh (#11237)
|
2025-10-05 17:35:22 -07:00 |
|
|
Cheng Wan
|
6535fda127
|
[Profile] dump memory trace when cuda graph profile is enabled (#11083)
|
2025-09-29 17:36:48 -07:00 |
|
|
Cheng Wan
|
4844fac91d
|
Refactor TopK to ensure readability and extensibility (#9338)
|
2025-09-14 19:16:25 -07:00 |
|
|
Cheng Wan
|
2f8ba6fe82
|
[Fix] MoE: fix w8a8_fp8 MoE and add tests to cover this code path (#10429)
|
2025-09-14 17:34:28 -07:00 |
|
|
Cheng Wan
|
5a7e10fe4c
|
[MoE] fix: incorrect weight initialization for cutlass_fused_experts_fp8 (#10144)
|
2025-09-07 19:43:59 -07:00 |
|
|
Cheng Wan
|
a5a03209e9
|
Fix circular import (#10107)
|
2025-09-06 01:34:17 -07:00 |
|
|
Cheng Wanandgemini-code-assist[bot]
|
21af5c0404
|
[Fix] Compatibility between DP attention and pipeline parallelism (#10100)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-09-06 01:34:10 -07:00 |
|
|
Cheng Wan
|
3fa62da78c
|
[7/N] MoE Refactor: the implementation of new framework (#9269)
|
2025-09-05 21:09:09 -07:00 |
|
|
Cheng Wan
|
453511acc7
|
Save memory for expert model parallel (#9957)
|
2025-09-04 13:31:47 -07:00 |
|
|
Cheng Wan
|
84b006b278
|
Cleanup MoE Refactor (#9223)
|
2025-08-15 02:28:33 -07:00 |
|
|
Cheng Wan
|
e3e75a786a
|
Fix the deprecation warning for enable_flashinfer_mxfp4_moe (#9214)
|
2025-08-14 23:59:35 -07:00 |
|
|
Cheng Wan
|
295895120d
|
[6/N] MoE Refactor: Cleanup MoE-related configs (#8849)
|
2025-08-14 21:14:53 -07:00 |
|
|
Cheng Wan
|
b87aacb5c5
|
[DP Attention] Refactor: adding some utility functions (#9136)
|
2025-08-13 21:08:06 -07:00 |
|
|
Cheng Wan
|
5f5b3b2449
|
[5/n] DP Enhancement: Correct num_token_non_padded (#9107)
|
2025-08-12 12:23:46 -07:00 |
|
|
Cheng Wanandgemini-code-assist[bot]
|
f003cd3548
|
[CI] Fix CI tests (#9050)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
2025-08-10 23:52:05 -07:00 |
|
|
Cheng Wan
|
5018809222
|
[DP] fix: engine crash when decode batch is padded (#8995)
|
2025-08-09 01:29:29 -07:00 |
|
|
Cheng Wan
|
a47baff12c
|
[hotfix] use the original implementation in 8785 (#8994)
|
2025-08-08 21:47:25 -07:00 |
|
|
Cheng Wan
|
fd7e15b76d
|
Revert "[bug fix] Ensure local token and global token buffers are pointing to different storage " (#8993)
|
2025-08-08 21:34:17 -07:00 |
|
|
Cheng Wan
|
1d24db8348
|
Expert Parallelism for GPT-OSS (#8944)
|
2025-08-08 00:46:42 -07:00 |
|
|
Cheng Wan
|
5b6acc1495
|
fix glm4 moe (#8883)
|
2025-08-06 18:02:31 -07:00 |
|
|
Cheng Wan
|
78aad91037
|
[CI] fix pip upgrade (#8881)
|
2025-08-06 15:02:32 -07:00 |
|
|
Cheng Wan
|
b102353f8f
|
[MoE] Enable renormalize=False in Triton kernels (#8735)
|
2025-08-03 17:03:04 -07:00 |
|
|
Cheng Wan
|
0e0eef00ce
|
[DP] fix the compatibility issue between DP attention and --attention-backend triton (#8723)
|
2025-08-03 03:06:57 -07:00 |
|
|
Cheng Wan
|
cb099d2095
|
[CUDA Graph] save cuda graph memory by using next_token_logits_buffer (#8579)
|
2025-08-03 03:06:47 -07:00 |
|
|
Cheng Wan
|
7a91330149
|
Save cuda graph memory for fa3 (#8567)
|
2025-08-03 03:06:31 -07:00 |
|
|
Cheng Wan
|
a437aa9987
|
[hotfix] fix mixtral with tensor-level compressed-tensor quantization (#8721)
|
2025-08-02 22:59:25 -07:00 |
|
|
Cheng Wan
|
6c88f6c8d9
|
[5/N] MoE Refactor: Update MoE parallelism arguments (#8658)
|
2025-08-01 01:20:03 -07:00 |
|
|
Cheng Wan
|
7a1f7fc504
|
[Feature] Hybrid EP and TP (#8590)
|
2025-07-31 02:53:25 -07:00 |
|
|
Cheng Wan
|
32fa1e9cc2
|
[4/N] MoE Refactor: Unified Triton Kernel for FusedMoE and EPMoE (#8515)
|
2025-07-31 02:34:02 -07:00 |
|
|
Cheng Wan
|
e179e0b797
|
update sgl-kernel for EP: python part (#8550)
|
2025-07-31 00:14:39 -07:00 |
|
|
|
a5f5ab4030
|
update sgl-kernel for EP: kernel part (#8514)
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
|
2025-07-30 22:19:55 -07:00 |
|
|
Cheng Wan
|
9effeb5bdd
|
Support EPLB in FusedMoE (#8448)
|
2025-07-29 16:02:41 -07:00 |
|
|
Cheng Wan
|
9c138a0445
|
[3/N] MoE Refactor: Simplify DeepEP Output (#8421)
|
2025-07-28 11:37:17 -07:00 |
|
|
Cheng Wan
|
4d921f2b79
|
[hotfix] fix merge conflicts in FlashInferEPMoE (#8405)
|
2025-07-27 01:24:10 -07:00 |
|
|
Cheng Wan
|
bf0f448fe5
|
[2/N] MoE Refactor: Unify weight loader and quant methods (#8397)
|
2025-07-27 01:00:21 -07:00 |
|
|
Cheng Wan
|
e236d8fee8
|
Save peak memory in logits processor (#8343)
|
2025-07-26 01:46:42 -07:00 |
|
|
Cheng Wan
|
c0fb25e949
|
DP Enhancement (#8280)
|
2025-07-24 21:36:21 -07:00 |
|
|
Cheng Wan
|
abda2542d5
|
Fix tuning_fused_moe_triton.py (#8175)
|
2025-07-19 17:33:50 -07:00 |
|
|
Cheng Wan
|
15ad6c9086
|
[1/N] MoE Refactor: refactor select_experts (#7966)
|
2025-07-19 00:51:15 -07:00 |
|
|
Cheng Wan
|
49b8777460
|
Refactor: move all quantization-related code to srt/layer/quantization (#7989)
|
2025-07-17 00:47:07 -07:00 |
|
|
Cheng Wan
|
02404a1e35
|
[ci] recover 8-gpu deepep test (#8105)
|
2025-07-17 00:46:40 -07:00 |
|
|
Cheng Wan
|
9069884b51
|
[ci] disable memory imbalance check for draft worker (#8108)
|
2025-07-16 20:41:47 -07:00 |
|
|
Cheng Wan
|
98aa836bbf
|
Overlap the gating function with shared experts in DeepSeek (#7978)
|
2025-07-12 13:41:50 -07:00 |
|
|
Cheng Wan
|
475a249bb8
|
temporarily disable deepep-8-gpu and activate two small tests (#7961)
|
2025-07-11 14:22:05 -07:00 |
|
|
Cheng Wan
|
d487555f84
|
[CI] Add deepep tests to CI (#7872)
|
2025-07-09 01:49:47 -07:00 |
|
|
Cheng Wan
|
6c903611ca
|
Fix incorrect spec_num_draft_tokens in draft_extend (#7757)
|
2025-07-05 02:18:16 -07:00 |
|
|
Cheng Wan
|
8fc910db03
|
DP Attention with Auto DeepEP Dispatch (#7222)
|
2025-07-05 01:54:24 -07:00 |
|
|
Cheng Wan
|
cb432f1770
|
saving hidden_states.clone() (#7705)
|
2025-07-04 20:07:42 -07:00 |
|
|
Cheng Wan
|
1b8cf77b01
|
[Fix] incorrect assert in EPLB (#7575)
|
2025-06-26 14:59:20 -07:00 |
|
|
Cheng Wan
|
f8d48fd311
|
Fix dtype for idle input in spec decoding (#7456)
|
2025-06-23 11:23:25 -07:00 |
|