 fzyzcjy
|
44ffe2cb72
|
Install py-spy by default for containers for easier debugging (#9649)
|
2025-08-26 10:40:52 -07:00 |
|
|
fzyzcjy
|
9dcdf5da03
|
Tiny fix wrong comments (#9589)
|
2025-08-25 03:08:10 -07:00 |
|
|
fzyzcjy
|
71a7f1d86f
|
Offload tensors by sharding on GPU (#9536)
|
2025-08-25 00:02:49 -07:00 |
|
|
fzyzcjy
|
433266c125
|
Reintroduce memory usage fix (#9535)
|
2025-08-25 00:02:31 -07:00 |
|
|
fzyzcjy
|
b1b3f0b38f
|
Partially unify triton per token group quant kernels (#9485)
|
2025-08-23 02:07:31 -07:00 |
|
|
fzyzcjy
|
34e5e11f0f
|
Tiny make device_loading_context more static (#9478)
|
2025-08-23 02:07:15 -07:00 |
|
|
fzyzcjy
|
2600fc0d47
|
Overlapped weight offload (#8034)
|
2025-08-23 02:06:46 -07:00 |
|
|
fzyzcjy
|
0374304a2c
|
Add enable_flashinfer_mxfp4_bf16_moe for higher precision and slower moe backend (#9004)
|
2025-08-23 15:38:40 +08:00 |
|
|
fzyzcjy
|
e85cb1ce9d
|
Fix quant kernel test errors and benchmark wrong output speeds (#7604)
|
2025-08-21 03:48:41 -07:00 |
|
|
fzyzcjy
|
55d336cb08
|
Refactor weight offloading logic (#8521)
|
2025-08-21 03:48:13 -07:00 |
|
|
fzyzcjy
|
42c8704560
|
Add PDL support for quant kernel and rope kernel (#9106)
|
2025-08-20 01:56:29 -07:00 |
|
|
fzyzcjy
|
fe43e889f8
|
Fix mini lb timeout issue (#9369)
|
2025-08-19 20:15:16 -07:00 |
|
|
fzyzcjy
|
5626e20b2b
|
Tiny fix CI (#9306)
|
2025-08-18 16:54:36 -07:00 |
|
|
fzyzcjy
|
4c0bb411e5
|
Further fix memory pool leak error (#9298)
|
2025-08-18 00:58:06 -07:00 |
|
|
fzyzcjy
|
b498cd21d7
|
Tiny make fp4 moe method parameters more static (#8520)
|
2025-08-17 13:26:02 -07:00 |
|
|
fzyzcjy
|
b3c1f2e4f2
|
Fix memory pool leak error (#9271)
|
2025-08-17 12:53:34 -07:00 |
|
|
fzyzcjy
|
f8644a5632
|
Tiny update tmux history limit on dev container (#9218)
|
2025-08-15 00:22:08 -07:00 |
|
|
fzyzcjy
|
392de007cb
|
Minor fix docker container DeepEP on multi platforms (#9205)
|
2025-08-14 17:41:49 -07:00 |
|
|
fzyzcjy
|
b3363cc1aa
|
Fix docker container DeepEP error on Blackwell (#9171)
|
2025-08-13 21:06:48 -07:00 |
|
|
fzyzcjy
|
9394ed6386
|
Fix gpt-oss ~2x memory consumption issue (#9146)
|
2025-08-13 00:11:43 -07:00 |
|
|
fzyzcjy
|
9aea255522
|
Fuse writing KV buffer into rope kernel (part 1: sgl-kernel) (#9077)
|
2025-08-12 01:46:40 -07:00 |
|
|
fzyzcjyandQiaolin-Yu
|
5190ba7f42
|
Fuse two kernels of hidden states padding into quantization kernel (#9005)
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
|
2025-08-12 01:20:13 -07:00 |
|
|
fzyzcjy
|
d3e67deb1b
|
Fix redundant kernel in sink dtype conversion (#8966)
|
2025-08-09 00:34:49 -07:00 |
|
|
fzyzcjy
|
442534aa44
|
Add CI for gpt-oss model on hopper (#8851)
|
2025-08-09 00:34:23 -07:00 |
|
|
fzyzcjy
|
774b47f3f1
|
Reduce scheduler recv requests overhead (#8947)
|
2025-08-08 00:10:05 -07:00 |
|
|
fzyzcjy
|
482c3db29f
|
Fix sgl-kernel arch and missing package in CI (#8869)
|
2025-08-07 02:08:15 -07:00 |
|
|
fzyzcjy
|
b114a8105b
|
Support B200 in CI (#8861)
|
2025-08-06 21:42:44 +08:00 |
|
|
fzyzcjy
|
0e612dbf12
|
Tiny fix CI pytest error (#8524)
|
2025-08-02 22:48:42 -07:00 |
|
|
fzyzcjy
|
8ada1ab6c7
|
Fix triton moe error caused by TopK refactor (#8705)
|
2025-08-02 18:49:47 -07:00 |
|
|
fzyzcjy
|
403566bcca
|
Remove assertions about per group quant fp8 (#8717)
|
2025-08-02 17:08:40 -07:00 |
|
|
fzyzcjy
|
0ce84c822b
|
Support colocating requests (#7973)
|
2025-07-28 22:51:49 -07:00 |
|
|
fzyzcjy
|
59d0bf012f
|
Tiny add warnings for DeepEP when it is suboptimal (#8426)
|
2025-07-28 22:51:38 -07:00 |
|
|
fzyzcjy
|
7df2c0c2db
|
Reduce memory usage for fp4 moe (#8413)
|
2025-07-28 22:51:23 -07:00 |
|
|
fzyzcjy
|
b58c3c285e
|
Support ue8m0 for triton quant kernel (#7603)
|
2025-07-27 13:04:35 -07:00 |
|
|
fzyzcjy
|
e34cf6ad75
|
Fix bench script making input data on L2 cache (#7739)
|
2025-07-27 00:30:24 -07:00 |
|
|
fzyzcjy
|
62222bd27e
|
Minor tool for comparison of benchmark results (#7974)
|
2025-07-27 00:27:50 -07:00 |
|
|
fzyzcjy
|
ed0fdbf35b
|
Tool to dump and compare internal activation tensors (#7976)
|
2025-07-27 00:27:21 -07:00 |
|
|
fzyzcjy
|
da0c026084
|
Tiny assert EPLB is used together with expert parallel (#8381)
|
2025-07-26 03:20:39 -07:00 |
|
|
fzyzcjy
|
519ff5c8e6
|
Super tiny fix typo (#8046)
|
2025-07-17 21:15:51 +08:00 |
|
|
fzyzcjy
|
c46e069d34
|
Tiny fix mooncake log warning wrong output (#7952)
|
2025-07-12 21:22:44 -07:00 |
|
|
fzyzcjy
|
2a2d3478af
|
Fix wrong gemm branch cause 250us slower (#7969)
|
2025-07-11 19:45:09 -07:00 |
|
|
fzyzcjy
|
880221bd3b
|
Revert "[PD Disaggregation] replace transfer with batch transfer for better performance (#7236)" (#7968)
|
2025-07-11 19:03:01 -07:00 |
|
|
fzyzcjy
|
a291439a59
|
Support logprobs in two-batch overlap (#7709)
|
2025-07-05 19:05:32 -07:00 |
|
|
fzyzcjy
|
0c9c6c75a8
|
Move files related to EPLB (#7580)
|
2025-06-29 15:39:38 -07:00 |
|
|
fzyzcjy
|
00c7b1ad07
|
Let EP prefill support new DeepGEMM (#7310)
|
2025-06-28 01:45:30 -07:00 |
|
|
fzyzcjy
|
82eccae44e
|
Let ep_scatter support arbitrary strides / ue8m0 format (#7309)
|
2025-06-28 01:38:33 -07:00 |
|
|
fzyzcjy
|
9efb2993da
|
Tiny add logs for expert location updater (#7308)
|
2025-06-27 19:12:33 -07:00 |
|
|
fzyzcjy
|
3d7cdb2ebd
|
Fix MTP error when enabling two-batch overlap (#7569)
|
2025-06-26 15:40:54 -07:00 |
|
|
fzyzcjy
|
edc21cc8ae
|
Tiny add logging for GC (#7406)
|
2025-06-22 12:40:02 +08:00 |
|
|
fzyzcjy
|
650127a173
|
Reintroduce tiny fix sampler error when prob is not contiguous (#7354)
|
2025-06-19 09:39:34 -07:00 |
|
|
fzyzcjy
|
9c6a0656a3
|
Fix profiler error when there are idle passes (#7003)
|
2025-06-18 10:55:01 -07:00 |
|
|
fzyzcjy
|
91a066ec6a
|
Tiny remove comments about DeepEP on H20 (#7234)
|
2025-06-16 09:13:57 -07:00 |
|
|
fzyzcjy
|
349bb2c92a
|
Fix error when disabling new DeepGEMM (#7198)
|
2025-06-14 19:24:54 -07:00 |
|
|
fzyzcjy
|
0b8939bc2c
|
Fix NCCL 2.27.3 not in docker image (#7195)
|
2025-06-14 18:36:11 -07:00 |
|
|
fzyzcjy
|
b57d87c297
|
Fix shared experts fusion + weight requant (#7177)
|
2025-06-14 02:35:18 -07:00 |
|
|
fzyzcjy
|
93cec4335f
|
Support new DeepGEMM (#7172)
|
2025-06-13 23:00:17 -07:00 |
|
|
fzyzcjy
|
b4c41f7276
|
Refactor DeepGEMM integration (#7150)
|
2025-06-13 20:41:03 -07:00 |
|
|
fzyzcjy
|
8b8f2e7463
|
Support new DeepGEMM input format in silu_and_mul_masked_post_quant_fwd (#7153)
|
2025-06-13 20:40:24 -07:00 |
|
|
fzyzcjy
|
5b1afa7814
|
Re-quantize DeepSeek model weights to support DeepGEMM new input format (#7156)
|
2025-06-13 15:57:45 -07:00 |
|
|
fzyzcjy
|
c49c1d9226
|
Remove 200us slow concat kernel (part 2: srt) (#7020)
|
2025-06-13 15:19:31 -07:00 |
|
|
fzyzcjy
|
0f1dfa1efe
|
Tiny add sanity checks for DeepGEMM inputs (#7157)
|
2025-06-13 14:36:27 -07:00 |
|
|
fzyzcjy
|
e3ec6bf4b6
|
Minor speed up block_quant_dequant (#6814)
|
2025-06-13 14:32:46 -07:00 |
|
|
fzyzcjy
|
b04df75acd
|
Fix DeepEP error in some environments (#7154)
|
2025-06-13 14:27:25 -07:00 |
|
|
fzyzcjy
|
bec3e48402
|
Support new DeepGEMM format in per token group quant (part 2: srt) (#7155)
|
2025-06-13 14:25:40 -07:00 |
|
|
fzyzcjy
|
5c66c4424f
|
Support new DeepGEMM format in per token group quant (#7146)
|
2025-06-13 02:00:22 -07:00 |
|
|
fzyzcjy
|
aa46ed34d2
|
Remove 200us slow concat kernel (part 1: kernel) (#7145)
|
2025-06-13 01:58:29 -07:00 |
|
|
fzyzcjy
|
da47621ccc
|
Minor speedup topk postprocessing (#7058)
|
2025-06-13 00:50:18 -07:00 |
|
|
fzyzcjy
|
b02df20a8d
|
Minor add utility to read expert distribution recorder output (#7134)
|
2025-06-12 16:50:05 -07:00 |
|
|
fzyzcjy
|
fe55947acd
|
Add sanity checks when a test file is not added to CI (#6947)
|
2025-06-10 12:34:57 -07:00 |
|
|
fzyzcjy
|
19995dd78e
|
Tiny fix cutlass_mla_get_workspace_size stub incorrect signature (#7057)
|
2025-06-10 12:27:57 -07:00 |
|
|
fzyzcjy
|
f6ebba537a
|
Support both approximate and exact expert distribution collection (#6964)
|
2025-06-09 20:56:17 -07:00 |
|
|
fzyzcjy
|
e58423b2b9
|
Fix cutlass MLA gets almost zero accuracy (#6998)
|
2025-06-09 10:16:29 -07:00 |
|
|
fzyzcjy
|
de1350ea20
|
Minor remove one kernel for DeepSeek (#6977)
|
2025-06-08 17:41:35 -07:00 |
|
|
fzyzcjy
|
86fe943bc3
|
Fix expert distribution dumping causes OOM (#6967)
|
2025-06-08 17:41:14 -07:00 |
|
|
fzyzcjy
|
2fc1299562
|
Remove unnecessary kernels of num_token_non_padded (#6965)
|
2025-06-08 05:09:17 -07:00 |
|
|
fzyzcjy
|
f5599ef124
|
Refactor global_server_args_dict (#6866)
|
2025-06-07 03:10:35 -07:00 |
|
|
fzyzcjy
|
c499591ac8
|
Add canary for EPLB rebalancing (#6895)
|
2025-06-07 03:09:33 -07:00 |
|
|
fzyzcjy
|
d5c097a2f9
|
Tiny re-introduce profile id logging (#6912)
|
2025-06-07 02:32:50 -07:00 |
|
|
fzyzcjy
|
2f715f51cc
|
Minor compile fused topk (#6944)
|
2025-06-07 01:40:38 -07:00 |
|
|
fzyzcjy
|
61ce91ed28
|
Tiny support customize DeepEP max dispatch tokens per rank (#6934)
|
2025-06-06 17:18:35 -07:00 |
|
|
fzyzcjy
|
35b65cf0ca
|
Use deepgemm instead of triton for fused_qkv_a_proj_with_mqa (#6890)
|
2025-06-05 11:37:05 -07:00 |
|
|
fzyzcjy
|
0166403c20
|
Support Blackwell DeepEP docker images (#6868)
|
2025-06-05 00:07:53 -07:00 |
|
|
fzyzcjy
|
bcf66ef3e1
|
Tiny allow profiler API to auto create directory (#6865)
|
2025-06-05 00:07:03 -07:00 |
|
|
fzyzcjy
|
0de5e7d40f
|
Support layerwise rebalancing experts (#6851)
|
2025-06-05 00:05:52 -07:00 |
|
|
fzyzcjy
|
72a110f664
|
Tiny update error hints (#6846)
|
2025-06-05 00:05:28 -07:00 |
|
|
fzyzcjy
|
5aff1e9392
|
Fix Qwen3MoE missing token padding optimization (#6820)
|
2025-06-05 00:04:59 -07:00 |
|
|
fzyzcjy
|
b6d0ce9f78
|
Minor add metrics to expert location updater (#6816)
|
2025-06-02 23:59:11 -07:00 |
|
|
fzyzcjy
|
0ea330ca34
|
Fix wrong weight reference in dynamic EPLB (#6818)
|
2025-06-02 23:26:04 -07:00 |
|
|
fzyzcjy
|
df7f61ee7d
|
Speed up rebalancing when using non-static dispatch algorithms (#6812)
|
2025-06-02 11:18:17 -07:00 |
|
|
fzyzcjy
|
ef21729c1d
|
Fix profiles do not have consistent names (#6811)
|
2025-06-02 11:17:22 -07:00 |
|
|
fzyzcjy
|
f5159315b2
|
Add simple utility to dump tensors for debugging (#6815)
|
2025-06-02 11:15:31 -07:00 |
|
|
fzyzcjy
|
6d7b6696d4
|
Tiny fix EPLB assertion about rebalancing period and recorder window size (#6813)
|
2025-06-02 11:13:33 -07:00 |
|
|
fzyzcjy
|
6376b632eb
|
Tiny log prefill time (#6780)
|
2025-06-02 10:28:27 -07:00 |
|
|
fzyzcjy
|
e05e29d178
|
Refactor CustomOp to avoid confusing bugs (#5382)
|
2025-06-02 10:27:36 -07:00 |
|
|
fzyzcjy
|
2c3b71d678
|
Improve EPLB logical to physical dispatch map (#6727)
|
2025-05-29 19:23:54 -07:00 |
|
|
fzyzcjy
|
6df81e8a39
|
Support tuning DeepEP configs (#6742)
|
2025-05-29 08:12:22 -07:00 |
|
|
fzyzcjy
|
3ab7d9b55e
|
Support picking variants of EPLB algorithms (#6728)
|
2025-05-29 08:12:01 -07:00 |
|
|
fzyzcjy
|
7e5071c92a
|
Super tiny enable sole usage of expert distribution metrics and update doc (#6680)
|
2025-05-29 08:11:38 -07:00 |
|
|
fzyzcjyandCheng Wan
|
31589e177e
|
Speed up when having padding tokens two-batch overlap (#6668)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-05-28 16:00:58 -07:00 |
|
|
fzyzcjy
|
ae6a5b2950
|
Minor refactor two-batch overlap (#6682)
|
2025-05-28 15:54:17 -07:00 |
|