 fzyzcjy
|
2a3763c335
|
Tiny fix sgl-kernel related CI installing the wrong binary (#12283)
|
2025-10-28 10:29:06 -07:00 |
|
|
|
691c8534cf
|
Support releasing CUDA graph memory when paused (#7873)
Co-authored-by: ryang-max <y1cunhui.yang@gmail.com>
Co-authored-by: ryang <38470282+ryang-max@users.noreply.github.com>
|
2025-10-28 14:40:50 +08:00 |
|
|
fzyzcjy
|
326c84c493
|
Compiling rope while preserving true on policy (#12161)
|
2025-10-28 08:02:17 +08:00 |
|
|
fzyzcjy
|
0103f374ba
|
Support DeepGEMM for deterministic inference (#12142)
|
2025-10-26 22:36:17 +08:00 |
|
|
fzyzcjy
|
c001deba37
|
Make bmm batch invariant injection optional (#12118)
|
2025-10-26 10:18:35 +08:00 |
|
|
fzyzcjy
|
20bd2271e2
|
Support true on-policy (#12058)
|
2025-10-25 10:23:42 +08:00 |
|
|
fzyzcjy
|
d7056c5236
|
Enhance tests in deterministic kernels (#12070)
|
2025-10-25 08:53:22 +08:00 |
|
|
fzyzcjy
|
e04340bf48
|
Fix multi processing serializer bug (#11958)
|
2025-10-24 22:53:45 +08:00 |
|
|
fzyzcjy
|
2342605ef0
|
Tiny cleanup send_single (#12056)
|
2025-10-23 23:53:42 -07:00 |
|
|
fzyzcjy
|
0f0c430e93
|
Install numactl in Dockerfile for GH200/GB200/GB300 (#11853)
|
2025-10-23 21:39:10 -07:00 |
|
|
fzyzcjy
|
8612811d85
|
Bump grace blackwell DeepEP version (#11990)
|
2025-10-22 21:08:12 -07:00 |
|
|
fzyzcjy
|
0917c5da8c
|
Support mixing cutedsl and deepgemm backend (#11807)
|
2025-10-21 07:38:35 +08:00 |
|
|
fzyzcjy
|
9e3be1fa2a
|
Tiny bump DeepEP version in ARM blackwell (#11810)
|
2025-10-20 08:15:14 +08:00 |
|
|
fzyzcjy
|
a8ba32798e
|
Fix triton_kernels import error on some hardwares (#11831)
|
2025-10-20 08:14:47 +08:00 |
|
|
fzyzcjy
|
12eb02e982
|
Change bf16 to fp8 for some gemms in attention for DeepSeek ckpt v2 (#11805)
|
2025-10-19 16:15:13 +08:00 |
|
|
fzyzcjyandYineng Zhang
|
002d037359
|
Avoid generation gets hanging when user specifies multiple event loops (#5162)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-10-19 16:12:49 +08:00 |
|
|
fzyzcjy
|
a27825ae01
|
Support not officially supported high sgl-kernel version with low srt version (#11786)
|
2025-10-19 16:11:59 +08:00 |
|
|
fzyzcjy
|
ce399e154c
|
Make single-batch overlap compatible with NextN (#11804)
|
2025-10-19 16:10:44 +08:00 |
|
|
fzyzcjy
|
ea6275dfbc
|
Tiny add hints when users send requests to wrong place (#11808)
|
2025-10-19 16:10:20 +08:00 |
|
|
fzyzcjy
|
a7043c6f0d
|
Bump torch_memory_saver to avoid installing pre-release versions (#11797)
|
2025-10-18 01:20:42 -07:00 |
|
|
fzyzcjy
|
dbbd4e1891
|
Try add back no-commit-to-branch (#11799)
|
2025-10-18 12:05:12 +08:00 |
|
|
fzyzcjy
|
6c7c92eb02
|
Enable lint on main (#11794)
|
2025-10-17 19:08:50 -07:00 |
|
|
fzyzcjy
|
33e9bbec35
|
Make single-batch overlap compatible with offloading (#11614)
|
2025-10-18 08:45:54 +08:00 |
|
|
fzyzcjy
|
dcb8f090ad
|
Super tiny fix CI (#11788)
|
2025-10-17 17:41:58 -07:00 |
|
|
fzyzcjy
|
8af8491298
|
Support casting bf16 NextN moe to fp8 (#11613)
|
2025-10-18 08:02:15 +08:00 |
|
|
fzyzcjy
|
505329cab0
|
Support shared experts overlap in cutlass moe (#11611)
|
2025-10-18 07:59:40 +08:00 |
|
|
fzyzcjy
|
8a382fd399
|
Super tiny fix missing input throughput (#11607)
|
2025-10-18 07:58:48 +08:00 |
|
|
fzyzcjy
|
32803fb279
|
Super tiny improve FA3 import error message (#11590)
|
2025-10-14 22:06:31 -07:00 |
|
|
fzyzcjy
|
cb8ed2c09a
|
Make DeepEP combine recv do not overlap (#11535)
|
2025-10-13 18:40:42 -07:00 |
|
|
fzyzcjy
|
065ce81574
|
Tiny cleanup fp4 gemm calls (#11537)
|
2025-10-13 14:48:22 -07:00 |
|
|
fzyzcjy
|
bf3e7149be
|
Fix enable_v2 in int8 quant (#11470)
|
2025-10-11 21:56:30 +08:00 |
|
|
fzyzcjy
|
d957177a22
|
Super tiny delete unused openai router in sgl-router (#11448)
|
2025-10-11 15:59:30 +08:00 |
|
|
fzyzcjyandYineng Zhang
|
21337b22b9
|
Reland [1/2] Optimizations and refactors about quant kernel (#10312)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-10-11 15:59:03 +08:00 |
|
|
+2
|
efbc687c28
|
Support DeepSeek V3.2 Exp (#11061)
Co-authored-by: Stefan He <11166516+hebiao064@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <95566987+hnyls2002@users.noreply.github.com>
Co-authored-by: Baizhou Zhang <56809903+fridge003@users.noreply.github.com>
Co-authored-by: DarkSharpness <76582120+darksharpness@users.noreply.github.com>
Co-authored-by: ZhengdQin <46387172+zhengdqin@users.noreply.github.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Zhengda Qin <zhengdqin@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-10-06 00:24:15 -07:00 |
|
|
fzyzcjy
|
2f80bd9f0e
|
Bump torch_memory_saver 0.0.9rc2 (#11252)
|
2025-10-05 20:26:20 -07:00 |
|
|
fzyzcjy
|
fdc4e1e570
|
Tiny move files to utils folder (#11166)
|
2025-10-03 22:40:06 +08:00 |
|
|
fzyzcjy
|
6794d21051
|
Tiny add PD disaggregation + DP attention test (#11167)
|
2025-10-03 14:15:46 +08:00 |
|
|
fzyzcjy
|
afcd3e1089
|
Tiny remove duplicated code (#11164)
|
2025-10-02 21:56:31 +08:00 |
|
|
fzyzcjy
|
12d6818380
|
Tiny fix ep_gather behavior different in CI (#11130)
|
2025-10-02 21:55:53 +08:00 |
|
|
fzyzcjy
|
b65db0287b
|
Tiny cleanup deepseek_v2.py (#11163)
|
2025-10-02 21:54:52 +08:00 |
|
|
fzyzcjy
|
5e786cca3a
|
Support single batch overlap (#10422)
|
2025-10-02 18:04:36 +08:00 |
|
|
fzyzcjyandKaixi Hou
|
0b9dfba787
|
Support dispatch low latency (#10263)
Co-authored-by: Kaixi Hou <4001424+kaixih@users.noreply.github.com>
|
2025-10-02 18:02:19 +08:00 |
|
|
fzyzcjy
|
2ac453b07f
|
Tiny detect slow ranks (#10508)
|
2025-10-02 18:00:33 +08:00 |
|
|
fzyzcjy
|
f35def8652
|
Fuse quantize and rope in trtllm_mla MTP (#10779)
|
2025-10-02 17:59:37 +08:00 |
|
|
fzyzcjy
|
d61615fe93
|
Tiny fix missing alt stream in nextn layer (#10768)
|
2025-10-02 17:58:23 +08:00 |
|
|
fzyzcjy
|
b1ccaf01cd
|
Tiny improve dumper (#11132)
|
2025-10-02 17:55:01 +08:00 |
|
|
fzyzcjy
|
44b1fbe258
|
Fix DeepSeek chunked prefill memory issue (#11149)
|
2025-10-01 23:56:59 -07:00 |
|
|
fzyzcjy
|
063c3791fe
|
Fix trtllm_mla slow concat kernel in MTP (#10777)
|
2025-09-22 22:47:49 -07:00 |
|
|
fzyzcjy
|
720c1c8ca3
|
Super tiny fix extra logs (#10697)
|
2025-09-20 21:30:54 -07:00 |
|
|
fzyzcjy
|
311de47bb7
|
[2/2] Speed up trtllm_mla attention backend (#10474)
|
2025-09-16 15:49:22 -07:00 |
|
|
fzyzcjy
|
8df7353af3
|
Support sgl-router parallel_batch in bench_one_batch_server (#10506)
|
2025-09-16 02:52:57 -07:00 |
|
|
fzyzcjy
|
ae4be601c2
|
Fix CI when sgl-kernel is changed but srt is not changed (#10515)
|
2025-09-16 02:49:54 -07:00 |
|
|
fzyzcjy
|
3b25dc127a
|
[1/2] Speed up trtllm_mla attention backend (>10% e2e) (#10473)
|
2025-09-15 11:53:21 -07:00 |
|
|
fzyzcjy
|
059c13de5c
|
Fix trtllm_moe wrong correction bias (#10440)
|
2025-09-15 01:02:05 -07:00 |
|
|
fzyzcjy
|
010181388c
|
Tiny fix wrong naming (#10437)
|
2025-09-14 19:24:41 -07:00 |
|
|
fzyzcjy
|
ca63f075b7
|
Revert "Fix FA4 import cause moe_fused_gate output be illegal memory" (#10432)
|
2025-09-14 19:03:27 -07:00 |
|
|
fzyzcjy
|
258d02c86d
|
Fix correction bias undefined behavior for nvfp4 models (#10426)
|
2025-09-14 18:41:09 -07:00 |
|
|
fzyzcjy
|
e3cf812f7d
|
Fix sgl-kernel + srt CI (#10419)
|
2025-09-14 01:44:47 -07:00 |
|
|
fzyzcjy
|
4da5533682
|
Support profile args in Engine API (#6539)
|
2025-09-14 01:21:10 -07:00 |
|
|
fzyzcjy
|
ac964d2e58
|
Support global scale in addition to per expert scale for cutedsl moe (#10270)
|
2025-09-14 01:17:00 -07:00 |
|
|
fzyzcjy
|
fa46e2bd40
|
Support offloading in fp8 (#9948)
|
2025-09-14 01:14:28 -07:00 |
|
|
fzyzcjy
|
b047b553c2
|
[2/2] Speed up prefill mla attention concat (#10157)
|
2025-09-14 01:12:04 -07:00 |
|
|
fzyzcjy
|
a0f844ed5a
|
Let sgl-kernel changes be tested on srt (#10313)
|
2025-09-14 01:09:17 -07:00 |
|
|
fzyzcjy
|
2df532ef20
|
Fix the global scale fix does not support EPLB and improve enabling condition (#10369)
|
2025-09-14 01:07:47 -07:00 |
|
|
fzyzcjy
|
abea9250da
|
Auto determine sgl kernel version in blackwell CI (#10318)
|
2025-09-14 01:06:30 -07:00 |
|
|
fzyzcjy
|
72dfa96aeb
|
Fix cutlass moe accuracy drop caused by attention UB from DP padding mode (#10414)
|
2025-09-13 22:29:09 -07:00 |
|
|
fzyzcjy
|
efedbe6ca9
|
Fix global input scale incompatible with CuTe DSL moe (#10370)
|
2025-09-12 03:22:49 -07:00 |
|
|
fzyzcjy
|
3a77c80b26
|
Fix FA4 import cause moe_fused_gate output be illegal memory (#10368)
|
2025-09-12 03:21:26 -07:00 |
|
|
fzyzcjy
|
0096798ed6
|
[1/2] Speed up prefill mla attention (#10156)
|
2025-09-08 09:00:33 -07:00 |
|
|
fzyzcjy
|
bc5fc332f7
|
Fix slow fused add RMSNorm (#10141)
|
2025-09-07 20:20:39 -07:00 |
|
|
fzyzcjy
|
bd7f882142
|
Support copying tensor from cpu to gpu without using copy engines (#10007)
|
2025-09-05 20:07:19 +08:00 |
|
|
fzyzcjyandYineng Zhang
|
5e5c30d9ab
|
Tiny let DeepGEMM scale checks cover more cases (#7182)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-09-05 19:52:32 +08:00 |
|
|
fzyzcjy
|
9f00ec44eb
|
Fix and enhance dumper (#8725)
|
2025-09-05 19:51:09 +08:00 |
|
|
fzyzcjy
|
8e85ee887e
|
Support simple evals in text comparator (#8867)
|
2025-09-05 19:50:21 +08:00 |
|
|
fzyzcjy
|
adf73175d6
|
Forbid DeepEP racing condition when too many tokens (#9567)
|
2025-09-05 19:47:05 +08:00 |
|
|
fzyzcjy
|
df97b31f37
|
Tiny support setting numa nodes for different ranks (#10006)
|
2025-09-05 19:01:27 +08:00 |
|
|
fzyzcjy
|
339f8eef09
|
[1/2] Optimizations and refactors about quant kernel (#9534)
|
2025-09-05 18:45:08 +08:00 |
|
|
fzyzcjy
|
065e523d7b
|
Tiny allow DeepGEMM on cu12.9 (#9858)
|
2025-08-31 23:29:56 -07:00 |
|
|
fzyzcjy
|
44ffe2cb72
|
Install py-spy by default for containers for easier debugging (#9649)
|
2025-08-26 10:40:52 -07:00 |
|
|
fzyzcjy
|
9dcdf5da03
|
Tiny fix wrong comments (#9589)
|
2025-08-25 03:08:10 -07:00 |
|
|
fzyzcjy
|
71a7f1d86f
|
Offload tensors by sharding on GPU (#9536)
|
2025-08-25 00:02:49 -07:00 |
|
|
fzyzcjy
|
433266c125
|
Reintroduce memory usage fix (#9535)
|
2025-08-25 00:02:31 -07:00 |
|
|
fzyzcjy
|
b1b3f0b38f
|
Partially unify triton per token group quant kernels (#9485)
|
2025-08-23 02:07:31 -07:00 |
|
|
fzyzcjy
|
34e5e11f0f
|
Tiny make device_loading_context more static (#9478)
|
2025-08-23 02:07:15 -07:00 |
|
|
fzyzcjy
|
2600fc0d47
|
Overlapped weight offload (#8034)
|
2025-08-23 02:06:46 -07:00 |
|
|
fzyzcjy
|
0374304a2c
|
Add enable_flashinfer_mxfp4_bf16_moe for higher precision and slower moe backend (#9004)
|
2025-08-23 15:38:40 +08:00 |
|
|
fzyzcjy
|
e85cb1ce9d
|
Fix quant kernel test errors and benchmark wrong output speeds (#7604)
|
2025-08-21 03:48:41 -07:00 |
|
|
fzyzcjy
|
55d336cb08
|
Refactor weight offloading logic (#8521)
|
2025-08-21 03:48:13 -07:00 |
|
|
fzyzcjy
|
42c8704560
|
Add PDL support for quant kernel and rope kernel (#9106)
|
2025-08-20 01:56:29 -07:00 |
|
|
fzyzcjy
|
fe43e889f8
|
Fix mini lb timeout issue (#9369)
|
2025-08-19 20:15:16 -07:00 |
|
|
fzyzcjy
|
5626e20b2b
|
Tiny fix CI (#9306)
|
2025-08-18 16:54:36 -07:00 |
|
|
fzyzcjy
|
4c0bb411e5
|
Further fix memory pool leak error (#9298)
|
2025-08-18 00:58:06 -07:00 |
|
|
fzyzcjy
|
b498cd21d7
|
Tiny make fp4 moe method parameters more static (#8520)
|
2025-08-17 13:26:02 -07:00 |
|
|
fzyzcjy
|
b3c1f2e4f2
|
Fix memory pool leak error (#9271)
|
2025-08-17 12:53:34 -07:00 |
|
|
fzyzcjy
|
f8644a5632
|
Tiny update tmux history limit on dev container (#9218)
|
2025-08-15 00:22:08 -07:00 |
|
|
fzyzcjy
|
392de007cb
|
Minor fix docker container DeepEP on multi platforms (#9205)
|
2025-08-14 17:41:49 -07:00 |
|
|
fzyzcjy
|
b3363cc1aa
|
Fix docker container DeepEP error on Blackwell (#9171)
|
2025-08-13 21:06:48 -07:00 |
|
|
fzyzcjy
|
9394ed6386
|
Fix gpt-oss ~2x memory consumption issue (#9146)
|
2025-08-13 00:11:43 -07:00 |
|
|
fzyzcjy
|
9aea255522
|
Fuse writing KV buffer into rope kernel (part 1: sgl-kernel) (#9077)
|
2025-08-12 01:46:40 -07:00 |
|
|
fzyzcjyandQiaolin-Yu
|
5190ba7f42
|
Fuse two kernels of hidden states padding into quantization kernel (#9005)
Co-authored-by: Qiaolin-Yu <liin1211@outlook.com>
|
2025-08-12 01:20:13 -07:00 |
|