 fzyzcjy
|
45264554f3
|
Super tiny fix typo (#14219)
|
2025-12-01 20:19:17 +08:00 |
|
|
fzyzcjy
|
d815d00248
|
Tiny call cudaProfilerStart only on first rank in node (#14211)
|
2025-12-01 18:18:45 +08:00 |
|
|
fzyzcjy
|
de153cf76a
|
Fix speculative decoding error when retracting (#14180)
|
2025-12-01 15:30:13 +08:00 |
|
|
fzyzcjy
|
f4a0c5c76b
|
Try to remove wrong logic about max total token in spec decoding (#14167)
|
2025-12-01 15:29:58 +08:00 |
|
|
fzyzcjy
|
e8ba5a668c
|
Support profiling only prefill or decode without the other (#14182)
|
2025-12-01 14:46:30 +08:00 |
|
|
fzyzcjy
|
a2960bdd6b
|
Super tiny allow millisecond precision in logging (#14183)
|
2025-12-01 14:46:09 +08:00 |
|
|
fzyzcjy
|
487c8d4df3
|
Tiny add several args to bench serving (#14181)
|
2025-12-01 14:45:47 +08:00 |
|
|
fzyzcjy
|
f87b8eab23
|
Tiny fix transform_scale_ue8m0 wrong output in some scenarios (#14003)
|
2025-12-01 14:45:27 +08:00 |
|
|
fzyzcjy
|
340c613ab5
|
Support numactl bind for CPU and memory before process starts (#14156)
|
2025-11-30 17:00:33 +08:00 |
|
|
fzyzcjy
|
36b729c2b8
|
Implement profiler v2 and fix stage mixture bug (#14148)
|
2025-11-30 16:59:52 +08:00 |
|
|
fzyzcjy
|
0ae4b1ad81
|
Show errors when misusing env variables (#14154)
|
2025-11-30 10:57:35 +08:00 |
|
|
fzyzcjy
|
94cd64a7b0
|
Support checking fp8 params in weight_checker (#14147)
|
2025-11-30 09:08:59 +08:00 |
|
|
fzyzcjy
|
b870271a50
|
Fix spec v2 does not support RL update weights from tensor (#14146)
|
2025-11-30 09:08:05 +08:00 |
|
|
fzyzcjy
|
22ee9b0111
|
Super tiny add more info in dumper (#14145)
|
2025-11-30 09:07:39 +08:00 |
|
|
fzyzcjy
|
9d0e5f1f74
|
Tiny fix DeepGEMM precompile rank check (#14136)
|
2025-11-30 09:07:17 +08:00 |
|
|
fzyzcjy
|
4cafc835d3
|
Super tiny fix typo (#14131)
|
2025-11-29 21:08:31 +08:00 |
|
|
fzyzcjy
|
45cf575852
|
Fix overlap scheduler not take effect when outputing logprobs (#14096)
|
2025-11-28 18:15:56 +08:00 |
|
|
fzyzcjy
|
21af8e73ad
|
Super tiny add comments to SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK (#14048)
|
2025-11-27 22:16:43 +08:00 |
|
|
fzyzcjy
|
25758647b1
|
Support sanity checking weight consistency especially for RL (#13854)
|
2025-11-27 20:25:12 +08:00 |
|
|
fzyzcjy
|
2bc8ee8b74
|
Tiny support 3D tensors in inverse_transform_scale_ue8m0 (#14002)
|
2025-11-27 20:20:45 +08:00 |
|
|
fzyzcjy
|
7130ad3a29
|
Fix SGLANG_ENABLE_HEALTH_ENDPOINT_GENERATION not working (#13961)
|
2025-11-25 21:58:55 -08:00 |
|
|
fzyzcjy
|
64a11303ce
|
Fix update weight error for blackwell DeepGEMM (#13910)
|
2025-11-25 13:28:12 -08:00 |
|
|
fzyzcjy
|
94216a9cc4
|
Fix quantized moe checker fail for Qwen3 dense fp8 model (#13853)
|
2025-11-24 11:16:50 -08:00 |
|
|
fzyzcjyandBaizhou Zhang
|
45c572c58f
|
Support torch 12.9 + DeepEP by removing custom nvshmem (#12949)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-11-21 11:11:43 -08:00 |
|
|
fzyzcjy
|
1776dce557
|
Super tiny delete unused files (#13734)
|
2025-11-21 22:26:39 +08:00 |
|
|
fzyzcjy
|
3f1cfd87b6
|
Super tiny remove unused MiniMaxM2MLP class (#13659)
|
2025-11-21 07:35:12 +08:00 |
|
|
fzyzcjy
|
f3e9336dcb
|
Support weight update for blackwell DeepGEMM (#13324)
|
2025-11-17 15:55:26 +08:00 |
|
|
fzyzcjy
|
d971f22898
|
Super tiny expose transform_scale_ue8m0 API for RL frameworks (#13323)
|
2025-11-15 17:31:04 +08:00 |
|
|
fzyzcjy
|
33f08a98b0
|
Tiny refactor condition to requant scale ue8m0 (#13286)
|
2025-11-15 16:36:00 +08:00 |
|
|
fzyzcjy
|
8e6083bfcf
|
Support inverse transform ue8m0 scale (#13285)
|
2025-11-15 16:34:32 +08:00 |
|
|
fzyzcjy
|
2fbc78a083
|
Support fast gemm when in batch invariant DeepGEMM fallback (#13259)
|
2025-11-15 16:34:15 +08:00 |
|
|
fzyzcjy
|
af9f71f9c5
|
Add script to create a model with fewer layers for debugging (#13284)
|
2025-11-14 22:13:27 +08:00 |
|
|
fzyzcjy
|
15264232ee
|
Super tiny fix CI (#13283)
|
2025-11-14 21:45:10 +08:00 |
|
|
fzyzcjy
|
3701f34dab
|
Tiny add utility to parse server logs (#12605)
|
2025-11-14 17:34:22 +08:00 |
|
|
fzyzcjy
|
821fb060c3
|
Enhance dumper comparator with tensor unifier and location finder (#12623)
|
2025-11-14 17:34:08 +08:00 |
|
|
fzyzcjy
|
ace27c0c01
|
Tiny enhance dumper with ctx and enable flags (#12622)
|
2025-11-14 17:33:49 +08:00 |
|
|
fzyzcjy
|
ed1d18d472
|
Tiny fix update version logic location (#12620)
|
2025-11-14 17:33:36 +08:00 |
|
|
fzyzcjy
|
1f134f850a
|
fix outdated router doc (#13255)
|
2025-11-14 14:00:42 +09:00 |
|
|
fzyzcjy
|
86255f27b4
|
Revert "fallback to triton mm_persistent kernel when deepGemm fail" (#13178)
|
2025-11-12 22:03:35 -08:00 |
|
|
fzyzcjy
|
b0ee99dd03
|
Super tiny fix typo (#13001)
|
2025-11-11 00:47:45 +08:00 |
|
|
fzyzcjy
|
2b6c4257a0
|
Fix sending all requests to the first rank in DP attention (#12832)
|
2025-11-09 00:18:53 +08:00 |
|
|
fzyzcjy
|
b7d7041190
|
Add sanity checks when a test file is not added to CI (reland) (#12594)
|
2025-11-04 18:04:26 +08:00 |
|
|
fzyzcjy
|
ff0b64e1e6
|
Ensure GPU work is finished when release memory occupation call is finished (#12592)
|
2025-11-04 18:01:27 +08:00 |
|
|
fzyzcjy
|
d84790db39
|
Support aggregating engine metrics in sgl-router (#11456)
|
2025-11-04 01:59:50 -08:00 |
|
|
fzyzcjy
|
60b0754cc9
|
Tiny fix ExpertDistributionReq error (#11760)
|
2025-11-04 13:39:25 +08:00 |
|
|
fzyzcjy
|
193fbb0bce
|
Super tiny add UT for copy_to_gpu_no_ce (#12270)
|
2025-11-04 09:40:51 +08:00 |
|
|
fzyzcjy
|
8834260739
|
Super tiny dump server info such as args in bench for post analysis (#12550)
|
2025-11-03 14:24:08 -08:00 |
|
|
fzyzcjy
|
fd7a72d62d
|
Super tiny allow profile activities in bench_serving (#12549)
|
2025-11-03 14:23:18 -08:00 |
|
|
fzyzcjy
|
385599cb04
|
Fix error when calling quantization (#12548)
|
2025-11-03 10:17:43 -08:00 |
|
|
fzyzcjy
|
c9db79117f
|
Super tiny fix naming in bench serving scripts (#12515)
|
2025-11-02 12:43:10 -08:00 |
|
|
fzyzcjy
|
30ad107028
|
Try to allow NCCL cumem for multi node nvlink case (#11987)
|
2025-10-31 12:48:25 -07:00 |
|
|
fzyzcjy
|
25257d8e00
|
Tiny assert no running requests when releasing memory to avoid IMA (#12341)
|
2025-11-01 01:28:53 +08:00 |
|
|
fzyzcjy
|
df5192cffa
|
Enable fast silu-and-mul-and-quant fused kernel (#11806)
|
2025-10-30 18:15:39 +08:00 |
|
|
fzyzcjy
|
fb52d35f63
|
Super tiny fix AMD ci (#12378)
|
2025-10-29 23:25:18 -07:00 |
|
|
fzyzcjy
|
25c5049870
|
Super tiny add tag for benchmark scripts (#12340)
|
2025-10-30 11:19:14 +08:00 |
|
|
fzyzcjy
|
29195aaa6e
|
Super tiny fix expert distribution dump error (#12271)
|
2025-10-28 15:20:55 -07:00 |
|
|
fzyzcjy
|
2a3763c335
|
Tiny fix sgl-kernel related CI installing the wrong binary (#12283)
|
2025-10-28 10:29:06 -07:00 |
|
|
|
691c8534cf
|
Support releasing CUDA graph memory when paused (#7873)
Co-authored-by: ryang-max <y1cunhui.yang@gmail.com>
Co-authored-by: ryang <38470282+ryang-max@users.noreply.github.com>
|
2025-10-28 14:40:50 +08:00 |
|
|
fzyzcjy
|
326c84c493
|
Compiling rope while preserving true on policy (#12161)
|
2025-10-28 08:02:17 +08:00 |
|
|
fzyzcjy
|
0103f374ba
|
Support DeepGEMM for deterministic inference (#12142)
|
2025-10-26 22:36:17 +08:00 |
|
|
fzyzcjy
|
c001deba37
|
Make bmm batch invariant injection optional (#12118)
|
2025-10-26 10:18:35 +08:00 |
|
|
fzyzcjy
|
20bd2271e2
|
Support true on-policy (#12058)
|
2025-10-25 10:23:42 +08:00 |
|
|
fzyzcjy
|
d7056c5236
|
Enhance tests in deterministic kernels (#12070)
|
2025-10-25 08:53:22 +08:00 |
|
|
fzyzcjy
|
e04340bf48
|
Fix multi processing serializer bug (#11958)
|
2025-10-24 22:53:45 +08:00 |
|
|
fzyzcjy
|
2342605ef0
|
Tiny cleanup send_single (#12056)
|
2025-10-23 23:53:42 -07:00 |
|
|
fzyzcjy
|
0f0c430e93
|
Install numactl in Dockerfile for GH200/GB200/GB300 (#11853)
|
2025-10-23 21:39:10 -07:00 |
|
|
fzyzcjy
|
8612811d85
|
Bump grace blackwell DeepEP version (#11990)
|
2025-10-22 21:08:12 -07:00 |
|
|
fzyzcjy
|
0917c5da8c
|
Support mixing cutedsl and deepgemm backend (#11807)
|
2025-10-21 07:38:35 +08:00 |
|
|
fzyzcjy
|
9e3be1fa2a
|
Tiny bump DeepEP version in ARM blackwell (#11810)
|
2025-10-20 08:15:14 +08:00 |
|
|
fzyzcjy
|
a8ba32798e
|
Fix triton_kernels import error on some hardwares (#11831)
|
2025-10-20 08:14:47 +08:00 |
|
|
fzyzcjy
|
12eb02e982
|
Change bf16 to fp8 for some gemms in attention for DeepSeek ckpt v2 (#11805)
|
2025-10-19 16:15:13 +08:00 |
|
|
fzyzcjyandYineng Zhang
|
002d037359
|
Avoid generation gets hanging when user specifies multiple event loops (#5162)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-10-19 16:12:49 +08:00 |
|
|
fzyzcjy
|
a27825ae01
|
Support not officially supported high sgl-kernel version with low srt version (#11786)
|
2025-10-19 16:11:59 +08:00 |
|
|
fzyzcjy
|
ce399e154c
|
Make single-batch overlap compatible with NextN (#11804)
|
2025-10-19 16:10:44 +08:00 |
|
|
fzyzcjy
|
ea6275dfbc
|
Tiny add hints when users send requests to wrong place (#11808)
|
2025-10-19 16:10:20 +08:00 |
|
|
fzyzcjy
|
a7043c6f0d
|
Bump torch_memory_saver to avoid installing pre-release versions (#11797)
|
2025-10-18 01:20:42 -07:00 |
|
|
fzyzcjy
|
dbbd4e1891
|
Try add back no-commit-to-branch (#11799)
|
2025-10-18 12:05:12 +08:00 |
|
|
fzyzcjy
|
6c7c92eb02
|
Enable lint on main (#11794)
|
2025-10-17 19:08:50 -07:00 |
|
|
fzyzcjy
|
33e9bbec35
|
Make single-batch overlap compatible with offloading (#11614)
|
2025-10-18 08:45:54 +08:00 |
|
|
fzyzcjy
|
dcb8f090ad
|
Super tiny fix CI (#11788)
|
2025-10-17 17:41:58 -07:00 |
|
|
fzyzcjy
|
8af8491298
|
Support casting bf16 NextN moe to fp8 (#11613)
|
2025-10-18 08:02:15 +08:00 |
|
|
fzyzcjy
|
505329cab0
|
Support shared experts overlap in cutlass moe (#11611)
|
2025-10-18 07:59:40 +08:00 |
|
|
fzyzcjy
|
8a382fd399
|
Super tiny fix missing input throughput (#11607)
|
2025-10-18 07:58:48 +08:00 |
|
|
fzyzcjy
|
32803fb279
|
Super tiny improve FA3 import error message (#11590)
|
2025-10-14 22:06:31 -07:00 |
|
|
fzyzcjy
|
cb8ed2c09a
|
Make DeepEP combine recv do not overlap (#11535)
|
2025-10-13 18:40:42 -07:00 |
|
|
fzyzcjy
|
065ce81574
|
Tiny cleanup fp4 gemm calls (#11537)
|
2025-10-13 14:48:22 -07:00 |
|
|
fzyzcjy
|
bf3e7149be
|
Fix enable_v2 in int8 quant (#11470)
|
2025-10-11 21:56:30 +08:00 |
|
|
fzyzcjy
|
d957177a22
|
Super tiny delete unused openai router in sgl-router (#11448)
|
2025-10-11 15:59:30 +08:00 |
|
|
fzyzcjyandYineng Zhang
|
21337b22b9
|
Reland [1/2] Optimizations and refactors about quant kernel (#10312)
Co-authored-by: Yineng Zhang <me@zhyncs.com>
|
2025-10-11 15:59:03 +08:00 |
|
|
+2
|
efbc687c28
|
Support DeepSeek V3.2 Exp (#11061)
Co-authored-by: Stefan He <11166516+hebiao064@users.noreply.github.com>
Co-authored-by: Liangsheng Yin <95566987+hnyls2002@users.noreply.github.com>
Co-authored-by: Baizhou Zhang <56809903+fridge003@users.noreply.github.com>
Co-authored-by: DarkSharpness <76582120+darksharpness@users.noreply.github.com>
Co-authored-by: ZhengdQin <46387172+zhengdqin@users.noreply.github.com>
Co-authored-by: DarkSharpness <2040703891@qq.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Zhengda Qin <zhengdqin@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-10-06 00:24:15 -07:00 |
|
|
fzyzcjy
|
2f80bd9f0e
|
Bump torch_memory_saver 0.0.9rc2 (#11252)
|
2025-10-05 20:26:20 -07:00 |
|
|
fzyzcjy
|
fdc4e1e570
|
Tiny move files to utils folder (#11166)
|
2025-10-03 22:40:06 +08:00 |
|
|
fzyzcjy
|
6794d21051
|
Tiny add PD disaggregation + DP attention test (#11167)
|
2025-10-03 14:15:46 +08:00 |
|
|
fzyzcjy
|
afcd3e1089
|
Tiny remove duplicated code (#11164)
|
2025-10-02 21:56:31 +08:00 |
|
|
fzyzcjy
|
12d6818380
|
Tiny fix ep_gather behavior different in CI (#11130)
|
2025-10-02 21:55:53 +08:00 |
|
|
fzyzcjy
|
b65db0287b
|
Tiny cleanup deepseek_v2.py (#11163)
|
2025-10-02 21:54:52 +08:00 |
|
|
fzyzcjy
|
5e786cca3a
|
Support single batch overlap (#10422)
|
2025-10-02 18:04:36 +08:00 |
|
|
fzyzcjyandKaixi Hou
|
0b9dfba787
|
Support dispatch low latency (#10263)
Co-authored-by: Kaixi Hou <4001424+kaixih@users.noreply.github.com>
|
2025-10-02 18:02:19 +08:00 |
|
|
fzyzcjy
|
2ac453b07f
|
Tiny detect slow ranks (#10508)
|
2025-10-02 18:00:33 +08:00 |
|
|
fzyzcjy
|
f35def8652
|
Fuse quantize and rope in trtllm_mla MTP (#10779)
|
2025-10-02 17:59:37 +08:00 |
|