 Qiaolin Yu
|
aeff0d386b
|
Fix amd rope definition (#14556)
|
2025-12-07 23:47:03 -08:00 |
|
|
Qiaolin Yu
|
661e9775d0
|
[2/2] Add rope kernel in sgl-kernel (#14452)
|
2025-12-07 21:37:29 -08:00 |
|
|
Qiaolin Yu
|
cb8df87fc1
|
[1/2] Add rope kernel in sgl-kernel (#14334)
|
2025-12-04 16:45:44 +08:00 |
|
|
Qiaolin YuandKangyan-Zhou
|
7cb04dc0e5
|
Use trtllm mha decode kernel for target_verify in speculative decoding (#13976)
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
|
2025-11-26 20:40:34 -08:00 |
|
|
Qiaolin Yu
|
4b45d556a7
|
Overlap glm moe gemms in two cuda streams (#13786)
|
2025-11-24 18:15:24 -08:00 |
|
|
Qiaolin Yu
|
2892265d4c
|
Tune fp8_w8a8 fused triton moe for GLM-4.6-FP8 (#13815)
|
2025-11-23 13:29:54 -08:00 |
|
|
Qiaolin Yu
|
681b9e6425
|
Revert "enable csgmv automatically on cuda" (#13707)
|
2025-11-21 10:37:41 -08:00 |
|
|
Qiaolin YuandKangyan-Zhou
|
78a4b446c6
|
Fix dpsk-r1-fp4 tp8 by reverting two commits (#13162 and #13341) (#13348)
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
|
2025-11-15 21:31:36 -08:00 |
|
|
Qiaolin Yu
|
4a2768a86b
|
Fix spec decoding acc length for dpsk-r1-fp4 tp8 (2nd attempt) (#12915)
|
2025-11-11 13:30:33 -08:00 |
|
|
Qiaolin Yu
|
f290e8016e
|
Revert "Fix spec decoding acc length for dpsk-r1-fp4 tp8" (#12900)
|
2025-11-08 20:42:42 -08:00 |
|
|
Qiaolin Yu
|
9299a62fcb
|
Fix spec decoding acc length for dpsk-r1-fp4 tp8 (#12896)
|
2025-11-08 19:55:45 -08:00 |
|
|
Qiaolin Yu
|
59cce5941a
|
Use sgl fp4 quant kernel by default (#12482)
|
2025-10-31 19:51:28 -07:00 |
|
|
Qiaolin Yu
|
d98b81e260
|
Accelerate deepseek fp4 b200 ci (#11993)
|
2025-10-25 15:29:14 -07:00 |
|
|
Qiaolin Yu
|
71d41212e4
|
Fix dpsk-r1-fp4 launching crash (#12063)
|
2025-10-24 17:04:50 +08:00 |
|
|
Qiaolin Yu
|
36a4cad7b0
|
Support overlap-spec-v2 with trtllm_mla attention backend (#11821)
|
2025-10-23 16:55:35 +08:00 |
|
|
Qiaolin Yu
|
d9a20fd28a
|
Use trtllm_mla decode kernel for draft extend in speculative decoding (#11664)
|
2025-10-21 11:42:09 +08:00 |
|
|
Qiaolin YuandLiangsheng Yin
|
f7b1d8c5ab
|
Fix acc len and gen throughput metrics when enabling overlap-spec (#11823)
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
|
2025-10-21 01:34:38 +08:00 |
|
|
Qiaolin Yu
|
ebda73dc72
|
Use cutlass fp4 gemm by default (#11813)
|
2025-10-18 14:10:15 -07:00 |
|
|
Qiaolin Yu
|
547003bdd0
|
fix command line usage of profiling (#11793)
|
2025-10-18 12:54:36 +08:00 |
|
|
Qiaolin Yu
|
e4358a4585
|
Add fused_moe_triton config: triton_3_4_0/E=256,N=256,device_name=NVIDIA_B200.json (#11587)
|
2025-10-14 13:24:43 +08:00 |
|
|
Qiaolin Yu
|
43f80884c5
|
Fix accept rate in speculative decoding metrics (#11572)
|
2025-10-13 16:35:50 -07:00 |
|
|
Qiaolin Yu
|
a2b3d9b90b
|
Update DeepSeek-R1-FP4 default config on blackwell (#11512)
|
2025-10-12 20:32:11 -07:00 |
|
|
|
e2ac7888b8
|
[2/2] Support deterministic inference for temperature > 0 (#10678)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
|
2025-09-21 19:36:08 -07:00 |
|
|
Qiaolin Yu
|
8cda5a622c
|
Standalone speculative decoding (#10090)
|
2025-09-07 20:55:09 -07:00 |
|
|
Qiaolin Yu
|
4a4772ae03
|
Support speculative decoding in hybrid attention backend (#9573)
|
2025-08-28 01:11:42 -07:00 |
|
|
Qiaolin Yuandispobock
|
9c0c1e30b2
|
Disable torch.compile for get_last_loc_large_page_size_large_top_k (#9507)
Co-authored-by: ispobock <ispobaoke@gmail.com>
|
2025-08-22 02:05:02 -07:00 |
|
|
Qiaolin Yuandispobock
|
9ec314c6ac
|
Support speculative decoding in the trtllm_mha attention backend (#9331)
Co-authored-by: ispobock <ispobaoke@gmail.com>
|
2025-08-21 23:53:35 -07:00 |
|
|
Qiaolin Yu
|
af1973b871
|
Fix max_seq_len_k in trtllm_mha attention backend (#9416)
|
2025-08-20 19:17:13 -07:00 |
|
|
Qiaolin Yu
|
fc8c8e5041
|
Integrate triton_kernels in sgl-kernel (#8762)
|
2025-08-04 12:12:14 -07:00 |
|
|
Qiaolin Yu
|
484d0e021d
|
doc: add bench_one_batch_server in the benchmark doc (#8441)
|
2025-07-27 23:07:54 -07:00 |
|
|
|
2810338401
|
[feat] Support different attention backends for prefill and decode (#6338)
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-07-28 11:42:29 +08:00 |
|
|
Qiaolin Yu
|
bbcfbc1a02
|
feat: add h200 tp 16 kimi k2 moe config (#8183)
|
2025-07-19 23:30:08 -07:00 |
|
|
Qiaolin Yu
|
69f453e5a4
|
Use device_group for all_gather when disabling overlap scheduling (#8001)
|
2025-07-15 19:38:58 -07:00 |
|
|
Qiaolin Yu
|
3bc43c683e
|
Fix different device type adjustment in PP (#7760)
|
2025-07-15 19:37:14 -07:00 |
|
|
Qiaolin YuandCheng Wan
|
41650b0d70
|
feat: support compatibility between MTP and two-batch-overlap (#7225)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-06-27 01:10:27 -07:00 |
|
|
Qiaolin Yu
|
b8df43ab9c
|
Fix gathered_buffer issues in tbo (#7531)
|
2025-06-25 14:42:21 -07:00 |
|
|
Qiaolin Yu
|
f20f70003d
|
Fix torch version in blackwell dockerfile (#7017)
|
2025-06-09 17:06:50 -07:00 |
|
|
Qiaolin Yu
|
7dc0e39442
|
Bump torch to 2.7.0 (#6788)
|
2025-05-31 14:43:12 -07:00 |
|
|
Qiaolin Yu
|
0b9557fcd7
|
Disable compiling arch below sm_90 in aarch64 by default (#6380)
|
2025-05-27 15:50:02 -07:00 |
|
|
Qiaolin Yu
|
aff584fa54
|
Fix sgl-kernel build on aarch64 platforms (#6062)
|
2025-05-06 16:10:57 -07:00 |
|
|
Qiaolin Yuandzhaochenyang20
|
357671e216
|
Add examples for server token-in-token-out (#4103)
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
|
2025-03-05 13:16:31 -08:00 |
|
|
Qiaolin Yu
|
4725e3f652
|
Add examples for returning hidden states when using the server (#4074)
|
2025-03-04 19:31:50 -08:00 |
|
|
Qiaolin Yu
|
57a404fd55
|
Remove outdated test utils and fix links for the doc of sampling params (#3999)
|
2025-03-03 09:41:38 -08:00 |
|
|
Qiaolin YuandBeichen-Ma
|
40782f05d7
|
Refactor: Move return_hidden_states to the generate input (#3985)
Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
|
2025-03-01 17:51:29 -08:00 |
|
|
Qiaolin Yu
|
d38878523d
|
Fix the doc link for sampling params (#3861)
|
2025-02-27 13:31:43 -08:00 |
|
|
|
d6898dd253
|
Add return hidden state in the native API (#3897)
Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
|
2025-02-26 22:06:54 -08:00 |
|