 Qiaolin Yu
|
4659b08fcf
|
update CI_PERMISSIONS.json (#20551)
|
2026-03-13 14:52:06 -07:00 |
|
|
Qiaolin Yu
|
9c6f166600
|
tiny fix the usage of test_logprobs (#20473)
|
2026-03-12 15:37:45 -07:00 |
|
|
|
09a118fafe
|
Support return_logprob for spec v2 (overlap safe) (#19801)
Co-authored-by: Ratish1 <ratish1501@gmail.com>
Co-authored-by: Ratish1 <formula733@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-03-10 15:38:27 -07:00 |
|
|
Qiaolin Yu
|
bd460e9565
|
add logprob related params in bench_serving (#20218)
|
2026-03-10 15:04:57 -07:00 |
|
|
Qiaolin Yu
|
a3d88a247b
|
Enable piecewise-cuda-graph when logprob_start_len = -1 (#19453)
|
2026-03-10 12:50:57 -07:00 |
|
|
Qiaolin Yu
|
925185f9ec
|
Fix flashinfer backend with pcg (#20061)
|
2026-03-06 20:01:43 -08:00 |
|
|
Qiaolin Yu
|
46dced64ea
|
Adjust padding size to improve triton_kernels moe performance (#19174)
|
2026-03-05 14:50:40 -08:00 |
|
|
Qiaolin Yu
|
d80c884a27
|
Use single mma warp group for short q_len in FA to optimize decoding performance (#18985)
|
2026-02-23 17:01:28 +08:00 |
|
|
Qiaolin Yu
|
42b1019881
|
Fix bench_one_batch_server by moving the print statements (#19175)
|
2026-02-22 22:06:25 -08:00 |
|
|
Qiaolin Yu
|
8cf003c44b
|
Fix spec v2+dp attention in nsa backend (#19134)
|
2026-02-22 13:46:15 +08:00 |
|
|
Qiaolin Yu
|
96bae2355e
|
Add generated-shared-prefix dataset in bench_one_batch (#18986)
|
2026-02-20 13:33:10 -08:00 |
|
|
Qiaolin Yu
|
90d5e27f79
|
Enable fa3 PDL by compiling it with corresponding flags (#18756)
|
2026-02-18 17:12:05 +08:00 |
|
|
Qiaolin Yu
|
3c601db031
|
Fix generated-shared-prefix bench_serving (#18769)
|
2026-02-17 14:00:22 -08:00 |
|
|
Qiaolin Yu
|
4a1b50bb2d
|
Fix idle batch predict dtype in spec v2 (#18379)
|
2026-02-10 10:29:13 +08:00 |
|
|
Qiaolin Yu
|
4f6f5d25c8
|
Support fa4 decoding (#16034)
|
2026-01-21 22:54:02 +08:00 |
|
|
|
76b06bee03
|
[New Model] GLM4.7-Flash (#17247)
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: JustinTong0323 <justinning0323@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2026-01-20 23:44:16 +08:00 |
|
|
Qiaolin Yu
|
e3a95077bc
|
Add dpsk-r1-fp4 in nightly perf ci (#16882)
|
2026-01-15 16:13:35 -08:00 |
|
|
Qiaolin Yu
|
206db66f5c
|
tiny refactor pcg split op registration (#16863)
|
2026-01-10 23:45:28 +08:00 |
|
|
Qiaolin Yu
|
dd93e4451c
|
Support returning customized info (#16144)
|
2026-01-02 15:32:18 +08:00 |
|
|
Qiaolin Yu
|
6a3e709253
|
Tiny rename test_deepseek_v3_fp4_mtp_stage_b.py (#16141)
|
2025-12-30 15:55:07 +08:00 |
|
|
Qiaolin Yu
|
aef7ca7cf2
|
Raise the accept length bar in dpsk-r1-fp4 spec decoding tests (#15705)
|
2025-12-23 18:38:02 -08:00 |
|
|
Qiaolin Yu
|
677930c276
|
Add glm-4.6-fp8 with/without mtp in nightly ci (#15566)
|
2025-12-21 22:27:02 -08:00 |
|
|
Qiaolin Yu
|
a92de891b8
|
Split dpsk fp4 4 gpu tests and move the mtp part to real stage b (#15553)
|
2025-12-21 15:51:43 -08:00 |
|
|
Qiaolin Yu
|
254de6d2fd
|
Add triton_fused_moe config for GLM-4.6-FP8 tp8 blackwell (#15569)
|
2025-12-21 13:41:22 -08:00 |
|
|
Qiaolin Yu
|
e220da1723
|
tiny fix sampling seed for completion api (#15498)
|
2025-12-19 20:32:13 -08:00 |
|
|
Qiaolin Yu
|
173940927f
|
Add customized sampler registration (#15423)
|
2025-12-18 23:10:23 -08:00 |
|
|
Qiaolin Yu
|
729529190d
|
[ci] Move dpsk-r1-fp4 b200 test to stage b (#15084)
|
2025-12-13 23:14:08 -08:00 |
|
|
Qiaolin Yu
|
77873343c4
|
tiny update: use rope kernel in sgl-kernel for amd (#14955)
|
2025-12-12 18:48:12 -08:00 |
|
|
Qiaolin Yu
|
44fd701732
|
Tune triton fused moe for the case of glm-4.6-fp8 b200 tp4 (#15020)
|
2025-12-12 17:04:56 -08:00 |
|
|
Qiaolin Yu
|
aeff0d386b
|
Fix amd rope definition (#14556)
|
2025-12-07 23:47:03 -08:00 |
|
|
Qiaolin Yu
|
661e9775d0
|
[2/2] Add rope kernel in sgl-kernel (#14452)
|
2025-12-07 21:37:29 -08:00 |
|
|
Qiaolin Yu
|
cb8df87fc1
|
[1/2] Add rope kernel in sgl-kernel (#14334)
|
2025-12-04 16:45:44 +08:00 |
|
|
Qiaolin YuandKangyan-Zhou
|
7cb04dc0e5
|
Use trtllm mha decode kernel for target_verify in speculative decoding (#13976)
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
|
2025-11-26 20:40:34 -08:00 |
|
|
Qiaolin Yu
|
4b45d556a7
|
Overlap glm moe gemms in two cuda streams (#13786)
|
2025-11-24 18:15:24 -08:00 |
|
|
Qiaolin Yu
|
2892265d4c
|
Tune fp8_w8a8 fused triton moe for GLM-4.6-FP8 (#13815)
|
2025-11-23 13:29:54 -08:00 |
|
|
Qiaolin Yu
|
681b9e6425
|
Revert "enable csgmv automatically on cuda" (#13707)
|
2025-11-21 10:37:41 -08:00 |
|
|
Qiaolin YuandKangyan-Zhou
|
78a4b446c6
|
Fix dpsk-r1-fp4 tp8 by reverting two commits (#13162 and #13341) (#13348)
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
|
2025-11-15 21:31:36 -08:00 |
|
|
Qiaolin Yu
|
4a2768a86b
|
Fix spec decoding acc length for dpsk-r1-fp4 tp8 (2nd attempt) (#12915)
|
2025-11-11 13:30:33 -08:00 |
|
|
Qiaolin Yu
|
f290e8016e
|
Revert "Fix spec decoding acc length for dpsk-r1-fp4 tp8" (#12900)
|
2025-11-08 20:42:42 -08:00 |
|
|
Qiaolin Yu
|
9299a62fcb
|
Fix spec decoding acc length for dpsk-r1-fp4 tp8 (#12896)
|
2025-11-08 19:55:45 -08:00 |
|
|
Qiaolin Yu
|
59cce5941a
|
Use sgl fp4 quant kernel by default (#12482)
|
2025-10-31 19:51:28 -07:00 |
|
|
Qiaolin Yu
|
d98b81e260
|
Accelerate deepseek fp4 b200 ci (#11993)
|
2025-10-25 15:29:14 -07:00 |
|
|
Qiaolin Yu
|
71d41212e4
|
Fix dpsk-r1-fp4 launching crash (#12063)
|
2025-10-24 17:04:50 +08:00 |
|
|
Qiaolin Yu
|
36a4cad7b0
|
Support overlap-spec-v2 with trtllm_mla attention backend (#11821)
|
2025-10-23 16:55:35 +08:00 |
|
|
Qiaolin Yu
|
d9a20fd28a
|
Use trtllm_mla decode kernel for draft extend in speculative decoding (#11664)
|
2025-10-21 11:42:09 +08:00 |
|
|
Qiaolin YuandLiangsheng Yin
|
f7b1d8c5ab
|
Fix acc len and gen throughput metrics when enabling overlap-spec (#11823)
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
|
2025-10-21 01:34:38 +08:00 |
|
|
Qiaolin Yu
|
ebda73dc72
|
Use cutlass fp4 gemm by default (#11813)
|
2025-10-18 14:10:15 -07:00 |
|
|
Qiaolin Yu
|
547003bdd0
|
fix command line usage of profiling (#11793)
|
2025-10-18 12:54:36 +08:00 |
|
|
Qiaolin Yu
|
e4358a4585
|
Add fused_moe_triton config: triton_3_4_0/E=256,N=256,device_name=NVIDIA_B200.json (#11587)
|
2025-10-14 13:24:43 +08:00 |
|
|
Qiaolin Yu
|
43f80884c5
|
Fix accept rate in speculative decoding metrics (#11572)
|
2025-10-13 16:35:50 -07:00 |
|
|
Qiaolin Yu
|
a2b3d9b90b
|
Update DeepSeek-R1-FP4 default config on blackwell (#11512)
|
2025-10-12 20:32:11 -07:00 |
|
|
|
e2ac7888b8
|
[2/2] Support deterministic inference for temperature > 0 (#10678)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
|
2025-09-21 19:36:08 -07:00 |
|
|
Qiaolin Yu
|
8cda5a622c
|
Standalone speculative decoding (#10090)
|
2025-09-07 20:55:09 -07:00 |
|
|
Qiaolin Yu
|
4a4772ae03
|
Support speculative decoding in hybrid attention backend (#9573)
|
2025-08-28 01:11:42 -07:00 |
|
|
Qiaolin Yuandispobock
|
9c0c1e30b2
|
Disable torch.compile for get_last_loc_large_page_size_large_top_k (#9507)
Co-authored-by: ispobock <ispobaoke@gmail.com>
|
2025-08-22 02:05:02 -07:00 |
|
|
Qiaolin Yuandispobock
|
9ec314c6ac
|
Support speculative decoding in the trtllm_mha attention backend (#9331)
Co-authored-by: ispobock <ispobaoke@gmail.com>
|
2025-08-21 23:53:35 -07:00 |
|
|
Qiaolin Yu
|
af1973b871
|
Fix max_seq_len_k in trtllm_mha attention backend (#9416)
|
2025-08-20 19:17:13 -07:00 |
|
|
Qiaolin Yu
|
fc8c8e5041
|
Integrate triton_kernels in sgl-kernel (#8762)
|
2025-08-04 12:12:14 -07:00 |
|
|
Qiaolin Yu
|
484d0e021d
|
doc: add bench_one_batch_server in the benchmark doc (#8441)
|
2025-07-27 23:07:54 -07:00 |
|
|
|
2810338401
|
[feat] Support different attention backends for prefill and decode (#6338)
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-07-28 11:42:29 +08:00 |
|
|
Qiaolin Yu
|
bbcfbc1a02
|
feat: add h200 tp 16 kimi k2 moe config (#8183)
|
2025-07-19 23:30:08 -07:00 |
|
|
Qiaolin Yu
|
69f453e5a4
|
Use device_group for all_gather when disabling overlap scheduling (#8001)
|
2025-07-15 19:38:58 -07:00 |
|
|
Qiaolin Yu
|
3bc43c683e
|
Fix different device type adjustment in PP (#7760)
|
2025-07-15 19:37:14 -07:00 |
|
|
Qiaolin YuandCheng Wan
|
41650b0d70
|
feat: support compatibility between MTP and two-batch-overlap (#7225)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-06-27 01:10:27 -07:00 |
|
|
Qiaolin Yu
|
b8df43ab9c
|
Fix gathered_buffer issues in tbo (#7531)
|
2025-06-25 14:42:21 -07:00 |
|
|
Qiaolin Yu
|
f20f70003d
|
Fix torch version in blackwell dockerfile (#7017)
|
2025-06-09 17:06:50 -07:00 |
|
|
Qiaolin Yu
|
7dc0e39442
|
Bump torch to 2.7.0 (#6788)
|
2025-05-31 14:43:12 -07:00 |
|
|
Qiaolin Yu
|
0b9557fcd7
|
Disable compiling arch below sm_90 in aarch64 by default (#6380)
|
2025-05-27 15:50:02 -07:00 |
|
|
Qiaolin Yu
|
aff584fa54
|
Fix sgl-kernel build on aarch64 platforms (#6062)
|
2025-05-06 16:10:57 -07:00 |
|
|
Qiaolin Yuandzhaochenyang20
|
357671e216
|
Add examples for server token-in-token-out (#4103)
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
|
2025-03-05 13:16:31 -08:00 |
|
|
Qiaolin Yu
|
4725e3f652
|
Add examples for returning hidden states when using the server (#4074)
|
2025-03-04 19:31:50 -08:00 |
|
|
Qiaolin Yu
|
57a404fd55
|
Remove outdated test utils and fix links for the doc of sampling params (#3999)
|
2025-03-03 09:41:38 -08:00 |
|
|
Qiaolin YuandBeichen-Ma
|
40782f05d7
|
Refactor: Move return_hidden_states to the generate input (#3985)
Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
|
2025-03-01 17:51:29 -08:00 |
|
|
Qiaolin Yu
|
d38878523d
|
Fix the doc link for sampling params (#3861)
|
2025-02-27 13:31:43 -08:00 |
|
|
|
d6898dd253
|
Add return hidden state in the native API (#3897)
Co-authored-by: Beichen-Ma <mabeichen12@gmail.com>
Co-authored-by: Chayenne <zhaochen20@outlook.com>
|
2025-02-26 22:06:54 -08:00 |
|