 Qiaolin Yu
|
982f67d9a6
|
Suppress cutlass-dsl noisy warning (#26169)
|
2026-05-23 13:19:14 -07:00 |
|
|
Qiaolin Yu
|
cb7b57955d
|
fix tokenspeed_mla attn kernel jit (#26170)
|
2026-05-23 03:24:33 -07:00 |
|
|
Qiaolin Yu
|
c112f7623a
|
Skip init_mha_chunk_metadata in trtllm_mla when not needed (#26017)
|
2026-05-22 16:34:16 -07:00 |
|
|
Qiaolin Yu
|
1a17d753f1
|
[perf] prepare_prefill_qkv hook + fp8 quantize jit kernel (#25460)
|
2026-05-20 14:20:49 -07:00 |
|
|
Qiaolin Yu
|
1f185c6ba8
|
Support draft extend cuda graph for tokenspeed_mla attention backend (#25489)
|
2026-05-18 11:26:16 -07:00 |
|
|
Qiaolin Yu
|
f5049709b3
|
fix(eagle3): drop +1 offset on aux layer ids when first id != 1 (#25454)
|
2026-05-18 11:25:51 -07:00 |
|
|
Qiaolin Yu
|
9869ef0849
|
Revert "[attn backend] avoid initing parent class's workspace buffer" (#25488)
|
2026-05-16 13:05:42 -07:00 |
|
|
Qiaolin Yu
|
2f81718773
|
[attn backend] avoid initing parent class's workspace buffer (#25321)
|
2026-05-16 03:30:33 -07:00 |
|
|
Qiaolin Yu
|
7af4320d67
|
[perf] fix kimi tokenizer to improve ttft (#25265)
|
2026-05-14 19:11:25 -07:00 |
|
|
Qiaolin Yu
|
421179c453
|
[perf] avoid hidden states d2h when return_hidden_states=false (#25155)
|
2026-05-13 23:47:17 -07:00 |
|
|
Qiaolin Yu
|
7618ad7075
|
[attn backend] Integrate tokenspeed_mla prefill/decode kernels (fp8 kv cache, blackwell) (#24925)
|
2026-05-13 17:36:17 -07:00 |
|
|
Qiaolin Yu
|
cfc41d5b15
|
Fix kimi k2.5 mla eagle + dp attention (#25033)
|
2026-05-11 20:58:00 -07:00 |
|
|
Qiaolin Yu
|
a87fb399de
|
[spec decoding] support kimi-k2.5-eagle3-mla (#24826)
|
2026-05-09 23:57:39 -07:00 |
|
|
Qiaolin Yu
|
a965f886bf
|
[misc] update CI_PERMISSIONS.json (#24468)
|
2026-05-05 18:37:18 -07:00 |
|
|
Qiaolin Yu
|
4197c55968
|
[spec decoding] add tests for chain-style multi layer eagle + return_logprob (#24192)
|
2026-05-01 01:48:48 -07:00 |
|
|
Qiaolin Yu
|
583929c0a1
|
fix the compatibility between --moe-dense-tp-size 1 and piecewise cuda graph (#23972)
|
2026-04-30 02:12:13 -07:00 |
|
|
Qiaolin Yu
|
2bbd30a27a
|
relax the threshold in test_step3p5_flash_chain_mtp (#24105)
|
2026-04-29 16:53:35 -07:00 |
|
|
Qiaolin Yu
|
ea794deefc
|
[misc] fix lint in main branch (#24095)
|
2026-04-29 15:01:35 -07:00 |
|
|
Qiaolin Yu
|
79dbfe4505
|
Use spec v2 by default (#21062)
|
2026-04-29 13:40:42 -07:00 |
|
|
Qiaolin Yu
|
f57ec8d6ef
|
[spec decoding] add extra attribute 'spec_hidden_size' (#23890)
|
2026-04-28 19:54:50 -07:00 |
|
|
Qiaolin YuandYuzhen Zhou
|
c560326884
|
[perf] support return_routed_experts with overlap scheduling (#22911)
Co-authored-by: Yuzhen Zhou <82826991+zyzshishui@users.noreply.github.com>
|
2026-04-21 14:42:49 -07:00 |
|
|
Qiaolin Yu
|
a87806a65f
|
[misc] refine outdated comments for chain-style multi-layer MTP (#22996)
|
2026-04-16 14:49:43 -07:00 |
|
|
Qiaolin Yu
|
12266cf953
|
[misc] update .github/CODEOWNERS (#22993)
|
2026-04-16 14:19:41 -07:00 |
|
|
Qiaolin Yu
|
0b1b07db72
|
[misc] fix ray folder lint (#22905)
|
2026-04-15 15:08:18 -07:00 |
|
|
Qiaolin Yu
|
f41c810a2d
|
[misc] update CI_PERMISSIONS.json (#22570)
|
2026-04-10 18:58:55 -07:00 |
|
|
Qiaolin Yuand0xNullPath
|
d8831355a3
|
Fix multi_layer_eagle_worker_v2 draft extend selection, add chain style multi layer mtp test (#22340)
Co-authored-by: 0xNullPath <luyan@nvidia.com>
|
2026-04-10 12:44:52 -07:00 |
|
|
Qiaolin YuandLiangsheng Yin
|
117508dcd7
|
Switch eagle_infer_beta to EAGLE3 (#22303)
Co-authored-by: Liangsheng Yin <hnyls2002@users.noreply.github.com>
|
2026-04-07 18:43:48 -07:00 |
|
|
Qiaolin Yu
|
0bc4e0ea75
|
[misc] update CI_PERMISSIONS.json (#22207)
|
2026-04-06 14:11:49 -07:00 |
|
|
Qiaolin Yu
|
93f38fe410
|
tiny fix chain-style multi layer eagle comments (#22206)
|
2026-04-06 13:49:03 -07:00 |
|
|
Qiaolin Yu
|
f407461ec8
|
Tiny fix trtllm_fp8_per_tensor_scale_moe_wrapper router_logits dtype (#22006)
|
2026-04-05 21:11:45 -07:00 |
|
|
Qiaolin Yu
|
ef13031243
|
Tiny fix step3.5-flash launch crash (#22076)
|
2026-04-03 22:25:25 -07:00 |
|
|
Qiaolin Yu
|
b684b0b72f
|
Fix spec v2 + logprob when max_num_token is set (#20799)
|
2026-04-02 01:55:16 -07:00 |
|
|
Qiaolin Yu
|
d8db3077ca
|
Fix draft extend cuda graph when spec_step=1 (#21709)
|
2026-03-31 18:29:56 -07:00 |
|
|
Qiaolin Yu
|
5f6250769a
|
Reduce redundant speculative decoding CI tests (#21779)
|
2026-03-31 17:40:20 -07:00 |
|
|
Qiaolin Yu
|
a27651d5e0
|
Remove sync when enabling return_logprob (#20972)
|
2026-03-27 16:36:28 -07:00 |
|
|
Qiaolin Yu
|
4a41aec844
|
Fix flaky test_pp_single_node (#21564)
|
2026-03-27 14:33:46 -07:00 |
|
|
Qiaolin Yu
|
8c3ccef2d9
|
Fix Kimi K2.5 dp attention+ spec decoding launch crash (#21391)
|
2026-03-26 14:40:26 -07:00 |
|
|
Qiaolin Yu
|
3837588a4c
|
Temporarily disable TestReturnRoutedExperts since it's flaky (#21267)
|
2026-03-23 21:45:30 -07:00 |
|
|
Qiaolin Yu
|
3f883ebf2e
|
Fix test_qwen35_models (#21081)
|
2026-03-21 00:22:38 -07:00 |
|
|
Qiaolin Yu
|
eea9e19c13
|
fix lint introduced in #20708 (#20886)
|
2026-03-18 15:38:52 -07:00 |
|
|
Qiaolin Yu
|
c5d2528bff
|
Revert "[AMD][MORI] Fix MTP crash with FP4/FP8 dispatch and add NEXTN dispatch env vars." (#20797)
|
2026-03-17 17:28:09 -07:00 |
|
|
Qiaolin Yu
|
71a54c1c42
|
update CODEOWNERS (#20733)
|
2026-03-16 19:41:26 -07:00 |
|
|
Qiaolin Yu
|
7d87a6a071
|
Fix spec v1 token_ids_logprobs (#20718)
|
2026-03-16 19:23:28 -07:00 |
|
|
Qiaolin Yu
|
4659b08fcf
|
update CI_PERMISSIONS.json (#20551)
|
2026-03-13 14:52:06 -07:00 |
|
|
Qiaolin Yu
|
9c6f166600
|
tiny fix the usage of test_logprobs (#20473)
|
2026-03-12 15:37:45 -07:00 |
|
|
|
09a118fafe
|
Support return_logprob for spec v2 (overlap safe) (#19801)
Co-authored-by: Ratish1 <ratish1501@gmail.com>
Co-authored-by: Ratish1 <formula733@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-03-10 15:38:27 -07:00 |
|
|
Qiaolin Yu
|
bd460e9565
|
add logprob related params in bench_serving (#20218)
|
2026-03-10 15:04:57 -07:00 |
|
|
Qiaolin Yu
|
a3d88a247b
|
Enable piecewise-cuda-graph when logprob_start_len = -1 (#19453)
|
2026-03-10 12:50:57 -07:00 |
|
|
Qiaolin Yu
|
925185f9ec
|
Fix flashinfer backend with pcg (#20061)
|
2026-03-06 20:01:43 -08:00 |
|
|
Qiaolin Yu
|
46dced64ea
|
Adjust padding size to improve triton_kernels moe performance (#19174)
|
2026-03-05 14:50:40 -08:00 |
|
|
Qiaolin Yu
|
d80c884a27
|
Use single mma warp group for short q_len in FA to optimize decoding performance (#18985)
|
2026-02-23 17:01:28 +08:00 |
|
|
Qiaolin Yu
|
42b1019881
|
Fix bench_one_batch_server by moving the print statements (#19175)
|
2026-02-22 22:06:25 -08:00 |
|
|
Qiaolin Yu
|
8cf003c44b
|
Fix spec v2+dp attention in nsa backend (#19134)
|
2026-02-22 13:46:15 +08:00 |
|
|
Qiaolin Yu
|
96bae2355e
|
Add generated-shared-prefix dataset in bench_one_batch (#18986)
|
2026-02-20 13:33:10 -08:00 |
|
|
Qiaolin Yu
|
90d5e27f79
|
Enable fa3 PDL by compiling it with corresponding flags (#18756)
|
2026-02-18 17:12:05 +08:00 |
|
|
Qiaolin Yu
|
3c601db031
|
Fix generated-shared-prefix bench_serving (#18769)
|
2026-02-17 14:00:22 -08:00 |
|
|
Qiaolin Yu
|
4a1b50bb2d
|
Fix idle batch predict dtype in spec v2 (#18379)
|
2026-02-10 10:29:13 +08:00 |
|
|
Qiaolin Yu
|
4f6f5d25c8
|
Support fa4 decoding (#16034)
|
2026-01-21 22:54:02 +08:00 |
|
|
|
76b06bee03
|
[New Model] GLM4.7-Flash (#17247)
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: JustinTong0323 <justinning0323@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2026-01-20 23:44:16 +08:00 |
|
|
Qiaolin Yu
|
e3a95077bc
|
Add dpsk-r1-fp4 in nightly perf ci (#16882)
|
2026-01-15 16:13:35 -08:00 |
|
|
Qiaolin Yu
|
206db66f5c
|
tiny refactor pcg split op registration (#16863)
|
2026-01-10 23:45:28 +08:00 |
|
|
Qiaolin Yu
|
dd93e4451c
|
Support returning customized info (#16144)
|
2026-01-02 15:32:18 +08:00 |
|
|
Qiaolin Yu
|
6a3e709253
|
Tiny rename test_deepseek_v3_fp4_mtp_stage_b.py (#16141)
|
2025-12-30 15:55:07 +08:00 |
|
|
Qiaolin Yu
|
aef7ca7cf2
|
Raise the accept length bar in dpsk-r1-fp4 spec decoding tests (#15705)
|
2025-12-23 18:38:02 -08:00 |
|
|
Qiaolin Yu
|
677930c276
|
Add glm-4.6-fp8 with/without mtp in nightly ci (#15566)
|
2025-12-21 22:27:02 -08:00 |
|
|
Qiaolin Yu
|
a92de891b8
|
Split dpsk fp4 4 gpu tests and move the mtp part to real stage b (#15553)
|
2025-12-21 15:51:43 -08:00 |
|
|
Qiaolin Yu
|
254de6d2fd
|
Add triton_fused_moe config for GLM-4.6-FP8 tp8 blackwell (#15569)
|
2025-12-21 13:41:22 -08:00 |
|
|
Qiaolin Yu
|
e220da1723
|
tiny fix sampling seed for completion api (#15498)
|
2025-12-19 20:32:13 -08:00 |
|
|
Qiaolin Yu
|
173940927f
|
Add customized sampler registration (#15423)
|
2025-12-18 23:10:23 -08:00 |
|
|
Qiaolin Yu
|
729529190d
|
[ci] Move dpsk-r1-fp4 b200 test to stage b (#15084)
|
2025-12-13 23:14:08 -08:00 |
|
|
Qiaolin Yu
|
77873343c4
|
tiny update: use rope kernel in sgl-kernel for amd (#14955)
|
2025-12-12 18:48:12 -08:00 |
|
|
Qiaolin Yu
|
44fd701732
|
Tune triton fused moe for the case of glm-4.6-fp8 b200 tp4 (#15020)
|
2025-12-12 17:04:56 -08:00 |
|
|
Qiaolin Yu
|
aeff0d386b
|
Fix amd rope definition (#14556)
|
2025-12-07 23:47:03 -08:00 |
|
|
Qiaolin Yu
|
661e9775d0
|
[2/2] Add rope kernel in sgl-kernel (#14452)
|
2025-12-07 21:37:29 -08:00 |
|
|
Qiaolin Yu
|
cb8df87fc1
|
[1/2] Add rope kernel in sgl-kernel (#14334)
|
2025-12-04 16:45:44 +08:00 |
|
|
Qiaolin YuandKangyan-Zhou
|
7cb04dc0e5
|
Use trtllm mha decode kernel for target_verify in speculative decoding (#13976)
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
|
2025-11-26 20:40:34 -08:00 |
|
|
Qiaolin Yu
|
4b45d556a7
|
Overlap glm moe gemms in two cuda streams (#13786)
|
2025-11-24 18:15:24 -08:00 |
|
|
Qiaolin Yu
|
2892265d4c
|
Tune fp8_w8a8 fused triton moe for GLM-4.6-FP8 (#13815)
|
2025-11-23 13:29:54 -08:00 |
|
|
Qiaolin Yu
|
681b9e6425
|
Revert "enable csgmv automatically on cuda" (#13707)
|
2025-11-21 10:37:41 -08:00 |
|
|
Qiaolin YuandKangyan-Zhou
|
78a4b446c6
|
Fix dpsk-r1-fp4 tp8 by reverting two commits (#13162 and #13341) (#13348)
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
|
2025-11-15 21:31:36 -08:00 |
|
|
Qiaolin Yu
|
4a2768a86b
|
Fix spec decoding acc length for dpsk-r1-fp4 tp8 (2nd attempt) (#12915)
|
2025-11-11 13:30:33 -08:00 |
|
|
Qiaolin Yu
|
f290e8016e
|
Revert "Fix spec decoding acc length for dpsk-r1-fp4 tp8" (#12900)
|
2025-11-08 20:42:42 -08:00 |
|
|
Qiaolin Yu
|
9299a62fcb
|
Fix spec decoding acc length for dpsk-r1-fp4 tp8 (#12896)
|
2025-11-08 19:55:45 -08:00 |
|
|
Qiaolin Yu
|
59cce5941a
|
Use sgl fp4 quant kernel by default (#12482)
|
2025-10-31 19:51:28 -07:00 |
|
|
Qiaolin Yu
|
d98b81e260
|
Accelerate deepseek fp4 b200 ci (#11993)
|
2025-10-25 15:29:14 -07:00 |
|
|
Qiaolin Yu
|
71d41212e4
|
Fix dpsk-r1-fp4 launching crash (#12063)
|
2025-10-24 17:04:50 +08:00 |
|
|
Qiaolin Yu
|
36a4cad7b0
|
Support overlap-spec-v2 with trtllm_mla attention backend (#11821)
|
2025-10-23 16:55:35 +08:00 |
|
|
Qiaolin Yu
|
d9a20fd28a
|
Use trtllm_mla decode kernel for draft extend in speculative decoding (#11664)
|
2025-10-21 11:42:09 +08:00 |
|
|
Qiaolin YuandLiangsheng Yin
|
f7b1d8c5ab
|
Fix acc len and gen throughput metrics when enabling overlap-spec (#11823)
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
|
2025-10-21 01:34:38 +08:00 |
|
|
Qiaolin Yu
|
ebda73dc72
|
Use cutlass fp4 gemm by default (#11813)
|
2025-10-18 14:10:15 -07:00 |
|
|
Qiaolin Yu
|
547003bdd0
|
fix command line usage of profiling (#11793)
|
2025-10-18 12:54:36 +08:00 |
|
|
Qiaolin Yu
|
e4358a4585
|
Add fused_moe_triton config: triton_3_4_0/E=256,N=256,device_name=NVIDIA_B200.json (#11587)
|
2025-10-14 13:24:43 +08:00 |
|
|
Qiaolin Yu
|
43f80884c5
|
Fix accept rate in speculative decoding metrics (#11572)
|
2025-10-13 16:35:50 -07:00 |
|
|
Qiaolin Yu
|
a2b3d9b90b
|
Update DeepSeek-R1-FP4 default config on blackwell (#11512)
|
2025-10-12 20:32:11 -07:00 |
|
|
|
e2ac7888b8
|
[2/2] Support deterministic inference for temperature > 0 (#10678)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
|
2025-09-21 19:36:08 -07:00 |
|
|
Qiaolin Yu
|
8cda5a622c
|
Standalone speculative decoding (#10090)
|
2025-09-07 20:55:09 -07:00 |
|
|
Qiaolin Yu
|
4a4772ae03
|
Support speculative decoding in hybrid attention backend (#9573)
|
2025-08-28 01:11:42 -07:00 |
|
|
Qiaolin Yuandispobock
|
9c0c1e30b2
|
Disable torch.compile for get_last_loc_large_page_size_large_top_k (#9507)
Co-authored-by: ispobock <ispobaoke@gmail.com>
|
2025-08-22 02:05:02 -07:00 |
|
|
Qiaolin Yuandispobock
|
9ec314c6ac
|
Support speculative decoding in the trtllm_mha attention backend (#9331)
Co-authored-by: ispobock <ispobaoke@gmail.com>
|
2025-08-21 23:53:35 -07:00 |
|
|
Qiaolin Yu
|
af1973b871
|
Fix max_seq_len_k in trtllm_mha attention backend (#9416)
|
2025-08-20 19:17:13 -07:00 |
|