 Qiaolin Yu
|
a965f886bf
|
[misc] update CI_PERMISSIONS.json (#24468)
|
2026-05-05 18:37:18 -07:00 |
|
|
Qiaolin Yu
|
4197c55968
|
[spec decoding] add tests for chain-style multi layer eagle + return_logprob (#24192)
|
2026-05-01 01:48:48 -07:00 |
|
|
Qiaolin Yu
|
583929c0a1
|
fix the compatibility between --moe-dense-tp-size 1 and piecewise cuda graph (#23972)
|
2026-04-30 02:12:13 -07:00 |
|
|
Qiaolin Yu
|
2bbd30a27a
|
relax the threshold in test_step3p5_flash_chain_mtp (#24105)
|
2026-04-29 16:53:35 -07:00 |
|
|
Qiaolin Yu
|
ea794deefc
|
[misc] fix lint in main branch (#24095)
|
2026-04-29 15:01:35 -07:00 |
|
|
Qiaolin Yu
|
79dbfe4505
|
Use spec v2 by default (#21062)
|
2026-04-29 13:40:42 -07:00 |
|
|
Qiaolin Yu
|
f57ec8d6ef
|
[spec decoding] add extra attribute 'spec_hidden_size' (#23890)
|
2026-04-28 19:54:50 -07:00 |
|
|
Qiaolin YuandYuzhen Zhou
|
c560326884
|
[perf] support return_routed_experts with overlap scheduling (#22911)
Co-authored-by: Yuzhen Zhou <82826991+zyzshishui@users.noreply.github.com>
|
2026-04-21 14:42:49 -07:00 |
|
|
Qiaolin Yu
|
a87806a65f
|
[misc] refine outdated comments for chain-style multi-layer MTP (#22996)
|
2026-04-16 14:49:43 -07:00 |
|
|
Qiaolin Yu
|
12266cf953
|
[misc] update .github/CODEOWNERS (#22993)
|
2026-04-16 14:19:41 -07:00 |
|
|
Qiaolin Yu
|
0b1b07db72
|
[misc] fix ray folder lint (#22905)
|
2026-04-15 15:08:18 -07:00 |
|
|
Qiaolin Yu
|
f41c810a2d
|
[misc] update CI_PERMISSIONS.json (#22570)
|
2026-04-10 18:58:55 -07:00 |
|
|
Qiaolin Yuand0xNullPath
|
d8831355a3
|
Fix multi_layer_eagle_worker_v2 draft extend selection, add chain style multi layer mtp test (#22340)
Co-authored-by: 0xNullPath <luyan@nvidia.com>
|
2026-04-10 12:44:52 -07:00 |
|
|
Qiaolin YuandLiangsheng Yin
|
117508dcd7
|
Switch eagle_infer_beta to EAGLE3 (#22303)
Co-authored-by: Liangsheng Yin <hnyls2002@users.noreply.github.com>
|
2026-04-07 18:43:48 -07:00 |
|
|
Qiaolin Yu
|
0bc4e0ea75
|
[misc] update CI_PERMISSIONS.json (#22207)
|
2026-04-06 14:11:49 -07:00 |
|
|
Qiaolin Yu
|
93f38fe410
|
tiny fix chain-style multi layer eagle comments (#22206)
|
2026-04-06 13:49:03 -07:00 |
|
|
Qiaolin Yu
|
f407461ec8
|
Tiny fix trtllm_fp8_per_tensor_scale_moe_wrapper router_logits dtype (#22006)
|
2026-04-05 21:11:45 -07:00 |
|
|
Qiaolin Yu
|
ef13031243
|
Tiny fix step3.5-flash launch crash (#22076)
|
2026-04-03 22:25:25 -07:00 |
|
|
Qiaolin Yu
|
b684b0b72f
|
Fix spec v2 + logprob when max_num_token is set (#20799)
|
2026-04-02 01:55:16 -07:00 |
|
|
Qiaolin Yu
|
d8db3077ca
|
Fix draft extend cuda graph when spec_step=1 (#21709)
|
2026-03-31 18:29:56 -07:00 |
|
|
Qiaolin Yu
|
5f6250769a
|
Reduce redundant speculative decoding CI tests (#21779)
|
2026-03-31 17:40:20 -07:00 |
|
|
Qiaolin Yu
|
a27651d5e0
|
Remove sync when enabling return_logprob (#20972)
|
2026-03-27 16:36:28 -07:00 |
|
|
Qiaolin Yu
|
4a41aec844
|
Fix flaky test_pp_single_node (#21564)
|
2026-03-27 14:33:46 -07:00 |
|
|
Qiaolin Yu
|
8c3ccef2d9
|
Fix Kimi K2.5 dp attention+ spec decoding launch crash (#21391)
|
2026-03-26 14:40:26 -07:00 |
|
|
Qiaolin Yu
|
3837588a4c
|
Temporarily disable TestReturnRoutedExperts since it's flaky (#21267)
|
2026-03-23 21:45:30 -07:00 |
|
|
Qiaolin Yu
|
3f883ebf2e
|
Fix test_qwen35_models (#21081)
|
2026-03-21 00:22:38 -07:00 |
|
|
Qiaolin Yu
|
eea9e19c13
|
fix lint introduced in #20708 (#20886)
|
2026-03-18 15:38:52 -07:00 |
|
|
Qiaolin Yu
|
c5d2528bff
|
Revert "[AMD][MORI] Fix MTP crash with FP4/FP8 dispatch and add NEXTN dispatch env vars." (#20797)
|
2026-03-17 17:28:09 -07:00 |
|
|
Qiaolin Yu
|
71a54c1c42
|
update CODEOWNERS (#20733)
|
2026-03-16 19:41:26 -07:00 |
|
|
Qiaolin Yu
|
7d87a6a071
|
Fix spec v1 token_ids_logprobs (#20718)
|
2026-03-16 19:23:28 -07:00 |
|
|
Qiaolin Yu
|
4659b08fcf
|
update CI_PERMISSIONS.json (#20551)
|
2026-03-13 14:52:06 -07:00 |
|
|
Qiaolin Yu
|
9c6f166600
|
tiny fix the usage of test_logprobs (#20473)
|
2026-03-12 15:37:45 -07:00 |
|
|
|
09a118fafe
|
Support return_logprob for spec v2 (overlap safe) (#19801)
Co-authored-by: Ratish1 <ratish1501@gmail.com>
Co-authored-by: Ratish1 <formula733@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-03-10 15:38:27 -07:00 |
|
|
Qiaolin Yu
|
bd460e9565
|
add logprob related params in bench_serving (#20218)
|
2026-03-10 15:04:57 -07:00 |
|
|
Qiaolin Yu
|
a3d88a247b
|
Enable piecewise-cuda-graph when logprob_start_len = -1 (#19453)
|
2026-03-10 12:50:57 -07:00 |
|
|
Qiaolin Yu
|
925185f9ec
|
Fix flashinfer backend with pcg (#20061)
|
2026-03-06 20:01:43 -08:00 |
|
|
Qiaolin Yu
|
46dced64ea
|
Adjust padding size to improve triton_kernels moe performance (#19174)
|
2026-03-05 14:50:40 -08:00 |
|
|
Qiaolin Yu
|
d80c884a27
|
Use single mma warp group for short q_len in FA to optimize decoding performance (#18985)
|
2026-02-23 17:01:28 +08:00 |
|
|
Qiaolin Yu
|
42b1019881
|
Fix bench_one_batch_server by moving the print statements (#19175)
|
2026-02-22 22:06:25 -08:00 |
|
|
Qiaolin Yu
|
8cf003c44b
|
Fix spec v2+dp attention in nsa backend (#19134)
|
2026-02-22 13:46:15 +08:00 |
|
|
Qiaolin Yu
|
96bae2355e
|
Add generated-shared-prefix dataset in bench_one_batch (#18986)
|
2026-02-20 13:33:10 -08:00 |
|
|
Qiaolin Yu
|
90d5e27f79
|
Enable fa3 PDL by compiling it with corresponding flags (#18756)
|
2026-02-18 17:12:05 +08:00 |
|
|
Qiaolin Yu
|
3c601db031
|
Fix generated-shared-prefix bench_serving (#18769)
|
2026-02-17 14:00:22 -08:00 |
|
|
Qiaolin Yu
|
4a1b50bb2d
|
Fix idle batch predict dtype in spec v2 (#18379)
|
2026-02-10 10:29:13 +08:00 |
|
|
Qiaolin Yu
|
4f6f5d25c8
|
Support fa4 decoding (#16034)
|
2026-01-21 22:54:02 +08:00 |
|
|
|
76b06bee03
|
[New Model] GLM4.7-Flash (#17247)
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: JustinTong0323 <justinning0323@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2026-01-20 23:44:16 +08:00 |
|
|
Qiaolin Yu
|
e3a95077bc
|
Add dpsk-r1-fp4 in nightly perf ci (#16882)
|
2026-01-15 16:13:35 -08:00 |
|
|
Qiaolin Yu
|
206db66f5c
|
tiny refactor pcg split op registration (#16863)
|
2026-01-10 23:45:28 +08:00 |
|
|
Qiaolin Yu
|
dd93e4451c
|
Support returning customized info (#16144)
|
2026-01-02 15:32:18 +08:00 |
|
|
Qiaolin Yu
|
6a3e709253
|
Tiny rename test_deepseek_v3_fp4_mtp_stage_b.py (#16141)
|
2025-12-30 15:55:07 +08:00 |
|
|
Qiaolin Yu
|
aef7ca7cf2
|
Raise the accept length bar in dpsk-r1-fp4 spec decoding tests (#15705)
|
2025-12-23 18:38:02 -08:00 |
|
|
Qiaolin Yu
|
677930c276
|
Add glm-4.6-fp8 with/without mtp in nightly ci (#15566)
|
2025-12-21 22:27:02 -08:00 |
|
|
Qiaolin Yu
|
a92de891b8
|
Split dpsk fp4 4 gpu tests and move the mtp part to real stage b (#15553)
|
2025-12-21 15:51:43 -08:00 |
|
|
Qiaolin Yu
|
254de6d2fd
|
Add triton_fused_moe config for GLM-4.6-FP8 tp8 blackwell (#15569)
|
2025-12-21 13:41:22 -08:00 |
|
|
Qiaolin Yu
|
e220da1723
|
tiny fix sampling seed for completion api (#15498)
|
2025-12-19 20:32:13 -08:00 |
|
|
Qiaolin Yu
|
173940927f
|
Add customized sampler registration (#15423)
|
2025-12-18 23:10:23 -08:00 |
|
|
Qiaolin Yu
|
729529190d
|
[ci] Move dpsk-r1-fp4 b200 test to stage b (#15084)
|
2025-12-13 23:14:08 -08:00 |
|
|
Qiaolin Yu
|
77873343c4
|
tiny update: use rope kernel in sgl-kernel for amd (#14955)
|
2025-12-12 18:48:12 -08:00 |
|
|
Qiaolin Yu
|
44fd701732
|
Tune triton fused moe for the case of glm-4.6-fp8 b200 tp4 (#15020)
|
2025-12-12 17:04:56 -08:00 |
|
|
Qiaolin Yu
|
aeff0d386b
|
Fix amd rope definition (#14556)
|
2025-12-07 23:47:03 -08:00 |
|
|
Qiaolin Yu
|
661e9775d0
|
[2/2] Add rope kernel in sgl-kernel (#14452)
|
2025-12-07 21:37:29 -08:00 |
|
|
Qiaolin Yu
|
cb8df87fc1
|
[1/2] Add rope kernel in sgl-kernel (#14334)
|
2025-12-04 16:45:44 +08:00 |
|
|
Qiaolin YuandKangyan-Zhou
|
7cb04dc0e5
|
Use trtllm mha decode kernel for target_verify in speculative decoding (#13976)
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
|
2025-11-26 20:40:34 -08:00 |
|
|
Qiaolin Yu
|
4b45d556a7
|
Overlap glm moe gemms in two cuda streams (#13786)
|
2025-11-24 18:15:24 -08:00 |
|
|
Qiaolin Yu
|
2892265d4c
|
Tune fp8_w8a8 fused triton moe for GLM-4.6-FP8 (#13815)
|
2025-11-23 13:29:54 -08:00 |
|
|
Qiaolin Yu
|
681b9e6425
|
Revert "enable csgmv automatically on cuda" (#13707)
|
2025-11-21 10:37:41 -08:00 |
|
|
Qiaolin YuandKangyan-Zhou
|
78a4b446c6
|
Fix dpsk-r1-fp4 tp8 by reverting two commits (#13162 and #13341) (#13348)
Co-authored-by: Kangyan-Zhou <zky314343421@gmail.com>
|
2025-11-15 21:31:36 -08:00 |
|
|
Qiaolin Yu
|
4a2768a86b
|
Fix spec decoding acc length for dpsk-r1-fp4 tp8 (2nd attempt) (#12915)
|
2025-11-11 13:30:33 -08:00 |
|
|
Qiaolin Yu
|
f290e8016e
|
Revert "Fix spec decoding acc length for dpsk-r1-fp4 tp8" (#12900)
|
2025-11-08 20:42:42 -08:00 |
|
|
Qiaolin Yu
|
9299a62fcb
|
Fix spec decoding acc length for dpsk-r1-fp4 tp8 (#12896)
|
2025-11-08 19:55:45 -08:00 |
|
|
Qiaolin Yu
|
59cce5941a
|
Use sgl fp4 quant kernel by default (#12482)
|
2025-10-31 19:51:28 -07:00 |
|
|
Qiaolin Yu
|
d98b81e260
|
Accelerate deepseek fp4 b200 ci (#11993)
|
2025-10-25 15:29:14 -07:00 |
|
|
Qiaolin Yu
|
71d41212e4
|
Fix dpsk-r1-fp4 launching crash (#12063)
|
2025-10-24 17:04:50 +08:00 |
|
|
Qiaolin Yu
|
36a4cad7b0
|
Support overlap-spec-v2 with trtllm_mla attention backend (#11821)
|
2025-10-23 16:55:35 +08:00 |
|
|
Qiaolin Yu
|
d9a20fd28a
|
Use trtllm_mla decode kernel for draft extend in speculative decoding (#11664)
|
2025-10-21 11:42:09 +08:00 |
|
|
Qiaolin YuandLiangsheng Yin
|
f7b1d8c5ab
|
Fix acc len and gen throughput metrics when enabling overlap-spec (#11823)
Co-authored-by: Liangsheng Yin <lsyincs@gmail.com>
|
2025-10-21 01:34:38 +08:00 |
|
|
Qiaolin Yu
|
ebda73dc72
|
Use cutlass fp4 gemm by default (#11813)
|
2025-10-18 14:10:15 -07:00 |
|
|
Qiaolin Yu
|
547003bdd0
|
fix command line usage of profiling (#11793)
|
2025-10-18 12:54:36 +08:00 |
|
|
Qiaolin Yu
|
e4358a4585
|
Add fused_moe_triton config: triton_3_4_0/E=256,N=256,device_name=NVIDIA_B200.json (#11587)
|
2025-10-14 13:24:43 +08:00 |
|
|
Qiaolin Yu
|
43f80884c5
|
Fix accept rate in speculative decoding metrics (#11572)
|
2025-10-13 16:35:50 -07:00 |
|
|
Qiaolin Yu
|
a2b3d9b90b
|
Update DeepSeek-R1-FP4 default config on blackwell (#11512)
|
2025-10-12 20:32:11 -07:00 |
|
|
|
e2ac7888b8
|
[2/2] Support deterministic inference for temperature > 0 (#10678)
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>
|
2025-09-21 19:36:08 -07:00 |
|
|
Qiaolin Yu
|
8cda5a622c
|
Standalone speculative decoding (#10090)
|
2025-09-07 20:55:09 -07:00 |
|
|
Qiaolin Yu
|
4a4772ae03
|
Support speculative decoding in hybrid attention backend (#9573)
|
2025-08-28 01:11:42 -07:00 |
|
|
Qiaolin Yuandispobock
|
9c0c1e30b2
|
Disable torch.compile for get_last_loc_large_page_size_large_top_k (#9507)
Co-authored-by: ispobock <ispobaoke@gmail.com>
|
2025-08-22 02:05:02 -07:00 |
|
|
Qiaolin Yuandispobock
|
9ec314c6ac
|
Support speculative decoding in the trtllm_mha attention backend (#9331)
Co-authored-by: ispobock <ispobaoke@gmail.com>
|
2025-08-21 23:53:35 -07:00 |
|
|
Qiaolin Yu
|
af1973b871
|
Fix max_seq_len_k in trtllm_mha attention backend (#9416)
|
2025-08-20 19:17:13 -07:00 |
|
|
Qiaolin Yu
|
fc8c8e5041
|
Integrate triton_kernels in sgl-kernel (#8762)
|
2025-08-04 12:12:14 -07:00 |
|
|
Qiaolin Yu
|
484d0e021d
|
doc: add bench_one_batch_server in the benchmark doc (#8441)
|
2025-07-27 23:07:54 -07:00 |
|
|
|
2810338401
|
[feat] Support different attention backends for prefill and decode (#6338)
Co-authored-by: tianqilin.99 <tianqilin.99@bytedance.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-07-28 11:42:29 +08:00 |
|
|
Qiaolin Yu
|
bbcfbc1a02
|
feat: add h200 tp 16 kimi k2 moe config (#8183)
|
2025-07-19 23:30:08 -07:00 |
|
|
Qiaolin Yu
|
69f453e5a4
|
Use device_group for all_gather when disabling overlap scheduling (#8001)
|
2025-07-15 19:38:58 -07:00 |
|
|
Qiaolin Yu
|
3bc43c683e
|
Fix different device type adjustment in PP (#7760)
|
2025-07-15 19:37:14 -07:00 |
|
|
Qiaolin YuandCheng Wan
|
41650b0d70
|
feat: support compatibility between MTP and two-batch-overlap (#7225)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
|
2025-06-27 01:10:27 -07:00 |
|
|
Qiaolin Yu
|
b8df43ab9c
|
Fix gathered_buffer issues in tbo (#7531)
|
2025-06-25 14:42:21 -07:00 |
|
|
Qiaolin Yu
|
f20f70003d
|
Fix torch version in blackwell dockerfile (#7017)
|
2025-06-09 17:06:50 -07:00 |
|
|
Qiaolin Yu
|
7dc0e39442
|
Bump torch to 2.7.0 (#6788)
|
2025-05-31 14:43:12 -07:00 |
|
|
Qiaolin Yu
|
0b9557fcd7
|
Disable compiling arch below sm_90 in aarch64 by default (#6380)
|
2025-05-27 15:50:02 -07:00 |
|
|
Qiaolin Yu
|
aff584fa54
|
Fix sgl-kernel build on aarch64 platforms (#6062)
|
2025-05-06 16:10:57 -07:00 |
|
|
Qiaolin Yuandzhaochenyang20
|
357671e216
|
Add examples for server token-in-token-out (#4103)
Co-authored-by: zhaochenyang20 <zhaochen20@outlook.com>
|
2025-03-05 13:16:31 -08:00 |
|