 Qiaolin Yu
|
43390f63f5
|
[qwen 3.8 next] change the testing model in test_qwen4_exp_models.py (#39662)
|
2026-09-16 12:47:45 -07:00 |
|
|
Qiaolin Yu
|
2c0a70960c
|
[qwen 3.8 next] reuse old cuda stream instead of endlessly creating streams (#39474)
|
2026-09-15 01:36:14 -07:00 |
|
|
Qiaolin Yuandehuaa
|
d72e59508b
|
Reland fix(qsa): clamp the compress gather to the rows (#38346) (#39446)
Co-authored-by: ehuaa <ehuamail1@gmail.com>
|
2026-09-14 14:05:41 -07:00 |
|
|
Qiaolin Yu
|
6953dae005
|
[Qwen 3.8 Next] Remove unused tokenwise QSA implementation and tests (#38960)
|
2026-09-12 01:32:01 -07:00 |
|
|
Qiaolin Yu
|
559c7fa75b
|
Revert "PD disaggregation, isolated transfer, prefill OOM fixed." (#38572)
|
2026-09-08 15:57:44 -07:00 |
|
|
+3
|
52fecfdf09
|
support qwen 3.8 flash next (#37500)
Co-authored-by: ch-wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: ispobock <26454835+ispobock@users.noreply.github.com>
Co-authored-by: JustinTong0323 <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: samuellees <26428561+samuellees@users.noreply.github.com>
Co-authored-by: YAMY1234 <74099316+YAMY1234@users.noreply.github.com>
Co-authored-by: yhyang201 <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: yizhang2077 <25844240+yizhang2077@users.noreply.github.com>
Co-authored-by: zijiexia <37504505+zijiexia@users.noreply.github.com>
Co-authored-by: Shinto C V <cshintov@gmail.com>
Co-authored-by: Julian Huang <huangzhilin.hzl@antgroup.com>
Co-authored-by: Xiaoyu Zhang <1182563586@qq.com>
Co-authored-by: yhyang201 <yhyang201@gmail.com>
|
2026-09-08 13:56:21 -07:00 |
|
|
+8
|
5f216fc33f
|
qwen 3.8 rebase (#35758)
Co-authored-by: cherichy <cherichy@outlook.com>
Co-authored-by: guangyunh-nv <guangyunh@nvidia.com>
Co-authored-by: jiahanc <jiahanc@nvidia.com>
Co-authored-by: jinyangyuan-nvidia <joyuan@nvidia.com>
Co-authored-by: Cheng Hang <chang@nvidia.com>
Co-authored-by: Yicheng Qiang <yqiang@nvidia.com>
Co-authored-by: Sam Li <lsam@nvidia.com>
Co-authored-by: Tom-Zheng <tizheng@nvidia.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: xiaoweiw-nv <xiaoweiw@nvidia.com>
Co-authored-by: Zheng Li <lizheng.cs@zju.edu.cn>
Co-authored-by: yizhang2077 <1109276519@qq.com>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
|
2026-08-28 20:41:34 -07:00 |
|
|
Qiaolin Yu
|
733c05c887
|
[spec decoding] support inkling dspark (#31847)
|
2026-08-10 13:21:35 -07:00 |
|
|
Qiaolin Yu
|
9303e26f03
|
[ci] add qwen 3.5 mtp + replayssm + flashinfer gdn test (#33607)
|
2026-08-04 18:08:42 -07:00 |
|
|
Qiaolin Yu
|
b1754a8f3b
|
[gdn] fused replayssm ring write into flashinfer gdn mtp verify kernel (#33102)
|
2026-08-02 21:31:22 -07:00 |
|
|
Qiaolin Yu
|
f3fd869494
|
[gdn] support replayssm with extra buffer (#32692)
|
2026-07-30 21:34:37 -07:00 |
|
|
Qiaolin YuandZijie Xia
|
9c0dbf508f
|
[cookbook] add inkling dspark command (#32465)
Co-authored-by: Zijie Xia <zijie.xia@radixark.ai>
|
2026-07-28 14:33:38 -07:00 |
|
|
Qiaolin Yu
|
b8b9f3c8f5
|
[misc] update CI_PERMISSIONS.json (#32679)
|
2026-07-28 14:12:29 -07:00 |
|
|
Qiaolin Yu
|
4ececf2b1d
|
[chore] Add verbose flag to twine upload command (#32349)
|
2026-07-24 14:54:07 -07:00 |
|
|
Qiaolin Yu
|
82fe0f041a
|
Fix stale flashinfer-MLA fallback poisoning spec verify capture (trtllm_mla + tc_piecewise) (#32288)
|
2026-07-24 13:24:26 -07:00 |
|
|
Qiaolin Yu
|
15d73f1e03
|
Fix dynamo recompile limit in allreduce and bf16 gemm (#32239)
|
2026-07-23 22:24:01 -07:00 |
|
|
Qiaolin Yu
|
a0728ea502
|
[spec decoding] fix inkling multi layer mtp draft extend cuda graph (#32254)
|
2026-07-23 14:52:30 -07:00 |
|
|
Qiaolin Yu
|
378aea1385
|
Fix nvfp4 online scale with pcg (#32246)
|
2026-07-23 14:42:37 -07:00 |
|
|
Qiaolin Yu
|
01a96720c6
|
[spec decoding] replace torch.multinomial with several native torch op in rejection sampling (#31620)
|
2026-07-17 16:58:11 -07:00 |
|
|
Qiaolin Yu
|
632adff9fd
|
refactor logprob processor layer (#20071)
|
2026-07-17 16:28:10 -07:00 |
|
|
Qiaolin Yu
|
e2d2e8d07e
|
[spec decoding] fix multi_layer_eagle rotate_input_ids kernel registration (#31614)
|
2026-07-17 15:18:01 -07:00 |
|
|
Qiaolin Yu
|
801571e949
|
[spec decoding] support rejection sampling in multi layer eagle (#30303)
|
2026-07-07 14:50:24 -07:00 |
|
|
Qiaolin Yu
|
e26bceb81e
|
[perf] simplify _apply_cuda_graph_metadata for draft extend in trtllm_mla backend (#29077)
|
2026-06-24 15:45:26 -07:00 |
|
|
Qiaolin Yu
|
c01ad10d16
|
[perf] tiny optimize select_index op for draft extend (#29078)
|
2026-06-24 12:27:13 -07:00 |
|
|
Qiaolin Yu
|
2411737244
|
[spec decoding] fully overlap spec decoding for hybrid linear attention backend (#28579)
|
2026-06-18 13:31:12 -07:00 |
|
|
Qiaolin Yu
|
753aa89a83
|
[spec decoding] fix mrope_positions in draft extend (#28464)
|
2026-06-17 13:25:31 -07:00 |
|
|
Qiaolin Yu
|
2ad00faae1
|
[ci] add kimi nvfp4 nightly tests (#28467)
|
2026-06-16 16:49:51 -07:00 |
|
|
Qiaolin Yuandshuwenn
|
e068355831
|
[spec decoding] supports step 0 in adaptive spec decoding (updating draft kv cache without draft decoding) (#23994)
Co-authored-by: shuwenn <2508695655@qq.com>
|
2026-06-15 22:21:26 -07:00 |
|
|
Qiaolin Yu
|
f293ddf3ce
|
[perf] reduce overhead of fill_ids list reconstruction and decref (#27965)
|
2026-06-14 00:41:11 -07:00 |
|
|
Qiaolin Yu
|
e02f7ca482
|
[perf] remove several h2d sync (#28076)
|
2026-06-12 20:30:17 -07:00 |
|
|
Qiaolin Yu
|
4b0f629082
|
[perf] reduce radix cache match overhead by changing the match algorithm (#27364)
|
2026-06-06 15:40:28 -07:00 |
|
|
Qiaolin Yu
|
8c47b7678a
|
[attn backend] clean legacy init_mha_chunk_metadata in trtllm_mla backend (#27403)
|
2026-06-05 23:30:21 -07:00 |
|
|
Qiaolin Yu
|
bd47869ba4
|
[perf] parallelize create_flashmla_kv_indices over page-blocks (#27320)
|
2026-06-04 22:11:43 -07:00 |
|
|
Qiaolin Yu
|
c55548ba11
|
[perf] Replicate embed_tokens to drop the post-embed all-reduce (#26970)
|
2026-06-02 16:48:18 -07:00 |
|
|
Qiaolin Yu
|
4151a04d1a
|
[Perf][Spec Decoding] Skip cat/topk/sort/gather in draft_forward for topk=1 (#26424)
|
2026-06-01 15:37:47 -07:00 |
|
|
Qiaolin Yu
|
3bce192bd2
|
[misc] update adaptive spec decoding code owners (#26965)
|
2026-06-01 14:08:09 -07:00 |
|
|
Qiaolin Yu
|
118465f5b5
|
[attn backend] Make spec_v2 seq_lens_cpu optional in trtllm_mla backend (#26824)
|
2026-05-31 20:29:50 -07:00 |
|
|
Qiaolin Yu
|
3cecc77ccb
|
[perf] Fuse NVFP4 gate_up_gemm + swiglu + output FP4 quant (#26626)
|
2026-05-29 13:16:24 -07:00 |
|
|
Qiaolin Yu
|
561e54f803
|
Update kimi k25 launch command in cookbook (#26511)
|
2026-05-27 16:04:04 -07:00 |
|
|
Qiaolin Yu
|
dd6f073377
|
Reland "[perf][spec decoding] Skip full-vocab softmax in EAGLE draft when topk == 1 (#26235)" (#26397)
|
2026-05-26 14:14:48 -07:00 |
|
|
Qiaolin Yu
|
a77449f86d
|
[perf][spec decoding] Skip full-vocab softmax in EAGLE draft when topk == 1 (#26235)
|
2026-05-25 02:06:48 -07:00 |
|
|
Qiaolin Yu
|
ec6fcb93cb
|
[perf][spec decoding] Skip common_template in TRTLLMMLAMultiStepDraftBackend init (#26241)
|
2026-05-24 21:36:16 -07:00 |
|
|
Qiaolin Yu
|
982f67d9a6
|
Suppress cutlass-dsl noisy warning (#26169)
|
2026-05-23 13:19:14 -07:00 |
|
|
Qiaolin Yu
|
cb7b57955d
|
fix tokenspeed_mla attn kernel jit (#26170)
|
2026-05-23 03:24:33 -07:00 |
|
|
Qiaolin Yu
|
c112f7623a
|
Skip init_mha_chunk_metadata in trtllm_mla when not needed (#26017)
|
2026-05-22 16:34:16 -07:00 |
|
|
Qiaolin Yu
|
1a17d753f1
|
[perf] prepare_prefill_qkv hook + fp8 quantize jit kernel (#25460)
|
2026-05-20 14:20:49 -07:00 |
|
|
Qiaolin Yu
|
1f185c6ba8
|
Support draft extend cuda graph for tokenspeed_mla attention backend (#25489)
|
2026-05-18 11:26:16 -07:00 |
|
|
Qiaolin Yu
|
f5049709b3
|
fix(eagle3): drop +1 offset on aux layer ids when first id != 1 (#25454)
|
2026-05-18 11:25:51 -07:00 |
|
|
Qiaolin Yu
|
9869ef0849
|
Revert "[attn backend] avoid initing parent class's workspace buffer" (#25488)
|
2026-05-16 13:05:42 -07:00 |
|
|
Qiaolin Yu
|
2f81718773
|
[attn backend] avoid initing parent class's workspace buffer (#25321)
|
2026-05-16 03:30:33 -07:00 |
|
|
Qiaolin Yu
|
7af4320d67
|
[perf] fix kimi tokenizer to improve ttft (#25265)
|
2026-05-14 19:11:25 -07:00 |
|
|
Qiaolin Yu
|
421179c453
|
[perf] avoid hidden states d2h when return_hidden_states=false (#25155)
|
2026-05-13 23:47:17 -07:00 |
|
|
Qiaolin Yu
|
7618ad7075
|
[attn backend] Integrate tokenspeed_mla prefill/decode kernels (fp8 kv cache, blackwell) (#24925)
|
2026-05-13 17:36:17 -07:00 |
|
|
Qiaolin Yu
|
cfc41d5b15
|
Fix kimi k2.5 mla eagle + dp attention (#25033)
|
2026-05-11 20:58:00 -07:00 |
|
|
Qiaolin Yu
|
a87fb399de
|
[spec decoding] support kimi-k2.5-eagle3-mla (#24826)
|
2026-05-09 23:57:39 -07:00 |
|
|
Qiaolin Yu
|
a965f886bf
|
[misc] update CI_PERMISSIONS.json (#24468)
|
2026-05-05 18:37:18 -07:00 |
|
|
Qiaolin Yu
|
4197c55968
|
[spec decoding] add tests for chain-style multi layer eagle + return_logprob (#24192)
|
2026-05-01 01:48:48 -07:00 |
|
|
Qiaolin Yu
|
583929c0a1
|
fix the compatibility between --moe-dense-tp-size 1 and piecewise cuda graph (#23972)
|
2026-04-30 02:12:13 -07:00 |
|
|
Qiaolin Yu
|
2bbd30a27a
|
relax the threshold in test_step3p5_flash_chain_mtp (#24105)
|
2026-04-29 16:53:35 -07:00 |
|
|
Qiaolin Yu
|
ea794deefc
|
[misc] fix lint in main branch (#24095)
|
2026-04-29 15:01:35 -07:00 |
|
|
Qiaolin Yu
|
79dbfe4505
|
Use spec v2 by default (#21062)
|
2026-04-29 13:40:42 -07:00 |
|
|
Qiaolin Yu
|
f57ec8d6ef
|
[spec decoding] add extra attribute 'spec_hidden_size' (#23890)
|
2026-04-28 19:54:50 -07:00 |
|
|
Qiaolin YuandYuzhen Zhou
|
c560326884
|
[perf] support return_routed_experts with overlap scheduling (#22911)
Co-authored-by: Yuzhen Zhou <82826991+zyzshishui@users.noreply.github.com>
|
2026-04-21 14:42:49 -07:00 |
|
|
Qiaolin Yu
|
a87806a65f
|
[misc] refine outdated comments for chain-style multi-layer MTP (#22996)
|
2026-04-16 14:49:43 -07:00 |
|
|
Qiaolin Yu
|
12266cf953
|
[misc] update .github/CODEOWNERS (#22993)
|
2026-04-16 14:19:41 -07:00 |
|
|
Qiaolin Yu
|
0b1b07db72
|
[misc] fix ray folder lint (#22905)
|
2026-04-15 15:08:18 -07:00 |
|
|
Qiaolin Yu
|
f41c810a2d
|
[misc] update CI_PERMISSIONS.json (#22570)
|
2026-04-10 18:58:55 -07:00 |
|
|
Qiaolin Yuand0xNullPath
|
d8831355a3
|
Fix multi_layer_eagle_worker_v2 draft extend selection, add chain style multi layer mtp test (#22340)
Co-authored-by: 0xNullPath <luyan@nvidia.com>
|
2026-04-10 12:44:52 -07:00 |
|
|
Qiaolin YuandLiangsheng Yin
|
117508dcd7
|
Switch eagle_infer_beta to EAGLE3 (#22303)
Co-authored-by: Liangsheng Yin <hnyls2002@users.noreply.github.com>
|
2026-04-07 18:43:48 -07:00 |
|
|
Qiaolin Yu
|
0bc4e0ea75
|
[misc] update CI_PERMISSIONS.json (#22207)
|
2026-04-06 14:11:49 -07:00 |
|
|
Qiaolin Yu
|
93f38fe410
|
tiny fix chain-style multi layer eagle comments (#22206)
|
2026-04-06 13:49:03 -07:00 |
|
|
Qiaolin Yu
|
f407461ec8
|
Tiny fix trtllm_fp8_per_tensor_scale_moe_wrapper router_logits dtype (#22006)
|
2026-04-05 21:11:45 -07:00 |
|
|
Qiaolin Yu
|
ef13031243
|
Tiny fix step3.5-flash launch crash (#22076)
|
2026-04-03 22:25:25 -07:00 |
|
|
Qiaolin Yu
|
b684b0b72f
|
Fix spec v2 + logprob when max_num_token is set (#20799)
|
2026-04-02 01:55:16 -07:00 |
|
|
Qiaolin Yu
|
d8db3077ca
|
Fix draft extend cuda graph when spec_step=1 (#21709)
|
2026-03-31 18:29:56 -07:00 |
|
|
Qiaolin Yu
|
5f6250769a
|
Reduce redundant speculative decoding CI tests (#21779)
|
2026-03-31 17:40:20 -07:00 |
|
|
Qiaolin Yu
|
a27651d5e0
|
Remove sync when enabling return_logprob (#20972)
|
2026-03-27 16:36:28 -07:00 |
|
|
Qiaolin Yu
|
4a41aec844
|
Fix flaky test_pp_single_node (#21564)
|
2026-03-27 14:33:46 -07:00 |
|
|
Qiaolin Yu
|
8c3ccef2d9
|
Fix Kimi K2.5 dp attention+ spec decoding launch crash (#21391)
|
2026-03-26 14:40:26 -07:00 |
|
|
Qiaolin Yu
|
3837588a4c
|
Temporarily disable TestReturnRoutedExperts since it's flaky (#21267)
|
2026-03-23 21:45:30 -07:00 |
|
|
Qiaolin Yu
|
3f883ebf2e
|
Fix test_qwen35_models (#21081)
|
2026-03-21 00:22:38 -07:00 |
|
|
Qiaolin Yu
|
eea9e19c13
|
fix lint introduced in #20708 (#20886)
|
2026-03-18 15:38:52 -07:00 |
|
|
Qiaolin Yu
|
c5d2528bff
|
Revert "[AMD][MORI] Fix MTP crash with FP4/FP8 dispatch and add NEXTN dispatch env vars." (#20797)
|
2026-03-17 17:28:09 -07:00 |
|
|
Qiaolin Yu
|
71a54c1c42
|
update CODEOWNERS (#20733)
|
2026-03-16 19:41:26 -07:00 |
|
|
Qiaolin Yu
|
7d87a6a071
|
Fix spec v1 token_ids_logprobs (#20718)
|
2026-03-16 19:23:28 -07:00 |
|
|
Qiaolin Yu
|
4659b08fcf
|
update CI_PERMISSIONS.json (#20551)
|
2026-03-13 14:52:06 -07:00 |
|
|
Qiaolin Yu
|
9c6f166600
|
tiny fix the usage of test_logprobs (#20473)
|
2026-03-12 15:37:45 -07:00 |
|
|
|
09a118fafe
|
Support return_logprob for spec v2 (overlap safe) (#19801)
Co-authored-by: Ratish1 <ratish1501@gmail.com>
Co-authored-by: Ratish1 <formula733@gmail.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
|
2026-03-10 15:38:27 -07:00 |
|
|
Qiaolin Yu
|
bd460e9565
|
add logprob related params in bench_serving (#20218)
|
2026-03-10 15:04:57 -07:00 |
|
|
Qiaolin Yu
|
a3d88a247b
|
Enable piecewise-cuda-graph when logprob_start_len = -1 (#19453)
|
2026-03-10 12:50:57 -07:00 |
|
|
Qiaolin Yu
|
925185f9ec
|
Fix flashinfer backend with pcg (#20061)
|
2026-03-06 20:01:43 -08:00 |
|
|
Qiaolin Yu
|
46dced64ea
|
Adjust padding size to improve triton_kernels moe performance (#19174)
|
2026-03-05 14:50:40 -08:00 |
|
|
Qiaolin Yu
|
d80c884a27
|
Use single mma warp group for short q_len in FA to optimize decoding performance (#18985)
|
2026-02-23 17:01:28 +08:00 |
|
|
Qiaolin Yu
|
42b1019881
|
Fix bench_one_batch_server by moving the print statements (#19175)
|
2026-02-22 22:06:25 -08:00 |
|
|
Qiaolin Yu
|
8cf003c44b
|
Fix spec v2+dp attention in nsa backend (#19134)
|
2026-02-22 13:46:15 +08:00 |
|
|
Qiaolin Yu
|
96bae2355e
|
Add generated-shared-prefix dataset in bench_one_batch (#18986)
|
2026-02-20 13:33:10 -08:00 |
|
|
Qiaolin Yu
|
90d5e27f79
|
Enable fa3 PDL by compiling it with corresponding flags (#18756)
|
2026-02-18 17:12:05 +08:00 |
|
|
Qiaolin Yu
|
3c601db031
|
Fix generated-shared-prefix bench_serving (#18769)
|
2026-02-17 14:00:22 -08:00 |
|
|
Qiaolin Yu
|
4a1b50bb2d
|
Fix idle batch predict dtype in spec v2 (#18379)
|
2026-02-10 10:29:13 +08:00 |
|
|
Qiaolin Yu
|
4f6f5d25c8
|
Support fa4 decoding (#16034)
|
2026-01-21 22:54:02 +08:00 |
|