 strgrb
|
65ba5ab8b1
|
add cpp files for cpp_radix_tree to pyproject.toml. (#14052)
|
2025-11-30 13:05:04 +08:00 |
|
|
strgrb
|
94d26d850d
|
use non_blocking h2d in ForwardBatch.prepare_mlp_sync_batch. (#11605)
|
2025-10-14 11:30:59 -07:00 |
|
|
|
fac07c9b08
|
Support LingV2 model (#10359)
Co-authored-by: 羽癫 <yudian.zy@antgroup.com>
Co-authored-by: guoyuhong <yuhong.gyh@antgroup.com>
|
2025-09-11 23:53:52 -07:00 |
|
|
strgrbandZhang Kaihong
|
88fbc31b50
|
Support trtllm_allreduce_fusion in flashinfer for cuda<12.8 (#9339)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-08-20 16:54:30 -07:00 |
|
|
strgrbandZhang Kaihong
|
1f9d65f57d
|
use fast math for per_token_group_quant_8bit. (#9177)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-08-14 22:19:56 -07:00 |
|
|
strgrbandZhang Kaihong
|
fb4ce17de6
|
Fix per_token_group_quant_8bit when hidden_dim // group_size is not divided by 4. (#8449)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-07-28 01:32:46 -07:00 |
|
|
strgrbandZhang Kaihong
|
194841e329
|
remove kv_a.congigous in DeepseekV2AttentionMLA (#8058)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-07-15 18:20:41 -07:00 |
|
|
strgrbandZhang Kaihong
|
ceba0ce4f6
|
support return logprobs for pipeline (#7356)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-06-19 23:50:45 -07:00 |
|
|
strgrbandZhang Kaihong
|
b2a189dd11
|
use sglang_per_token_group_quant_fp8 from sgl-kernel instead of trion kernel (#5473)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-04-18 00:05:24 -07:00 |
|
|
strgrbandZhang Kaihong
|
668ecc6c5b
|
Fix ut mla-test-1-gpu-amd (#4813)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-27 08:27:51 -07:00 |
|
|
strgrbandZhang Kaihong
|
886fcbdd09
|
Use apply_rope_with_cos_sin_cache_inplace for DeepSeek (#4764)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-27 01:45:37 -07:00 |
|
|
strgrbandZhang Kaihong
|
df7014a8d2
|
avoid cudaStreamSynchronize in DeepSeekV2AttentionMLA (#4577)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-19 10:02:26 -07:00 |
|
|
strgrbandZhang Kaihong
|
f9c53cbb42
|
Create col-major and tma-aligned x_scale for deep_gemm.gemm_fp8_fp8_bf16_nt (#4515)
Co-authored-by: Zhang Kaihong <zhangkaihong.zkh@alibaba-inc.com>
|
2025-03-19 00:02:43 -07:00 |
|