 b8zhong
|
42028af614
|
enable csgmv automatically on cuda (#13600)
|
2025-11-20 12:53:02 -08:00 |
|
|
b8zhong
|
92ad2ff9ce
|
Flashinfer TRTLLM-GEN-MoE + Qwen3 (#13489)
|
2025-11-18 14:18:29 -08:00 |
|
|
b8zhong
|
f33860777c
|
[Piecewise CUDA Graph] Support ModelOpt FP8 (#13094)
|
2025-11-18 12:46:24 +08:00 |
|
|
b8zhong
|
d5fa58c4dd
|
fix nightly docker build (#13386)
|
2025-11-16 11:21:09 -08:00 |
|
|
b8zhong
|
f35f7f1245
|
[Piecewise CUDA Graph] Support W4A8 (#13179)
|
2025-11-16 11:53:50 +08:00 |
|
|
b8zhong
|
24a25ffa20
|
[Piecewise CUDA Graph] Support ModelOpt FP4 (#13101)
|
2025-11-16 11:03:19 +08:00 |
|
|
b8zhong
|
7b79cc4fe2
|
ci: speed up b200 ci (#13237)
|
2025-11-13 23:57:33 -08:00 |
|
|
b8zhong
|
e523e2167a
|
remove deprecated tile_tokens_dim (#13186)
|
2025-11-13 17:09:27 -08:00 |
|
|
b8zhong
|
cc2e36c352
|
overlap shared + routed expert computation in kimi linear (#12660)
|
2025-11-11 14:52:58 -08:00 |
|
|
b8zhong
|
49653c8896
|
use fast stream instead of torch.cuda.current_stream in llama 4 shared experts overlap (#12811)
|
2025-11-08 15:04:37 -08:00 |
|
|
b8zhongandBrayden Zhong
|
55e8e3999c
|
add back flashinfer jit cache to dev docker (#12851)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-11-07 14:51:24 -08:00 |
|
|
b8zhong
|
32ec68faf9
|
keep attention backend document up to date (#12741)
|
2025-11-05 20:41:20 -08:00 |
|
|
b8zhong
|
d31d48b341
|
update usage of trtllm_fp8_per_tensor_scale_moe (#12569)
|
2025-11-03 14:25:32 -08:00 |
|
|
b8zhong
|
a076ec1a7a
|
Revert "fix llama4 kv cache layout" (#12437)
|
2025-10-30 22:33:37 -07:00 |
|
|
b8zhongandBrayden Zhong
|
34c286b811
|
Fix the shared expert & routed expert overlap in Llama 4 (#12405)
Co-authored-by: Brayden Zhong <b8zhong@users.noreply.github.com>
|
2025-10-30 20:14:49 -07:00 |
|
|
b8zhong
|
7ed8ba05cb
|
[CI] Add Llama 3.1 8B FP4 to B200 CI (#12182)
|
2025-10-29 14:56:41 -07:00 |
|
|
b8zhong
|
bacb3825fe
|
fix: llama 4 + trtllm gen + fp8 kv cache incompatibility (#12347)
|
2025-10-29 11:31:02 -07:00 |
|
|
b8zhong
|
83a7c89c3f
|
followup fix for llama 4 trtllm flashinfer backend (#12314)
|
2025-10-28 22:17:08 -07:00 |
|
|
b8zhong
|
c143f416ce
|
fix: Llama 4 BF16 load on Blackwell (#12308)
|
2025-10-28 18:59:01 -07:00 |
|
|
b8zhong
|
77225d602a
|
Use Flashinfer TRT-LLM as Llama 4 compatible MoE backend (#11928)
|
2025-10-28 10:39:43 -07:00 |
|
|
b8zhong
|
f80371ff8c
|
Use flashinfer_trtllm moe runner backend to gain around 10% perf on b200 fp8 dpsk (#11816)
|
2025-10-23 19:12:15 -07:00 |
|
|
b8zhong
|
47e12e082e
|
Enable Llama 4 + TRTLLM MHA (#12003)
|
2025-10-23 18:22:58 -07:00 |
|
|
b8zhong
|
8ae9d4bb41
|
Revert "[ROCm] Remove vLLM rope dependency & use AITER impl" (#12028)
|
2025-10-23 12:42:59 -07:00 |
|
|
b8zhong
|
4d4feccbb2
|
[ROCm] Remove vLLM rope dependency & use AITER impl (#11322)
|
2025-10-22 19:17:34 -07:00 |
|
|
b8zhong
|
d0a64c7e2c
|
vlm: enforce pybase64 for image and str encode/decode (#10700)
|
2025-10-21 19:05:32 +08:00 |
|
|
b8zhong
|
f4f8a1b4d8
|
ci: update lmms-eval to speed up multimodal CI (#11000)
|
2025-10-19 02:51:19 +08:00 |
|
|
b8zhong
|
f9a7d9b3dc
|
support server arg override KV cache to bf16 to avoid slow cases (#11749)
|
2025-10-19 02:49:48 +08:00 |
|
|
b8zhong
|
6bc503af73
|
[Doc] Update support matrix for attn and hybrid attn (#11293)
|
2025-10-14 22:43:11 -07:00 |
|
|
b8zhong
|
a2faf8940c
|
[1/n] Enable DCA CUDA graph capture (#9537)
|
2025-10-03 11:30:00 +08:00 |
|
|
b8zhong
|
948278f173
|
fix cpp JIT compilation issue of ngram speculative decoding (#10837)
|
2025-10-02 21:05:01 +08:00 |
|
|
b8zhong
|
b2435be682
|
Cache the result of is_blackwell platform check (#10498)
|
2025-09-15 22:30:28 -07:00 |
|
|
b8zhong
|
716e682721
|
[Fix] Add undefined update_tensor_inplace function (#6307)
|
2025-08-18 11:11:00 +08:00 |
|
|
Brayden Zhong
|
bc938ea13f
|
Fix DP load for embedding (#9165)
|
2025-08-15 23:58:44 -07:00 |
|
|
Brayden Zhong
|
2871eacc05
|
Add Triton Fused MoE kernel config for E=16 on B200 (#7004)
|
2025-08-13 23:12:27 -07:00 |
|
|
Brayden Zhong
|
4a9f3eef90
|
Tiny Llama4 type error in constructor (#6752)
|
2025-08-09 01:03:59 -07:00 |
|
|
Brayden Zhong
|
4acf690206
|
[Optimization][Perf] Disable the GC during CUDA graph capture to speed up by up to 3x (#8577)
|
2025-07-31 11:31:21 -07:00 |
|
|
Brayden Zhong
|
a37e1247c1
|
[Multimodal][Perf] Use pybase64 instead of base64 (#7724)
|
2025-07-08 14:00:58 -07:00 |
|
|
Brayden ZhongandCinjon Resnick
|
ca9291181d
|
[Feature] Add Logit Bias (#6579)
Co-authored-by: Cinjon Resnick <cinjon.resnick@gmail.com>
|
2025-06-10 15:39:25 -07:00 |
|
|
Brayden Zhong
|
1aa0fbf416
|
Add note to add supported model to documentation (#6640)
|
2025-05-27 13:18:46 +08:00 |
|
|
Brayden Zhong
|
006ead9dcb
|
[FA][Test] Fix Sparse FA test (#6306)
|
2025-05-26 01:27:48 -07:00 |
|
|
Brayden Zhong
|
9a91fa0ed1
|
docs: fix a bad redirect (#6300)
|
2025-05-14 10:27:19 -07:00 |
|
|
Brayden Zhong
|
3c32895cbe
|
[Llama4] Add docs note about enable multimodal (#6235)
|
2025-05-13 10:05:47 +08:00 |
|
|
Brayden Zhong
|
12319a6787
|
[Docs] Add docs for SGLANG_ and SGL_ environment variables (#6206)
|
2025-05-13 01:45:41 +08:00 |
|
|
Brayden Zhongandvincent-4
|
43fb95c2fa
|
[Model] Support ArcticForCausalLM architecture (Snowflake/snowflake-arctic-instruct) (#5078)
Co-authored-by: vincent-4 <vincentzhongy+githubvincent4@gmail.com>
|
2025-04-25 15:24:09 +08:00 |
|
|
Brayden Zhong
|
b868526d94
|
Fix one more issue reported by torchfix (#4859)
|
2025-04-20 17:49:27 -07:00 |
|
|
Brayden Zhong
|
e84f4ba0ab
|
[Misc] Fix issues reported by torchfix (#4837)
|
2025-03-27 20:10:32 -07:00 |
|
|
Brayden Zhong
|
b149b39353
|
[CI] Remove unused imports with Ruff to pre-commit config, only to benchmarks/docs/examples folder (#3969)
|
2025-03-27 19:45:02 -07:00 |
|