 Stefan Heandnanjiangwill
|
3589aa79b0
|
[RL] Fix illegal memory for _import_static_state (#7733)
Co-authored-by: nanjiangwill <willjiang2018@gmail.com>
|
2025-07-06 16:25:21 -07:00 |
|
|
Stefan He
|
00fbd8a484
|
Fix typo of flash_cache (#7513)
|
2025-06-25 02:04:41 -07:00 |
|
|
Stefan He
|
3774f07825
|
Multi-Stage Awake: Support Resume and Pause KV Cache and Weights separately (#7099)
|
2025-06-19 00:56:37 -07:00 |
|
|
Stefan He
|
1ab14c4c5c
|
[VERL Use Case] Add torch_memory_saver into deps (#6247)
|
2025-05-12 19:09:03 -07:00 |
|
|
Stefan He
|
087751a8f2
|
Remove unecessary is_fa3_supported check (#6112)
|
2025-05-08 14:45:33 -07:00 |
|
|
Stefan He
|
24c13ca950
|
Clean up fa3 test from 8 gpus (#6105)
|
2025-05-07 18:38:40 -07:00 |
|
|
Stefan HeandLifu Huang
|
6b0fae797a
|
Fix Phi3 serving which was broke by earlier change (#5991)
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
|
2025-05-03 00:28:47 -07:00 |
|
|
Stefan He
|
6fc175968c
|
Optimize a pad operation to accelerate 25us (#5945)
|
2025-05-01 10:48:55 -07:00 |
|
|
Stefan He
|
408ba02218
|
Add Llama 4 to FA3 test (#5509)
|
2025-04-26 19:49:31 -07:00 |
|
|
Stefan He
|
5db37c8626
|
[metrics] Add in queue metrics (#4444)
|
2025-04-09 17:19:27 -07:00 |
|
|
Stefan HeandQingquan Song
|
93470a1411
|
Refactor and Optimize FA3 Code (#5090)
Co-authored-by: Qingquan Song <ustcsqq@gmail.com>
|
2025-04-07 11:52:42 -07:00 |
|
|
|
ca8d02abd5
|
FA3 Spec Decoding to support top k = 1 and add cuda graph support (#5050)
Co-authored-by: Qingquan Song <ustcsqq@gmail.com>
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
|
2025-04-04 23:03:59 -07:00 |
|
|
|
26c0f13126
|
Support Page Size > 1 for FA3 (#4832)
Co-authored-by: Qingquan Song <ustcsqq@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-03-27 22:07:14 -07:00 |
|
|
Stefan HeandYubo Wang
|
1b9175cb23
|
[FA3 Attn Backend] Remove Unnecessary Device Sync for FA3 (#4745)
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
|
2025-03-27 00:45:11 -07:00 |
|
|
Stefan He
|
4c584fc632
|
Fix circular imports in gptq.py and unblock test explorer (#4736)
|
2025-03-24 18:07:08 -07:00 |
|
|
|
5d7edc8e55
|
Support FA3 as Attention backend by using --attention-backend fa3 (#4680)
Co-authored-by: qsong <qsong@linkedin.com>
Co-authored-by: qingquansong <ustcsqq@gmail.com>
|
2025-03-23 23:28:11 -07:00 |
|
|
Stefan He
|
ef3c2dd08e
|
Support Online Quantization for W8A8 (#4485)
|
2025-03-17 00:28:56 -07:00 |
|
|
Stefan HeandStefan He
|
e0917e6bd0
|
Remove vllm ops scaled fp8 quant and accelerate per token quant by 20-28% (#4215)
Co-authored-by: Stefan He <bhe@linkedin.com>
|
2025-03-12 00:08:03 -07:00 |
|
|
Stefan He
|
dceb256f1b
|
[docs] Unhide production metrics page (#4193)
|
2025-03-08 23:41:40 -08:00 |
|
|
Stefan He
|
3c3eb374b2
|
Remove non-existent AMD header include (#4166)
|
2025-03-06 23:29:30 -08:00 |
|
|
Stefan He
|
95085d65e9
|
[Refactor] Reducing code duplication across FP8 CUDA quantization kernels (#4163)
|
2025-03-06 22:58:52 -08:00 |
|
|
Stefan He
|
63ee26d162
|
Add sgl_per_token_quant_fp8 (#4089)
|
2025-03-06 20:53:05 -08:00 |
|
|
Stefan He
|
0194948fd9
|
Optimize Triton Kernel of Group GEMM in DeepGEMM Benchmark (#4014)
|
2025-03-02 23:29:55 -08:00 |
|
|
Stefan He
|
b7e274f2d9
|
Add Benchmark for DeepGEMM Group GEMM (#3993)
|
2025-03-02 17:47:21 -08:00 |
|
|
Stefan He
|
d8a98a2cad
|
[Docs] Improve DPSK docs in dark mode (#3914)
|
2025-02-27 00:13:04 -08:00 |
|