 Stefan HeandBiao He
|
9578ba1b57
|
[Utils] Refactor device cache emptying (#24861)
Co-authored-by: Biao He <biao@Biaos-MacBook-Air.local>
|
2026-05-09 21:28:00 -07:00 |
|
|
Stefan He
|
43779f27b7
|
Disable PCG for draft worker (#16354)
|
2026-01-17 09:56:56 +08:00 |
|
|
Stefan He
|
9416464682
|
Fix Qwen Next GDN w/ Radix Cache (#16053)
|
2025-12-29 21:06:06 -08:00 |
|
|
Stefan He
|
f1bbd26ff7
|
Clean up GDN Init (#14855)
|
2025-12-13 02:56:54 -08:00 |
|
|
Stefan He
|
8fe8b63576
|
Revert "Try to remove wrong logic about max total token in spec decoding" (#14259)
|
2025-12-01 18:18:03 -08:00 |
|
|
Stefan He
|
9f340ab1fb
|
[Piecewise] support disable decode cuda graph when enable piecewise cuda graph (#13965)
|
2025-11-26 18:35:59 -08:00 |
|
|
Stefan He
|
70c6f95107
|
Add CODEOWNERS entry for batch_invariant_ops (#14026)
|
2025-11-26 18:35:37 -08:00 |
|
|
Stefan He
|
b6312e62ea
|
Update CODEOWNERS for layer and executor files (#14020)
|
2025-11-26 12:59:04 -08:00 |
|
|
|
d754ce973e
|
[Piecewise Cuda Graph] rename, refactor and add more logging (#13675)
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
Co-authored-by: Ke Bao <ISPObaoke@163.com>
Co-authored-by: Oasis-Git <ayw.sirius19@gmail.com>
|
2025-11-21 13:28:38 +08:00 |
|
|
|
b5344b31b8
|
[Piecewise CUDA Graph] Fix recompile issue for Mixtral and Grok2 (#13667)
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
Co-authored-by: Ke Bao <ISPObaoke@163.com>
Co-authored-by: Oasis-Git <ayw.sirius19@gmail.com>
|
2025-11-20 14:20:11 -08:00 |
|
|
Stefan He
|
0997c78d2c
|
Support FP8 Per Token Quant Piecewise (#13272)
|
2025-11-14 11:40:12 -08:00 |
|
|
Stefan He
|
df08f346ce
|
Fix Duplicate Classmethod in spec_info.py (#12354)
|
2025-10-29 13:57:10 -07:00 |
|
|
|
4fff1ec1d9
|
Deterministic Mode: Add 1-stage triton kernel for prefill (#11147)
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
Co-authored-by: Binyao Jiang <bijiang@linkedin.com>
|
2025-10-20 01:47:36 +08:00 |
|
|
Stefan He
|
6b143d62a2
|
Clean up some Qwen3-Next and deterministic code (#11585)
|
2025-10-15 15:19:37 +08:00 |
|
|
Stefan He
|
eae9a9fb9d
|
Fix batch invariant ops (#11368)
|
2025-10-10 20:49:08 -07:00 |
|
|
Stefan He
|
86527a4799
|
[deterministic inference] Move batch invariant pkg to sglang (#10695)
|
2025-09-21 19:35:14 -07:00 |
|
|
Stefan He
|
cba0d8c309
|
[Feature] Support deterministic inference with FA3 backend (#10651)
|
2025-09-20 17:50:21 -07:00 |
|
|
Stefan HeandBinyao Jiang
|
6c18ab46a2
|
[Qwen3-Next] switch to triton and cache conv states to accelerate MTP from 300 tok/s to 341 tok/s (#10335)
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
|
2025-09-11 11:59:48 -07:00 |
|
|
Stefan He
|
a530b3ffdc
|
[RL] fix register the same ops multiple times (#9564)
|
2025-08-26 16:24:44 -07:00 |
|
|
Stefan He
|
cbc0e4d779
|
Fix lint for router (#9636)
|
2025-08-26 00:38:53 -07:00 |
|
|
Stefan He
|
cded039b57
|
[FA3] Init Spec Page Table only when Spec is enabled to save ~40MB (#9455)
|
2025-08-21 15:11:38 -07:00 |
|
|
Stefan He
|
e47800e176
|
Quick Fix GLM (#9264)
|
2025-08-16 23:43:41 -07:00 |
|
|
Stefan HeandXiaoyu Zhang
|
930fe467bd
|
Support Triton FP8 Gemm can handle hidden_dim not divisible by 16 (#9093)
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
|
2025-08-12 21:21:55 -07:00 |
|
|
Stefan He
|
8ecf6b9d24
|
Support Flatten Tensor Update Weights to speed up MOE Update Weights by 20% (#8079)
|
2025-08-10 16:08:59 -07:00 |
|
|
Stefan He
|
6345069f6c
|
[RL] Add test for /abort_request (#7626)
|
2025-08-10 09:14:19 -07:00 |
|
|
Stefan He
|
d3be97104b
|
correct the tp_plan logic (#8850)
|
2025-08-07 16:53:34 -07:00 |
|
|
Stefan He
|
aaf0ad8cdf
|
remove vllm fp8quant from fp8.py (#8937)
|
2025-08-07 15:50:52 -07:00 |
|
|
Stefan He
|
cbbb738371
|
[2/3] Optimize Slime Update Weights: Avoid GPU-to-CPU Device Sync when update expert weights (#8753)
|
2025-08-05 22:09:52 -07:00 |
|
|
Stefan He
|
89588179cf
|
[1/3] Optimize Slime Update Weights: Remove QWen3MOE Load Weight Overhead (#8751)
|
2025-08-05 22:07:54 -07:00 |
|
|
Stefan He
|
4ca43b061c
|
Add tensor.detach() back to update weight util (#8691)
|
2025-08-02 00:41:05 -07:00 |
|
|
Stefan He
|
db7343c992
|
fix per token cuda kernel hidden dim cannot divide by 16 (#8543)
|
2025-08-01 09:27:18 -07:00 |
|
|
Stefan He
|
c0fd77e839
|
bring back kimi vl ci (#8537)
|
2025-07-29 13:14:18 -07:00 |
|
|
Stefan HeandMinglei Zhu
|
74e7e45710
|
Fix DEEPEP BF16 compatibility for Deepseek Style model like GLM 4.5 (#8469)
Co-authored-by: Minglei Zhu <mingleizhu1122@gmail.com>
|
2025-07-28 14:36:08 -07:00 |
|
|
|
4ad9737045
|
chore: bump transformer to 4.54.0 (#8416)
Co-authored-by: Binyao Jiang <byjiang1996@gmail.com>
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
|
2025-07-27 21:27:25 -07:00 |
|
|
|
ce32bc2ba9
|
Extract update_weights from RL Engine to SGLang to keep simplicity and fix torch reduce (#8267)
Co-authored-by: CuiBo 82354186+SuperCB@users.noreply.github.com
Co-authored-by: GeLee 865038696@qq.com
Co-authored-by: 杨睿 yangruipis@163.com
|
2025-07-26 02:00:59 -07:00 |
|
|
Stefan Heandnanjiangwill
|
3589aa79b0
|
[RL] Fix illegal memory for _import_static_state (#7733)
Co-authored-by: nanjiangwill <willjiang2018@gmail.com>
|
2025-07-06 16:25:21 -07:00 |
|
|
Stefan He
|
00fbd8a484
|
Fix typo of flash_cache (#7513)
|
2025-06-25 02:04:41 -07:00 |
|
|
Stefan He
|
3774f07825
|
Multi-Stage Awake: Support Resume and Pause KV Cache and Weights separately (#7099)
|
2025-06-19 00:56:37 -07:00 |
|
|
Stefan He
|
1ab14c4c5c
|
[VERL Use Case] Add torch_memory_saver into deps (#6247)
|
2025-05-12 19:09:03 -07:00 |
|
|
Stefan He
|
087751a8f2
|
Remove unecessary is_fa3_supported check (#6112)
|
2025-05-08 14:45:33 -07:00 |
|
|
Stefan He
|
24c13ca950
|
Clean up fa3 test from 8 gpus (#6105)
|
2025-05-07 18:38:40 -07:00 |
|
|
Stefan HeandLifu Huang
|
6b0fae797a
|
Fix Phi3 serving which was broke by earlier change (#5991)
Co-authored-by: Lifu Huang <lifu.hlf@gmail.com>
|
2025-05-03 00:28:47 -07:00 |
|
|
Stefan He
|
6fc175968c
|
Optimize a pad operation to accelerate 25us (#5945)
|
2025-05-01 10:48:55 -07:00 |
|
|
Stefan He
|
408ba02218
|
Add Llama 4 to FA3 test (#5509)
|
2025-04-26 19:49:31 -07:00 |
|
|
Stefan He
|
5db37c8626
|
[metrics] Add in queue metrics (#4444)
|
2025-04-09 17:19:27 -07:00 |
|
|
Stefan HeandQingquan Song
|
93470a1411
|
Refactor and Optimize FA3 Code (#5090)
Co-authored-by: Qingquan Song <ustcsqq@gmail.com>
|
2025-04-07 11:52:42 -07:00 |
|
|
|
ca8d02abd5
|
FA3 Spec Decoding to support top k = 1 and add cuda graph support (#5050)
Co-authored-by: Qingquan Song <ustcsqq@gmail.com>
Co-authored-by: Chunan Zeng <zcnrex@gmail.com>
|
2025-04-04 23:03:59 -07:00 |
|
|
|
26c0f13126
|
Support Page Size > 1 for FA3 (#4832)
Co-authored-by: Qingquan Song <ustcsqq@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
|
2025-03-27 22:07:14 -07:00 |
|
|
Stefan HeandYubo Wang
|
1b9175cb23
|
[FA3 Attn Backend] Remove Unnecessary Device Sync for FA3 (#4745)
Co-authored-by: Yubo Wang <yubowang2019@gmail.com>
|
2025-03-27 00:45:11 -07:00 |
|
|
Stefan He
|
4c584fc632
|
Fix circular imports in gptq.py and unblock test explorer (#4736)
|
2025-03-24 18:07:08 -07:00 |
|
|
|
5d7edc8e55
|
Support FA3 as Attention backend by using --attention-backend fa3 (#4680)
Co-authored-by: qsong <qsong@linkedin.com>
Co-authored-by: qingquansong <ustcsqq@gmail.com>
|
2025-03-23 23:28:11 -07:00 |
|
|
Stefan He
|
ef3c2dd08e
|
Support Online Quantization for W8A8 (#4485)
|
2025-03-17 00:28:56 -07:00 |
|
|
Stefan HeandStefan He
|
e0917e6bd0
|
Remove vllm ops scaled fp8 quant and accelerate per token quant by 20-28% (#4215)
Co-authored-by: Stefan He <bhe@linkedin.com>
|
2025-03-12 00:08:03 -07:00 |
|
|
Stefan He
|
dceb256f1b
|
[docs] Unhide production metrics page (#4193)
|
2025-03-08 23:41:40 -08:00 |
|
|
Stefan He
|
3c3eb374b2
|
Remove non-existent AMD header include (#4166)
|
2025-03-06 23:29:30 -08:00 |
|
|
Stefan He
|
95085d65e9
|
[Refactor] Reducing code duplication across FP8 CUDA quantization kernels (#4163)
|
2025-03-06 22:58:52 -08:00 |
|
|
Stefan He
|
63ee26d162
|
Add sgl_per_token_quant_fp8 (#4089)
|
2025-03-06 20:53:05 -08:00 |
|
|
Stefan He
|
0194948fd9
|
Optimize Triton Kernel of Group GEMM in DeepGEMM Benchmark (#4014)
|
2025-03-02 23:29:55 -08:00 |
|
|
Stefan He
|
b7e274f2d9
|
Add Benchmark for DeepGEMM Group GEMM (#3993)
|
2025-03-02 17:47:21 -08:00 |
|
|
Stefan He
|
d8a98a2cad
|
[Docs] Improve DPSK docs in dark mode (#3914)
|
2025-02-27 00:13:04 -08:00 |
|