|
d918ab7985
|
Support NVFP4 quantized dense models on AMD CDNA2/CDNA3 GPUs (#7302)
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Sai Enduri <saimanas.enduri@amd.com>
|
2025-07-18 19:59:39 -07:00 |
|
|
Mick
|
3964b352c3
|
chore: tune mem fraction static for vlm (#6881)
|
2025-07-18 17:19:27 -07:00 |
|
|
Zhiqiang Xie
|
9d33fcfb8e
|
Hicache Storage Layer Prototype (#7704)
|
2025-07-18 15:20:19 +08:00 |
|
|
Yingchun LaiandStefan He
|
795668dc73
|
feat: add tp_rank, pp_rank and dp_rank labels for scheduler metrics (#7597)
Co-authored-by: Stefan He <hebiaobuaa@gmail.com>
|
2025-07-16 17:55:59 -07:00 |
|
|
ykcombat
|
d4d0c7c367
|
[Feature]TP Group Switching for PD-Multiplexing (#7653)
|
2025-07-15 02:35:46 +08:00 |
|
|
Lifu Huang
|
e2ed9d049a
|
Refactor dynamic LoRA update to fix incorrect handling of variant weight shapes (#7844)
|
2025-07-13 18:36:01 -07:00 |
|
|
Hanming LuandYing Sheng
|
9379da77de
|
SWA Prefix Cache (#7367)
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
|
2025-07-13 12:31:07 -07:00 |
|
|
Ying Sheng
|
42fc44100a
|
[minor] Add server_args check for Llama4 with hybrid (#7988)
|
2025-07-12 20:13:40 -07:00 |
|
|
ronnie_zhengandliupeng
|
86044712c6
|
[feature] kv transfer support of ascend npu (#7795)
Co-authored-by: liupeng <liupeng374@huawei.com>
|
2025-07-11 00:07:51 -07:00 |
|
|
Atream
|
615553079d
|
Support Kimi K2 (#7940)
|
2025-07-11 00:02:21 -07:00 |
|
|
Xinyuan Tong
|
e5888eddda
|
Fixes typo in assertion message (#7895)
Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>
|
2025-07-09 01:47:14 -07:00 |
|
|
SijiaYang
|
cb9d91ea8a
|
feat: support DeepSeek-R1-W4AFP8 model with ep-moe mode (#7762)
Signed-off-by: yangsijia.614 <yangsijia.614@bytedance.com>
|
2025-07-07 14:47:21 -07:00 |
|
|
Zhiqiang Xie
|
2fc824b84c
|
Kernels for efficient KV cache IO (#7313)
|
2025-07-06 22:53:36 -07:00 |
|
|
Yuan Luoandluoyuan.luo
|
253454de9b
|
Integrate triton moe kernel (#7689)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
|
2025-07-06 20:05:49 -07:00 |
|
|
Cheng Wan
|
8fc910db03
|
DP Attention with Auto DeepEP Dispatch (#7222)
|
2025-07-05 01:54:24 -07:00 |
|
|
Zilin Zhu
|
16a6b1d83a
|
[RL] Add --nccl-port to prevent port conflict (#7418)
|
2025-07-04 18:48:57 -07:00 |
|
|
Lianmin Zheng
|
14229ccf8f
|
Move mem_fraction_static adjustment for multimodal models to server_args.py & Fix session control & Other cleanups (#7748)
|
2025-07-04 16:33:33 -07:00 |
|
|
|
1e0e549766
|
Ascend attention backend(PA&MLA) (#7722)
Co-authored-by: Maksim <makcum888e@mail.ru>
Co-authored-by: VDV1985 <vladdv85@mail.ru>
|
2025-07-03 09:23:19 -07:00 |
|
|
Xiaoyu Zhang
|
8e64140e35
|
[b200] support trt-llm allreduce fuse rms_norm_add kernel (#7621)
|
2025-07-02 19:36:20 -07:00 |
|
|
Zilin Zhu
|
09e699bba4
|
[RL] add --skip-warmup (#7416)
|
2025-07-02 18:50:43 -07:00 |
|
|
Lifu Huang
|
1a08358aed
|
Improve error handling for requests with unloaded LoRA path(s) (#7642)
|
2025-07-01 20:05:34 -07:00 |
|
|
Lianmin ZhengandKan Wu
|
22352d47a9
|
Improve streaming, log_level, memory report, weight loading, and benchmark script (#7632)
Co-authored-by: Kan Wu <wukanustc@gmail.com>
|
2025-06-29 23:16:19 -07:00 |
|
|
Lianmin Zheng
|
78700893ee
|
[EAGLE] remove a wrong adjustment for page_size > 1 & topk > 1 in server_args.py (#7643)
|
2025-06-29 19:25:28 -07:00 |
|
|
|
eb6c2c1663
|
Hybrid kv cache for LLaMA4 (#6563)
Co-authored-by: Cheng Wan <54331508+ch-wan@users.noreply.github.com>
Co-authored-by: tarinkk <rt572@physics.rutger.edu>
Co-authored-by: tarinkk <rt572@rutgers.physics.edu>
Co-authored-by: Hanming Lu <69857889+hanming-lu@users.noreply.github.com>
|
2025-06-27 18:58:55 -07:00 |
|
|
zixuanzhang226
|
f3cbd24541
|
feat: send kvmetrics from sglang scheduler (#6721)
|
2025-06-25 01:57:49 -07:00 |
|
|
Chang Su
|
3562256bb2
|
fix: Add --model as an alias for --model-path in server_args (#7505)
|
2025-06-24 12:08:08 -07:00 |
|
|
xianzhiT
|
9f1787fa60
|
Support multi-thread model weight loading (#7277)
|
2025-06-24 10:39:10 -07:00 |
|
|
Yuhong Guo
|
e5afb88b1c
|
Support weight loading without mmap (#7469)
|
2025-06-23 15:13:59 -07:00 |
|
|
Lianmin Zheng
|
55e03b10c4
|
Fix a bug in BatchTokenIDOut & Misc style and dependency updates (#7457)
|
2025-06-23 06:20:39 -07:00 |
|
|
|
5962e70d8d
|
FlashInfer NVFP4 MoE with EP & 2-stream shared expert (#7327)
Co-authored-by: JieXin Liang <Alcanderian@users.noreply.github.com>
Co-authored-by: alcanderian <alcanderian@gmail.com>
|
2025-06-22 13:38:47 -07:00 |
|
|
Cheng Wan
|
e879d8b7a8
|
[Feature] Comprehensive Hybrid Parallelism Support (#6389)
|
2025-06-20 14:43:11 -07:00 |
|
|
KavioYuandkavioyu
|
873ae12cee
|
support custom weight loader for model runner (#7122)
Co-authored-by: kavioyu <kavioyu@tencent.com>
|
2025-06-16 16:28:15 -07:00 |
|
|
Byron Hsu
|
7d316991b2
|
[PD] Update prefill.py (#7190)
|
2025-06-14 15:59:54 -07:00 |
|
|
Povilas Kanapickas
|
bd7cfbd2f8
|
[Fix] Reduce busy polling when scheduler is idle (#6026)
|
2025-06-12 14:58:22 -07:00 |
|
|
Lianmin Zheng
|
dbdf76ca98
|
Clean up docs for server args and sampling parameters (generated by grok) (#7076)
|
2025-06-10 19:55:42 -07:00 |
|
|
Lianmin Zheng
|
6b12d6a8d5
|
Simplify the heuristics for setting --mem-fraction-static (#7054)
|
2025-06-10 19:01:39 -07:00 |
|
|
kyle-pena-kuzco
|
b56de8f943
|
Open AI API hidden states (#6716)
|
2025-06-10 14:37:29 -07:00 |
|
|
Lianmin Zheng
|
6406408a70
|
Clean up server_args.py (#7037)
|
2025-06-10 05:34:29 -07:00 |
|
|
fzyzcjy
|
f6ebba537a
|
Support both approximate and exact expert distribution collection (#6964)
|
2025-06-09 20:56:17 -07:00 |
|
|
Lianmin Zheng
|
dc0705a504
|
Simplify prepare_extend_after_decode (#6987)
|
2025-06-09 16:39:21 -07:00 |
|
|
Swipe4057
|
e1ce44cdb1
|
Disabling mixed chunked prefill when eagle is enabled (#6874)
|
2025-06-07 03:06:58 -07:00 |
|
|
Lianmin Zheng
|
60fdad7cf3
|
Sync the changes on cuda graph runners (#6932)
|
2025-06-06 18:23:52 -07:00 |
|
|
fzyzcjy
|
0de5e7d40f
|
Support layerwise rebalancing experts (#6851)
|
2025-06-05 00:05:52 -07:00 |
|
|
zyksir
|
8e3797be1c
|
support 1 shot allreduce in 1-node and 2-node using mscclpp (#6277)
|
2025-06-04 22:11:24 -07:00 |
|
|
Cheng Wan
|
81964328b7
|
Set num_fused_shared_experts as num_shared_experts when shared_experts fusion is not disabled (#6736)
|
2025-06-04 15:53:22 -07:00 |
|
|
Marc Sun
|
37f1547587
|
[FEAT] Add transformers backend support (#5929)
|
2025-06-03 21:05:29 -07:00 |
|
|
Cheng Wan
|
8a5480528d
|
[Refactor] Rename n_share_experts_fusion as num_fused_shared_experts (#6735)
|
2025-06-03 17:48:24 -07:00 |
|
|
Lianmin Zheng
|
2d72fc47cf
|
Improve profiler and integrate profiler in bench_one_batch_server (#6787)
|
2025-05-31 15:53:55 -07:00 |
|
|
|
888cb175a6
|
Add intel_amx backend for Radix Attention for CPU (#6408)
Co-authored-by: Chunyuan WU <chunyuan.wu@intel.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
|
2025-05-30 21:37:42 -07:00 |
|
|
Baizhou Zhang
|
73def253b5
|
Fix mem_fraction_static for AMD CI (#6748)
|
2025-05-29 12:37:30 -07:00 |
|