Commit Graph
6055 Commits
Author SHA1 Message Date
095a817612 [Bugfix][HiCache] measure load-back duration with CUDA events (#26411)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: vuuihc <vuuihc@users.noreply.github.com>
Co-authored-by: Zhiqiang Xie <xiezhq@stanford.edu>
2026-07-16 00:11:04 -07:00
Xiaoyu Zhang e73f323464 [JIT] Reduce MoE fused gate CI test sweep (#31400) 2026-07-16 14:42:45 +08:00
Ryan Zzzandzhujunyu 5af65d8542 [DeepSeek-V4] Support BF16 Compress State for Online C128 (#29609)
Co-authored-by: zhujunyu <zhujunyu.666@bytedance.com>
2026-07-15 23:17:29 -07:00
guzekai01 dee91c51cf perf(deepseek_v4): enable SGLANG_OPT_FP8_WO_A_GEMM on sm90 (Hopper) (#28983) 2026-07-16 11:16:32 +08:00
Mick d9003dd452 fix: skip unsafe automatic prefill graph capture (#31204) 2026-07-16 09:27:38 +08:00
Brayden ZhongandBrayden Zhong 7647a9d260 Fuse the preprocess kernels of trtllm-gen attention (#29690)
Co-authored-by: Brayden Zhong <brayden.zhong@radixark.ai>
2026-07-15 18:21:34 -07:00
Po-Han Huang (NVIDIA) 5d004a20c5 Fix FlashInfer A2A top-k ID dtype (#29929) 2026-07-15 17:56:11 -07:00
Liangsheng Yin b0b2dfbda1 [Spec] Extract the shared draft() tail into build_eagle_verify_input (#31375) 2026-07-15 15:59:31 -07:00
7a973c03a0 [Bugfix] Stamp capture-time num_tokens_per_req in multi-layer EAGLE; close jit_kernel CI filter gaps (#31367)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
2026-07-15 15:24:32 -07:00
Jorge António 26cb0fcdda Empty _REQ_TYPES_WITH_OPAQUE_FIELDS on the msgpack IPC path (#29465 Task 4) (#30182) 2026-07-15 14:55:06 -07:00
Michael 67148447a6 [AMD] Register 3 CPU-bound / triton unit + light-integration tests for AMD 1-GPU PR CI (#31088) 2026-07-15 14:36:05 -07:00
karverma-amd ec32590025 feat(moriep): add fp4 combine dtype (SGLANG_MORI_COMBINE_DTYPE=fp4) (#30706) 2026-07-15 14:21:41 -07:00
jacky.cheng e78051a419 [AMD] [Fix] Fix --attention-backend triton work for DeepSeek MLA on MI355 (null-K + decode dispatch + RoPE) (#30355) 2026-07-15 14:19:23 -07:00
Liangsheng Yin e76cc75cfa [CI] Remove nightly registrations redundant with scheduled stage runs (#31371) 2026-07-15 14:12:07 -07:00
Mohammad Miadh Angkad 18043aec20 [CI] Fix TRTLLM MHA graph metadata test fixture (#31332) 2026-07-15 12:44:51 -07:00
YAMYandXuwei Li 2d00e20a52 [Disagg][Qwen3.5] Fix heterogeneous attn-TP scatter transfer: GDN conv sub-block slice + GQA replicated-KV head map (#30997)
Co-authored-by: Xuwei Li <lixuwei.xy@gmail.com>
2026-07-16 02:31:37 +08:00
Liangsheng Yin dc078ddd2a [Spec] Extract stateless draft prepare helpers into eagle_worker_common (#31257) 2026-07-15 10:54:31 -07:00
Артем Савкинandronnie_zheng 8ed82afcc8 [MoE Refactor] [NPU] Refactor Ascend MoE implementation to reduce code duplication and align with community design (#25663)
Co-authored-by: ronnie_zheng <zl19940307@163.com>
2026-07-15 14:59:42 +03:00
Mick 947a14d617 feat: unify multimodal feature transport (#30904) 2026-07-15 17:42:38 +08:00
weireweireandweireweire f2c875d1c8 [PD] Route PD server warmup to every DP rank (#30748)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-07-15 15:59:41 +08:00
Mohammad Miadh Angkad 5af670284e [CI] Lower GLM-5.2 NVFP4 MTP speed threshold (#31289) 2026-07-15 00:39:58 -07:00
Sam ShleiferandAlex Nails dec0836302 Fix processor config loading for object-storage model paths (#31211)
Co-authored-by: Alex Nails <alex.nails@radixark.ai>
2026-07-15 00:21:31 -07:00
skyandBrayden Zhong 980acd6eca Fix MoE TP allreduce to use NCCL symmetric memory via in-pool output allocation (#29007)
Signed-off-by: wangfakang <fakangwang@gmail.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
2026-07-15 15:06:37 +08:00
Ma Mingfei 41e0b4b369 [CPU] add fused input proj for qwen3.5 (#31171) 2026-07-15 15:06:24 +08:00
Yuan Luoandluoyuan.luo a649b5a9db [KDA] Add FlashInfer SM100 KDA decode + MTP (target_verify) backend (#30113)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-07-15 15:04:20 +08:00
fzyzcjy 1afab30577 Fix bookkeeping fields not encapsulated with real allocations in normal alloc, PD pre-alloc, DFlash and EAGLE (#29432) 2026-07-15 14:52:21 +08:00
fzyzcjy e789ca24a7 Lightweight extract allocation logic from mem_cache/common.py to more clearly show nearly parallel variants (#29431) 2026-07-15 14:49:37 +08:00
fzyzcjy 2d979f1d8c Let the presence of req.kv indicate the existence of owned kv resources (#29429) 2026-07-15 14:47:16 +08:00
fzyzcjy 27256aee5b Let cache backend do not couple with owned committed kv details and avoid kv_committed_freed/kv_overallocated_freed fields (#29428) 2026-07-15 14:43:10 +08:00
fzyzcjy d8d76c4d12 Introduce req.kv container for coupled owned kv field lifecycle (#29427) 2026-07-15 14:40:38 +08:00
fzyzcjy 21c62b9830 Rewrite pause_generation retract path as req-level release and requeue for clarity (#30675) 2026-07-15 14:32:12 +08:00
fzyzcjy 1967b9ec99 Fix missed hisparse release and stale field cleanup in pause retract (#30674) 2026-07-15 14:31:11 +08:00
fzyzcjy b6cc897fea Fix non-existent abort mode in Scheduler.pause_generation and inline retract_all (#30673) 2026-07-15 14:27:48 +08:00
fzyzcjy 52a88fb212 Avoid mutating ScheduleBatch fields in place (#30672) 2026-07-15 14:27:08 +08:00
fzyzcjy e77d95c3d5 Pass per-forward overrides to ForwardBatch.init_new as explicit arguments (#30670) 2026-07-15 14:25:59 +08:00
fzyzcjy 861d97d24d Remove dead ScheduleBatch fields and avoid inplace seq_lens bump (#30669) 2026-07-15 14:23:31 +08:00
haoyangli0109 0832d856ca [Bugfix] fix quickreduce acc error in cudagraph mode (#29508)
Signed-off-by: Haoyang Li <lihaoyang0109@gmail.com>
2026-07-14 22:16:02 -07:00
Xiaoyu ZhangandClaude Fable 5 4aadf94146 [Kernel] Relocate vendored fla and mamba kernel trees to sglang.kernels (RFC #29630, Phase 2.5, 7/7) (#30795)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 12:52:15 +08:00
ANSHUMAN TRIPATHY 23f2b77d82 Make UTs compatible for XPU (#27106) 2026-07-15 12:35:56 +08:00
Xiaoyu ZhangandClaude Fable 5 c00131ebaa [Kernel] Migrate linear-attention, MiniMax-sparse and diffusion kernels to sglang.kernels (RFC #29630, Phase 2.5, 6/7) (#30793)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 11:21:36 +08:00
Xiaoyu ZhangandClaude Fable 5 ba5be86d42 [Kernel] Migrate DSA + DSV4 attention kernels to sglang.kernels (RFC #29630, Phase 2.5, 5/7) (#30792)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 11:11:22 +08:00
Mick 43124cdd90 fix: fix image benchmark backend parity (#30867) 2026-07-15 10:11:22 +08:00
weireweireandweireweire a9cf5e68e6 [DSV4] Remove per-step seqlen D2H from speculative to make overlap scheduler work (#30365)
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
2026-07-14 18:01:22 -07:00
Liangsheng Yin 50d1edaa7f [misc] Move SchedulerRecvSkipper into scheduler_components (#31222) 2026-07-14 16:35:04 -07:00
Alison Shao 771e386332 Disable flaky DSV4-Flash FP4 BCG determinism test (nondeterminism from #30898 idle-rank dummy extend) (#31125) 2026-07-14 16:21:10 -07:00
paulzhang-tm 463a3f4248 [Mamba] Support configurable conv-window layouts (#31059) 2026-07-14 14:41:10 -07:00
0d89564d27 Support scheduler_recv_interval (recv skipper) under DP-attention (#30457)
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: hnyls2002 <lsyincs@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
2026-07-14 14:02:58 -07:00
hunhokimandHun-ho Kim a5c3e0283f [Bug fix] Account for KV replication fan-out in transfer-byte metrics (#30351)
Co-authored-by: Hun-ho Kim <hunho.kim@samsung.com>
2026-07-14 10:08:24 -07:00
Mohammad Miadh Angkad 271e5ef5c3 [CI] Fix Flash MLA SM120 test import path (#31199) 2026-07-14 09:01:23 -07:00
Mick 04af94d150 fix: avoid tilelang cuda runtime pollution (#30870) 2026-07-14 22:30:27 +08:00