Commit Graph
263 Commits
Author SHA1 Message Date
Anupa SajikumarandMa Mingfei 2ff52e3ceb [Intel][XPU]Enable chunked prefill scnearios for XPU with UT (#33804)
Co-authored-by: Ma Mingfei <mingfei.ma@intel.com>
2026-09-29 13:28:25 +08:00
Xiaoyu Zhang 38ec649048 [Refactor] Restore logical kernel groups and test organization (#41243) 2026-09-26 12:30:44 +08:00
Olga Miroshnichenko 0e2aac500b [ROCm][Perf] aiter: page-level KV view for gfx950 fp8 page-64 asm prefill (#36505) 2026-09-25 19:23:46 -07:00
Liangsheng Yin 516ab77619 [Test] Route all sgl-eval benchmarks through run_sgl_eval and deprecate run_eval (#41280) 2026-09-25 15:48:38 -07:00
Liangsheng Yin f88572a57c [Test] Add in-process sgl-eval adapter and move validated GSM8K tests to it (#41216) 2026-09-25 13:59:17 -07:00
Liangsheng Yin f337f01b80 [Test] Remove dead eval modules and point GSM8K/MMLU docs to sgl-eval (#41215) 2026-09-25 12:50:31 -07:00
Yoray Zack de123f38bb [3/N] elastic-ep: Recapture decode CUDA graphs after scale-up (#33723) 2026-09-23 01:51:30 -07:00
ANSHUMAN TRIPATHY eb5e8c84be Speculative Decoding with NGRAM support for XPU (#31362) 2026-09-23 14:04:20 +08:00
Liangsheng Yin 78980a3b0b [misc] Remove deprecated endpoints, env vars and aliases past two releases (#40795) 2026-09-22 18:14:45 -07:00
Cheng Wan 91c329cc86 Take the model config out of the parallel group build, and finish retiring the parallel getters (#40707) 2026-09-22 10:55:21 -07:00
Dayananda VandClaude Opus 5 35eb7cf8d6 [Intel][XPU][KVCanary] Enable KV Canary on Intel XPU (#33520)
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 09:19:01 +08:00
Cheng Wan bccf691b22 Bringing the parallel runtime up becomes a phase, not a side effect (#40345) 2026-09-21 12:29:50 -07:00
Cheng Wan 970e946e4f Retire the per-runner parallel record (#40343) 2026-09-21 12:26:40 -07:00
Cheng Wan 2d0e94e3a3 Check the topology identities where the layout is written, and build at the published widths (#40340) 2026-09-21 12:22:59 -07:00
HuangJi 2a0cb2f04e [Diffusion][MiniMax-H3] Add SM120 Sage compute for SubBlock sparse attention (#40116) 2026-09-20 16:40:28 +08:00
Ke Bao a8a4d86be9 Remove swa and mamba radix cache (#40313) 2026-09-20 16:16:27 +08:00
Liangsheng Yin d507accadc [Test] Drop dead and strictly-subsumed CI test registrations (#40264) 2026-09-18 17:49:13 -07:00
iridiumine f86f60081d [NPU] Adapt hicache for K3 hybrid models (#39415) 2026-09-18 14:48:20 +08:00
Jensen 3ce3b4969f [NPU] Avoid repeated BF16 wo_a weight transposes in DeepSeek-V4 decode (#39919) 2026-09-18 09:10:43 +03:00
Cheng Wan 406c9c71d8 [Feature] Support BF16 and batch-invariant inference with DeepEP v2 (#38160) 2026-09-15 14:30:50 -07:00
Cheng Wan fa663e7297 config: delete the redundant full stamp in initialize_model_parallel (#39202) 2026-09-12 17:27:21 -07:00
iridiumine a984c78330 [NPU] Support DFlash speculative decoding for MiMo-V2.5-Pro (mxfp4) (#37565) 2026-09-12 11:59:23 +08:00
Miaomiao Jiang 690428b470 [XPU] Add xpu forward in Gemma3RMSNorm & Add test and benchmark for Gemma3RMSNorm (#36278) 2026-09-11 10:32:25 +08:00
ANSHUMAN TRIPATHYandmingfeima 2adb2e8485 [XPU] Adapt device agnostic API usage (#32093)
Co-authored-by: mingfeima <mingfei.ma@intel.com>
2026-09-11 10:03:12 +08:00
1b77f498a0 [NVIDIA] Support flashinfer Mega Moe (#31470)
Co-authored-by: djns99 <40156487+djns99@users.noreply.github.com>
Co-authored-by: 云挚 <ningyunxiao.nyx@antgroup.com>
Co-authored-by: Yangmin Li <yangminl@nvidia.com>
Co-authored-by: Po-Han Huang (NVIDIA) <53919306+nvpohanh@users.noreply.github.com>
2026-09-10 00:22:47 -07:00
Cheng Wan db272201a2 [Config] Retire get_global_server_args, and clear the deprecated flags that have a replacement (#38375) 2026-09-08 16:42:12 -07:00
Anupa Sajikumar 91a45ea37e [Intel][XPU][LoRA] Enable LoRA on Intel XPU (#30345) 2026-09-08 12:46:41 +08:00
Polisetty V R K Jyothendra Varma e59a576f03 fix test/manual/test_forward_split_prefill.py UT due to many refactors and design changes (#36617)
Signed-off-by: P V R K Jyothendra Varma <polisetty.v.r.k.jyothendra.varma@intel.com>
2026-09-03 01:48:55 -07:00
Alex NailsandAlison Shao 28262c20df [CI][RFC] Replace black-jupyter with ruff-format (#37210)
Co-authored-by: Alison Shao <a.shao@wustl.edu>
2026-09-02 19:46:08 -07:00
Liangsheng Yin a522c8a4b6 [misc] Extract PP dynamic chunk sizing into a DynamicChunkSizer scheduler component (#37674) 2026-09-02 18:35:43 -07:00
Yuan Luoandluoyuan.luo 5b04408784 [MoE] Add FlashInfer SM90 MXFP4 W4A8 CUTLASS MoE (#34967)
Co-authored-by: luoyuan.luo <luoyuan.luo@antgroup.com>
2026-08-31 20:04:41 -07:00
Xiaoyu Zhang 52e1c24744 [Diffusion] Fuse FLUX.2 token concatenation and NVFP4 quantization (#37141) 2026-08-31 21:33:02 +08:00
Liangsheng Yin 007ef5e23a [mem_cache] Move req_pool_idx into ReqKvInfo (#37094) 2026-08-30 14:46:21 -07:00
Liangsheng Yin 0438b16154 [mem_cache] Move kv_committed_len into ReqKvInfo (#37078) 2026-08-29 22:42:25 -07:00
Liangsheng Yin 6be767c2d2 [mem_cache] Move cache_protected_len and swa_evict_floor into ReqKvInfo (#36982) 2026-08-29 19:37:53 -07:00
Liangsheng Yin 9a489f8d2f [Test] Move gpqa and aime25 onto sgl-eval, drop unused eval paths (#36979) 2026-08-29 17:36:13 -07:00
Jonny Kongandjonnykong 4d78d59e51 [MoE] Make simulated expert routing support DP>1, and fuse into one triton kernel (#29718)
Co-authored-by: jonnykong <jonnykong@fb.com>
2026-08-28 14:22:45 -07:00
Lianmin Zheng 94183a8d2b Move server args config parser under utils (#36681) 2026-08-27 03:34:50 -07:00
Lianmin Zheng 3402265989 [Core] Refactor server argument choices (#36586) 2026-08-27 01:56:03 -07:00
ec4bdbfa4a [Feature] Beam search support (#31626)
Co-authored-by: cswuyg <cswuyg@gmail.com>
Co-authored-by: cswuyg <496090217@qq.com>
Co-authored-by: Vedant Jhaveri <vedantjh2@gmail.com>
Co-authored-by: Vedant Jhaveri <vjhaveri@linkedin.com>
2026-08-26 16:56:15 -07:00
Cheng Wan 937af8538b config: the runtime readers take the published bags (#36254) 2026-08-26 05:08:25 -07:00
Cheng Wan d7b144f64e config: publishing is the process entry's job (#36251) 2026-08-26 04:58:41 -07:00
829138a31e [HiCache] Fix PP inconsistency with HiCache L3 (#22607) (#27010)
Co-authored-by: ybyang <ybyang7@iflytek.com>
Co-authored-by: hzh0425 <hzh0425@apache.org>
Co-authored-by: shangmingc <csmthu@gmail.com>
Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>
2026-08-25 20:49:57 +08:00
Baizhou Zhang 833be86c15 [CP V1 Deprecation 1/5] Migrate tests to strategy-based prefill CP (#36222) 2026-08-24 20:15:34 -07:00
Xun Sunandsiyu a90d770c40 [Weight Cache] Support static DP/EP layouts (#33684)
Co-authored-by: siyu <liusy58@linux.alibaba.com>
2026-08-23 19:57:52 -07:00
Hồ Sỹ Thế eec794bce0 [AMD][Spec] Fix aiter GQA packing + split-KV routing in NEXTN spec attention (verify & draft_extend) (#30105) 2026-08-22 14:22:43 -07:00
Yanbin Jiang 7d893255c3 [Spec][LoRA] Support multi-adapter LoRA with EAGLE/NEXTN/DFLASH/DSPARK speculative decoding (#34337) 2026-08-21 14:21:53 -07:00
vikram singh shekhawatandClaude Sonnet 4.5 f7a404e9c3 Fix rope config compatibility and VL/transformers-fallback weight loading (#31575)
Co-authored-by: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
2026-08-17 14:53:59 +08:00
Brayden ZhongandBrayden Zhong 5e65dd01a7 Remove the torchao integration (--torchao-config) (#34304)
Co-authored-by: Brayden Zhong <brayden@radixark.ai>
2026-08-14 21:49:11 +08:00
jthomson04 903439044a perf(kv-events): coalesce cache events (#31479) 2026-08-13 13:36:53 -07:00