31 Commits
Author SHA1 Message Date
AMD-yanfeiwang 678913a0f0 [Feat] sgl-model-gateway: Add --decode-dispatch-limit to pick PD decode workers near KV arrival (#44689) 2026-10-20 02:16:21 +08:00
AMD-yanfeiwang f86a04a331 [Feat] sgl-router: Add --decode-dispatch-limit to pick decode DP ranks near KV arrival (#44402) 2026-10-19 15:11:40 +08:00
AMD-yanfeiwang a3abea3560 [Fix] sgl-router: Resend on a closed pooled connection; add --disable-circuit-breaker (#44075) 2026-10-18 18:23:18 +08:00
2adc76ad61 [Perf] [AMD] dsv4: reuse the q_lora fp8 quant for the C4 indexer wq_b (4/8) (#41903)
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:55:25 -07:00
f05da43d44 [Feat] [AMD] disagg: Support DSV4 decode radix cache on ROCm unified KV (bf16/fp8, DSpark) (#30929)
Co-authored-by: HAI <hixiao@gmail.com>
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-10-16 02:39:27 -07:00
a57707377f [Fix] sgl-router: Send disagg_prefill_dp_rank to decode under --dp-aware (#44072)
Co-authored-by: Shangming Cai <csmthu@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-15 22:42:33 +08:00
AMD-yanfeiwang 08f15e76bb [Feat] sgl-router: Add --dp-rank-policy round_robin for --dp-aware (#43511) 2026-10-15 12:03:01 +08:00
AMD-yanfeiwang 17c3b259f5 [router] Fix power_of_two with --dp-aware (#42027) 2026-10-13 14:15:36 +08:00
AMD-yanfeiwangandThomas Wang 9ddbba50dd [AMD][DSV4] feat: enable PCP with fp8 unified_kv on gfx950 (#39923)
Co-authored-by: Thomas Wang <thomawan@amd.com>
2026-10-06 22:09:18 -07:00
AMD-yanfeiwang 3dc7f4b816 fix(cuda-graph): remove unnecessary CP batch alignment (#40543) 2026-10-03 12:14:28 -07:00
AMD-yanfeiwang e83c95b650 [AMD] Fix AITER weight slicing for CP decode TP (#39950) 2026-10-03 12:09:08 -07:00
AMD-yanfeiwang f327f92424 [AMD] Port CP V2 to the DeepSeek-V4 HIP backend (#34200) 2026-10-03 12:08:17 -07:00
AMD-yanfeiwang 0a59830114 [AMD][DSV4] fp8 unified_kv decode: wave-aware split count past 40 tokens (#40878)
Signed-off-by: Yanfei Wang <yanfwang@crsuse2-m2m-v2-017.us-east2-a.compute.internal>
2026-09-24 03:41:23 -07:00
AMD-yanfeiwangandDuyi-Wang a813224e78 [ROCm][DSV4] Enable breakable CUDA graph prefill (#37810)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-09-16 08:52:11 -07:00
AMD-yanfeiwang 0163f8ff74 [AMD] Fix registered HiCache host pointer aliases (#35233) 2026-09-14 15:49:05 -07:00
AMD-yanfeiwang 69aa46b2fa [ROCm] Raise HiCache JIT block quota for mapped-host throughput (#39036) 2026-09-11 00:12:24 -07:00
AMD-yanfeiwang 2e8c03e2c7 Fix inflated row pitch when a CP round-robin shard has a single row (#34142) 2026-09-06 15:40:14 -07:00
AMD-yanfeiwangandkk 7f89cc5286 [AMD] Skip unused TOPK v2 plan kernel on ROCm (#37580)
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
2026-09-04 00:00:38 -07:00
AMD-yanfeiwangandkk 7825e5ffca [AMD] Fix DSV4 unified attention sink TP slice (#35092)
Co-authored-by: kk <43161300+kkHuang-amd@users.noreply.github.com>
2026-09-03 22:59:53 -07:00
AMD-yanfeiwang 2ded8a6aea [AMD] Enable moe_a2a_backend=mori for DeepSeek-V4 prefill context parallelism (#35611) 2026-08-27 02:31:41 -07:00
AMD-yanfeiwang cbfe54fba8 [AMD] Use fast exponentials in C4 and C128 ROCm kernels (#34296) 2026-08-26 19:49:01 -07:00
AMD-yanfeiwang 24bce93c93 [AMD][MORI] Deduplicate CP-replicated state transfers (#36025) 2026-08-24 14:43:52 -07:00
AMD-yanfeiwangandZhangheng a34f81251f fix(hicache/umbp): support DeepSeek-V4 hybrid HostPoolGroup (multi-po… (#30762)
Co-authored-by: Zhangheng <hzh0425@apache.org>
2026-08-13 15:40:07 +08:00
AMD-yanfeiwang ca0f8a0f4c perf(hisparse): fuse the DSv4 value and scale swap-in copy on ROCm (#33484) 2026-08-10 14:30:15 -07:00
AMD-yanfeiwang 1a8e4876b6 perf(hisparse): 128-bit non-temporal swap-in copy on ROCm (#33085) 2026-08-10 13:27:15 -07:00
AMD-yanfeiwang a3194d3585 [AMD] Remove ROCm page_first+kernel -> layer_first HiCache fallback (follow-up to #28534) (#30622) 2026-07-14 22:48:16 -07:00
AMD-yanfeiwangandDuyi-Wang d74619b373 [AMD] Enable JIT staged HiCache write-back and fix CPU-index crash (#28534)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-07-09 01:22:37 -07:00
AMD-yanfeiwang 5141d8ae21 [AMD]fix: use CUDA event for targeted draft-to-verify sync in EAGLE overlap (#21940) 2026-04-26 21:58:34 -07:00
AMD-yanfeiwang 966ae87d02 [AMD] avoid correction_bias_dtype dtype convert (#20692) 2026-03-17 02:55:05 -07:00
AMD-yanfeiwangandDuyi-Wang f0153ad225 [AMD][Feature] support fp4 dispatch and fp8 combine in moriep (#19757)
Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
2026-03-09 12:52:05 -07:00
AMD-yanfeiwangandroot f69ca93d49 [AMD] remove redundancy H2D op in aiter attention backend (#19416)
Co-authored-by: root <root@mia1-p01-g28.mia.tensorwave.lan>
2026-02-27 02:15:55 -08:00