dsv4.1: remaining model and runtime integration (#38798)

Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xiaoyu Zhang <xiaoyu.zhang@radixark.ai>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: Khoa Pham <khoa.pham@radixark.ai>
Co-authored-by: Yuhao Yang <47235274+yhyang201@users.noreply.github.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
Co-authored-by: Ke Bao <ispobaoke@gmail.com>
This commit is contained in:
Liangsheng Yin
2026-09-18 02:55:30 -07:00
committed by GitHub
co-authored by BBuf Claude Opus 5 Xiaoyu Zhang Yuwei An Khoa Pham Yuhao Yang Zhichen Zeng Ke Bao
parent 1b200ffaaa
commit a6cf05817f
103 changed files with 8807 additions and 723 deletions
@@ -86,6 +86,7 @@ def _prefill_queue(sched):
def _decode_queue(sched):
q = SimpleNamespace(
scheduler=sched,
token_to_kv_pool_allocator=MagicMock(),
retracted_queue=[],
pending_reqs=[],
_check_if_req_exceed_kv_capacity=MagicMock(return_value=False),
@@ -906,7 +906,7 @@ def _buf_infos(*ptrs):
def _make_dsv4_target(*, unified, mapping=None):
pool = object.__new__(DeepSeekV4TokenToKVPool)
pool.compression_ratios = [0, 4, 128]
pool.compression_ratios = [0, 2, 1, 4, 128]
pool._unified_kv = unified
pool.page_size = 256
pool.sliding_window = 128