G9 P0 fix: /query_cache_hit 绕过 _tokenize_one_request 的 ReqState 依赖

_tokenize_one_request._create_tokenized_object 会读 rid_to_state[obj.rid]
(正常生成路径在 tokenize 前注册 ReqState),纯查询无 rid → KeyError: None。
改为:dsv41 chat encoding 直接渲染的 prompt_ids 直接用;否则对渲染后
text 调 _tokenize_texts(与真实路径同一分词入口)。tok8 环境实测 200。

(cherry picked from commit 71c6d8bc55)
This commit is contained in:
2026-10-19 10:46:23 +08:00
parent 319ee3c056
commit 77e7555ae3
@@ -1178,12 +1178,26 @@ class OpenAIServingChat(OpenAIServingBase):
content={"error": "multimodal requests are not supported by /query_cache_hit"},
)
try:
tokenized = await self.tokenizer_manager._tokenize_one_request(internal_req)
# 只取 input_ids,不走 _tokenize_one_request——它要求 rid_to_state
# 里已有 ReqState(正常生成路径在 tokenize 前注册),纯查询没有。
# dsv41 系 chat encoding 直接渲染 prompt_ids;否则对渲染后的
# text 走 tokenizer(_tokenize_texts 与真实路径同一入口)。
if internal_req.input_ids is not None:
input_ids = list(internal_req.input_ids)
elif internal_req.text:
input_ids, _ = await self.tokenizer_manager._tokenize_texts(
internal_req.text, False
)
else:
return ORJSONResponse(
status_code=400,
content={"error": "request produced neither input_ids nor text"},
)
from sglang.srt.managers.io_struct import QueryCacheHitReqInput
result = await self.tokenizer_manager.query_cache_hit(
QueryCacheHitReqInput(
input_ids=list(tokenized.input_ids),
input_ids=list(input_ids),
extra_key=getattr(internal_req, "extra_key", None),
cache_salt=getattr(internal_req, "cache_salt", None),
)