3 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Zhao Chenand赵晨阳	d5fa019c36	feat: limit peak memory usage when computing logprobs (#6318) ... Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com> Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>	2025-11-03 23:53:20 -08:00
Zhao Chenandgemini-code-assist[bot]	0b24af4d79	test: support return logprobs in bench_offline_throughput test (#12462) ... Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>	2025-11-03 21:38:48 -08:00
Zhao Chen	3586b4cef2	feat: add production metric for retracted requests due to insufficient kvcache (#7030) ... Signed-off-by: Zhao Chen <zhaochen.zju@gmail.com>	2025-07-17 11:59:05 -07:00