3 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
harrisonlimh	c726d44cc7	Recapture cuda graph after model weight update to resolve IMA error (#11780)	2025-10-20 10:50:03 +08:00
harrisonlimh	14fdd52740	feat: add priority based scheduling with priority based request acceptance and preemption (#8746)	2025-09-16 17:10:10 -07:00
harrisonlimh	747dd45077	feat: throttle requests at scheduler based on --max_queued_requests (#7565)	2025-07-28 22:32:33 +08:00