-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Open
Labels
Description
- Context is in Fix ResponseStoreCleaner bugs causing unbounded response store growth and cleanup failures #17622.
- In cases where there's a very large number of cursor responses stored on the brokers, the periodic controller job that is responsible for cleanup can get bogged down because it currently does a
GETcall on each broker to retrieve a full list of cursor responses and then issues an individualDELETEcall for each cursor response that is past expiry. - We can consider adding a batch delete API to make this controller-based broker cleanup job more efficient. Basically, we can avoid even fetching the entire list of stored cursor responses - maybe through an API like
DELETE /responseStore?expiredBefore={currentTimeMs}which pushes the expiration logic to the broker where it belongs. The controller can act purely like a cron trigger in that case.
Reactions are currently unavailable