[Bug]: The batchSyncEnabled flag is not reverted back to false when batch processing ends #6445
Description
Version
3.6.2
What Kubernetes platforms are you running on?
GKE Google Cloud
What happened?
One of the goals in PR #4371 was to avoid unnecessary nginx reloads in batch processing when the tasks in the queue are of kind endpointslice and are not referenced in any of the watched resources.
This is done via the batchSyncEnabled flag in the controller.sync(task) function. The flag controls whether an nginx reload will be done at the end of a batch processing event.
The flag starts as false and is set to true in two scenarios (synced task is not an endpointslice, or synced task is an endpointslice but it's not referenced in other watched resources). Unfortunately, the flag is never reverted back to false once it's set to true. This means that any batch process from that point on will trigger an nginx reload.
This results in undesired nginx reloads in environments with the following conditions:
- There has been an event that set
batchSyncEnabledtotrue. - There is a high number of non-interesting (i.e. not referenced)
endpointsliceresources. - These resources change frequently, up to the point where is not unusual to enter into batch processing mode.
In these environments, one can observe one or more of these behaviors:
- Eventual reloads not associated with any relevant change.
- Reloads in loop every N seconds.
- Reloads in loop for a period of time after a NIC pod restarts.
Steps to reproduce
- Create a k8s stack with a high number of
endpointsliceobjects that also change frequently (*) - Monitor the NIC pod and verify that the pace of changes is enough for the system to enter into batch processing mode from time to time.
- Change existing config, or create new one (try to do a few changes at once, as it will increase the likelihood of observing the issue).
(*) Operators that implement endpointslicemirroring-controller.k8s.io, like https://github.com/zalando/postgres-operator, will help with that.
Expected behaviour
No response
Kubectl Describe output
No response
Log output
No response
Contributing Guidelines
- I confirm that I have read the Report a Bug section of the Contributing Guidelines