Improved logic for checking overflow workers #89
Improved logic for checking overflow workers #89comtihon wants to merge 14 commits intodevinus:masterfrom
Conversation
When workers are expensive to start and transactions are quick killing all workers that are checked in when in overflow is very expensive. This change allows delaying the termination of overflow workers when there is peak load and alleviates worker churn.
When workers are expensive to start and transactions are quick killing all workers that are checked in when in overflow is very expensive. This change allows delaying the termination of overflow workers when there is peak load and alleviates worker churn.
Conflicts: README.md
Get changes
|
@fishcakez I could really use another set of eyes on this here. |
|
When reaping in batches there is no need to read/store the monotonic time in the workers queue, only the time of the poll when it would be reaped or the time of the last reap. This need not be a clock time but could be a counter that increments once per reap, in a similar way to a timer wheel works. Using a counter (instead of monotonic time) may introduce some lag in the TTL but I don't think that will be an issue here because strict timing is not required. This would remove the OTP-18 requirement and should be faster. With OTP-18 strict timing would be possible using absolute timers. This branch needs to access both ends of the worker queue so you may want to consider a different data structure. The |
|
I am not sure how I feel about adding features to |
As discussed in #83 (comment) I made improvements in zugolosian`s changes.
In zugolosian#1 all commits are discussed.
Summing up: