Context
PR #576 added a re-enqueue scan in DrainBatchesForOneNode (RecordAccumulator.cs) to handle the case where a partition leader migrates between Ready() and Drain(). When GetPartitionsForNode(nodeId) returns empty, all non-empty partition deques are re-enqueued to _readyPartitions.
Concern
The O(n) scan over _partitionDeques runs inside the send loop. While the partitions.Count == 0 condition is expected to be transient (leader migration), it can also fire for:
- Newly-created topics where metadata hasn't populated yet
- Stale metadata after broker disconnection
For producers routing to many topics/partitions, this linear scan could add latency.
Possible improvements
- Narrow the condition: distinguish genuine leader migrations from stale metadata (e.g., check if the node was recently known to own partitions)
- Track which partitions Ready() consumed per node: only re-enqueue those specific partitions instead of scanning all deques
- Cap the scan: limit re-enqueue to partitions that were recently sealed (within the last N seconds)
Current behavior
The scan is harmless correctness-wise — duplicate enqueues are documented as safe, and the scan only fires on the rare partitions.Count == 0 path. This is a performance optimization, not a bug fix.
Introduced in PR #576, commit 09979d2.
Context
PR #576 added a re-enqueue scan in
DrainBatchesForOneNode(RecordAccumulator.cs) to handle the case where a partition leader migrates betweenReady()andDrain(). WhenGetPartitionsForNode(nodeId)returns empty, all non-empty partition deques are re-enqueued to_readyPartitions.Concern
The O(n) scan over
_partitionDequesruns inside the send loop. While thepartitions.Count == 0condition is expected to be transient (leader migration), it can also fire for:For producers routing to many topics/partitions, this linear scan could add latency.
Possible improvements
Current behavior
The scan is harmless correctness-wise — duplicate enqueues are documented as safe, and the scan only fires on the rare
partitions.Count == 0path. This is a performance optimization, not a bug fix.Introduced in PR #576, commit
09979d2.