100w-master#4958
Conversation
ti-chi-bot
Bot
added
release-note
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
📝 WalkthroughWalkthroughModified periodic event interval in the maintainer's event loop from 100ms to 120s, and adjusted slow-event logging for EventMessage to log only the message type instead of the entire message object. Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~5 minutes Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 0/1 reviews remaining, refill in 60 minutes.Comment |
ti-chi-bot
Bot
added
the
size/XS
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request increases the periodEventInterval from 100ms to 120s and updates event logging to record only the message type. Feedback suggests that the significant increase in the interval may negatively impact system responsiveness and replication lag. Additionally, it is recommended to use zap.Stringer for logging the message type to improve log readability.
|
|
||
| const ( | ||
| periodEventInterval = time.Millisecond * 100 | ||
| periodEventInterval = time.Second * 120 |
There was a problem hiding this comment.
Increasing periodEventInterval from 100ms to 120s is a drastic change that will significantly impact the system's responsiveness. This interval controls how frequently the maintainer calculates the global checkpoint and resends critical messages (such as bootstrap requests or barrier ACKs). A 2-minute delay in these operations will lead to excessive replication lag and extremely slow recovery from transient network issues or node failures. If this change was intended to reduce CPU overhead for large-scale changefeeds (e.g., 1 million tables), it should be made configurable or set to a more reasonable value like 1-5 seconds.
| zap.Int("eventType", event.eventType), | ||
| zap.Duration("duration", duration), | ||
| zap.Any("Message", event.message), | ||
| zap.Any("MessageType", event.message.Type), |
There was a problem hiding this comment.
Using zap.Stringer is preferred here since event.message.Type (of type IOType) implements the fmt.Stringer interface. This ensures the log contains the human-readable string representation of the message type rather than its raw integer value, which is much more helpful for debugging slow event processing. Additionally, logging only the type instead of the entire message is a good improvement for performance when dealing with large table counts.
| zap.Any("MessageType", event.message.Type), | |
| zap.Stringer("MessageType", event.message.Type), |
|
[FORMAT CHECKER NOTIFICATION] Notice: To remove the 📖 For more info, you can check the "Contribute Code" section in the development guide. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@maintainer/maintainer.go`:
- Around line 50-51: The single shared variable periodEventInterval was
increased to 120s but is used for both checkpoint calculation and EventPeriod
resend/housekeeping; split it into two clearly named intervals (e.g.,
periodCheckpointInterval for the checkpoint calculation used in the checkpoint
function referenced at Line 655, and periodEventInterval for EventPeriod
resend/housekeeping used around Lines 1178 and 1230) and update all references
so checkpoint logic uses periodCheckpointInterval (shorter, e.g.,
original/near-real-time value) while EventPeriod resend/housekeeping continues
to use the longer periodEventInterval; keep periodRedoInterval unchanged and
ensure names (periodCheckpointInterval, periodEventInterval, periodRedoInterval)
match across maintainer.go and any functions that read them.
- Line 312: In the defer slow-event logging path, guard against nil
event.message before accessing event.message.Type: update the zap logging call
that currently uses zap.Any("MessageType", event.message.Type) to first check if
event.message != nil and only include event.message.Type when non-nil (otherwise
log nil or omit the field) so the defer logger cannot nil-deref; locate this
change around the slow-event defer block and the usage of event.message and
"MessageType".
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 15a8564c-8b20-4472-b77d-509703c7cc7f
📒 Files selected for processing (1)
maintainer/maintainer.go
| periodEventInterval = time.Second * 120 | ||
| periodRedoInterval = time.Second * 1 |
There was a problem hiding this comment.
120s shared periodic tick can regress checkpoint freshness and resend recovery.
periodEventInterval now drives both checkpoint calculation (Line 655) and EventPeriod resend/housekeeping (Line 1230 + Line 1178). Moving it to 120s means both flows can lag by up to 2 minutes.
Suggested split of intervals
const (
- periodEventInterval = time.Second * 120
+ periodEventInterval = time.Second * 120 // resend/housekeeping
+ checkpointCalcInterval = time.Second // checkpoint advancement cadence
periodRedoInterval = time.Second * 1
)
@@
-func (m *Maintainer) calCheckpointTs(ctx context.Context) {
- ticker := time.NewTicker(periodEventInterval)
+func (m *Maintainer) calCheckpointTs(ctx context.Context) {
+ ticker := time.NewTicker(checkpointCalcInterval)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@maintainer/maintainer.go` around lines 50 - 51, The single shared variable
periodEventInterval was increased to 120s but is used for both checkpoint
calculation and EventPeriod resend/housekeeping; split it into two clearly named
intervals (e.g., periodCheckpointInterval for the checkpoint calculation used in
the checkpoint function referenced at Line 655, and periodEventInterval for
EventPeriod resend/housekeeping used around Lines 1178 and 1230) and update all
references so checkpoint logic uses periodCheckpointInterval (shorter, e.g.,
original/near-real-time value) while EventPeriod resend/housekeeping continues
to use the longer periodEventInterval; keep periodRedoInterval unchanged and
ensure names (periodCheckpointInterval, periodEventInterval, periodRedoInterval)
match across maintainer.go and any functions that read them.
| zap.Int("eventType", event.eventType), | ||
| zap.Duration("duration", duration), | ||
| zap.Any("Message", event.message), | ||
| zap.Any("MessageType", event.message.Type), |
There was a problem hiding this comment.
Guard event.message before logging MessageType in defer path.
event.message.Type can nil-deref in the slow-event logger. A small guard avoids panic in diagnostic code.
Nil-safe logging tweak
if event.eventType == EventMessage {
+ messageType := "nil"
+ if event.message != nil {
+ messageType = event.message.Type.String()
+ }
log.Info("maintainer is too slow",
zap.Stringer("changefeedID", m.changefeedID),
zap.Int("eventType", event.eventType),
zap.Duration("duration", duration),
- zap.Any("MessageType", event.message.Type),
+ zap.String("messageType", messageType),
)
} else {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| zap.Any("MessageType", event.message.Type), | |
| if event.eventType == EventMessage { | |
| messageType := "nil" | |
| if event.message != nil { | |
| messageType = event.message.Type.String() | |
| } | |
| log.Info("maintainer is too slow", | |
| zap.Stringer("changefeedID", m.changefeedID), | |
| zap.Int("eventType", event.eventType), | |
| zap.Duration("duration", duration), | |
| zap.String("messageType", messageType), | |
| ) | |
| } else { |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@maintainer/maintainer.go` at line 312, In the defer slow-event logging path,
guard against nil event.message before accessing event.message.Type: update the
zap logging call that currently uses zap.Any("MessageType", event.message.Type)
to first check if event.message != nil and only include event.message.Type when
non-nil (otherwise log nil or omit the field) so the defer logger cannot
nil-deref; locate this change around the slow-event defer block and the usage of
event.message and "MessageType".
1 participant
What problem does this PR solve?
Issue Number: close #xxx
What is changed and how it works?
Check List
Tests
Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
Summary by CodeRabbit