Skip to content

set default incremental checkpoint interval to 5m#25432

Open
jiangxinmeng1 wants to merge 5 commits into
matrixorigin:4.1-devfrom
jiangxinmeng1:jxm/ickp-interval-5min-4.1-dev
Open

set default incremental checkpoint interval to 5m#25432
jiangxinmeng1 wants to merge 5 commits into
matrixorigin:4.1-devfrom
jiangxinmeng1:jxm/ickp-interval-5min-4.1-dev

Conversation

@jiangxinmeng1

Copy link
Copy Markdown
Contributor

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

issue #24900

What this PR does / why we need it:

set default incremental checkpoint interval to 5m

@jiangxinmeng1 jiangxinmeng1 requested a review from XuPeng-SH as a code owner July 3, 2026 06:20
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@XuPeng-SH XuPeng-SH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes after reviewing latest head 0b98efc2a39ad1d080796fcd2305f76a0bacd189 against 4.1-dev.

The core Go default change is internally consistent, but the behavior is not closed end-to-end yet.

  1. The launch/embed configs still explicitly override the new default with incremental-interval = "180s".

This PR changes the zero-value default to 5 minutes, but all of these shipped startup templates still set an explicit 3 minute interval, so deployments using them will not see the new default:

  • pkg/embed/template.go
  • etc/launch-dynamic-with-proxy/tn.toml
  • etc/launch-dynamic-cn/tn.toml
  • etc/launch-multi-cn/tn.toml
  • etc/launch-with-proxy/tn.toml
  • etc/launch-minio-local/tn.toml
  • etc/launch-tae-compose/config/tn.toml

If the intended release behavior is 5m, those explicit configs need to be updated or removed so the Go default can apply. Otherwise this only changes code paths where the field is omitted, while the common packaged/dev launch configs keep the old value.

  1. The arena pool drain timer still assumes the old 1m checkpoint interval.

pkg/objectio/arena_pool.go documents arenaDrainDelay as 2x the default incremental checkpoint interval so the timer is reset by steady-state checkpointing and pools stay warm. The value is still 2 * time.Minute; after this PR, the default checkpoint interval becomes 5m, so the drain timer can fire between normal checkpoints and empty the arena free lists during active operation. That is a performance regression risk and contradicts the documented invariant. Please update this delay/design alongside the new default, or justify why the old 2m drain is still correct after changing checkpoint cadence.

  1. Please add focused regression coverage for the default value propagation.

At minimum, cover the three default-entry paths changed here:

  • tnservice.Config.Validate / SetDefaultValue
  • options.Options.FillDefaults
  • checkpoint.CheckpointCfg.FillDefaults

That would catch drift between TN config, TAE options, and checkpoint runtime defaults.

Local verification I ran:

  • go test -count=1 -timeout 240s ./pkg/vm/engine/tae/options ./pkg/vm/engine/tae/db/checkpoint ./pkg/tnservice
  • go test -count=1 -timeout 240s ./pkg/vm/engine/tae/options ./pkg/vm/engine/tae/db/checkpoint ./pkg/tnservice -run '^$'
  • go vet ./pkg/vm/engine/tae/options ./pkg/vm/engine/tae/db/checkpoint ./pkg/tnservice

Those passed locally, but the end-to-end config/behavior gaps above should be fixed before merge.

@XuPeng-SH XuPeng-SH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for one remaining inconsistency:

  • etc/docker-multi-cn-local-disk/generate-config.sh:442 still emits incremental-interval = "60s".
  • The rest of the PR consistently moves the default incremental checkpoint interval to 5 minutes (pkg/vm/engine/tae/options/types.go, pkg/tnservice/cfg.go, pkg/vm/engine/tae/db/checkpoint/cfg.go, the touched etc/launch-* TN configs, and pkg/embed/template.go), so this local docker config generation path is still on the old value.
  • If this divergence is intentional, it should be called out explicitly; otherwise it leaves one non-CI startup path silently outside the new default and defeats the stated goal of the PR.

Suggested fix: update the generator to 300s, or route it through the same shared default/template source as the other launcher configs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/S Denotes a PR that changes [10,99] lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants