mounter(ticdc): fix truncate table cause mounter failed (#12431)

[ti-chi-bot]

This is an automated cherry-pick of #12431

What problem does this PR solve?

Issue Number: close #12430

What is changed and how it works?

If the table is a partition table, we have to record the partition table id when executing the truncate table ddl.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)

1. create a changefeed
2. create table t (id INT AUTO_INCREMENT PRIMARY KEY, data VARCHAR(255)) partition by hash(id) partitions 5;
3. run script

function execute_ddls() {
	while true; do
		run_sql "TRUNCATE TABLE test.t;" ${UP_TIDB_HOST} ${UP_TIDB_PORT}
		sleep 1
	done
}

function execute_dml() {
	while true; do
		run_sql "INSERT INTO test.t (data) VALUES ('insert_$(date +%s)_${RANDOM}');"
	done
}

execute_ddls &
execute_dml &

Before this PR:

[ERROR] [mounter.go:179] ["can not found table schema"] [ts=462616963080781851] [key=7480000000000000865f72800000000000003e] [tableID=134]

After this PR:

[DEBUG] [mounter.go:176] ["skip the DML of truncated table"] [ts=462617109199585299] [tableID=244]

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Fix a bug in truncating the partition table that causes the change feed to fail.

[ti-chi-bot]

This cherry pick PR is for a release branch and has not yet been approved by triage owners.
Adding the do-not-merge/cherry-pick-not-approved label.

To merge this cherry pick:

1. It must be approved by the approvers firstly.
2. AFTER it has been approved by approvers, please wait for the cherry-pick merging approval from triage owners.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[gemini-code-assist]

Summary of Changes

Hello @ti-chi-bot, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a critical bug in the ticdc mounter component where truncating a partitioned table could lead to mounter failures. The fix ensures that the schema snapshot accurately tracks all physical partition IDs of a truncated partitioned table, allowing the system to correctly skip DMLs on these old, non-existent partitions and prevent errors.

Highlights

• Partitioned Table Truncation Fix: Correctly handles TRUNCATE TABLE operations on partitioned tables by ensuring all old partition IDs are marked as truncated.
• Error Prevention: Prevents "can not found table schema" errors in the mounter component when DMLs are applied to previously truncated partitions of a partitioned table.
• Schema Snapshot Logic Enhancement: Enhances the truncateTable function in the schema snapshot to identify and record individual physical partition IDs that are no longer active after a partitioned table truncation.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request addresses a bug where truncating a partitioned table caused the mounter to fail. The fix involves correctly identifying and marking the old physical partition IDs as truncated. The overall logic is sound and the test changes align with the fix. I have one suggestion to improve the robustness of the implementation and prevent a potential panic.

[gemini-code-assist]

This block has a potential nil pointer dereference if newPi is nil. Although TRUNCATE TABLE on a partitioned table should result in a new partitioned table, it's safer to handle the case where newPi could be nil for robustness.

Additionally, the map is sized using len(newPi.NewPartitionIDs), but it's populated from newPi.Definitions in the following loop. It would be more correct and robust to size it with len(newPi.Definitions).

newPartitionIDMap := make(map[int64]struct{})
		if newPi != nil {
			newPartitionIDMap = make(map[int64]struct{}, len(newPi.Definitions))
			for _, partition := range newPi.Definitions {
				newPartitionIDMap[partition.ID] = struct{}{}
			}
		}

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wk989898

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [wk989898]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

❌ Patch coverage is 81.25000% with 3 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (release-8.1@05534b3). Learn more about missing BASE report.

Additional details and impacted files

Components	Coverage Δ
cdc	61.4645% <0.0000%> (?)
dm	51.0408% <0.0000%> (?)
engine	63.3738% <0.0000%> (?)

Flag	Coverage Δ
unit	57.2632% <81.2500%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             release-8.1     #12447   +/-   ##
================================================
  Coverage               ?   57.2632%           
================================================
  Files                  ?        854           
  Lines                  ?     126706           
  Branches               ?          0           
================================================
  Hits                   ?      72556           
  Misses                 ?      48696           
  Partials               ?       5454           

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ti-chi-bot]

@ti-chi-bot: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cdc-integration-mysql-test	c657e6e	link	true	/test pull-cdc-integration-mysql-test
pull-cdc-integration-storage-test	c657e6e	link	true	/test pull-cdc-integration-storage-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.