[Bug] Data loss occurs when multiple Spark jobs write data concurrently

[MaxLinyun]

Search before asking

• I searched in the issues and found nothing similar.

Paimon version

1.3.2

Compute Engine

spark 3.3

Minimal reproduce step

1. Using the Filesystem Catalog with Alibaba Cloud OSS storage
2. When two Spark jobs are started simultaneously to write identical data into two different partitions of the same Paimon table, it may happen that both write operations report success, yet data only exists in one of the two partitions.

What doesn't meet your expectations?

both write operations report success, and data exists in both partitions.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!