bug: validate schema and partition field name conflicts during updates

[kevinjqliu]

Feature Request / Improvement

Description

Iceberg validates conflicts between schema field names and partition field names. For example, in checkAndAddPartitionName, name conflicts are rejected unless it’s an identity/void transform sourced from the same field.

We need to make this check when updating the schema (UpdateSchema) as well as updating the partition spec (UpdatePartitionSpec).

Found this edge case while we were implementing the same check in pyiceberg, apache/iceberg-python#2305

Query engine

None

Willingness to contribute

• I can contribute this improvement/feature independently
• I would be willing to contribute this improvement/feature with guidance from the Iceberg community
• I cannot contribute this improvement/feature at this time

[kevinjqliu]

this actually isnt an issue because table update will rebuild partition spec which will trigger the name collision check

iceberg/core/src/main/java/org/apache/iceberg/TableMetadata.java

Lines 1098 to 1100 in 0b599bc

	// rebuild all the partition specs and sort orders for the new current schema
	this.specs =
	Lists.newArrayList(Iterables.transform(specs, spec -> updateSpecSchema(schema, spec)));

[kevinjqliu]

Open #13834 to add a new test for this scenario

[Fokko]

This is not covered today:

That test should be failing 👍

[kevinjqliu]

thats a good one. Here's the call stack for this bug:
updateSpec() calls PartitionSpec.Builder.add(), which passes name and sourceId to checkAndAddPartitionName
Because sourceColumnId is provided, checkAndAddPartitionName goes to the if statement which handles identity transform.

This is a great illustration of why refactoring checkAndAddPartitionName to be more explicit in #13835 helps prevent bugs

[kevinjqliu]

the refactor in #13835 actually caught this bug 👍