Allow`write.metadata.previous-versions-max` table property to be 0 for empty metadata logs

[smaheshwar-pltr]

Feature Request / Improvement

write.metadata.previous-versions-max is currently floored to 1:

iceberg/core/src/main/java/org/apache/iceberg/TableMetadata.java

Lines 1801 to 1807 in 70b9033

	int maxSize =
	Math.max(
	1,
	PropertyUtil.propertyAsInt(
	properties,
	TableProperties.METADATA_PREVIOUS_VERSIONS_MAX,
	TableProperties.METADATA_PREVIOUS_VERSIONS_MAX_DEFAULT));

so a table always retains at least one previous metadata file in the metadata-log.

I was a bit confused by this, because I don't see a concrete reason to disallow empty metadata logs; I'd therefore like to know what folks think about permitting a value of 0 instead to enable this.

With write.metadata.delete-after-commit.enabled=true, this would then delete the superseded metadata file on every commit.

(My use case for this is from a REST catalog standpoint where an empty metadata log would allow metadata JSON to be "self-contained" and only reference its current snapshots, instead of indirectly reference snapshots in prior metadata files too)

Query engine

None

Willingness to contribute

• I can contribute this improvement/feature independently
• I would be willing to contribute this improvement/feature with guidance from the Iceberg community
• I cannot contribute this improvement/feature at this time

[smaheshwar-pltr]

I also think that from a spec standpoint

A list (optional) of timestamp and metadata file location pairs that encodes changes to the previous metadata files for the table. Each time a new metadata file is created, a new entry of the previous metadata file location should be added to the list. Tables can be configured to remove oldest metadata log entries and keep a fixed-size log of the most recent entries after a commit.

from https://iceberg.apache.org/spec/#table-metadata is worded consistently with a value of 0?

[smaheshwar-pltr]

cc @RussellSpitzer, I seem to remember seeing a thread from you at one point about the possibility of letting this value be zero, but for the life of me I cannot find it so perhaps I am hallucinating 😅

[RussellSpitzer]

I don't remember but that's possible. I don't have a problem with letting it be zero and I have no idea why it's currently limited to 1

[smaheshwar-pltr]

I don't remember but that's possible. I don't have a problem with letting it be zero and I have no idea why it's currently limited to 1

Thank you!

I've put up #16796 if folks could have a look here 🙏