feat(catalog): support column-level alter table

[TheR1sing3un]

Purpose

Linked issue: close #368

Implement column-level alter table on both FileSystemCatalog and RESTCatalog, and align the SchemaChange JSON wire format with Java Paimon.

Brief change log

• TableSchema::apply_changes: implement add / rename / drop column, update column type / nullability / comment / position, and update table comment, mirroring Java SchemaManager.generateTableSchema (field-id allocation, FIRST/AFTER/BEFORE/LAST moves, primary/partition-key guards). Operates on top-level columns; reuses ColumnAlreadyExist / ColumnNotExist. The method keeps its single-argument signature — the catalog fills in the table name on column errors.
• Align SchemaChange JSON with Java Paimon: internally tagged by action, with fieldNames arrays, comment / newDataType / keepNullability / newNullability / newComment fields, referenceFieldName move anchors, and FIRST/AFTER/BEFORE/LAST move types.
• Add AlterTableRequest + RESTApi::alter_table; implement RESTCatalog::alter_table.

Tests

• New unit tests in spec::schema_change (Java-format (de)serialization), spec::schema, and catalog::filesystem (add+move, rename syncing PK refs, drop rejecting PK columns, type/nullability/comment updates, reposition, and the ColumnAlreadyExist / ColumnNotExist / TableNotExist error paths).
• Existing paimon-datafusion alter-table tests pass.

API and Format

Changes the SchemaChange JSON wire format. This is effectively a fix: the previous format was incompatible with Java Paimon, and REST alter_table was never implemented, so no working interaction is broken. No public Rust API signature change (apply_changes keeps its arity).

Documentation

None required.

Out of scope (follow-ups)

Nested struct field paths; UpdateColumnType cast-compatibility validation; Java's dropPrimaryKey / updateColumnDefaultValue.

[TheR1sing3un]

Pushed a fix for the integration job: the existing test_alter_table_add_column (in crates/integrations/datafusion/tests/sql_context_tests.rs) asserted that ALTER TABLE ... ADD COLUMN fails because AddColumn was unsupported. Since this PR implements it, the test now asserts the statement succeeds and the new column is appended to the schema. Verified locally: cargo test -p paimon-datafusion --test sql_context_tests test_alter_table passes.

[JingsongLi]

Cross-PR note with #340: if merge-engine=aggregation lands before or together with this alter-table work, RenameColumn needs to update aggregation field-scoped options as well.

Right now the rename path rewrites primary_keys, bucket-key, and sequence.field, but it does not rename keys such as fields.<col>.aggregate-function or fields.<col>.list-agg-delimiter. After renaming an aggregation value column, those options would still point at the old column name. Since the final validation here reruns blob / partial-update / first-row checks but not aggregation validation, the stale option can be persisted and the renamed column may silently fall back to the default aggregator/delimiter at read time.

This is not visible against current main by itself, but it should be handled if this PR is intended to compose with #340.

[TheR1sing3un]

Thanks — confirmed. The rename path here only ports case 1 of Java's applyRenameColumnsToOptions (primary_keys / bucket-key / sequence.field); it doesn't rewrite the field-scoped fields..* keys (Java case 2: aggregate-function / ignore-retract / distinct / list-agg-delimiter, and case 3: sequence-group / nested-key). Worth noting this is already latent for partial-update tables on main, not only when composing with #340. To keep this PR scoped to the alter-table feature as reviewed, I'll track the field-scoped option rewriting as a follow-up: #383.
@JingsongLi