Skip to content

create-geoparquet drops collection-only properties from the collection metadata #26

Description

@cholmes

Found while regenerating the examples for vecorel/administrative-division-extension#6 with vecorel-cli 0.2.15.

vec create-geoparquet silently drops properties that are restricted to the collection level (via a schema's collection: {field: true} block) when writing the GeoParquet collection metadata.

Repro (using the multi-source-fields branch of the administrative-division-extension repo):

vec create-geoparquet examples/geojson -o example.parquet \
  -s https://vecorel.org/administrative-division-extension/v0.2.0/schema.yaml,schema/schema.yaml

The source GeoJSON FeatureCollection carries five collection-level properties (admin_source_name, admin_source_url, admin_source_version, admin_source_method, admin_source_processing). The resulting parquet's collection metadata contains only {schemas, collection, admin_continent, admin_country_code, admin_country_name} — the admin_source_* properties are gone from both the metadata and the columns.

Cause: merge_collections() (vecorel_cli/vecorel/ops.py, around line 45) rebuilds the collection dict from scratch with only schemas. Properties that are allowed at the feature level survive incidentally — read(hydrate=True) expands them into constant columns which dehydrate_to_collection() later folds back — but collection-only properties are never hydrated into columns, so they're discarded.

Validation can't catch the loss when the affected fields are optional, which makes this easy to miss. It will affect any extension that defines collection-level-only properties.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UTHtaFACrPMP2YuvUYgxwV

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions