Skip to content

fix: branch support on object stores, python branch ops, s3/ddb integration tests#7219

Merged
jackye1995 merged 3 commits into
lance-format:mainfrom
brendanclement:feat/pylance-namespace-branch-ops
Jun 11, 2026
Merged

fix: branch support on object stores, python branch ops, s3/ddb integration tests#7219
jackye1995 merged 3 commits into
lance-format:mainfrom
brendanclement:feat/pylance-namespace-branch-ops

Conversation

@brendanclement

Copy link
Copy Markdown
Contributor

Adds the first integration coverage for table branches (localstack S3 + DynamoDB) and fixes the four bugs it caught. All four were invisible to local-FS tests.

  • Branch locations broke uris that carry query strings. BranchLocation appended tree/ after the query string, so s3+ddb://...?ddbTableName=x produced a branch uri that re-parses to the main table path. Branches now work end to end over s3+ddb://.

  • create_branch dropped the dataset's store params. It passed store_params.unwrap_or_default() to CommitBuilder, so the commit's existence probe built an object store with no endpoint or region. It now falls back to the dataset's own params, matching shallow_clone.

  • The managed manifest store treated a missing chain as a fatal error. Probing a branch location before the branch exists surfaced the namespace's not-found instead of "no versions". get_latest_version now reports a missing table or branch chain as None, matching the ExternalManifestStore contract and the store's own behavior for empty main chains.

  • The Python namespace adapter flattened error identity. Exceptions from custom Python namespaces all arrived in Rust as opaque IO strings. The adapter now reconstructs the NamespaceError from the lance_namespace exception's error code, gated on the exception actually being a LanceNamespaceError so a foreign exception that happens to carry an integer code is never reinterpreted (it propagates as itself). Custom namespaces now raise the same typed exceptions as the native ones.

@github-actions github-actions Bot added A-python Python bindings A-deps Dependency updates bug Something isn't working labels Jun 10, 2026
@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.68421% with 6 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rust/lance/src/io/commit/namespace_manifest.rs 93.33% 3 Missing and 1 partial ⚠️
rust/lance/src/dataset/branch_location.rs 93.93% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

Comment thread python/python/tests/test_s3_ddb.py
Comment thread python/pyproject.toml Outdated
brendanclement added a commit to lance-format/lance-namespace that referenced this pull request Jun 11, 2026
## Description

The Rust implementation in lance-format/lance defines
TableBranchNotFound (22) and TableBranchAlreadyExists (23), but the spec
registry and the Python and Java interface modules stop at 21. Clients
converting errors by code (for example pylance via
lance_namespace.errors.from_error_code) currently reclassify branch
failures as InternalError, so deleting a missing branch surfaces as an
internal error instead of a branch-not-found error.

This adds the two branch error codes everywhere the registry lives:

- spec.yaml: error code registry gains 22 and 23, plus 21 (Throttling),
which was already present in every implementation but missing from the
spec list.
- Python: ErrorCode members, TableBranchNotFoundError /
TableBranchAlreadyExistsError, from_error_code mapping, and package
re-exports.
- Java (core and core-async): ErrorCode entries, ErrorFactory cases, and
the two exception classes in each module, mirroring the existing tag
exceptions.

## Testing

Python round-trip test for both codes (from_error_code returns the typed
exceptions with the right code values).

Surfaced by review feedback on lance-format/lance#7219, where the new
Python branch API bindings expose the gap.
@brendanclement brendanclement force-pushed the feat/pylance-namespace-branch-ops branch from 2796cd3 to ee35261 Compare June 11, 2026 17:07

@jackye1995 jackye1995 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the fixes!

@jackye1995 jackye1995 merged commit d3f0db3 into lance-format:main Jun 11, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-deps Dependency updates A-python Python bindings bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants