Skip to content

Fix vortex-jni local path handling, centralise logic in vortex-file#8650

Merged
robert3005 merged 1 commit into
developfrom
claude/vortex-spark-bare-paths-bja1ek
Jul 6, 2026
Merged

Fix vortex-jni local path handling, centralise logic in vortex-file#8650
robert3005 merged 1 commit into
developfrom
claude/vortex-spark-bare-paths-bja1ek

Conversation

@robert3005

Copy link
Copy Markdown
Contributor

Existing logic was duplicated between jni and duckdb with slight
differences. JNI logic didn't always handle local paths.

@robert3005 robert3005 requested a review from a team July 3, 2026 14:35
@codspeed-hq

codspeed-hq Bot commented Jul 3, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚡ 1 improved benchmark
❌ 2 regressed benchmarks
✅ 1604 untouched benchmarks
⏩ 42 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation bitwise_not_vortex_buffer_mut[128] 244.4 ns 273.6 ns -10.66%
Simulation chunked_varbinview_opt_canonical_into[(100, 100)] 305.7 µs 340.9 µs -10.33%
Simulation chunked_varbinview_canonical_into[(1000, 10)] 191.1 µs 154.8 µs +23.44%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/vortex-spark-bare-paths-bja1ek (79e94b4) with develop (570f7ed)2

Open in CodSpeed

Footnotes

  1. 42 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on develop (4959939) during the generation of this report, so 570f7ed was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@robert3005 robert3005 force-pushed the claude/vortex-spark-bare-paths-bja1ek branch from a2cfeab to ecc9af6 Compare July 3, 2026 14:55
@robert3005 robert3005 added the changelog/fix A bug fix label Jul 3, 2026
@robert3005 robert3005 changed the title Centralise path normalisation in vortex-file module Fix vortex-jni local path handling, centralise logic in vortex-file Jul 3, 2026
@robert3005 robert3005 force-pushed the claude/vortex-spark-bare-paths-bja1ek branch from ecc9af6 to 35ad5bf Compare July 3, 2026 15:10
@robert3005 robert3005 requested a review from AdamGS July 3, 2026 15:14
@robert3005 robert3005 enabled auto-merge (squash) July 3, 2026 15:14
Existing logic was duplicated between jni and duckdb with slight
differences. JNI logic didn't always handle local paths.

Signed-off-by: "robert" <robert@spiraldb.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005 robert3005 force-pushed the claude/vortex-spark-bare-paths-bja1ek branch from 35ad5bf to 79e94b4 Compare July 6, 2026 13:20
@robert3005 robert3005 merged commit 1743bb0 into develop Jul 6, 2026
70 of 72 checks passed
@robert3005 robert3005 deleted the claude/vortex-spark-bare-paths-bja1ek branch July 6, 2026 16:39
robert3005 pushed a commit that referenced this pull request Jul 6, 2026
… SQL

Prompted by discussion #8648, document how to get Vortex into Spark and
query it from Spark SQL:

- Add a README to java/vortex-spark with quick-start instructions for
  spark-shell/spark-submit/pyspark, the DataFrame API, and Spark SQL
  (CREATE TEMPORARY VIEW / CREATE TABLE ... USING vortex LOCATION).
- Rework docs/user-guide/spark.md: correct the Maven coordinates to the
  Scala-suffixed artifacts (vortex-spark_2.12 for Spark 3.5.x,
  vortex-spark_2.13 for Spark 4.x), document that the self-contained
  "all" classifier JAR is required (the thin JAR fails with
  NoClassDefFoundError on relocated classes, so --packages alone does
  not work), document accepted path forms (bare local paths work since
  #8650; releases up to 0.76.0 require URL-form paths), and add a
  Spark SQL section.
- Point the root README Maven badge at vortex-spark_2.13; the old
  unsuffixed artifact stopped publishing at 0.65.0.
- Link the new README from java/README.md.

The documented examples were verified against the published
dev.vortex:vortex-spark_2.13:0.76.0-all JAR with PySpark 4.1.2:
DataFrame read/write, CREATE TEMPORARY VIEW ... USING vortex,
CREATE TABLE ... USING vortex LOCATION + INSERT + SELECT, and CTAS
with LOCATION all pass; managed tables (no LOCATION) and direct file
queries (FROM vortex.`path`) fail and are called out as unsupported.
Bare local paths are covered by the connector test added in #8650.

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LM9s8M99nQHdwhkrba35PU
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants