Closed
Conversation
Implements StorageOperations adapter for SlateDB backend. - 19 tests pass (get, scan, put, delete, increment, batch) - 7 tests skipped (checkAndMutate not supported by SlateDB) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This reverts commit ec35907.
Replace manual native library path management with JAR-bundled native library from slatedb/slatedb#1329. NativeLibraryLoader automatically extracts and loads the platform-specific library from classpath resources. - Remove SlateDb.loadLibrary(path) calls (API removed in upstream PR) - Use SlateDb.initLogging(SlateDbConfig.LogLevel.INFO) for initialization - Remove libraryPath from SlateDbOptions, SlateDbConnections, tests, and config - Remove working directory hack in server bootRun task Tested: all 4 SlateDB test suites pass (SlateDbTableTest, SlateDbStorageTest, SlateDbHashLabelTest, SlateDbIndexedLabelTest)
Out-of-scope change (import reordering, expression body) that was unrelated to SlateDB integration.
- Restore MathOps.kt and math_ops.c (FFI examples belong to java25 branch) - Remove build-slatedb.sh (obsolete with JAR-bundled native library)
- Restore .gitignore (slatedb build artifacts no longer relevant) - Restore engine/build.gradle.kts (remove import reorder and broken native/lib/slatedb.jar file dependency) - Restore server/build.gradle.kts (Java 25 javaLauncher belongs in next/java25 branch)
Add native/build-slatedb-java.sh to clone, build, and publish slatedb-java to Maven local since it is not on Maven Central. Declare the dependency in engine and add mavenLocal() to repo resolution.
7 tasks
em3s
force-pushed
the
next/slatedb
branch
2 times, most recently
from
2b306fc to
e6773a5
Compare
- Add merge() to SlateDbTable interface and implementation - Add toSlateBytes()/toLong() helpers to eliminate duplicate ByteBuffer code - Define incrementMergeOperator once, reuse in connections and tests - Replace non-atomic read-modify-write with db.merge() in SlateDbHashLabel.incrby() - Use batch.merge() for BatchOperation.Increment in batch writes - Switch SlateDbConnections from SlateDb.open() to builder pattern with merge operator - Update build script to use java-merge-op branch (slatedb PR #1338) - Add merge operator tests for degree counting use case
7 tasks
The slatedb-java library is now published to Maven Central as io.slatedb:slatedb:0.11.0 with bundled native libraries. No local build is required anymore. Changes: - Switch dependency from 0.1.0-SNAPSHOT to 0.11.0 (Maven Central) - Remove mavenLocal() repository added for SNAPSHOT resolution - Configure org.gradle.jvm.version=25 on resolvable configurations so Gradle accepts the artifact's JVM 24+ requirement - Introduce SlateDbScheduler singleton to serialize native FFI calls; the C library's global Tokio runtime does not support concurrent block_on from multiple threads - Delete native/build-slatedb-java.sh (local build no longer needed) Ref: slatedb/slatedb#368 (comment)
- Add ByteBuddy experimental flag so BlockHound works on Java 25 - Skip HBase mini cluster tests on Java 18+: Subject.getSubject() was removed in Java 18 (Hadoop UserGroupInformation incompatibility)
MathOps.kt formatting, HBase Java 18+ skip, and BlockHound/Hadoop JVM args belong in next/java25, not next/slatedb. Revert those to the next/java25 baseline so this branch only contains SlateDB-specific changes.
The JVM version attribute fix belongs in next/java25 as a general Java 25 environment fix, not in next/slatedb. Also remove stale .gitignore entries for local native build artifacts (no longer needed since SlateDB is pulled from Maven Central).
Resolve conflict in GraphFixtures.kt: take java25 version which adds configBuilder parameter and BlockHound MockHTable allowlist.
Two bugs fixed in SlateDbConnections/SlateDbScheduler: 1. Use-after-close race: closeConnections() was running db.close() on boundedElastic while pending FFI ops (delete, merge) were still queued on slatedb-worker. Moving close to SlateDbScheduler.INSTANCE ensures it is enqueued after all pending ops, preventing the ClosedException that corrupted the Tokio runtime state and caused SIGABRT on the next test. 2. BlockHound conflict: Schedulers.newSingle() marks its thread as Reactor NonBlocking, which BlockHound monitors for blocking calls. The intentional FFI block_on calls triggered instrumentation conflicts on Java 25. Switching to Schedulers.fromExecutorService with a plain single-thread executor avoids the NonBlocking marker while preserving serialization guarantees.
Contributor
Author
|
We confirmed SlateDB's potential in this PR. We'll revisit after defining our Java 25 upgrade policy and V2→V3 engine migration.
Note: hit a concurrency issue in 0.11.0, worked around with cc. @criccomini |
Curious about this. If it's an issue with SlateDB, please do open an issue :) |
Contributor
Author
Opened slatedb/slatedb#1366 cc. @criccomini After investigation, the issue is blocking — not concurrency. The multi-thread Tokio runtime handles concurrent |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Experimental integration to explore the feasibility of SlateDB as an alternative storage layer.
Closes #155
Stages
Stage 0
Stage 1
slatedb-cslatedb/slatedb#1282..dylib/.so) separately, loaded viaSystem.loadLibrary.Stage 2
libslatedb_cin slatedb-java JAR slatedb/slatedb#1329.publishToMavenLocalvianative/build-slatedb-java.sh.Stage 3
SlateDb.builder().withMergeOperator()with an increment merge operator.BatchOperation.Incrementwithbatch.merge().SlateDbHashLabel.incrby()withdb.merge().Stage 4 (current)
io.slatedb:slatedb:0.11.0from Maven Central (published 2026-03-03).How to Test
To run SlateDB datastore compatibility tests (opt-in):
SLATEDB_TEST=true ./gradlew :engine:test --tests "*SlateDBDatastoreCompatibilityTest*"Notes
Subject.getSubject(AccessControlContext), which was removed in Java 18.SlateDbScheduler). Concurrent reads could theoretically be parallelized, but the C library's Tokio runtime model makes this risky without further investigation.