Initial commit

Signed-off-by: teodordelibasic-db <teodor.delibasic@databricks.com>

Author: teodordelibasic-db

.gitignore

@@ -19,7 +19,16 @@ buildNumber.properties
 .settings/
 .project
 .classpath
+.metals/
+.bsp/
+.bazelbsp/
 
 # OS
 .DS_Store
 Thumbs.db
+
+# Compiled class files
+*.class
+
+# Generated proto files in examples
+examples/proto/**/AirQualityProto.java

CHANGELOG.md

@@ -1,5 +1,73 @@
 # Version changelog
 
+## Release v0.2.0
+
+### Native Rust Backend (JNI Migration)
+- The SDK now uses JNI (Java Native Interface) to call the Zerobus Rust SDK instead of pure Java gRPC calls
+- Native library is automatically loaded from the classpath or system library path
+- Token management and background processing handled by native code
+
+### New Stream Classes
+
+**ZerobusProtoStream** - Protocol Buffer ingestion with method-level generics:
+- `ingestRecordOffset(T record)` - Auto-encoded: SDK encodes Message to bytes
+- `ingestRecordOffset(byte[] bytes)` - Pre-encoded: User provides encoded bytes
+- `ingestRecordsOffset(List<T> records)` - Batch auto-encoded ingestion
+- `ingestRecordsOffset(List<byte[]> bytes)` - Batch pre-encoded ingestion
+- `getUnackedRecords()` - Returns `List<byte[]>` of unacked records
+- `getUnackedRecords(Parser<T>)` - Returns parsed `List<T>` of unacked records
+- `getUnackedBatches()` - Returns `List<EncodedBatch>` preserving batch grouping
+
+**ZerobusJsonStream** - JSON ingestion without Protocol Buffer dependency:
+- `ingestRecordOffset(String json)` - Pre-serialized: User provides JSON string
+- `ingestRecordOffset(T obj, JsonSerializer<T>)` - Auto-serialized: SDK serializes object
+- `ingestRecordsOffset(List<String> jsons)` - Batch pre-serialized ingestion
+- `ingestRecordsOffset(List<T> objs, JsonSerializer<T>)` - Batch auto-serialized ingestion
+- `getUnackedRecords()` - Returns `List<String>` of unacked JSON records
+- `getUnackedRecords(JsonDeserializer<T>)` - Returns parsed `List<T>` of unacked records
+- `getUnackedBatches()` - Returns `List<EncodedBatch>` preserving batch grouping
+
+### New Factory Methods
+
+- `ZerobusSdk.createProtoStream(tableName, descriptorProto, clientId, clientSecret)` - Create proto stream
+- `ZerobusSdk.createProtoStream(tableName, descriptorProto, clientId, clientSecret, options)` - With options
+- `ZerobusSdk.createJsonStream(tableName, clientId, clientSecret)` - Create JSON stream
+- `ZerobusSdk.createJsonStream(tableName, clientId, clientSecret, options)` - With options
+- `ZerobusSdk.recreateStream(ZerobusProtoStream)` - Recreate proto stream with unacked record re-ingestion
+- `ZerobusSdk.recreateStream(ZerobusJsonStream)` - Recreate JSON stream with unacked record re-ingestion
+- `ZerobusSdk.recreateStream(ZerobusStream)` - Recreate legacy stream (deprecated)
+
+### New Supporting Types
+
+- `BaseZerobusStream` - Abstract base class with native JNI methods
+- `JsonSerializer<T>` - Functional interface for object-to-JSON serialization
+- `JsonDeserializer<T>` - Functional interface for JSON-to-object deserialization
+- `EncodedBatch` - Represents a batch of encoded records for recovery
+- `AckCallback` - Callback interface with `onAck(long)` and `onError(long, String)`
+
+### Deprecated (Backward Compatible)
+
+**ZerobusStream<T>** - Use `ZerobusProtoStream` instead:
+- `ingestRecord(T record)` - Returns `CompletableFuture<Void>`, use `ingestRecordOffset()` instead
+- `getStreamId()` - No longer exposed by native backend, returns empty string
+- `getState()` - Returns `OPENED` or `CLOSED` only
+- `getUnackedRecords()` - **Breaking:** Returns empty iterator (records stored in native, type erasure prevents deserialization). Use `ZerobusProtoStream.getUnackedRecords(Parser<T>)` for typed access, or use `recreateStream()` which handles re-ingestion automatically using cached raw bytes.
+
+**StreamConfigurationOptions**:
+- `setAckCallback(Consumer<IngestRecordResponse>)` - No longer invoked by native backend. Use `setAckCallback(AckCallback)` instead
+
+### Removed
+
+- `TokenFactory` - Token management now handled by native Rust SDK
+- `BackgroundTask` - Background processing now handled by native Rust SDK
+- `ZerobusSdkStubUtils` - gRPC stub utilities no longer needed with native backend
+
+### Platform Support
+
+- Linux x86_64: Supported
+- Windows x86_64: Supported
+- **macOS: Not yet supported**
+
 ## Release v0.1.0
 
 Initial release of the Databricks Zerobus Ingest SDK for Java.

NEXT_CHANGELOG.md

@@ -1,38 +1,13 @@
 # NEXT CHANGELOG
 
-## Release v0.2.0
+## Release v0.3.0
 
 ### New Features and Improvements
 
-- Updated Protocol Buffers from 3.24.0 to 4.33.0 for improved performance and latest features
-- Updated gRPC dependencies from 1.58.0 to 1.76.0 for enhanced stability and security
-- Updated SLF4J logging framework from 1.7.36 to 2.0.17 for modern logging capabilities
-
 ### Bug Fixes
 
 ### Documentation
 
-- Updated README.md with new dependency versions
-- Updated protoc compiler version recommendations
-- Updated Logback version compatibility for SLF4J 2.0
-
 ### Internal Changes
 
-- Updated maven-compiler-plugin from 3.11.0 to 3.14.1
-- All gRPC artifacts now consistently use version 1.76.0
-
 ### API Changes
-
-**Breaking Changes**
-
-- **Protocol Buffers 4.x Migration**: If you use the regular JAR (not the fat JAR), you must upgrade to protobuf-java 4.33.0 and regenerate any custom `.proto` files using protoc 4.x
-  - Download protoc 4.33.0 from: https://github.com/protocolbuffers/protobuf/releases/tag/v33.0
-  - Regenerate proto files: `protoc --java_out=src/main/java src/main/proto/record.proto`
-  - Protobuf 4.x is binary-compatible over the wire with 3.x, but generated Java code may differ
-
-- **SLF4J 2.0 Migration**: If you use a logging implementation, you may need to update it:
-  - `slf4j-simple`: Use version 2.0.17 or later
-  - `logback-classic`: Use version 1.4.14 or later (for SLF4J 2.0 compatibility)
-  - `log4j-slf4j-impl`: Use version 2.20.0 or later
-
-**Note**: If you use the fat JAR (`jar-with-dependencies`), all dependencies are bundled and no action is required.