Skip to content

Replace DefaultHBaseCluster with DefaultStorageBackendFactory#175

Open
em3s wants to merge 34 commits intomainfrom
feat/issue-173-engine-storage-backend-claude
Open

Replace DefaultHBaseCluster with DefaultStorageBackendFactory#175
em3s wants to merge 34 commits intomainfrom
feat/issue-173-engine-storage-backend-claude

Conversation

@em3s
Copy link
Contributor

@em3s em3s commented Feb 5, 2026

Summary

Replace DefaultHBaseCluster with DefaultStorageBackendFactory + HBaseStorageBackend (both already on main via PRs #191, #194, #192).

Before: DefaultHBaseCluster -- a monolithic singleton bundling connection management, mock branching, and table access.

After: DefaultStorageBackendFactory dispatches to pluggable backends (HBaseStorageBackend, MemoryStorageBackend, MockHBaseStorageBackend), each implementing the StorageBackend interface. The HBase backend can now be swapped for any other StorageBackend implementation.

Storage backend series: PR #191 -> PR #194 -> PR #192 (all merged) -> this PR

Known issue

This PR was started before the v3 abstractions landed on main, so it created v2-local duplicates (StorageBucket, DatastoreUri, DefaultStorageBackendFactory, etc.) instead of using the v3 classes directly. These should be removed and replaced with v3 imports.

Remove (v2 duplicate) Use instead (v3, on main)
v2/.../StorageBackend engine.storage.StorageBackend
v2/.../StorageBucket / StorageBuckets engine.storage.StorageTable
v2/.../DatastoreUri engine.storage.DatastoreUri
v2/.../DefaultStorageBackendFactory engine.storage.DefaultStorageBackendFactory
v2/.../HBaseStorageBackend engine.storage.hbase.HBaseStorageBackend
v2/.../MockHBaseStorageBackend engine.storage.hbase.MockHBaseStorageBackend
v2/.../MemoryStorageBackend engine.storage.memory.MemoryStorageBackend

Test plan

  • ./gradlew :engine:build passes
  • CI passes

Closes #173

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. enhancement New feature or request module:engine labels Feb 5, 2026
Step 1: Create StorageBackend, StorageBucket, StorageBuckets interfaces

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels Feb 5, 2026
@em3s em3s changed the title feat(engine): introduce StorageBackend abstraction layer feat(engine): introduce StorageBackend abstraction layer by Claude Feb 5, 2026
Step 2: Create MemoryStorageBucket with tests

- Implements StorageBucket interface using ByteArrayStore
- Add StorageBucketCompatibilityTest as base test class
- All 28 tests passing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Feb 5, 2026
Step 3: Create MemoryStorageBackend with tests

- Implements StorageBackend interface for in-memory storage
- All tests passing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Feb 5, 2026
em3s and others added 2 commits February 5, 2026 13:02
Step 4: Create HBaseStorageBucket with tests

- Implements StorageBucket interface wrapping HBaseTable
- Add supportsIncrement() to compatibility tests for Mock HBase limitations
- All available tests passing (some skipped due to Mock HBase)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Step 5: Create HBaseStorageBackend with tests

- Implements StorageBackend interface for HBase
- Supports HBase 2.4 and 2.5
- Supports Kerberos authentication
- All validation tests passing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Feb 5, 2026
em3s and others added 4 commits February 5, 2026 13:04
Step 6: Create DefaultStorageBackendFactory and MockStorageBackend

- Factory supports memory, embedded, and hbase backend types
- MockStorageBackend wraps HBase MockHTable for testing
- All tests passing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Step 7-8: Update GraphDefaults and Graph.kt to use StorageBackend

- GraphDefaults.datastore now uses StorageBackend interface
- Graph.kt uses DefaultStorageBackendFactory.initialize/INSTANCE/close
- Add getTable() method to StorageBackend for backward compatibility
- All existing tests pass

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ndFactory

- Update HBaseOptions.getTables() and getBuckets() to use DefaultStorageBackendFactory
- Add getEffectiveNamespace() to use DefaultHBaseCluster namespace as fallback
- Update MockStorageBackend to properly handle namespace and tableName parameters
- Initialize DefaultStorageBackendFactory in HBaseTestingClusterExtension for tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add EmbeddedStorageBackend that wraps HBase testing cluster connection
- Update DefaultStorageBackendFactory with initialize(backend) method
- Revert MockStorageBackend to use edges table for backward compatibility
- Update HBaseTestingClusterExtension to use EmbeddedStorageBackend

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
em3s and others added 6 commits February 5, 2026 14:32
More explicit naming since it uses HBase MockHTable for storage operations.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Each namespace:name combination now gets its own ByteArrayStore instance
instead of sharing a single store across all buckets.

Added tests:
- different buckets are isolated from each other
- same namespace and name returns same store

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
More logical package location since it's HBase-specific.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add defaultNamespace to DefaultStorageBackendFactory
- Update HBaseOptions to use DefaultStorageBackendFactory.defaultNamespace
- Update HBaseTestingClusterExtension to pass namespace to factory
- Delete DefaultHBaseCluster.kt and compat package

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ckend

Rename to better reflect that this backend wraps the HBase mini cluster
for integration testing, distinguishing it from MockHBaseStorageBackend
which uses in-memory MockHTable.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
… config

- Rename MiniHBaseStorageBackend to HBaseTestingStorageBackend for consistency
- Expose connectionMono in HBaseStorageBackend for admin access
- Update HBaseDatastoreBindingConfiguration to use DefaultStorageBackendFactory

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@em3s
Copy link
Contributor Author

em3s commented Feb 6, 2026

Code Review Round 3 - RESOLVED

  • (High) DefaultStorageBackendFactory.close() not synchronized, instance0 not nulled → added @Synchronized, null out instance0 on close
  • (High) MemoryStorageBackend.getTable() throws eagerly instead of Mono.error()changed to Mono.error()
  • (High) HBaseStorageBackend.connectionMono is public → made private, added getAdminMono() public method
  • (Medium) HBaseOptions.getBuckets()/getTables() log at INFO on every call → changed to DEBUG

Tracked separately:

All issues resolved in commit 3c04c7c

Reviewed by claude code (opus 4.6)

- Synchronize DefaultStorageBackendFactory.close() and null out instance
- Use Mono.error() instead of throw in MemoryStorageBackend.getTable()
- Encapsulate HBaseStorageBackend.connectionMono as private, expose getAdminMono()
- Change HBaseOptions log level from INFO to DEBUG for getBuckets()/getTables()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@em3s em3s self-assigned this Feb 6, 2026
@em3s
Copy link
Contributor Author

em3s commented Feb 9, 2026

Too large to review; splitting it up now.

em3s and others added 2 commits February 9, 2026 11:00
…-storage-backend-claude

# Conflicts:
#	engine/src/main/kotlin/com/kakao/actionbase/v2/engine/storage/hbase/HBaseStorageBackend.kt
Update test to reference HBaseStorageBackend instead of removed
DefaultHBaseCluster class. Apply spotless formatting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@em3s
Copy link
Contributor Author

em3s commented Feb 9, 2026

PR Split Plan

This PR (26 files, +1517/-143) has been split into 3 incremental PRs for easier review:

Part 1: Core abstractions + Memory backend — #191

Branch: feat/storage-backend-interfacesmain

  • StorageBackend interface: getStorageTable(namespace, name)Mono<StorageTable> (URI overload as default method)
  • StorageTable interface, DatastoreUri (strict validation: ^[a-z0-9_]+$)
  • MemoryStorageTable, MemoryStorageBackend implementation
  • StorageTableCompatibilityTest abstract contract test suite
  • All tests for the above
  • Purely additive, no existing code modified

Part 2: HBase backend + Factory — #192

Branch: feat/storage-backend-hbasefeat/storage-backend-interfaces (rebases to main after Part 1 merges)

  • HBaseStorageTable, HBaseStorageBackend, MockHBaseStorageBackend — all implement getStorageTable()
  • DefaultStorageBackendFactory
  • HBaseTestingStorageBackend test fixture
  • HBaseTestingClusterExtension modification (additive init call)
  • All tests for the above
  • Mostly additive, one test fixture modification

Part 3: Consumer migration — This PR (#175)

After Parts 1+2 merge to main, this PR rebases and shrinks to only the consumer migration:

  • Graph.kt — swap DefaultHBaseClusterDefaultStorageBackendFactory
  • GraphDefaults.kt — remove datastore field
  • DatastoreHashLabel.kt, DatastoreIndexedLabel.kt — use factory's getStorageTable()
  • HBaseOptions.kt — rewrite with getStorageTable()
  • HBaseDatastoreBindingConfiguration.kt — use factory
  • HBaseTestingClusterExtension.kt — remove old init
  • Delete compat/DefaultHBaseCluster.kt
  • Pure swap + cleanup

Merge order

PR #191 → main → PR #192 → main → PR #175 (now small) → main

em3s and others added 2 commits February 9, 2026 16:24
…-storage-backend-claude

# Conflicts:
#	engine/src/testFixtures/kotlin/com/kakao/actionbase/test/hbase/HBaseTestingClusterExtension.kt
#	engine/src/testFixtures/kotlin/com/kakao/actionbase/test/hbase/HBaseTestingStorageBackend.kt
Initialize both v2 and v3 DefaultStorageBackendFactory in
HBaseTestingClusterExtension to support tests using either API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@em3s em3s changed the title feat(engine): introduce StorageBackend abstraction layer by Claude feat(engine): migrate v2 engine to StorageBackend abstraction Feb 9, 2026
@em3s em3s changed the title feat(engine): migrate v2 engine to StorageBackend abstraction refactor(engine): migrate v2 engine from DefaultHBaseCluster to DefaultStorageBackendFactory Feb 9, 2026
@em3s em3s changed the title refactor(engine): migrate v2 engine from DefaultHBaseCluster to DefaultStorageBackendFactory refactor(engine): replace DefaultHBaseCluster with DefaultStorageBackendFactory Feb 9, 2026
@em3s
Copy link
Contributor Author

em3s commented Feb 9, 2026

Plan: Remove v2 duplicates, wire to v3 StorageBackend

PR #175 was started before v3 storage abstractions landed on main (PRs #191/#194/#192), so it created v2-local duplicates. This cleanup removes them and wires v2 engine directly to v3 engine.storage.*.

Key design: HBaseTablesProvider interface

v2 Labels need Mono<HBaseTables> for multi-column HBase operations (Filters, CellUtil), but v3 StorageBackend only returns Mono<StorageTable>. A new HBaseTablesProvider interface bridges this gap:

interface HBaseTablesProvider {
    fun getHBaseTables(namespace: String, name: String): Mono<HBaseTables>
}

Implemented by: HBaseStorageBackend, MockHBaseStorageBackend, HBaseTestingStorageBackend

Changes summary

Action Files Description
NEW 1 HBaseTablesProvider interface
MOD (v3) 5 Add HBaseTablesProvider impl + getAdminMono() to backends, widen DatastoreUri regex
MOD (v2) 6 Switch Graph, HBaseOptions, DatastoreHashLabel, DatastoreIndexedLabel, HBaseTestingClusterExtension, DefaultHBaseClusterTest to v3 imports
MOD (server) 1 Switch HBaseDatastoreBindingConfiguration to v3 imports
DELETE 17 v2-local duplicate src (10) + test (7) files

Files to delete

StorageBackend.kt, StorageBucket.kt, StorageBuckets.kt, DatastoreUri.kt, DefaultStorageBackendFactory.kt, HBaseStorageBackend.kt, HBaseStorageBucket.kt, MockHBaseStorageBackend.kt, MemoryStorageBackend.kt, MemoryStorageBucket.kt + 7 test files

…StorageBackend

Delete v2-local duplicate storage classes (StorageBackend, StorageBucket,
StorageBuckets, DatastoreUri, DefaultStorageBackendFactory, HBaseStorageBackend,
HBaseStorageBucket, MockHBaseStorageBackend, MemoryStorageBackend,
MemoryStorageBucket) and their tests, totaling ~1575 lines removed.

Add HBaseTablesProvider interface to bridge v2 Labels (which need
Mono<HBaseTables> for Filters/CellUtil) to v3 StorageBackend. Implement it
in HBaseStorageBackend, MockHBaseStorageBackend, and HBaseTestingStorageBackend.

Wire DatastoreHashLabel, DatastoreIndexedLabel, HBaseOptions, and Graph to
use v3 DefaultStorageBackendFactory + HBaseTablesProvider instead of the
deleted v2 classes. Relax v3 DatastoreUri regex to accept uppercase for
backward compatibility with existing data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:XXL This PR changes 1000+ lines, ignoring generated files. labels Feb 9, 2026
em3s and others added 3 commits February 9, 2026 17:17
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… for clarity and consistency

Aligns v2 Labels with v3 StorageBackend terminology. Adds @deprecated to HBaseTablesProvider for backward compatibility.
@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Feb 9, 2026
em3s and others added 3 commits February 9, 2026 17:59
…egation

Rename StorageTable.get(List) → getAll and StorageTable.batch(List) →
batchAll to avoid JVM type-erasure conflicts with HBaseTable methods.
HBaseStorageTable now implements both StorageTable and HBaseTable by table,
keeping all v2 HBaseTable call sites unchanged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ch for consistency

Align method names across StorageTable implementations to improve clarity and eliminate unnecessary verbosity. Update corresponding test cases to reflect the changes.
…atchAll

Resolve JVM type-erasure conflicts so HBaseStorageTable can implement
both StorageTable and HBaseTable by delegation. StorageTable method
names (get/batch) remain unchanged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@em3s em3s changed the title refactor(engine): replace DefaultHBaseCluster with DefaultStorageBackendFactory Replace DefaultHBaseCluster with DefaultStorageBackendFactory Feb 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant