ApexStore

High-performance, embedded Key-Value engine built with Rust 🦀
Implementing LSM-Tree architecture with a focus on SOLID principles, observability, and performance.

---

🎯 Overview

ApexStore is a modern, Rust-based storage engine designed for write-heavy workloads. It combines the durability of write-ahead logging (WAL) with the efficiency of Log-Structured Merge-Tree (LSM-Tree) architecture.

Built from the ground up using SOLID principles, it provides a production-grade storage solution that is easy to reason about, test, and maintain, while delivering the performance expected from a systems-level language.

⚖️ Why ApexStore?

While industry giants like RocksDB or LevelDB focus on extreme complexity, ApexStore offers:

• Educational Clarity: A clean, modular implementation of LSM-Tree that serves as a blueprint for high-performance systems.
• Strict SOLID Compliance: Leveraging Rust's ownership model to enforce clear boundaries between MemTable, WAL, and SSTable layers.
• Observability First: Built-in real-time metrics for memory, disk usage, and WAL health.
• Modern Defaults: Native LZ4 compression, Bloom Filters, and 35+ tunable parameters via environment variables.

📊 Performance Benchmarks

Run locally: cargo bench --all-features → HTML reports at target/criterion/

🤖 Latest CI Results

🤖 Auto-updated by CI on 2026-05-25 06:44 UTC — View run

No results parsed — check the run artifacts.

📋 YCSB Mixed Workload — mixed_bench

Measured on Intel Core i5-9300H @ 2.40GHz, 16 GB DDR4 2667 MHz, HDD SATA 1TB (v2.1.39) — cargo bench --bench mixed_bench -- --sample-size 10

Throughput (operations/second)

Benchmark	Size	Median	Throughput	Change vs previous
YCSB Type A (50% write / 50% read)	10K	952.83 µs	1.05 Mops/s	no change
YCSB Type A (50% write / 50% read)	100K	2.706 ms	369.6 Kops/s	✅ +49% throughput
YCSB Type B (5% write / 95% read)	10K	814.90 µs	1.23 Mops/s	⚠️ -18.6% throughput
YCSB Type B (5% write / 95% read)	100K	1.409 ms	710.0 Kops/s	⚠️ -20.1% throughput
YCSB Type C (100% read)	10K	334.70 µs	2.99 Mops/s	✅ +9.4% throughput
YCSB Type C (100% read)	100K	745.36 µs	1.34 Mops/s	✅ +12.4% throughput
YCSB Type C (100% read)	1M	1.290 ms	775.0 Kops/s	— (new)

Composite Workloads

Benchmark	Size	Median	Throughput	Change vs previous
Balanced (mixed workload)	10K	1.080 ms	925.9 Kops/s	⚠️ -8.7% throughput
Balanced (mixed workload)	100K	2.831 ms	353.2 Kops/s	— (no baseline)
Read Heavy (read-intensive)	10K	811.91 µs	1.23 Mops/s	✅ +15.7% throughput
Read Heavy (read-intensive)	100K	1.777 ms	562.7 Kops/s	— (no baseline)
Write Heavy (write-intensive)	10K	1.187 ms	82.3 KiB/s	⚠️ -6.7% throughput
Write Heavy (write-intensive)	100K	3.486 ms	28.0 KiB/s	— (no baseline)

Hardware note: results above are conservative — measured on HDD SATA (vs. NVMe). On NVMe, expect 2–4× better throughput for I/O-bound operations.

Key Insights:

• WAL_SYNC_MODE=async provides 16x throughput vs fsync (trade durability for speed)
• Cache hit rate > 80% when block_cache_size_mb > 256
• Bloom filter rejects 99.2% of non-existent key lookups
• Optimal memtable_max_size is 16-32MB for write-heavy workloads

✨ Key Features

🛠️ Storage Engine

• MemTable: In-memory BTreeMap with configurable size limits.
• Write-Ahead Log (WAL): ACID-compliant durability with configurable sync modes.
• SSTable V2: Block-based storage with Sparse Indexing and LZ4 Compression.
• Bloom Filters: Drastically reduces unnecessary disk I/O for read operations.
• Crash Recovery: Automatic WAL replay on startup to ensure zero data loss.

🔌 Access Patterns

• Interactive CLI: REPL interface for development and debugging.
• REST API: Full HTTP API with JSON payloads for microservices.
• Batch Operations: Efficient bulk inserts and updates.
• Search Capabilities: Prefix and substring search (Optimized iterators coming in v2.0).

🏗️ Architecture

The engine follows a modular architecture where each component has a single responsibility:

graph TB
    subgraph "Interface Layer"
        CLI[CLI / REPL]
        API[REST API Server]
    end

    subgraph "Core Domain"
        Engine[LSM Engine]
        MemTable[MemTable<br/>BTreeMap]
        LogRecord[LogRecord<br/>Data Model]
    end

    subgraph "Storage Layer"
        WAL[Write-Ahead Log<br/>Durability]
        SST[SSTable Manager<br/>V2 Format]
        Builder[SSTable Builder<br/>Compression]
    end

    subgraph "Infrastructure"
        Codec[Serialization<br/>Bincode]
        Error[Error Handling]
        Config[Configuration<br/>Environment]
    end

    CLI --> Engine
    API --> Engine
    Engine --> WAL
    Engine --> MemTable
    MemTable -->|Flush| Builder
    Builder --> SST
    Engine -->|Read| MemTable
    Engine -->|Read| SST
    WAL -.->|Recovery| MemTable
    
    Engine --> Config
    SST --> Codec
    Builder --> Codec
    WAL --> Codec

    style Engine fill:#f9a,stroke:#333,stroke-width:3px
    style WAL fill:#9cf,stroke:#333,stroke-width:2px
    style SST fill:#9cf,stroke:#333,stroke-width:2px

Loading

🚀 Quick Start

Prerequisites

• Rust 1.70+: Install via rustup.rs

Installation & Run

# Clone and enter
git clone https://github.com/ElioNeto/ApexStore.git && cd ApexStore

# Build and Start REPL
cargo run --release

# Available commands:
# > put user:1 "John Doe"
# > get user:1
# > stats

🐳 Docker Deployment

Run ApexStore as a standalone API server:

# Start with Docker Compose
docker-compose up -d

# Manual run with custom config
docker run -d \
  --name apexstore-server \
  -p 8080:8080 \
  -e MEMTABLE_MAX_SIZE=33554432 \
  -v apexstore-data:/data \
  elioneto/apexstore:latest

🌐 REST API Examples

Method	Endpoint	Description
POST	/keys	Insert/Update: {"key": "k1", "value": "v1"}
GET	/keys/{key}	Retrieve value
GET	/stats/all	Full telemetry (Memory, Disk, WAL)

📁 Project Structure

ApexStore/
├── src/
│   ├── core/      # LSM Engine, MemTable, Domain logic
│   ├── storage/   # WAL, SSTable V2, Block Builder
│   ├── infra/     # Codec, Error Handling, Config
│   ├── api/       # Actix-Web Server & Handlers
│   └── cli/       # REPL Implementation
├── docs/          # Detailed documentation & Architecture
├── tests/         # Integration test suite
└── Dockerfile     # Multi-stage build

🧪 Testing & Quality

cargo test                  # Run all tests
cargo clippy -- -D warnings # Linting
cargo fmt                   # Formatting

🚀 CI/CD & Development Workflow

ApexStore uses trunk-based development with automated releases:

graph LR
    A[Feature Branch] -->|Open PR| B[CI Validation]
    B -->|✅ Pass| C[Merge to main]
    C --> D[Auto Release]
    D --> E[v2.1.X]

Loading

Development Flow

1. Create feature branch from main
2. Open PR → CI runs cargo fmt, clippy, test, build
3. Merge PR → Auto-increments version in Cargo.toml, creates tag & GitHub release

📖 Read: MIGRATION_GUIDE.md for team workflow
📂 Details: .github/workflows/README.md

🗺️ Roadmap

• SSTable V2 with compression & Bloom Filters
• REST API & Feature Flags
• Global Block Cache
• Trunk-based CI/CD with auto-release
• v2.2: Storage iterators for range queries
• v2.3: Concurrent read optimization
• v3.0: Leveled/Tiered Compaction Strategies

🤝 Contributing

Contributions are what make the open-source community an amazing place! Please check our Contributing Guidelines.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feat/amazing-feature)
3. Commit your Changes (git commit -m 'feat: add amazing feature')
4. Push to the Branch (git push origin feat/amazing-feature)
5. Open a Pull Request to main
6. CI will auto-release on merge 🚀

📄 License

Distributed under the MIT License. See LICENSE for more information.

📧 Contact

Elio Neto - GitHub - netoo.elio@hotmail.com
Demo: lsm-admin-dev.up.railway.app

🌟 Star History

---

Built with 🦀 Rust and ❤️ for high-performance storage systems