Skip to content

PLAN.md

Latest commit

 

History

History
210 lines (163 loc) · 6.98 KB

PLAN.md

File metadata and controls

210 lines (163 loc) · 6.98 KB

RustStack Implementation Plan

Status: MVP Complete ✅

RustStack has achieved its MVP milestone. This document tracks what's been implemented and the roadmap for future work.

Completed (MVP)

✅ Phase 1: Project Setup & S3

Project Foundation:

  • Workspace structure with 7 crates
  • Core types, error handling, request ID generation
  • GitHub Actions CI/CD (format, clippy, test, integration)
  • Release workflow for Linux/macOS binaries

S3 Service:

  • ObjectStorage trait with pluggable backends
  • EphemeralStorage (in-memory with DashMap)
  • Full bucket operations (create, delete, list, head)
  • Full object operations (get, put, delete, head, copy)
  • ListObjectsV2 with prefix, delimiter, pagination
  • Multipart upload (create, upload part, complete, abort)
  • Correct ETag computation (MD5, multipart format)
  • XML error responses with proper AWS error codes
  • Metadata support (content-type, user metadata, etc.)
  • 60 unit tests

✅ Phase 2: DynamoDB

Native Rust Implementation (not DynamoDB Local proxy):

  • In-memory table storage
  • Table operations (create, delete, describe, list)
  • Item CRUD (GetItem, PutItem, DeleteItem, UpdateItem)
  • Query with KeyConditionExpression
  • Scan with FilterExpression
  • BatchGetItem, BatchWriteItem
  • Global Secondary Index (GSI) support
  • Local Secondary Index (LSI) support
  • Full expression parser and evaluator:
    • KeyConditionExpression
    • FilterExpression
    • UpdateExpression (SET, REMOVE, ADD, DELETE)
    • ConditionExpression
    • ProjectionExpression
  • Correct error codes (ResourceNotFoundException, ConditionalCheckFailedException, ValidationException)
  • 136 unit tests (74 storage + 62 expression)

✅ Phase 3: Lambda

Function Management:

  • CreateFunction (zip upload, handler config)
  • GetFunction, DeleteFunction, ListFunctions
  • UpdateFunctionCode, UpdateFunctionConfiguration
  • Environment variable support
  • Python runtime support (3.9, 3.10, 3.11, 3.12)

Invocation:

  • Synchronous invoke (RequestResponse)
  • Async invoke (Event)
  • Python subprocess execution
  • API Gateway v2 event format
  • Log capture and retrieval (LogType: Tail)
  • Timeout handling

✅ Phase 4: Integration & Polish

CloudWatch Logs:

  • CreateLogGroup, CreateLogStream
  • DescribeLogGroups, DescribeLogStreams
  • PutLogEvents, GetLogEvents
  • Lambda log storage and retrieval

HTTP Gateway:

  • Single server on port 4566
  • Service routing by headers/path
  • Health endpoints (/health, /_localstack/health)
  • x-amzn-requestid headers
  • Proper content-type handling (XML/JSON)

Testing:

  • 220+ tests across all crates
  • Integration tests with boto3
  • CI/CD with GitHub Actions

Documentation:

  • README with quickstart
  • pytest fixture examples
  • API compatibility matrix
  • Docker instructions

Project Stats

Metric Value
Lines of Rust ~17,500
Test count 240+
Crates 11
CI status ✅ Green

Future Roadmap

Phase 5: Secrets Manager & IAM ✅

  • Secrets Manager (CreateSecret, GetSecretValue, PutSecretValue, DeleteSecret, DescribeSecret, ListSecrets)
  • Secret versioning (AWSCURRENT, AWSPREVIOUS)
  • IAM roles (CreateRole, GetRole, DeleteRole, ListRoles)
  • IAM policies (CreatePolicy, GetPolicy, DeletePolicy)
  • Role-policy attachment (AttachRolePolicy, DetachRolePolicy, ListAttachedRolePolicies)

Phase 6: API Gateway & Firehose ✅

  • API Gateway V2 HTTP APIs (CreateApi, GetApi, DeleteApi, ListApis)
  • Routes (CreateRoute, GetRoute, DeleteRoute, ListRoutes)
  • Integrations (CreateIntegration, GetIntegration, DeleteIntegration, ListIntegrations)
  • Stages (CreateStage, GetStage, DeleteStage, ListStages)
  • Kinesis Firehose delivery streams (CreateDeliveryStream, DeleteDeliveryStream, DescribeDeliveryStream, ListDeliveryStreams)
  • PutRecord, PutRecordBatch

Phase 7: Persistence

  • File-system storage backend for S3
  • SQLite storage backend for DynamoDB
  • --data-dir CLI option
  • State recovery on restart

Phase 8: Enhanced Lambda

  • Docker container execution (alternative to subprocess)
  • Node.js runtime support
  • Lambda layers
  • Provisioned concurrency simulation

Phase 9: Additional Services

  • SQS (queues, messages)
  • SNS (topics, subscriptions)

Phase 10: Performance & Production

  • Benchmarks vs LocalStack
  • Memory optimization
  • Connection pooling
  • Graceful shutdown
  • Metrics endpoint

Architecture

┌────────────────────────────────────────────────────────────────────────────┐
│                         HTTP Gateway (Axum)                                 │
│                            Port 4566                                        │
│  Routes: /health, S3, DynamoDB, Lambda, Logs, Secrets, IAM, APIGW, Firehose│
└──┬────┬────┬────┬────┬─────────┬───┬────────┬─────────────────────────────┘
   │    │    │    │    │         │   │        │
┌──▼─┐┌─▼──┐┌▼───┐┌▼──┐┌▼──────┐┌▼─┐┌▼─────┐┌─▼──────┐
│ S3 ││DDB ││λ   ││Log││Secrets││IAM││APIGW ││Firehose│
└────┘└────┘└────┘└───┘└───────┘└───┘└──────┘└────────┘

Crate Structure

ruststack/
├── ruststack/              # Main binary, HTTP routing
├── ruststack-core/         # Shared types, errors, request IDs
├── ruststack-auth/         # SigV4 verification (scaffolded)
├── ruststack-s3/           # S3 service + storage backends
├── ruststack-dynamodb/     # DynamoDB service + expression parser
├── ruststack-lambda/       # Lambda service + invocation
├── ruststack-secretsmanager/ # Secrets Manager service
├── ruststack-iam/          # IAM roles & policies (stub)
├── ruststack-apigateway/   # API Gateway V2 HTTP APIs
├── ruststack-firehose/     # Kinesis Firehose delivery streams
└── tests/                  # Integration tests

Contributing

  1. Check the roadmap above for what's next
  2. Read ARCHITECTURE.md for design details
  3. Run cargo test --workspace before submitting PRs
  4. Ensure cargo clippy -- -D warnings passes

Release Process

# Tag a release
git tag v0.1.0
git push --tags

# GitHub Actions will:
# 1. Build binaries for Linux x86_64, macOS x86_64, macOS arm64
# 2. Create GitHub release with artifacts
# 3. Generate checksums