SagaIndexer - DeFi Liquidity Monitor

A high-performance microservices system for monitoring Uniswap V3 Concentrated Liquidity Pools on Ethereum, developed as part of an MBA thesis at USP/Esalq.

Architecture Overview

                           ┌─────────────────────────────────────┐
                           │         External Clients            │
                           └─────────────────┬───────────────────┘
                                             │ HTTPS (8080)
                           ┌─────────────────▼───────────────────┐
                           │       KrakenD API Gateway           │
                           │  • JWT Validation                   │
                           │  • Rate Limiting                    │
                           │  • CORS / Security                  │
                           └─────────────────┬───────────────────┘
                                             │
          ┌──────────────┬───────────────────┼───────────────────┬──────────────┐
          │              │                   │                   │              │
          ▼              ▼                   ▼                   ▼              ▼
   ┌─────────────┐ ┌───────────┐     ┌─────────────┐     ┌─────────────┐ ┌─────────────┐
   │    Auth     │ │  Tracker  │     │  Analytics  │     │    Saga     │ │    Redis    │
   │  (HTTP)     │ │  (gRPC)   │     │   (gRPC)    │     │Orchestrator │ │   (Cache)   │
   │   :8081     │ │  :50051   │     │   :50052    │     │             │ │    :6379    │
   └──────┬──────┘ └─────┬─────┘     └──────┬──────┘     └─────────────┘ └─────────────┘
          │              │                  │
          ▼              ▼                  │
   ┌─────────────┐ ┌─────────────┐          │          ┌─────────────────────────────┐
   │ PostgreSQL  │ │   SQLite    │◄─────────┘          │      Monitoring Stack       │
   │   (Auth)    │ │  (Events)   │                     │  Prometheus :9090           │
   └─────────────┘ └─────────────┘                     │  Grafana    :3000           │
                                                       └─────────────────────────────┘

Services

1. Tracker Service (Indexer)

Subscribes to Uniswap V3 pool events on Ethereum.

Features:

• Swap, Mint, Burn event decoding
• Checkpoint-based resumption (12-block intervals)
• Multi-pool support
• gRPC API for event queries

Tech: go-ethereum, SQLite, gRPC

2. Analytics Service

Computes on-chain metrics from indexed events.

Features:

• TWAP: Time-Weighted Average Price calculation
• Liquidity Distribution: Tick-range liquidity aggregation
• Pool Stats: Volume, fees, event counts

Tech: gRPC, Redis (caching)

3. Auth Service

Handles user authentication and JWT tokens.

Features:

• User registration with bcrypt password hashing
• JWT access tokens (HS256)
• Session management with refresh tokens

Tech: Go-Fiber, JWT

4. Saga Orchestrator

Manages distributed transactions with fault tolerance.

Features:

• Circuit Breaker: Prevents cascade failures
• Saga Pattern: Compensating transactions on failure
• Retry Logic: Exponential backoff

5. API Gateway (KrakenD)

Single entry point for all external requests.

Features:

• JWT validation
• Rate limiting (per IP/client)
• CORS configuration
• Request routing

Tech Stack

Layer	Technology
Language	Go 1.21+
API Gateway	KrakenD 2.5
Internal Communication	gRPC + Protocol Buffers
External API	REST (via gateway)
Databases	PostgreSQL (auth), SQLite (events), Redis (cache)
Authentication	JWT (HS256)
Security	mTLS (internal), Zero-Trust
Monitoring	Prometheus, Grafana
Infrastructure	Docker, Docker Compose

Quick Start

Prerequisites

• Go 1.21+
• Docker & Docker Compose
• OpenSSL (for certificate generation)

1. Clone and Setup

git clone <repository>
cd sagaindexer

# Generate mTLS certificates
cd gateway/certs
./generate.sh
cd ../..

2. Configure Environment

cp .env.example .env
# Edit .env with your values:
# - RPC_URL: Ethereum RPC endpoint (e.g., Infura)
# - POOL_ADDRESSES: Comma-separated pool addresses
# - JWT_SECRET: 32+ character secret

3. Run Tests

go test ./... -v

4. Start Services

docker-compose up --build

5. Access API

# Health check
curl http://localhost:8080/health

# Register user
curl -X POST http://localhost:8080/api/v1/auth/register \
  -H "Content-Type: application/json" \
  -d '{"email":"user@example.com","password":"securepass123"}'

# Login
curl -X POST http://localhost:8080/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email":"user@example.com","password":"securepass123"}'

# Get pool stats (with JWT)
curl http://localhost:8080/api/v1/pools/0x8ad599c3.../stats \
  -H "Authorization: Bearer <token>"

API Endpoints

Authentication (Public)

Method	Endpoint	Description
POST	/api/v1/auth/register	Register new user
POST	/api/v1/auth/login	Login, receive JWT
POST	/api/v1/auth/refresh	Refresh access token
POST	/api/v1/auth/logout	Invalidate session

Analytics (Protected)

Method	Endpoint	Description
GET	/api/v1/pools/{pool}/twap	Get TWAP for pool
GET	/api/v1/pools/{pool}/liquidity	Get liquidity distribution
GET	/api/v1/pools/{pool}/stats	Get pool statistics

Tracker (Protected)

Method	Endpoint	Description
GET	/api/v1/pools/{pool}/events	Get pool events
GET	/api/v1/checkpoints	Get indexing progress

Project Structure

sagaindexer/
├── gen/proto/                    # Generated protobuf code
│   ├── analytics/v1/
│   ├── auth/v1/
│   └── tracker/v1/
├── shared/proto/                 # Proto definitions
│   ├── analytics/v1/analytics.proto
│   ├── auth/v1/auth.proto
│   └── tracker/v1/tracker.proto
├── gateway/                      # API Gateway
│   ├── config/krakend.json
│   ├── certs/                    # mTLS certificates
│   └── Dockerfile
├── services/
│   ├── auth/                     # Auth Service
│   │   ├── cmd/main.go
│   │   └── internal/
│   │       ├── config/
│   │       ├── domain/
│   │       ├── handler/
│   │       ├── jwt/
│   │       └── repository/
│   ├── tracker/                  # Tracker Service
│   │   ├── cmd/main.go
│   │   └── internal/
│   │       ├── config/
│   │       ├── domain/
│   │       ├── grpc/
│   │       ├── http/
│   │       ├── repository/
│   │       └── subscriber/
│   ├── analytics/                # Analytics Service
│   │   ├── cmd/main.go
│   │   └── internal/
│   │       ├── calculator/
│   │       ├── config/
│   │       ├── grpc/
│   │       └── http/
│   └── saga/                     # Saga Orchestrator
│       ├── cmd/main.go
│       └── internal/
│           ├── circuitbreaker/
│           ├── orchestrator/
│           └── repository/
├── docker-compose.yml
├── go.mod
├── go.sum
├── Makefile
└── README.md

Testing

# Run all tests
go test ./... -v

# Run with coverage
go test ./... -cover

# Run specific service tests
go test ./services/tracker/... -v
go test ./services/analytics/... -v
go test ./services/auth/... -v
go test ./services/saga/... -v

Test Coverage: 236 tests across all services

Development

Make Commands

make proto       # Generate protobuf code
make test        # Run all tests
make test-cover  # Run tests with coverage
make build       # Build all service binaries
make certs       # Generate mTLS certificates
make docker-build  # Build Docker images
make docker-up     # Start all services
make docker-down   # Stop all services
make docker-logs   # Follow service logs
make clean       # Remove build artifacts

Generate Proto Files

make proto

Add New Pool to Track

Update POOL_ADDRESSES in .env:

POOL_ADDRESSES=0x8ad599c3...,0x88e6A0c2...

Circuit Breaker Configuration

cb := circuitbreaker.New("service-name", circuitbreaker.Config{
    FailureThreshold: 5,        // Opens after 5 failures
    ResetTimeout:     30*time.Second,  // Half-open after 30s
    HalfOpenMaxRequest: 1,      // Allow 1 request in half-open
})

Security

Zero-Trust Architecture

• All internal services communicate via gRPC with mTLS
• Only the API Gateway is exposed externally
• JWT validation on every protected endpoint
• Rate limiting prevents abuse

Certificate Generation

cd gateway/certs
./generate.sh

Generates:

• ca.crt/ca.key - Certificate Authority
• gateway.crt/gateway.key - Gateway certificate
• tracker.crt/tracker.key - Tracker certificate
• analytics.crt/analytics.key - Analytics certificate
• auth.crt/auth.key - Auth certificate

Monitoring

Prometheus + Grafana

The project includes built-in observability:

# Start all services including monitoring
make docker-up

# Access dashboards
# Prometheus: http://localhost:9090
# Grafana: http://localhost:3000 (admin/sagaindexer)

Available Metrics

Metric	Description
http_requests_total	Total HTTP requests by service/method/status
http_request_duration_seconds	Request latency histogram
grpc_requests_total	Total gRPC requests
tracker_events_processed_total	Blockchain events processed
tracker_blocks_indexed	Latest block indexed per pool
circuit_breaker_state	Circuit breaker state (0=closed, 1=open, 2=half-open)
saga_executions_total	Saga executions by status

Metrics Endpoints

Each service exposes /metrics for Prometheus scraping:

• Auth: http://auth:8081/metrics
• Tracker HTTP: http://tracker-http:8083/metrics
• Analytics HTTP: http://analytics-http:8082/metrics

Academic Context

This project was developed as part of an MBA thesis at USP/Esalq (Universidade de São Paulo - Escola Superior de Agricultura Luiz de Queiroz).

Research Topics

• Microservices Architecture
• Domain-Driven Design (DDD)
• Event Sourcing
• Circuit Breaker Pattern
• Saga Pattern for Distributed Transactions
• gRPC vs REST Performance
• Zero-Trust Security Model

License

MIT License - See LICENSE for details.

Contributing

1. Fork the repository
2. Create a feature branch
3. Write tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

---

Author: Developed for USP/Esalq MBA Program Year: 2025