Skip to content

Latest commit

 

History

History
443 lines (364 loc) · 12.3 KB

File metadata and controls

443 lines (364 loc) · 12.3 KB

Comprehensive Testing Plan for Forge Framework

This document outlines the testing strategy to ensure developer confidence and production reliability for the Forge microservice framework.

Current Testing Status

Test Coverage: 70.0% overall (target met)

Package Coverage
errors 100%
health 96.5%
prometheus 76.4%
httpclient 75.9%
config 73.8%
testutil 71.1%
framework 70.0%
configloader 66.7%
jwt 62.0%
redis 36.0%
postgresql 30.6%

Priority: Continue expanding integration test coverage for redis and postgresql bundles.

Testing Strategy Overview

1. Unit Testing (Priority: IMMEDIATE)

Target: 85%+ code coverage across all packages Scope: Individual functions, methods, and components in isolation

Framework Core (framework/)

  • app.go: Application lifecycle, component management, shutdown orchestration
  • http.go: HTTP server builder, middleware stack, security controls
  • logging.go: Logging manager, health logger adapter
  • observability.go: OpenTelemetry integration, metrics setup
  • shutdown.go: Graceful shutdown orchestration, signal handling

Configuration (config/)

  • base.go: Configuration validation, environment detection, security checks

Health Checks (health/)

  • check.go: Health check interfaces, basic implementations
  • registry.go: Concurrent health check execution, timeout handling
  • status.go: Health status aggregation, HTTP response formatting

Error Handling (errors/)

  • errors.go: Domain error patterns, error classification functions

2. Bundle Testing (Priority: IMMEDIATE)

Each bundle requires comprehensive testing with real dependencies:

PostgreSQL Bundle

// Test categories:
- Connection pool management and lifecycle
- Health check accuracy and timeout handling
- Configuration validation edge cases
- Error handling and recovery scenarios
- Connection cleanup and resource management

Redis Bundle

// Test categories:
- Cache operations with TTL validation
- Pub/sub subscription management and cleanup
- Distributed locking race conditions
- Rate limiting algorithm correctness
- Connection pool health and monitoring

JWT Bundle

// Test categories:
- Token generation and validation security
- gRPC interceptor authentication flow
- HTTP middleware path protection
- Token caching and expiry handling
- Service identity and permission validation

HTTP Client Bundle

// Test categories:
- Circuit breaker state transitions
- Retry logic with exponential backoff
- Authentication integration correctness
- TLS security configuration
- Request/response logging security

Prometheus Bundle

// Test categories:
- Metric registration and collection accuracy
- Thread safety with concurrent metric updates
- Custom metric validation and cardinality limits
- Health check metric integration
- Bundle metric integration with other components

Configuration Loader Bundle

// Test categories:
- Multi-source configuration loading priority
- Environment variable binding and validation
- File security validation (path traversal, permissions)
- Hot reload race condition handling
- Sensitive data redaction accuracy

3. Integration Testing (Priority: HIGH)

Real Infrastructure Testing:

Database Integration

# Test with real PostgreSQL
docker-compose up -d postgres
go test -tags=integration ./bundles/postgresql/...

# Test scenarios:
- Connection pool exhaustion and recovery
- Database connectivity failures and reconnection
- Health check accuracy under load
- Graceful shutdown with active connections

Redis Integration

# Test with real Redis
docker-compose up -d redis
go test -tags=integration ./bundles/redis/...

# Test scenarios:
- Cache operations under high concurrency
- Pub/sub message delivery guarantees
- Distributed lock correctness under contention
- Rate limiter accuracy under load

Service-to-Service Communication

# Test JWT authentication between services
go test -tags=integration ./bundles/jwt/...

# Test HTTP client resilience
go test -tags=integration ./bundles/httpclient/...

# Test scenarios:
- Multi-service authentication flow
- Circuit breaker behavior with real service failures
- Token propagation and validation
- Network failure handling and recovery

4. End-to-End Testing (Priority: HIGH)

Complete Application Testing:

Scenario 1: Multi-Bundle Service

// Test complete service with all bundles
- PostgreSQL + Redis + JWT + HTTP Client + Prometheus + Config
- Verify all integrations work together
- Test startup/shutdown with all bundles
- Verify metrics collection from all sources

Scenario 2: Production Deployment Simulation

# Test production-like scenarios
- Environment variable configuration
- TLS certificate validation
- Load balancer health checks
- Graceful shutdown under load
- Configuration hot reload safety

Scenario 3: Failure Recovery

# Test resilience patterns
- Database connection failures and recovery
- Redis connectivity issues
- External service failures (circuit breaker)
- Authentication service outages
- Partial service degradation

5. Security Testing (Priority: CRITICAL)

Authentication Security

// JWT bundle security tests:
- Token forgery attempts
- Token replay attacks
- Path traversal in skip paths
- Timing attacks on token validation
- Context isolation between services

Configuration Security

// Config loader security tests:
- Path traversal attacks
- File permission bypass attempts
- Sensitive data leak detection
- Environment variable injection
- Configuration file size limits

HTTP Security

// HTTP bundle security tests:
- CORS security validation
- TLS configuration enforcement
- Credential exposure prevention
- Request size limits
- Authentication bypass attempts

6. Performance Testing (Priority: HIGH)

Load Testing Targets

Framework Core:

  • Application startup time: < 5 seconds
  • Shutdown time: < 30 seconds
  • Memory usage: < 100MB base
  • Health check response: < 100ms

Bundle Performance:

  • Database operations: < 50ms p99
  • Redis operations: < 10ms p99
  • JWT validation: < 5ms p99
  • HTTP client requests: < 200ms p99
  • Metrics collection: < 1ms per metric

Stress Testing

# Load testing scenarios:
- 10,000 concurrent HTTP requests
- 1,000 concurrent gRPC calls
- 100 health check calls per second
- 1,000 metric updates per second
- Database connection pool exhaustion

7. Test Infrastructure

Testing Dependencies

// Add testing dependencies
go get github.com/stretchr/testify
go get github.com/golang/mock
go get github.com/testcontainers/testcontainers-go
go get github.com/ory/dockertest/v3

Test Utilities Package

// Package testutil for common testing patterns
package testutil

// Database testing
func SetupTestDB(t *testing.T) *sql.DB
func CleanupTestDB(t *testing.T, db *sql.DB)

// Redis testing
func SetupTestRedis(t *testing.T) redis.UniversalClient
func CleanupTestRedis(t *testing.T, client redis.UniversalClient)

// HTTP testing
func SetupTestServer(t *testing.T, handler http.Handler) *httptest.Server
func SetupTestApp(t *testing.T, options ...framework.AppOption) *framework.App

// Security testing
func GenerateTestJWT(t *testing.T, claims jwt.ServiceClaims) string
func SetupTLSTestServer(t *testing.T) *httptest.Server

8. CI/CD Testing Pipeline

GitHub Actions Workflow

# .github/workflows/test.yml
name: Comprehensive Tests

on: [push, pull_request]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version: '1.25'
      - run: go test -race -coverprofile=coverage.out ./...
      - run: go tool cover -html=coverage.out -o coverage.html

  integration-tests:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: test
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
      redis:
        image: redis:7
        options: >-
          --health-cmd "redis-cli ping"
          --health-interval 10s
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
      - run: go test -tags=integration ./...

  security-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: go install github.com/securecodewarrior/gosec/v2/cmd/gosec@latest
      - run: gosec ./...
      - run: go mod download
      - run: go list -json -m all | nancy sleuth

  performance-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: go test -bench=. -benchmem ./...

9. Test Coverage Targets

Minimum Coverage Requirements

  • Framework Core: 90% coverage
  • Bundles: 85% coverage
  • Health Checks: 95% coverage
  • Configuration: 90% coverage
  • Error Handling: 95% coverage

Coverage Exclusions

  • Example services (covered by E2E tests)
  • Generated code
  • Vendor dependencies

10. Testing Phases Implementation

Phase 1: Foundation (Week 1)

  • Set up testing infrastructure and CI/CD
  • Create test utilities package
  • Implement framework core unit tests
  • Add configuration and health check tests

Phase 2: Bundle Testing (Week 2-3)

  • PostgreSQL bundle comprehensive tests
  • Redis bundle comprehensive tests
  • JWT bundle security tests
  • HTTP client bundle resilience tests

Phase 3: Integration & Security (Week 4)

  • Multi-bundle integration tests
  • Security penetration testing
  • Performance benchmarking
  • End-to-end scenario testing

Phase 4: Production Readiness (Week 5)

  • Load testing with realistic scenarios
  • Chaos engineering tests
  • Documentation and examples testing
  • Final security audit

11. Testing Standards and Guidelines

Test Organization

package/
├── bundle.go           # Implementation
├── bundle_test.go      # Unit tests
├── integration_test.go # Integration tests (build tag)
├── benchmark_test.go   # Performance tests
└── testutil/          # Testing utilities

Test Naming Conventions

// Unit tests
func TestBundleName_MethodName_ExpectedBehavior(t *testing.T)
func TestPostgreSQLBundle_Initialize_ValidConfig(t *testing.T)

// Integration tests
func TestIntegration_DatabaseConnectivity(t *testing.T)

// Benchmarks
func BenchmarkMetricsCollection(b *testing.B)

Test Quality Standards

  • Deterministic: Tests must be reliable and repeatable
  • Isolated: Tests don't depend on external state
  • Fast: Unit tests complete in < 100ms each
  • Clear: Test intent is obvious from name and structure
  • Comprehensive: Edge cases and error scenarios covered

12. Success Metrics

Code Quality Metrics

  • Test Coverage: >85% overall, >90% for critical packages
  • Test Reliability: <1% flaky test rate
  • Performance: All benchmarks within target ranges
  • Security: Zero security vulnerabilities in tests

Developer Confidence Metrics

  • Documentation: All features have working test examples
  • Examples: All example services have integration tests
  • Regression Prevention: Breaking changes caught by tests
  • Contribution Quality: PRs include appropriate tests

Implementation Priority

Immediate (This Week)

  1. Framework Core Tests - Application lifecycle, health checks
  2. PostgreSQL Bundle Tests - Database connectivity patterns
  3. JWT Bundle Security Tests - Authentication security validation

High Priority (Next 2 Weeks)

  1. Integration Test Infrastructure - Docker-based testing
  2. Remaining Bundle Tests - Redis, HTTP Client, Prometheus, Config
  3. CI/CD Pipeline - Automated testing on all changes

Medium Priority (Month 2)

  1. Performance Benchmarking - Load testing and optimization
  2. Security Penetration Testing - Comprehensive security validation
  3. Chaos Engineering - Resilience testing under failure conditions

This comprehensive testing strategy will provide the confidence developers need to adopt Forge for production microservices while ensuring the framework maintains its high quality and security standards.