Module 4.md

Module 4: Golang API Gateway

What You'll Build

By the end of this module, you'll have:

✅ High-performance API Gateway written in Go (~17MB vs ~900MB Python)
✅ Production-ready reverse proxy with request forwarding
✅ Complete observability stack (structured logging + Prometheus metrics)
✅ Request ID tracking for distributed tracing
✅ Graceful shutdown handling for zero-downtime deployments
✅ Backend health monitoring and automatic failover
✅ Polyglot microservices architecture (Go + Python)

Real-World Impact:

67% reduction in memory usage vs Python-only architecture
Sub-millisecond request routing overhead
10x faster startup times for rapid scaling
Professional-grade observability for production monitoring
Separation of concerns: Go for I/O, Python for ML

Learning Objectives

By the end of this module, you will:

✅ Build a production-ready API Gateway in Go
✅ Understand polyglot architecture (Go + Python)
✅ Implement structured logging and metrics
✅ Deploy Go services to Kubernetes
✅ Monitor multi-language microservices

Part 1: Setup & Prerequisites

Why Go for the Gateway?

Aspect	Go Gateway	Python Alternative
Image Size	~17MB	~900MB
Memory Usage	~64MB	~200MB
Startup Time	~1 second	~5-10 seconds
CPU Efficiency	High (compiled)	Lower (interpreted)
Best For	Routing, I/O	ML, Data Science

Key Insight: Use the right tool for each job!

Go: Lightweight gateway/API layer
Python: ML inference where libraries matter

Prerequisites

Completed Module 3 (Kubernetes deployment)
Go 1.21+ installed
Docker and kind running
kubectl configured

Install Go

# macOS
brew install go

# Verify
go version

Part 2: Hands-On Exercise

Quick Start

1. Complete the Exercise

API Gateway

Goal: Build a complete API Gateway in Go with all essential features.

cd modules/module-4/starter

# Open the file
gateway.go 

# Find and fill in 20 TODOs
# Look for: // YOUR CODE HERE

# Download dependencies
go mod download

# Build and test locally (ensure ML service is running)
export BACKEND_URL=http://localhost:3000
go run gateway.go

# In another terminal, test it
curl http://localhost:8080/health
curl http://localhost:8080/metrics  # Prometheus metrics
curl -X POST http://localhost:8080/predict \
     -H "Content-Type: application/json" \
     -d '{"text":"Go is fast!"}'

# Validate your work
./tests/test_gateway.sh

Key TODOs (20 total):

Configuration & Metrics (3 TODOs):

Complete configuration loading
HTTP request counter metric
HTTP request duration metric

Middleware (3 TODOs): 4. Request ID middleware 5. Logging middleware completion 6. Metrics recording

Handlers (4 TODOs): 7. Create handler and register routes 8. Validate HTTP method in health handler 9. Backend health check request 10. Backend health status in response

Proxy Logic (6 TODOs): 11. Read request body 12. Create backend request 13. Copy headers to backend 14. Send backend request with metrics 15. Copy response headers 16. Copy status code and body

Server Setup (4 TODOs): 17. Setup structured logging 18. Create HTTP server 19. Start server in goroutine 20. Graceful shutdown

2. Build Docker Image

# Return to module-4 root
cd ..

# Build Docker image
docker build -t api-gateway:v1 .

# Load into kind
kind load docker-image api-gateway:v1 --name mlops-workshop

3. Deploy to Kubernetes

# Ensure ML service is running
kubectl get svc sentiment-api-service

# Deploy gateway
kubectl apply -f deployment.yaml

# Check deployment
kubectl get all -l app=api-gateway

# Port-forward to test
kubectl port-forward svc/api-gateway-service 8080:80

# Test
curl http://localhost:8080/health
curl -X POST http://localhost:8080/predict \
     -H "Content-Type: application/json" \
     -d '{"text":"Kubernetes + Go!"}'

4. Validate All Work

cd starter
./tests/test_gateway.sh

# If all pass:
🎉 ALL TESTS PASSED!

Part 3: Architecture & Concepts

Key Concepts Covered

Go Fundamentals

HTTP Server: Creating and configuring http.Server
Handlers: Request handling functions
Routing: ServeMux for URL pattern matching
Context: Request-scoped values

Reverse Proxy Pattern

Request Forwarding: Proxying requests to backend
Header Propagation: Copying headers between requests
Timeout Handling: Setting client and server timeouts
Error Handling: Graceful error responses

Production Features

Structured Logging: JSON logs with log/slog
Metrics: Prometheus instrumentation
Request Tracking: Correlation IDs (X-Request-ID)
Middleware: Composable request processing
Graceful Shutdown: Clean termination on SIGTERM

Observability

Logging: Structured JSON logs for parsing
Metrics: HTTP request counters and duration histograms
Tracing: Request ID propagation
Health Checks: Backend availability monitoring

Polyglot Architecture Benefits

Resource Efficiency

Without Gateway (Python only):

Python Service (10 replicas × 1Gi = 10Gi memory)
Cost: High

With Gateway (Go + Python):

Go Gateway (5 replicas × 64Mi = 320Mi)
    ↓
Python ML Service (3 replicas × 1Gi = 3Gi)

Total: 3.3Gi vs 10Gi = 67% reduction!

Separation of Concerns

Layer	Language	Responsibility	Scales Based On
Gateway	Go	Routing, auth	Request count
ML Service	Python	Inference	CPU usage

Common Commands

# Local development
go mod download                # Download dependencies
go run gateway.go              # Run locally
go build -o gateway gateway.go # Build binary

# Docker
docker build -t api-gateway:v1 .
kind load docker-image api-gateway:v1 --name mlops-workshop

# Kubernetes
kubectl apply -f deployment.yaml
kubectl get pods -l app=api-gateway
kubectl logs -l app=api-gateway -f
kubectl port-forward svc/api-gateway-service 8080:80

Part 4: Troubleshooting

Troubleshooting

Issue 1: Build fails - Missing dependencies

Symptoms:

go build gateway.go
# Error: package prometheus is not in GOROOT

Root Cause: Dependencies not downloaded

Solutions:

Step 1: Download dependencies

# Ensure you're in the starter directory
cd modules/module-4/starter

# Download all dependencies
go mod download

# Verify modules
go mod verify

# Tidy up (remove unused)
go mod tidy

Step 2: Check Go version

go version
# Should be >= 1.21

# If too old, upgrade Go
brew upgrade go

Step 3: Clear cache if corrupted

# Clean mod cache
go clean -modcache

# Re-download
go mod download

Issue 2: Build fails - Syntax errors

Symptoms:

gateway.go:45:2: syntax error: unexpected }

Root Cause: Incomplete TODO or syntax mistake

Solutions:

Check compilation:

# Build to see all errors
go build gateway.go

# Run with verbose errors
go build -v gateway.go

Use go fmt:

# Auto-format code (helps catch syntax issues)
go fmt gateway.go

# Check for common mistakes
go vet gateway.go

Issue 3: Cannot connect to backend service

Symptoms:

Error: dial tcp: lookup sentiment-api-service: no such host

Root Causes:

Backend service not running
Wrong service name or namespace
DNS resolution issues

Solutions:

Check 1: Verify backend is running

# Check if service exists
kubectl get svc sentiment-api-service

# Check if pods are ready
kubectl get pods -l app=sentiment-api

# If not running, deploy it first (Module 3)
cd ../module-3
kubectl apply -f deployment.yaml

Check 2: Test backend connectivity

# Port-forward backend
kubectl port-forward svc/sentiment-api-service 3000:80

# Test in another terminal
curl http://localhost:3000/health

Check 3: Verify service name in gateway

# Check BACKEND_URL environment variable
kubectl get deployment api-gateway -o yaml | grep BACKEND_URL

# Should be: http://sentiment-api-service:80

Check 4: Check gateway logs

# View logs
kubectl logs -l app=api-gateway --tail=50

# Look for connection errors
kubectl logs -l app=api-gateway | grep -i "error\|fail"

Issue 4: Gateway responds with 502 Bad Gateway

Symptoms:

curl http://localhost:8080/predict
# HTTP 502 Bad Gateway

Root Cause: Backend service is down or returning errors

Solutions:

Step 1: Check backend health

# Direct backend test
kubectl port-forward svc/sentiment-api-service 3000:80
curl http://localhost:3000/health

Step 2: Check gateway logs

# View detailed logs
kubectl logs -l app=api-gateway -f

# Look for proxy errors
kubectl logs -l app=api-gateway | grep proxy

Step 3: Verify backend response

# Exec into gateway pod
kubectl exec -it <gateway-pod> -- sh

# Test from inside pod
wget -qO- http://sentiment-api-service:80/health

Still stuck? Check the solution file

Part 5: Reference

Commands Cheat Sheet

Quick Start

# Navigate to module
cd modules/module-4/starter

# Download dependencies
go mod download

# Run locally (ensure backend is running)
export BACKEND_URL=http://localhost:3000
go run gateway.go

# In another terminal, test
curl http://localhost:8080/health

Docker Commands

# Build Docker image
docker build -t api-gateway:v1 .

# Build with specific tag
docker build -t api-gateway:v1.0.0 .

# Run locally
docker run -p 8080:8080 \
  -e BACKEND_URL=http://host.docker.internal:3000 \
  api-gateway:v1

# Check image size
docker images api-gateway:v1

# Inspect layers
docker history api-gateway:v1

# Load into kind
kind load docker-image api-gateway:v1 --name mlops-workshop

Kubernetes Commands

# Apply deployment
kubectl apply -f deployment.yaml

# Get all resources
kubectl get all -l app=api-gateway

# Get pods with details
kubectl get pods -l app=api-gateway -o wide

# Describe deployment
kubectl describe deployment api-gateway

# View logs
kubectl logs -l app=api-gateway

# Follow logs
kubectl logs -l app=api-gateway -f

# View logs from specific container
kubectl logs <pod-name> -c api-gateway

# Exec into pod
kubectl exec -it <pod-name> -- sh

# Port-forward
kubectl port-forward svc/api-gateway-service 8080:80

# Scale manually
kubectl scale deployment api-gateway --replicas=3

# Restart deployment
kubectl rollout restart deployment api-gateway

# Check rollout status
kubectl rollout status deployment api-gateway

# View rollout history
kubectl rollout history deployment api-gateway

Debugging Commands

# Check if gateway is running
ps aux | grep gateway

# Check port usage
lsof -i :8080  # macOS

# Test locally
curl http://localhost:8080/health
curl http://localhost:8080/metrics

# Test with verbose output
curl -v http://localhost:8080/health

# Test request ID
curl -H "X-Request-ID: test-123" http://localhost:8080/health

# Check Go environment
go env

# Check dependencies
go list -m all

# Why is package included?
go mod why github.com/prometheus/client_golang

# Dependency graph
go mod graph | grep prometheus

Metrics and Monitoring

# View Prometheus metrics
curl http://localhost:8080/metrics

# Filter specific metrics
curl http://localhost:8080/metrics | grep http_requests_total

# Parse metrics
curl -s http://localhost:8080/metrics | \
  grep http_requests_total | \
  grep method

# Watch metrics change
watch -n 1 'curl -s http://localhost:8080/metrics | grep http_requests_total'

Performance Testing

# Install hey
go install github.com/rakyll/hey@latest

# Basic load test
hey -n 1000 -c 10 http://localhost:8080/health

# POST requests
hey -n 1000 -c 10 -m POST \
    -H "Content-Type: application/json" \
    -d '{"text":"load test"}' \
    http://localhost:8080/predict

# With custom duration
hey -z 30s -c 20 http://localhost:8080/health

# Save results
hey -n 1000 -c 10 http://localhost:8080/health > results.txt

# Using Apache Bench
ab -n 1000 -c 10 http://localhost:8080/health

Cleanup Commands

# Stop running gateway
pkill gateway

# Delete Kubernetes resources
kubectl delete -f deployment.yaml

# Delete by label
kubectl delete all -l app=api-gateway

# Remove Docker image
docker rmi api-gateway:v1

# Clean Go cache
go clean -cache
go clean -modcache

# Remove built binaries
rm gateway gateway-linux

Solution File

If you get stuck, a complete reference implementation is available in solution/:

gateway_solution.go - All TODOs completed with detailed comments

Note: Try to complete the exercise on your own first!

Next Steps

Once you've completed the exercise and tests pass:

→ Module 5: Kubeflow Pipelines

In Module 5, you'll build ML pipelines for training automation!

Key Takeaways

What We Learned

✅ Go HTTP Server: Building production HTTP services
✅ Reverse Proxy: Forwarding requests to backend services
✅ Structured Logging: JSON logs for observability
✅ Metrics: Prometheus instrumentation
✅ Middleware: Composable request processing
✅ Graceful Shutdown: Clean termination
✅ Polyglot Architecture: Right tool for each job

Best Practices

Use Go for I/O-bound, routing, and gateway layers
Use Python for ML inference and data science
Always implement structured logging
Instrument code with Prometheus metrics
Propagate request IDs for tracing
Handle graceful shutdown
Set appropriate timeouts
Test backend connectivity in health checks

Real-World Production

This configuration is suitable for production ML APIs that need:

High Performance: Low latency, high throughput
Resource Efficiency: Minimal memory footprint
Observability: Structured logs and metrics
Reliability: Health checks and graceful shutdown
Scalability: Independent scaling of gateway and ML service

Having issues? Check the Troubleshooting section or review the solution file!

Navigation

Previous	Home	Next
← Module 3: Kubernetes Deployment	🏠 Home	Module 5: Kubeflow Pipelines & Model Serving →

Quick Links

MLOps Workshop | GitHub Repository

Module 4.md

Module 4: Golang API Gateway

What You'll Build

Learning Objectives

Part 1: Setup & Prerequisites

Why Go for the Gateway?

Prerequisites

Install Go

Part 2: Hands-On Exercise

Quick Start

1. Complete the Exercise

API Gateway

2. Build Docker Image

3. Deploy to Kubernetes

4. Validate All Work

Part 3: Architecture & Concepts

Key Concepts Covered

Go Fundamentals

Reverse Proxy Pattern

Production Features

Observability

Polyglot Architecture Benefits

Resource Efficiency

Separation of Concerns

Common Commands

Part 4: Troubleshooting

Troubleshooting

Issue 1: Build fails - Missing dependencies

Issue 2: Build fails - Syntax errors

Issue 3: Cannot connect to backend service

Issue 4: Gateway responds with 502 Bad Gateway

Part 5: Reference

Commands Cheat Sheet

Quick Start

Docker Commands

Kubernetes Commands

Debugging Commands

Metrics and Monitoring

Performance Testing

Cleanup Commands

Solution File

Next Steps

Key Takeaways

What We Learned

Best Practices

Real-World Production

Navigation

Quick Links

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally