feat: Optimize MCP Server Resource Usage (Phase 1)

[mkczarkowski]

Problem Statement

The MCP server was consuming 90% of the daily Cloudflare Durable
Objects free tier limit (100,000 rows_written), primarily due to:

• Architecture Issue: Each SSE request without a sessionId creates a
  new Durable Object, triggering 3-4 storage writes during initialization
• Bot Traffic: 31.44% error rate (158k/503k requests) indicates
  significant bot/crawler activity
• No Protection: Missing rate limiting, request validation, and session
  reuse mechanisms

Traffic Stats (Last Month):

• Total Requests: 503.62k (~16,700/day)
• Estimated Writes: ~90,000/day
• Actual Users: 10-500

Solution - Phase 1 Quick Wins

This PR implements immediate fixes to reduce Durable Object writes by
60-80% through four key optimizations:

1. ✅ Worker-Level Rate Limiting

Implementation (src/index.ts:43-93):

• In-memory rate limiter: 10 requests per minute per IP
• Automatic cleanup to prevent memory leaks
• Blocks requests BEFORE Durable Object creation
• Returns 429 with Retry-After header

Expected Impact: Reduce bot traffic by 40-50%

2. ✅ Health Check Endpoint

Implementation (src/index.ts:121-132):

• Lightweight /health and / endpoints
• No Durable Object creation
• Returns JSON status response
• Redirects monitoring bots away from resource-intensive routes

Expected Impact: Reduce unnecessary DO creations by 20-30%

3. ✅ SSE Request Validation

Implementation (src/index.ts:96-110, 154-164):

• Validates Accept header (must include text/event-stream or /)
• Validates User-Agent (minimum 5 characters)
• Rejects invalid requests with 400/406 status codes
• Prevents malformed requests from reaching Durable Objects

Expected Impact: Reduce malformed requests by 20-30%

4. ✅ Session Reuse Strategy

Implementation (src/index.ts:166-186):

• Adds X-Session-Reuse and X-Session-Info headers to responses
• Encourages clients to save and reuse sessionIds
• Documents best practices in README.md

Documentation (README.md:123-146):

• Comprehensive "Best Practices for Session Management" section
• Explains session reuse benefits
• Provides example URLs with sessionId parameter
• Documents rate limits and health check usage

Expected Impact: Reduce new DO creation by 50-70% (for legitimate
clients)

Changes Summary

Modified Files:

• mcp-server/src/index.ts - Added rate limiting, health checks, request
  validation, session reuse headers
• mcp-server/README.md - Added session management best practices
  documentation

Key Metrics:

• Lines Added: ~150
• New Classes: RateLimiter
• New Functions: validateSSERequest
• New Endpoints: /health, improved /sse

Expected Outcomes

After Phase 1 Deployment:

• Writes Reduction: 60-80%
• Daily Writes: 18,000-36,000 (18-36% of limit) ⬇️ from 90,000
• Error Rate: Reduced from 31.44% to ~20%
• Monthly Cost: Stay within free tier ✅

Performance Improvements:

• Faster response times for legitimate users
• Better resource utilization
• Reduced Cloudflare Worker CPU time
• More predictable scaling behavior

Testing Recommendations

Manual Testing:

Test health check (should not create DO)

curl https://10x-rules-mcp-server.przeprogramowani.workers.dev/health

Test rate limiting (11th request should fail)

for i in {1..12}; do
curl -s https://10x-rules-mcp-server.przeprogramowani.workers.dev/sse
done

Test request validation (should fail with 406)

curl -H "Accept: text/html"
https://10x-rules-mcp-server.przeprogramowani.workers.dev/sse

Test session reuse headers

curl -I https://10x-rules-mcp-server.przeprogramowani.workers.dev/sse

[github-actions]

✅ All checks have passed successfully!

• Lint: ✅
• Unit Tests: ✅
• E2E Tests: ✅

Coverage reports have been uploaded as artifacts.