Gmail Buddy

A robust, enterprise-ready Spring Boot application for Gmail management with exceptional performance (96% faster bulk operations), comprehensive email operations, dual authentication support, and modern architecture patterns.

✨ Features

Core Functionality

📧 Email Management: List, search, delete, and modify Gmail messages
🔍 Advanced Filtering: Filter messages by sender, recipient, subject, and custom Gmail queries
🏷️ Label Operations: Bulk label modification and management
📖 Message Content: Extract and display email body content
📚 Bulk Operations: Mass delete and label operations with validation
⚡ Native Batch Processing: Gmail API native batchDelete() for up to 1000 messages per batch (99% quota reduction)
🚀 High-Performance Operations: Adaptive batch sizing with circuit breaker protection
🔄 Intelligent Rate Limiting: Exponential backoff with adaptive algorithms

Architecture & Security

🔐 Dual Authentication: OAuth2 for browsers + Bearer token support for API clients (Postman, curl)
🛡️ Security Context Decoupling: Repository layer independent of Spring Security infrastructure
🔒 Token Validation: Google TokenInfo endpoint integration for secure API client access
🕵️ Security Logging: Comprehensive audit logging with automatic credential masking
✅ Input Validation: Comprehensive validation framework with custom validators
⚡ Exception Handling: Structured error responses with correlation IDs and detailed batch operation results
⚙️ Configuration Management: Centralized properties with environment-specific configs
🛡️ Security Hardening: CORS, security headers, and protection against common vulnerabilities

Performance & Reliability

🚀 96% Performance Improvement: Bulk operations complete in 8 seconds vs 210 seconds (1000 messages)
💰 99% Quota Reduction: Native batchDelete uses 50 units vs 5,000+ units for individual operations
🔄 Adaptive Rate Limiting: Dynamic batch sizing based on API response patterns
🛡️ Circuit Breaker Protection: Automatic cooling off periods during rate limit scenarios
📊 Detailed Operation Tracking: Success/failure tracking with retry counts and duration metrics

Developer Experience

🧪 High Test Coverage: 85%+ test coverage with unit and integration tests
📋 Structured Logging: JSON-formatted logs with correlation tracking and security masking
🔄 Retry Logic: Intelligent retry mechanisms with exponential backoff for Gmail API interactions
📊 Health Checks: Application and dependency health monitoring
🔧 API Client Ready: Full Postman support with Bearer token authentication

🚀 Quick Start

Prerequisites

Java 17+ (OpenJDK recommended)
Maven 3.6+
Google Cloud Project with Gmail API enabled
OAuth2 Credentials configured in Google Cloud Console

Installation

Clone the repository

git clone https://github.com/aucontraire/gmail-buddy.git
cd gmail-buddy

Configure OAuth2 Credentials

Create a .env file in the project root:
```
cp .env.example .env
```
Update the .env file with your Google OAuth credentials:
```
GOOGLE_CLIENT_ID=your_google_client_id_here
GOOGLE_CLIENT_SECRET=your_google_client_secret_here
```
Set up Google Cloud Console
- Enable the Gmail API in your Google Cloud project
- Configure OAuth2 consent screen
- Add http://localhost:8020/login/oauth2/code/google to authorized redirect URIs
Run the application
```
./mvnw spring-boot:run
```
Access the application

Open http://localhost:8020 and authenticate with Google.

📡 API Endpoints

Authentication

Gmail Buddy supports dual authentication modes:

Browser Users: OAuth2 flow with automatic redirect to Google's sign-in page
API Clients: Bearer token authentication for programmatic access (Postman, curl, scripts)

For API Testing: See the API Testing Guide for detailed instructions on using Bearer tokens with Postman, curl, and automation scripts.

Core Endpoints

Method	Endpoint	Description	Request Body	Performance
`GET`	`/dashboard`	Dashboard page (web interface)	-	-
`GET`	`/api/v1/gmail/messages`	List all messages	-	Standard
`GET`	`/api/v1/gmail/messages/latest`	List latest 50 messages	-	Standard
`POST`	`/api/v1/gmail/messages/filter`	Filter messages by criteria	`FilterCriteriaDTO`	Standard
`GET`	`/api/v1/gmail/messages/{id}/body`	Get message body content	-	Standard
`DELETE`	`/api/v1/gmail/messages/{id}`	Delete specific message	-	Standard
`PUT`	`/api/v1/gmail/messages/{id}/read`	Mark message as read	-	Standard
`DELETE`	`/api/v1/gmail/messages/filter`	High-performance bulk delete	`FilterCriteriaDTO`	99% quota reduction
`POST`	`/api/v1/gmail/messages/filter/modifyLabels`	Bulk modify labels	`FilterCriteriaWithLabelsDTO`	Batch optimized

High-Performance Batch Operations

Gmail Buddy's bulk delete operation uses Gmail's native batchDelete() API for exceptional performance:

Bulk Delete (DELETE /api/v1/gmail/messages/filter):

Uses native Gmail batchDelete() endpoint
Processes up to 1000 messages per batch chunk
Only 50 Gmail API quota units per batch (vs 5,000+ for individual deletes)
Automatic retry with exponential backoff
Circuit breaker protection against rate limits
Returns detailed BulkOperationResult with success/failure breakdown

Performance Example:

# Delete 1000 messages matching filter
curl -X DELETE http://localhost:8020/api/v1/gmail/messages/filter \
  -H "Authorization: Bearer ya29.a0ARrdaM..." \
  -H "Content-Type: application/json" \
  -d '{
    "from": "notifications@example.com",
    "olderThan": "30d"
  }'

# Response includes detailed metrics:
# {
#   "operationType": "BATCH_DELETE",
#   "totalOperations": 1000,
#   "successCount": 1000,
#   "failureCount": 0,
#   "successRate": 100.0,
#   "batchesProcessed": 1,
#   "batchesRetried": 0,
#   "durationMs": 8234
# }

Request/Response Examples

Filter Messages:

curl -X POST http://localhost:8020/api/v1/gmail/messages/filter \
  -H "Authorization: Bearer ya29.a0ARrdaM..." \
  -H "Content-Type: application/json" \
  -d '{
    "from": "notifications@example.com",
    "subject": "Weekly Report",
    "maxResults": 10
  }'

Modify Labels:

curl -X POST http://localhost:8020/api/v1/gmail/messages/filter/modifyLabels \
  -H "Authorization: Bearer ya29.a0ARrdaM..." \
  -H "Content-Type: application/json" \
  -d '{
    "from": "newsletter@example.com",
    "labelsToAdd": ["Newsletter", "Archive"],
    "labelsToRemove": ["INBOX"]
  }'

Note: Replace ya29.a0ARrdaM... with your actual Bearer token. See API Testing Guide for instructions on obtaining tokens.

Error Responses

All errors follow a consistent format with correlation IDs for tracking:

{
  "code": "VALIDATION_ERROR",
  "message": "Validation failed for input parameters",
  "category": "CLIENT_ERROR",
  "correlationId": "abc123-def456-ghi789",
  "timestamp": "2024-01-15T10:30:00Z",
  "details": {
    "from": "Must be a valid email address",
    "subject": "Subject must not exceed 255 characters"
  }
}

🏗️ Architecture

Project Structure

src/
├── main/java/com/aucontraire/gmailbuddy/
│   ├── client/         # Gmail API client layer (batch operations)
│   ├── config/         # Configuration classes and properties
│   ├── controller/     # REST controllers
│   ├── dto/            # Data Transfer Objects with validation
│   ├── exception/      # Exception hierarchy and handlers
│   ├── repository/     # Gmail data access layer
│   ├── security/       # Authentication and token validation
│   ├── service/        # Business logic layer
│   └── validation/     # Custom validators
├── main/resources/
│   ├── application.properties    # Main configuration
│   └── application-{env}.properties  # Environment-specific configs
└── test/               # Comprehensive test suite (120+ tests)

Key Components

Batch Processing Layer (NEW)

GmailBatchClient: Native Gmail API batch operations with circuit breaker protection
- Implements batchDelete() for up to 1000 messages per call (50 quota units flat fee)
- Adaptive batch sizing algorithm (5-50 operations per batch)
- Exponential backoff retry logic with configurable parameters
- Circuit breaker pattern with 30-second cooling off periods
BulkOperationResult: Thread-safe operation tracking with success/failure details
- Real-time success/failure tracking
- Retry attempt counting and batch metrics
- Duration and success rate calculations
BatchOperationException: Comprehensive batch operation error handling
- Partial failure support (HTTP 207 Multi-Status)
- Complete failure reporting (HTTP 502 Bad Gateway)
- Detailed operation-level error messages

Authentication & Security Layer

TokenProvider: Clean abstraction for token management (decoupled from Spring Security)
OAuth2TokenProvider: Dual authentication implementation
- Browser OAuth2 flow via SecurityContextHolder
- API client Bearer token support via HttpServletRequest
GoogleTokenValidator: Token validation using Google's TokenInfo endpoint
- Validates opaque OAuth2 tokens (not JWTs)
- Scope verification for Gmail API permissions
- Security logging with automatic credential masking

Exception Hierarchy

GmailBuddyException: Base exception with correlation IDs
ValidationException: Input validation errors (400)
ResourceNotFoundException: Missing resources (404)
GmailApiException: Gmail API integration errors (502)
AuthenticationException: OAuth2 authentication failures (401)
BatchOperationException: Batch operation failures with detailed results (207/502)

Validation Framework

Custom Email Validator: Validates email format patterns
Gmail Query Validator: Sanitizes and validates Gmail search queries
Label Validation: Enforces Gmail label naming constraints
Size Limits: Prevents oversized requests and bulk operations

Configuration Management

Centralized configuration using @ConfigurationProperties (GmailBuddyProperties):

Gmail API settings (rate limits, retry policies, batch operations)
Batch operations (delays, backoff, adaptive sizing)
OAuth2 configuration
Security settings (CORS, headers, token validation)
Validation rules and patterns

Authentication Architecture

Dual Authentication Support: OAuth2 for browsers, Bearer tokens for API clients (Postman, curl)
Security Context Decoupling: Repository layer independent of Spring Security context
Token Validation: Google TokenInfo endpoint integration for Bearer tokens
Graceful Fallback: API requests fall back to OAuth2 when Bearer token invalid
Credential Masking: Automatic redaction of sensitive data in logs

Design Patterns

Gmail Buddy implements modern architectural patterns for reliability and maintainability:

Batch Processing Patterns

Native API Batch Operations: Uses Gmail's batchDelete() endpoint (not REST API batching)
- Processes up to 1000 message IDs per batch
- Single API call with 50 quota unit flat fee
- All-or-nothing transactional semantics
Chunking Strategy: Splits large operations into optimal batch sizes
- Configurable chunk sizes (default: 1000 for delete, 50 for modify)
- Automatic batch creation and processing
- Progress tracking across multiple chunks

Resilience Patterns

Circuit Breaker: Protects against cascading failures
- Triggers after 3 consecutive failures
- 30-second cooling off period
- Automatic state reset on success
Exponential Backoff: Intelligent retry delays
- Initial delay: 2 seconds
- Multiplier: 2.5x per retry
- Maximum delay: 60 seconds
- Up to 4 retry attempts
Adaptive Rate Limiting: Dynamic batch size adjustment
- Starts at 15 operations per batch
- Increases gradually on success (up to 50)
- Decreases aggressively on failure (down to 5)
- Prevents rate limit violations

Repository Pattern

Clean Abstraction: Gmail API access isolated in repository layer
Token Provider Injection: Decoupled from Spring Security
Testable Design: Easy mocking for unit tests

DTO Pattern

Request/Response Separation: API contracts independent of internal models
Validation Integration: Bean Validation annotations on DTOs
Type Safety: Strong typing for filter criteria and label operations

Builder Pattern

GmailQueryBuilder: Fluent API for Gmail search query construction
Type-safe Query Building: Prevents invalid Gmail query syntax
Readable Code: Declarative query construction

Dependency Injection

Spring-managed Components: All services, repositories, and clients
Constructor Injection: Immutable dependencies with final fields
Configuration Properties: Externalized configuration via @ConfigurationProperties

Exception Handling Patterns

Hierarchical Exceptions: Base exception with specialized subtypes
Correlation IDs: Unique identifiers for request tracking
Detailed Error Context: Rich error information for debugging
HTTP Status Mapping: Appropriate status codes for each error type

🚀 Performance

Batch Operation Performance Metrics

Gmail Buddy achieves exceptional performance improvements through native Gmail API batch operations:

Performance Comparison (1000 Messages)

Metric	Old Approach	New Approach	Improvement
Execution Time	210 seconds	8 seconds	96% faster
Gmail Quota Used	5,000+ units	50 units	99% reduction
API Calls	1000+ calls	1 call	99.9% reduction
Batch Size	10 messages	1000 messages	100x larger
Inter-batch Delay	2000ms	500ms	75% faster

Real-World Performance Benefits

Bulk Delete Operations:

510 messages: ~5 seconds (was 210 seconds)
1000 messages: ~8 seconds (was 350+ seconds)
5000 messages: ~40 seconds (was 30+ minutes)

Quota Efficiency:

Native batchDelete(): 50 quota units for up to 1000 messages
Individual deletes: 5,000+ quota units for 1000 messages
99% quota savings enables more operations within Gmail API limits

Adaptive Performance Features

Intelligent Rate Limiting

Dynamic Batch Sizing: Automatically adjusts from 5-50 operations based on API success rates
Circuit Breaker: 30-second cooling off period after 3 consecutive failures
Exponential Backoff: Initial 2s delay, 2.5x multiplier, max 60s backoff
Micro-delays: 10ms between operations within batches to reduce concurrent pressure

Retry Logic

Configurable Retries: Up to 4 retry attempts with exponential backoff
Smart Failure Detection: Distinguishes retryable errors (rate limits, timeouts) from permanent failures
Batch-level Retries: Failed batches automatically retry with reduced size

Operation Tracking

Success Rate Monitoring: Real-time success/failure tracking per operation
Duration Metrics: Precise timing for performance analysis
Batch Statistics: Total batches processed, retried, and failed
Detailed Logging: Operation-level logs for debugging and auditing

Performance Configuration

All performance parameters are configurable via application.properties:

# Batch operation performance tuning
gmail-buddy.gmail-api.rate-limit.batch-operations.delay-between-batches-ms=500
gmail-buddy.gmail-api.rate-limit.batch-operations.max-batch-size=50
gmail-buddy.gmail-api.rate-limit.batch-operations.max-retry-attempts=4
gmail-buddy.gmail-api.rate-limit.batch-operations.initial-backoff-ms=2000
gmail-buddy.gmail-api.rate-limit.batch-operations.backoff-multiplier=2.5
gmail-buddy.gmail-api.rate-limit.batch-operations.max-backoff-ms=60000
gmail-buddy.gmail-api.rate-limit.batch-operations.micro-delay-between-operations-ms=10

⚙️ Configuration

Environment Variables

Variable	Description	Required	Default
`GOOGLE_CLIENT_ID`	Google OAuth2 Client ID	✅	-
`GOOGLE_CLIENT_SECRET`	Google OAuth2 Client Secret	✅	-
`SERVER_PORT`	Application port	❌	8020
`LOG_LEVEL`	Logging level	❌	INFO

Application Properties

Key configuration sections in application.properties:

# Gmail API Configuration
gmail-buddy.gmail-api.default-max-results=50
gmail-buddy.gmail-api.default-latest-messages-limit=50
gmail-buddy.gmail-api.batch-delete-max-results=500
gmail-buddy.gmail-api.rate-limit.default-retry-seconds=60

# Batch Operations Configuration (High Performance)
gmail-buddy.gmail-api.rate-limit.batch-operations.delay-between-batches-ms=500
gmail-buddy.gmail-api.rate-limit.batch-operations.max-batch-size=50
gmail-buddy.gmail-api.rate-limit.batch-operations.max-retry-attempts=4
gmail-buddy.gmail-api.rate-limit.batch-operations.initial-backoff-ms=2000
gmail-buddy.gmail-api.rate-limit.batch-operations.backoff-multiplier=2.5
gmail-buddy.gmail-api.rate-limit.batch-operations.max-backoff-ms=60000
gmail-buddy.gmail-api.rate-limit.batch-operations.micro-delay-between-operations-ms=10

# Security Configuration
gmail-buddy.security.permit-all-patterns=/login**,/oauth2/**
gmail-buddy.security.oauth2-security.default-success-url=/dashboard
gmail-buddy.security.oauth2-security.authorization-base-uri=/oauth2/authorization

# OAuth2 Configuration
gmail-buddy.oauth2.client-registration-id=google
gmail-buddy.oauth2.token.prefix=Bearer

# Validation Configuration
gmail-buddy.validation.email.pattern=^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$
gmail-buddy.validation.gmail-query.dangerous-pattern=.*[<>"'&;\\|\\*\\?\\[\\]\\(\\)\\{\\}\\$`\\\\].*

Batch Operations Tuning

The batch operations system is highly configurable for different performance requirements:

Conservative Settings (Safer for rate limits):

gmail-buddy.gmail-api.rate-limit.batch-operations.delay-between-batches-ms=1000
gmail-buddy.gmail-api.rate-limit.batch-operations.max-batch-size=25

Aggressive Settings (Maximum performance):

gmail-buddy.gmail-api.rate-limit.batch-operations.delay-between-batches-ms=100
gmail-buddy.gmail-api.rate-limit.batch-operations.max-batch-size=50

Recommended Production Settings (Balanced):

gmail-buddy.gmail-api.rate-limit.batch-operations.delay-between-batches-ms=500
gmail-buddy.gmail-api.rate-limit.batch-operations.max-batch-size=50

🧪 Testing

Running Tests

# Run all tests
./mvnw test

# Run specific test class
./mvnw test -Dtest=GmailServiceTest

# Run tests with coverage report
./mvnw test jacoco:report

Test Coverage

The project maintains high test coverage across all layers:

Unit Tests: Service logic, validation, and utilities
Integration Tests: Controller endpoints and OAuth2 flow
Validation Tests: Input validation and error scenarios

View coverage reports at target/site/jacoco/index.html after running tests.

Test Configuration

The test suite includes:

Mock OAuth2 Authentication: Simulates Google authentication
Test Configuration Properties: Isolated test configurations
Custom Test Utilities: Builders and factories for test data

🔒 Security

OAuth2 Implementation

Secure Token Storage: Encrypted token persistence
Scope Validation: Minimal required Gmail permissions
Session Management: Secure session handling with HttpOnly cookies

Input Validation

XSS Prevention: HTML/script tag filtering in queries
SQL Injection Protection: Parameterized Gmail API queries
Rate Limiting: Per-user request throttling

Security Headers

CORS Configuration: Controlled cross-origin access
Security Headers: CSP, X-Frame-Options, X-Content-Type-Options
Cookie Security: Secure, HttpOnly session cookies

🚀 Development

Prerequisites for Development

Java 17+ (OpenJDK recommended)
Maven 3.6+
IDE with Spring Boot support (IntelliJ IDEA, VS Code)
Git

Setting Up Development Environment

Clone and setup

git clone https://github.com/aucontraire/gmail-buddy.git
cd gmail-buddy
cp .env.example .env
# Update .env with your credentials

Import into IDE
- Import as Maven project
- Ensure Java 17+ is configured
- Install Spring Boot plugins if needed

Run in development mode

./mvnw spring-boot:run -Dspring-boot.run.profiles=dev

Code Quality

The project uses several tools to maintain code quality:

# Run linting and formatting
./mvnw spotless:apply

# Run security scan
./mvnw org.owasp:dependency-check-maven:check

# Run all quality checks
./mvnw verify

📚 Documentation

API Documentation

When running the application, interactive API documentation is available at:

Swagger UI: http://localhost:8020/swagger-ui.html
OpenAPI JSON: http://localhost:8020/v3/api-docs

Project Documentation

API Testing Guide: Bearer token authentication and Postman setup
Project Plan: Detailed development roadmap
Configuration Guide: Complete configuration reference
Deployment Guide: Production deployment instructions

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Workflow

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Make your changes with tests
Run the test suite (./mvnw test)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Code Standards

Follow existing code style and patterns
Include unit tests for new functionality
Update documentation for API changes
Ensure all tests pass before submitting PR

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🆘 Support

Common Issues

OAuth2 Client Not Found / Invalid Client

See the Google Client Verification Guide for step-by-step troubleshooting
Verify you're in the correct Google Cloud project
Ensure OAuth2 client ID and secret are properly configured

OAuth2 Redirect URI Mismatch

Ensure http://localhost:8020/login/oauth2/code/google is added to Google Cloud Console
Verify the redirect URI matches exactly (including port)

Insufficient Gmail Permissions

Check that Gmail API is enabled in Google Cloud Console
Verify OAuth2 scopes include gmail.readonly and gmail.modify
Re-authenticate to refresh token permissions

Test User Restrictions

Add your Google account as a test user in OAuth consent screen
Ensure app is not in production mode unless verified

Getting Help

Issues: GitHub Issues
Discussions: GitHub Discussions

🚀 What's Next

Check out our Project Plan for upcoming features:

Enhanced Monitoring: Real-time metrics dashboard for batch operations
Async Operations: Background processing with job queuing
Caching Layer: Intelligent caching for frequently accessed messages
Advanced Filtering: Additional Gmail query operators and search capabilities
Webhook Support: Real-time Gmail push notifications

🏗️ Architecture Documentation

For detailed architectural decisions and implementation details, see:

ADR-001: Foundation Architecture Improvements
- TokenProvider abstraction and security context decoupling
- Testing strategy and dependency injection patterns
ADR-002: OAuth2 Security Context Decoupling
- Dual authentication implementation (OAuth2 + Bearer tokens)
- Google TokenInfo endpoint integration
- API client authentication support

⭐ Star this repo if you find it useful!

Made with ❤️ by the Gmail Buddy Team

Name		Name	Last commit message	Last commit date
Latest commit History 41 Commits
docs		docs
src		src
.env.example		.env.example
.gitattributes		.gitattributes
.gitignore		.gitignore
CONFIGURATION.md		CONFIGURATION.md
README.md		README.md
pom.xml		pom.xml

Folders and files

Latest commit

History

Repository files navigation