Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,19 @@ ACTIVE_MASTER_KEY_ID=default
# Migration note: Prior to v0.5.0, default was 86400 (24 hours)
AUTH_TOKEN_EXPIRATION_SECONDS=14400

# Rate limiting configuration
# Protects against abuse and denial-of-service attacks
# Rate limiting configuration (authenticated endpoints)
# Protects against abuse and denial-of-service attacks on authenticated routes
RATE_LIMIT_ENABLED=true
RATE_LIMIT_REQUESTS_PER_SEC=10.0
RATE_LIMIT_BURST=20

# Token endpoint rate limiting (IP-based, unauthenticated)
# Applies to POST /v1/token endpoint to prevent credential stuffing and brute force attacks
# Stricter limits recommended as this endpoint is unauthenticated and commonly targeted
RATE_LIMIT_TOKEN_ENABLED=true
RATE_LIMIT_TOKEN_REQUESTS_PER_SEC=5.0
RATE_LIMIT_TOKEN_BURST=10

# CORS configuration
# ⚠️ SECURITY WARNING: CORS is disabled by default for server-to-server API
# Enable only if browser-based access is required
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,9 @@ jobs:

python3 docs/tools/check_docs_metadata.py

- name: Docs release image tag checks
run: python3 docs/tools/check_release_image_tags.py

- name: OpenAPI validation
run: |
set -euo pipefail
Expand Down
113 changes: 113 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -694,6 +694,119 @@ v1 := router.Group("/api/v1")
v1.Use(authMiddleware)
```

### Rate Limiting Middleware

The project implements two types of rate limiting middleware to protect against abuse:

#### 1. Client-Based Rate Limiting (Authenticated Endpoints)

**File:** `/internal/auth/http/rate_limit_middleware.go`

**Purpose:** Protects authenticated endpoints from abuse by limiting requests per authenticated client.

**Usage:**
```go
// Create middleware with configuration
rateLimitMiddleware := authHTTP.RateLimitMiddleware(
cfg.RateLimitRequestsPerSec, // e.g., 10.0 requests/second
cfg.RateLimitBurst, // e.g., 20 burst capacity
logger,
)

// Apply to authenticated route groups
clients := v1.Group("/clients")
clients.Use(authMiddleware) // Must come first
clients.Use(rateLimitMiddleware) // Rate limit per client
```

**Key Features:**
- **Requires authentication:** Must be used after `AuthenticationMiddleware`
- **Per-client limits:** Each authenticated client (by client ID) gets independent rate limiter
- **Token bucket algorithm:** Uses `golang.org/x/time/rate` for smooth rate limiting
- **Automatic cleanup:** Removes stale limiters after 1 hour of inactivity
- **Configurable:** Controlled by `RATE_LIMIT_ENABLED`, `RATE_LIMIT_REQUESTS_PER_SEC`, `RATE_LIMIT_BURST`

**Response:**
- Returns `429 Too Many Requests` with `Retry-After` header when limit exceeded
- Error response: `{"error": "rate_limit_exceeded", "message": "Too many requests. Please retry after the specified delay."}`

#### 2. IP-Based Rate Limiting (Unauthenticated Endpoints)

**File:** `/internal/auth/http/token_rate_limit_middleware.go`

**Purpose:** Protects unauthenticated endpoints (e.g., token issuance) from credential stuffing and brute force attacks.

**Usage:**
```go
// Create middleware with configuration
tokenRateLimitMiddleware := authHTTP.TokenRateLimitMiddleware(
cfg.RateLimitTokenRequestsPerSec, // e.g., 5.0 requests/second
cfg.RateLimitTokenBurst, // e.g., 10 burst capacity
logger,
)

// Apply to unauthenticated endpoints
if tokenRateLimitMiddleware != nil {
v1.POST("/token", tokenRateLimitMiddleware, tokenHandler.IssueTokenHandler)
}
```

**Key Features:**
- **No authentication required:** Works on unauthenticated endpoints
- **Per-IP limits:** Each IP address gets independent rate limiter
- **Automatic IP detection:** Uses `c.ClientIP()` which handles:
- `X-Forwarded-For` header (takes first IP)
- `X-Real-IP` header
- Direct connection remote address
- **Token bucket algorithm:** Uses `golang.org/x/time/rate` for smooth rate limiting
- **Automatic cleanup:** Removes stale limiters after 1 hour of inactivity
- **Configurable:** Controlled by `RATE_LIMIT_TOKEN_ENABLED`, `RATE_LIMIT_TOKEN_REQUESTS_PER_SEC`, `RATE_LIMIT_TOKEN_BURST`

**Response:**
- Returns `429 Too Many Requests` with `Retry-After` header when limit exceeded
- Error response: `{"error": "rate_limit_exceeded", "message": "Too many token requests from this IP. Please retry after the specified delay."}`

**Security Considerations:**

*Strengths:*
- Protects against credential stuffing and brute force attacks
- Stricter default limits (5 req/sec, burst 10) than authenticated endpoints
- No overhead on authenticated endpoints

*Limitations & Mitigations:*
- **Shared IPs (NAT, corporate proxies):** May affect legitimate users behind same IP
- Mitigation: Reasonable burst capacity (10 requests) handles legitimate retries
- Mitigation: Can be disabled via `RATE_LIMIT_TOKEN_ENABLED=false` if needed
- **IP Spoofing via X-Forwarded-For:** Attacker could rotate IPs in header
- Mitigation: Configure Gin's trusted proxy settings in production
- Mitigation: Deploy behind proper reverse proxy/load balancer

**Configuration Example (.env):**
```bash
# Authenticated endpoint rate limiting (per client)
RATE_LIMIT_ENABLED=true
RATE_LIMIT_REQUESTS_PER_SEC=10.0
RATE_LIMIT_BURST=20

# Token endpoint rate limiting (per IP, unauthenticated)
RATE_LIMIT_TOKEN_ENABLED=true
RATE_LIMIT_TOKEN_REQUESTS_PER_SEC=5.0
RATE_LIMIT_TOKEN_BURST=10
```

**Testing:**
Both middleware implementations include comprehensive test coverage:
- Requests within limit allowed
- Requests exceeding limit blocked with 429
- Retry-After header present
- Independent limits per client/IP
- Burst capacity handling
- Automatic cleanup of stale entries

**Reference:**
- Client-based: `/internal/auth/http/rate_limit_middleware.go` and `rate_limit_middleware_test.go`
- IP-based: `/internal/auth/http/token_rate_limit_middleware.go` and `token_rate_limit_middleware_test.go`

## Authentication & Authorization HTTP Layer

### HTTP Handler Organization Pattern
Expand Down
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,22 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.7.0] - 2026-02-20

### Added
- Added IP-based rate limiting middleware for unauthenticated `POST /v1/token`
- Added token endpoint rate-limit configuration via `RATE_LIMIT_TOKEN_ENABLED`, `RATE_LIMIT_TOKEN_REQUESTS_PER_SEC`, and `RATE_LIMIT_TOKEN_BURST`

### Changed
- Token issuance endpoint can now return `429 Too Many Requests` with `Retry-After` when per-IP limits are exceeded

### Security
- Hardened token issuance path against credential stuffing and brute-force request bursts

### Documentation
- Added `docs/releases/v0.7.0.md` release notes and `docs/releases/v0.7.0-upgrade.md` upgrade guide
- Updated docs for token endpoint throttling behavior, configuration, and troubleshooting guidance

## [0.6.0] - 2026-02-19

### Added
Expand Down
7 changes: 6 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: help build run test lint clean migrate-up migrate-down docker-build docker-run mocks docs-lint docs-check-examples docs-check-metadata
.PHONY: help build run test lint clean migrate-up migrate-down docker-build docker-run mocks docs-lint docs-check-examples docs-check-metadata docs-check-release-tags

APP_NAME := app
BINARY_DIR := bin
Expand Down Expand Up @@ -78,11 +78,16 @@ docs-check-metadata: ## Validate docs metadata and API markers
@echo "Running docs metadata checks..."
@python3 docs/tools/check_docs_metadata.py

docs-check-release-tags: ## Validate pinned release image tags in current docs
@echo "Running docs release image tag checks..."
@python3 docs/tools/check_release_image_tags.py

docs-lint: ## Run markdown lint and offline link checks
@echo "Running markdownlint-cli2..."
@docker run --rm -v "$(PWD):/workdir" -w /workdir davidanson/markdownlint-cli2:v0.18.1 README.md "docs/**/*.md" ".github/pull_request_template.md"
@$(MAKE) docs-check-examples
@$(MAKE) docs-check-metadata
@$(MAKE) docs-check-release-tags
@echo "Running lychee offline link checks..."
@docker run --rm -v "$(PWD):/input" lycheeverse/lychee:latest --offline --include-fragments --no-progress "/input/README.md" "/input/docs/**/*.md" "/input/.github/pull_request_template.md"

Expand Down
27 changes: 13 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Secrets is inspired by **HashiCorp Vault** ❤️, but it is intentionally **muc
The default way to run Secrets is the published Docker image:

```bash
docker pull allisson/secrets:v0.6.0
docker pull allisson/secrets:v0.7.0
```

Use pinned tags for reproducible setups. `latest` is available for dev-only fast iteration.
Expand All @@ -29,21 +29,20 @@ Then follow the Docker setup guide in [docs/getting-started/docker.md](docs/gett
1. 🐳 **Run with Docker image (recommended)**: [docs/getting-started/docker.md](docs/getting-started/docker.md)
2. 💻 **Run locally for development**: [docs/getting-started/local-development.md](docs/getting-started/local-development.md)

## 🆕 What's New in v0.6.0
## 🆕 What's New in v0.7.0

- ☁️ Added KMS integration for master key encryption at rest (`KMS_PROVIDER`, `KMS_KEY_URI`)
- 🔁 Added `rotate-master-key` CLI command for safer master key lifecycle operations
- 🧭 Added provider-specific KMS setup and migration runbook documentation
- ✅ Added KMS migration checklist: [docs/operations/kms-migration-checklist.md](docs/operations/kms-migration-checklist.md)
- 📘 Added release notes: [docs/releases/v0.6.0.md](docs/releases/v0.6.0.md)
- ⬆️ Added upgrade guide: [docs/releases/v0.6.0-upgrade.md](docs/releases/v0.6.0-upgrade.md)
- 📦 Updated pinned Docker docs/examples to `allisson/secrets:v0.6.0`
- 🛡️ Added IP-based rate limiting for unauthenticated token issuance (`POST /v1/token`)
- ⚙️ Added token endpoint configuration: `RATE_LIMIT_TOKEN_ENABLED`, `RATE_LIMIT_TOKEN_REQUESTS_PER_SEC`, `RATE_LIMIT_TOKEN_BURST`
- 🚦 Added token endpoint `429` + `Retry-After` behavior for burst/abuse control
- 📘 Added release notes: [docs/releases/v0.7.0.md](docs/releases/v0.7.0.md)
- ⬆️ Added upgrade guide: [docs/releases/v0.7.0-upgrade.md](docs/releases/v0.7.0-upgrade.md)
- 📦 Updated pinned Docker docs/examples to `allisson/secrets:v0.7.0`

Release history quick links:

- Current: [v0.6.0 release notes](docs/releases/v0.6.0.md)
- Previous: [v0.5.1 release notes](docs/releases/v0.5.1.md)
- Previous upgrade guide: [v0.5.1 upgrade guide](docs/releases/v0.5.1-upgrade.md)
- Current: [v0.7.0 release notes](docs/releases/v0.7.0.md)
- Previous: [v0.6.0 release notes](docs/releases/v0.6.0.md)
- Previous upgrade guide: [v0.6.0 upgrade guide](docs/releases/v0.6.0-upgrade.md)

## 📚 Docs Map

Expand All @@ -54,8 +53,8 @@ Release history quick links:
- 🧰 **Troubleshooting**: [docs/getting-started/troubleshooting.md](docs/getting-started/troubleshooting.md)
- ✅ **Smoke test script**: [docs/getting-started/smoke-test.md](docs/getting-started/smoke-test.md)
- 🧪 **CLI commands reference**: [docs/cli/commands.md](docs/cli/commands.md)
- 🚀 **v0.6.0 release notes**: [docs/releases/v0.6.0.md](docs/releases/v0.6.0.md)
- ⬆️ **v0.6.0 upgrade guide**: [docs/releases/v0.6.0-upgrade.md](docs/releases/v0.6.0-upgrade.md)
- 🚀 **v0.7.0 release notes**: [docs/releases/v0.7.0.md](docs/releases/v0.7.0.md)
- ⬆️ **v0.7.0 upgrade guide**: [docs/releases/v0.7.0-upgrade.md](docs/releases/v0.7.0-upgrade.md)
- 🔁 **Release compatibility matrix**: [docs/releases/compatibility-matrix.md](docs/releases/compatibility-matrix.md)

- **By Topic**
Expand Down
35 changes: 34 additions & 1 deletion docs/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,39 @@
# 🗒️ Documentation Changelog

> Last updated: 2026-02-19
> Last updated: 2026-02-20

## 2026-02-20 (docs v13 - v0.7.0 release prep)

- Added release notes page: `docs/releases/v0.7.0.md`
- Added upgrade guide: `docs/releases/v0.7.0-upgrade.md`
- Updated docs metadata source (`docs/metadata.json`) to `current_release: v0.7.0`
- Updated root README and docs index to promote `v0.7.0` release links
- Updated compatibility matrix with `v0.6.0 -> v0.7.0` upgrade path
- Updated API docs to document token endpoint rate limiting and `POST /v1/token` `429` behavior
- Updated environment variable docs for `RATE_LIMIT_TOKEN_ENABLED`, `RATE_LIMIT_TOKEN_REQUESTS_PER_SEC`, and `RATE_LIMIT_TOKEN_BURST`
- Updated troubleshooting and security hardening docs with token endpoint throttling guidance
- Updated pinned Docker image examples from `allisson/secrets:v0.6.0` to `allisson/secrets:v0.7.0`
- Added token endpoint throttling runbook section to production deployment guide
- Added token-endpoint-specific `429` response example and optional smoke test verification flow
- Expanded monitoring queries and alert starters for `/v1/token` throttling signals
- Added docs CI guard for current-release pinned image tag consistency
- Added operator quick card runbook (`docs/operations/operator-quick-card.md`) for rollout/incident triage
- Added trusted proxy reference guide (`docs/operations/trusted-proxy-reference.md`) for source-IP safety checks
- Added release note and upgrade guide templates (`docs/releases/_template.md`, `docs/releases/_upgrade-template.md`)
- Added auth docs retry handling snippets for token endpoint `429` and `Retry-After`
- Added docs architecture map updates for CI docs guards and local validation workflow
- Added Phase 3 planning roadmap (`docs/development/docs-phase-3-roadmap.md`)
- Expanded Phase 3 roadmap with prioritized backlog (`S/M/L`), dependencies, and execution order
- Added Phase 4 micro-roadmap (`docs/development/docs-phase-4-roadmap.md`) with 3 PR plan and CI guard proposals
- Added incident decision tree and first-15-minutes incident playbook runbooks
- Added known limitations page for rate limiting, proxy trust, and KMS startup tradeoffs
- Added versioned examples index by release (`docs/examples/versioned-by-release.md`)
- Added day-0 onboarding walkthroughs for operator and developer personas
- Added persona landing pages (`docs/personas/operator.md`, `docs/personas/developer.md`, `docs/personas/security.md`)
- Added docs KPI page and postmortem-to-doc feedback loop guidance
- Added consolidated docs master backlog (`docs/development/docs-master-backlog.md`)
- Added search alias shortcuts in docs index for faster incident/runbook discovery
- Added command verification markers to key rollout/troubleshooting/smoke docs

## 2026-02-19 (docs v12 - v0.6.0 release prep)

Expand Down
35 changes: 32 additions & 3 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 📚 Secrets Documentation

> Last updated: 2026-02-19
> Last updated: 2026-02-20

Metadata source for release/API labels: `docs/metadata.json`

Expand All @@ -10,6 +10,8 @@ Welcome to the full documentation for Secrets. Pick a path and dive in 🚀

- 🐳 [getting-started/docker.md](getting-started/docker.md) (recommended)
- 💻 [getting-started/local-development.md](getting-started/local-development.md)
- 🧭 [getting-started/day-0-operator.md](getting-started/day-0-operator.md)
- 💻 [getting-started/day-0-developer.md](getting-started/day-0-developer.md)
- 🧰 [getting-started/troubleshooting.md](getting-started/troubleshooting.md)
- ✅ [getting-started/smoke-test.md](getting-started/smoke-test.md)
- 🧪 [cli/commands.md](cli/commands.md)
Expand All @@ -22,6 +24,12 @@ Welcome to the full documentation for Secrets. Pick a path and dive in 🚀
4. Apply production hardening checklist: [operations/production.md](operations/production.md)
5. Use runbook hub for rollout and incidents: [operations/runbook-index.md](operations/runbook-index.md)

## 👥 Persona Paths

- 👷 [personas/operator.md](personas/operator.md)
- 👨‍💻 [personas/developer.md](personas/developer.md)
- 🛡️ [personas/security.md](personas/security.md)

## 📖 Documentation by Topic

- ⚙️ [configuration/environment-variables.md](configuration/environment-variables.md)
Expand All @@ -32,15 +40,25 @@ Welcome to the full documentation for Secrets. Pick a path and dive in 🚀
- ☁️ [operations/kms-setup.md](operations/kms-setup.md)
- ✅ [operations/kms-migration-checklist.md](operations/kms-migration-checklist.md)
- 🚀 [operations/production-rollout.md](operations/production-rollout.md)
- ⚡ [operations/operator-quick-card.md](operations/operator-quick-card.md)
- 🌲 [operations/incident-decision-tree.md](operations/incident-decision-tree.md)
- ⏱️ [operations/first-15-minutes.md](operations/first-15-minutes.md)
- 📊 [operations/monitoring.md](operations/monitoring.md)
- 🧯 [operations/operator-drills.md](operations/operator-drills.md)
- 🏭 [operations/production.md](operations/production.md)
- 🌐 [operations/trusted-proxy-reference.md](operations/trusted-proxy-reference.md)
- ⚠️ [operations/known-limitations.md](operations/known-limitations.md)
- 🚑 [operations/failure-playbooks.md](operations/failure-playbooks.md)
- 🧪 [operations/policy-smoke-tests.md](operations/policy-smoke-tests.md)
- 🧭 [operations/runbook-index.md](operations/runbook-index.md)
- 🛠️ [development/testing.md](development/testing.md)
- 🧾 [development/docs-release-checklist.md](development/docs-release-checklist.md)
- 🗺️ [development/docs-architecture-map.md](development/docs-architecture-map.md)
- 📈 [development/docs-quality-kpis.md](development/docs-quality-kpis.md)
- 🔁 [development/postmortem-doc-loop.md](development/postmortem-doc-loop.md)
- 🗂️ [development/docs-master-backlog.md](development/docs-master-backlog.md)
- 🛣️ [development/docs-phase-3-roadmap.md](development/docs-phase-3-roadmap.md)
- 🧭 [development/docs-phase-4-roadmap.md](development/docs-phase-4-roadmap.md)
- 🤝 [contributing.md](contributing.md)
- 🗒️ [CHANGELOG.md](CHANGELOG.md)

Expand Down Expand Up @@ -70,6 +88,14 @@ Welcome to the full documentation for Secrets. Pick a path and dive in 🚀
- 🧩 [api/versioning-policy.md](api/versioning-policy.md)
- 📄 [openapi.yaml](openapi.yaml)

## 🔎 Search Aliases

- `401 403 429 decision tree` -> [operations/incident-decision-tree.md](operations/incident-decision-tree.md)
- `first 15 minutes incident` -> [operations/first-15-minutes.md](operations/first-15-minutes.md)
- `trusted proxy retry-after token 429` -> [operations/trusted-proxy-reference.md](operations/trusted-proxy-reference.md)
- `known limitations` -> [operations/known-limitations.md](operations/known-limitations.md)
- `versioned examples` -> [examples/versioned-by-release.md](examples/versioned-by-release.md)

OpenAPI scope note:

- `openapi.yaml` is a baseline subset for common API flows in the current release (`docs/metadata.json`)
Expand All @@ -78,8 +104,10 @@ OpenAPI scope note:

## 🚀 Releases

- 📦 [releases/v0.6.0.md](releases/v0.6.0.md)
- ⬆️ [releases/v0.6.0-upgrade.md](releases/v0.6.0-upgrade.md)
- 📦 [releases/v0.7.0.md](releases/v0.7.0.md)
- ⬆️ [releases/v0.7.0-upgrade.md](releases/v0.7.0-upgrade.md)
- 📦 [releases/v0.6.0.md](releases/v0.6.0.md) (historical)
- ⬆️ [releases/v0.6.0-upgrade.md](releases/v0.6.0-upgrade.md) (historical)
- 📦 [releases/v0.5.1.md](releases/v0.5.1.md) (historical)
- ⬆️ [releases/v0.5.1-upgrade.md](releases/v0.5.1-upgrade.md) (historical)
- 📦 [releases/v0.5.0.md](releases/v0.5.0.md) (historical)
Expand All @@ -105,6 +133,7 @@ OpenAPI scope note:

## 💡 Practical Examples

- 🧭 [examples/versioned-by-release.md](examples/versioned-by-release.md)
- 🧪 [examples/curl.md](examples/curl.md)
- 🐍 [examples/python.md](examples/python.md)
- 🟨 [examples/javascript.md](examples/javascript.md)
Expand Down
Loading
Loading