Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,3 @@ RATE_LIMIT_TOKEN_BURST=10
# Never use "*" for CORS_ALLOW_ORIGINS in production
CORS_ENABLED=false
CORS_ALLOW_ORIGINS=

# Worker configuration
WORKER_INTERVAL=5
WORKER_BATCH_SIZE=10
WORKER_MAX_RETRIES=3
WORKER_RETRY_INTERVAL=1
11 changes: 1 addition & 10 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -195,16 +195,7 @@ jobs:
uses: actions/setup-go@v6
with:
go-version: "1.25.5"

- name: Cache Go modules
uses: actions/cache@v4
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
cache: true

- name: Download dependencies
run: go mod download
Expand Down
177 changes: 177 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1345,6 +1345,183 @@ return ErrKMSProviderNotSet or ErrKMSKeyURINotSet

**Reference:** `/internal/crypto/domain/master_key.go:287-315` (LoadMasterKeyChain)

## Audit Log Cryptographic Signing

The project implements HMAC-SHA256 cryptographic signing for audit logs to detect tampering and meet PCI DSS Requirement 10.2.2.

### Architecture Pattern

**Service Layer** (`internal/auth/service/audit_signer.go`):
- Implements `AuditSigner` interface with `Sign()` and `Verify()` methods
- Uses HKDF-SHA256 to derive signing key from KEK (separates encryption and signing usage)
- Canonical log serialization with length-prefixed encoding for variable fields

**Use Case Layer** (`internal/auth/usecase/audit_log_usecase.go`):
- `Create()` automatically signs logs if `KekChain` and `AuditSigner` available
- `VerifyBatch()` validates signatures for time range with KEK chain lookup
- `VerifyAuditLog()` validates single log signature

**Repository Layer**:
- Stores `signature` (BYTEA), `kek_id` (UUID FK), `is_signed` (BOOLEAN)
- Foreign key constraints prevent orphaned client/KEK references

### Signature Algorithm

**Key Derivation (HKDF-SHA256):**
```go
info := []byte("audit-log-signing-v1")
hash := sha256.New
hkdf := hkdf.New(hash, kekKey, nil, info)
signingKey := make([]byte, 32)
io.ReadFull(hkdf, signingKey)
```

**Canonical Log Format:**
```
request_id (16 bytes) ||
client_id (16 bytes) ||
len(capability) (4 bytes) || capability (variable) ||
len(path) (4 bytes) || path (variable) ||
len(metadata_json) (4 bytes) || metadata_json (variable) ||
created_at_unix_nano (8 bytes)
```

**HMAC-SHA256 Signature:**
```go
mac := hmac.New(sha256.New, signingKey)
mac.Write(canonicalBytes)
signature := mac.Sum(nil) // 32 bytes
```

### Testing with Foreign Key Constraints

Migration 000003 adds FK constraints requiring valid client and KEK references.

**Test Helpers** (`internal/testutil/database.go`):
```go
// Create FK-compliant test client
client := testutil.CreateTestClient(t, db, "postgresql", "test-client")

// Create FK-compliant test KEK
kek := testutil.CreateTestKek(t, db, "postgresql", "test-kek")

// Create both client and KEK
client, kek := testutil.CreateTestClientAndKek(t, db, "postgresql", "test")
```

**Pattern for Audit Log Tests:**
```go
func TestAuditLogRepository_Create(t *testing.T) {
db := setupTestDB(t)

// Create required FK references FIRST
client := testutil.CreateTestClient(t, db, "postgresql", "test-client")
kek := testutil.CreateTestKek(t, db, "postgresql", "test-kek")

// Create audit log with valid FK references
auditLog := &authDomain.AuditLog{
ID: uuid.Must(uuid.NewV7()),
ClientID: client.ID, // Valid FK reference
KekID: &kek.ID, // Valid FK reference
IsSigned: true,
// ... other fields
}

err := repo.Create(ctx, auditLog)
assert.NoError(t, err)
}
```

**Driver-Agnostic UUID Handling:**
- PostgreSQL: Native UUID type
- MySQL: BINARY(16) with hex conversion
- Test helpers abstract driver differences

### CLI Command Pattern

**Command Implementation** (`cmd/app/commands/verify_audit_logs.go`):
```go
func RunVerifyAuditLogs(ctx context.Context, startDate, endDate string, format string) error {
// Parse and validate inputs
start, err := parseDate(startDate)
end, err := parseDate(endDate)

// Load config and create container
cfg := config.Load()
container := app.NewContainer(cfg)
defer closeContainer(container, logger)

// Execute verification
auditLogUseCase, err := container.AuditLogUseCase()
report, err := auditLogUseCase.VerifyBatch(ctx, start, end)

// Output based on format
if format == "json" {
outputVerifyJSON(report)
} else {
outputVerifyText(report, start, end)
}

// Exit with error if integrity failed
if report.InvalidCount > 0 {
return fmt.Errorf("integrity check failed: %d invalid signature(s)", report.InvalidCount)
}

return nil
}
```

**Key Patterns:**
- Separate unexported helpers for parsing and output formatting
- Graceful container shutdown with `closeContainer()`
- Exit code indicates verification status (0=pass, 1=fail)
- Support both human-readable and JSON output

### Migration Testing Guidelines

When adding migrations that introduce FK constraints:

1. **Update all existing repository tests** to create required FK references
2. **Use testutil helpers** for consistent test data creation
3. **Test both PostgreSQL and MySQL** with identical logic
4. **Verify FK constraint enforcement** with negative tests

Example negative test:
```go
func TestAuditLogRepository_Create_FKViolation(t *testing.T) {
db := setupTestDB(t)

// Create audit log with non-existent client_id (FK violation)
auditLog := &authDomain.AuditLog{
ID: uuid.Must(uuid.NewV7()),
ClientID: uuid.Must(uuid.NewV7()), // Does not exist in clients table
// ... other fields
}

err := repo.Create(ctx, auditLog)
assert.Error(t, err)
assert.Contains(t, err.Error(), "foreign key constraint")
}
```

### Performance Considerations

**Signing Performance:**
- HKDF derivation: ~5-10µs per log
- HMAC-SHA256: ~1-2µs per log
- Total overhead: ~10-15µs per audit log (negligible)

**Verification Performance:**
- KEK lookup from chain: O(1) with map
- Signature verification: ~1-2µs per log
- Batch verification of 10k logs: ~20-30ms

**Benchmarks** (`internal/auth/service/audit_signer_benchmark_test.go`):
```
BenchmarkSign-8 100000 10234 ns/op 1024 B/op 12 allocs/op
BenchmarkVerify-8 200000 5123 ns/op 512 B/op 6 allocs/op
```

## See also

- [Repository README](README.md)
Expand Down
39 changes: 39 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,45 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.9.0] - 2026-02-20

### Added
- Added cryptographic audit log signing with HMAC-SHA256 for tamper detection (PCI DSS Requirement 10.2.2)
- Added HKDF-SHA256 key derivation to separate encryption and signing key usage
- Added `verify-audit-logs` CLI command for batch integrity verification with text/JSON output
- Added database columns: `signature` (BYTEA), `kek_id` (UUID FK), `is_signed` (BOOLEAN)
- Added foreign key constraints: `fk_audit_logs_client_id` and `fk_audit_logs_kek_id` to prevent orphaned records
- Added `AuditSigner` service for canonical log serialization and HMAC generation
- Added test infrastructure: `CreateTestClient()` and `CreateTestKek()` helpers for FK-compliant testing

### Changed
- Audit logs now automatically signed on creation when KEK chain is available
- Audit log API responses now include signature metadata (`signature`, `kek_id`, `is_signed`)
- Database migration 000003 required (adds signature columns and FK constraints)

### Fixed
- Fixed 46 audit log repository tests to comply with FK constraints

### Security
- Enhanced audit log tamper detection with cryptographic integrity verification
- Enforced data integrity with FK constraints preventing orphaned client/KEK references

### Documentation
- Added `docs/releases/v0.9.0-upgrade.md` upgrade guide with pre/post-migration checks
- Updated `docs/cli-commands.md` with `verify-audit-logs` command
- Updated `docs/api/observability/audit-logs.md` with signature field documentation
- Added AGENTS.md guidelines for audit signer architecture and FK testing patterns

## [0.8.0] - 2026-02-20

### Documentation
- Documentation consolidation: reduced from 77 to 47 markdown files (39% reduction)
- Established 8 new Architecture Decision Records (ADR 0003-0010) covering key architectural decisions
- Restructured API documentation with themed subdirectories (auth/, data/, observability/)
- Consolidated operations documentation with centralized runbook hub
- Merged all development documentation into contributing.md
- Comprehensive cross-reference updates throughout documentation (182+ updates)

## [0.7.0] - 2026-02-20

### Added
Expand Down
56 changes: 41 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,14 @@ Then follow the Docker setup guide in [docs/getting-started/docker.md](docs/gett
1. 🐳 **Run with Docker image (recommended)**: [docs/getting-started/docker.md](docs/getting-started/docker.md)
2. 💻 **Run locally for development**: [docs/getting-started/local-development.md](docs/getting-started/local-development.md)

## 🆕 What's New in v0.8.0
## 🆕 What's New in v0.9.0

- 📚 Major documentation consolidation: 77 → 47 files (39% reduction)
- 🏛️ Established 8 new Architecture Decision Records (ADR 0003-0010)
- 📂 Restructured API docs with themed organization (auth/, data/, observability/)
- 📖 Consolidated operations documentation with centralized runbook hub
- 🔗 Comprehensive cross-reference updates throughout documentation
- 📘 See [v0.8.0 release notes](docs/releases/RELEASES.md#080---2026-02-20)
- 🔐 Cryptographic audit log signing with HMAC-SHA256 for tamper detection (PCI DSS Requirement 10.2.2)
- ✅ New `verify-audit-logs` CLI command for integrity verification (text/JSON output)
- 🔑 HKDF-SHA256 key derivation separates encryption and signing key usage
- 🗄️ Database migration 000003 adds signature columns and FK constraints
- 🛡️ Foreign key constraints prevent orphaned audit log references
- 📘 See [v0.9.0 release notes](docs/releases/RELEASES.md#090---2026-02-20) and [upgrade guide](docs/releases/v0.9.0-upgrade.md)

Release history:

Expand Down Expand Up @@ -96,14 +96,40 @@ All detailed guides include practical use cases and copy/paste-ready examples.

## ✨ What You Get

- 🔐 Envelope encryption (`Master Key -> KEK -> DEK -> Secret Data`)
- 🔑 **KMS Integration** for master key encryption at rest (supports Google Cloud KMS, AWS KMS, Azure Key Vault, HashiCorp Vault, and local secrets for testing)
- 🚄 Transit encryption (`/v1/transit/keys/*`) for encrypt/decrypt as a service (decrypt input uses `<version>:<base64-ciphertext>`; see [Transit API docs](docs/api/data/transit.md), [create vs rotate](docs/api/data/transit.md#create-vs-rotate), and [error matrix](docs/api/data/transit.md#endpoint-error-matrix))
- 🎫 Tokenization API (`/v1/tokenization/*`) for token generation, detokenization, validation, and revocation
- 👤 Token-based authentication and policy-based authorization
- 📦 Versioned secrets by path (`/v1/secrets/*path`)
- 📜 Audit logs with request correlation (`request_id`) and filtering
- 📊 OpenTelemetry metrics with Prometheus-compatible `/metrics` export
**Core Cryptography:**

- 🔐 **Envelope encryption** (`Master Key → KEK → DEK → Secret Data`) with [key rotation](docs/operations/kms/key-management.md)
- 🔑 **KMS integration** for master key encryption at rest (Google Cloud KMS, AWS KMS, Azure Key Vault, HashiCorp Vault) - [v0.6.0+](docs/operations/kms/setup.md)
- 🔄 **Dual algorithm support** (AES-GCM and ChaCha20-Poly1305) for envelope encryption

**Authentication & Authorization:**

- 🎫 **Token-based authentication** with Argon2id password hashing (memory-hard, GPU-resistant)
- 🛡️ **Capability-based authorization** with [path-matching policies](docs/api/auth/policies.md) (exact, wildcard, prefix)
- 🎭 **Policy templates** for common personas (read-only, CI writer, key operator, break-glass admin)
- 🚦 **Dual-scope rate limiting** (per-client for authenticated endpoints, per-IP for token issuance)

**Data Services:**

- 📦 **Versioned secrets** by path (`/v1/secrets/*path`) with automatic versioning
- 🚄 **Transit encryption** (`/v1/transit/*`) for encrypt/decrypt as a service with [key rotation](docs/api/data/transit.md#create-vs-rotate)
- 🎫 **Tokenization API** (`/v1/tokenization/*`) with token generation, detokenization, validation, revocation, and TTL expiration

**Security & Compliance:**

- 🔏 **Cryptographic audit log signing** with HMAC-SHA256 for tamper detection (PCI DSS 10.2.2) - [v0.9.0+](docs/releases/RELEASES.md#090---2026-02-20)
- 📜 **Comprehensive audit logs** with request correlation (`request_id`), filtering, and [integrity verification](docs/cli-commands.md#verify-audit-logs)
- 🧹 **Memory safety** with sensitive key material zeroing in critical paths
- 🔒 **AEAD encryption** for authenticated encryption with associated data

**Operations & Observability:**

- 🗄️ **Dual database support** (PostgreSQL 12+ and MySQL 8.0+) with driver-agnostic migrations
- 📊 **OpenTelemetry metrics** with Prometheus-compatible `/metrics` export
- 🧪 **CLI tooling** (`verify-audit-logs`, `rotate-kek`, `create-master-key`, `rotate-master-key`)
- 🌐 **CORS support** (configurable, disabled by default)
- 🏥 **Health endpoints** (`/health`, `/ready`) for Kubernetes/Docker health checks
- 🧯 **Comprehensive documentation** with [runbooks](docs/operations/runbooks/README.md), [incident response guides](docs/operations/observability/incident-response.md), and [operator drills](docs/operations/runbooks/README.md#operator-drills-quarterly)

## 🌐 API Overview

Expand Down
Loading