Enterprise mode: External object storage (S3/GCS/Azure Blob)

[y0s3ph]

Context

Enterprise environments need external object storage instead of local PVCs for cost efficiency, durability, and scalability. Loki and optionally Prometheus (via Thanos) should support S3, GCS, and Azure Blob Storage.

Requirements

Configuration

```yaml
mode: enterprise

storage:
type: s3 # s3 | gcs | azure
s3:
endpoint: s3.amazonaws.com
region: eu-west-1
bucket: k8scope-data
accessKeyId: "" # or use IRSA/workload identity
secretAccessKey: "" # or reference to K8s Secret
# insecure: false
# pathStyle: false
gcs:
bucket: k8scope-data
serviceAccountKey: "" # or use workload identity
azure:
container: k8scope-data
accountName: ""
accountKey: "" # or use managed identity
```

Loki with external storage

• Switch from filesystem to object storage backend
• Separate bucket prefixes for chunks, ruler, and admin
• Configure compactor for retention on object storage

Prometheus with Thanos (optional)

• Deploy Thanos sidecar on Prometheus pods
• Upload blocks to object storage for long-term retention
• Deploy Thanos Query for unified query across replicas

Authentication methods (cloud-native)

• AWS: IAM Roles for Service Accounts (IRSA) preferred over static keys
• GCP: Workload Identity preferred over service account keys
• Azure: Managed Identity preferred over account keys
• Fall back to static credentials stored in Kubernetes Secrets

Acceptance criteria

• Loki stores chunks in S3/GCS/Azure instead of PVC
• Data survives pod restarts and even PVC deletion
• IRSA / Workload Identity authentication works
• Static credentials work as fallback
• Thanos integration is optional and configurable
• Documentation for each cloud provider