feat(redis): add Cluster, Sentinel, and TLS/SSL support#37
Open
gafda wants to merge 2 commits intoaron-muon:mainfrom
Open
feat(redis): add Cluster, Sentinel, and TLS/SSL support#37gafda wants to merge 2 commits intoaron-muon:mainfrom
gafda wants to merge 2 commits intoaron-muon:mainfrom
Conversation
Contributor
Author
|
@aron-muon : Third PR from original PR #33 |
There was a problem hiding this comment.
Pull request overview
Adds first-class support for Redis Cluster and Sentinel (with optional TLS), introduces a Redis key-prefix mechanism applied across services, and expands developer tooling/docs for local cluster testing and deployment configuration.
Changes:
- Implement Redis multi-mode configuration (standalone/cluster/sentinel) with TLS helpers, validators, and a unified async RedisPool.
- Update services to use the shared RedisPool and apply a configurable key prefix; ensure Redis pipelines are cluster-compatible (
transaction=False). - Add extensive docs/examples, Helm chart values/templates, and docker-compose + cert tooling for local Redis Cluster (TLS and non-TLS) integration testing.
Reviewed changes
Copilot reviewed 30 out of 31 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/config/redis.py |
Introduces RedisConfig with mode (standalone/cluster/sentinel), TLS options, key prefix, and sanitizing validators. |
src/config/__init__.py |
Plumbs new Redis/env settings through Settings, adds validators, and adds service_version override. |
src/core/pool.py |
Implements RedisPool supporting standalone/cluster/sentinel + TLS, key prefixing, and updated pool stats. |
src/utils/config_validator.py |
Validates Redis connectivity per mode (cluster/sentinel/standalone) and forwards TLS kwargs. |
src/utils/logging.py |
Logs effective version using runtime SERVICE_VERSION override when present. |
src/main.py |
Uses effective_version consistently for startup logs, FastAPI version, and health endpoint. |
src/services/session.py |
Applies key prefix via redis_pool.make_key() and uses non-transactional pipelines for cluster compatibility. |
src/services/state.py |
Applies key prefix via redis_pool.make_key() and uses non-transactional pipelines for cluster compatibility. |
src/services/api_key_manager.py |
Applies key prefix to API key storage keys and uses non-transactional pipelines for cluster compatibility. |
src/services/detailed_metrics.py |
Applies key prefix to detailed metrics keys and uses non-transactional pipelines for cluster compatibility. |
src/services/metrics.py |
Applies key prefix to persisted metrics keys. |
src/services/health.py |
Uses prefixed Redis key for read/write health checks. |
src/services/file.py |
Switches to RedisPool client and applies key prefix to file metadata keys. |
tests/unit/test_settings_validators.py |
Adds tests for empty-string-to-None sanitization for Redis passwords and node lists. |
tests/unit/test_session_service.py |
Fixes Redis pipeline mocking to reflect redis.asyncio pipeline behavior (sync method). |
tests/unit/test_core_pool.py |
Updates pool initialization behavior expectations and adds cluster kwargs compatibility test. |
tests/unit/test_cluster_pipeline_compat.py |
Adds regression tests enforcing transaction=False pipelines across services for Redis Cluster. |
tests/integration/test_redis_cluster.py |
Adds optional integration tests against a local 6-node Redis Cluster. |
tests/integration/test_redis_cluster_tls.py |
Adds optional integration tests against a local 6-node TLS Redis Cluster and production-like TLS config. |
tests/tls-certs/generate.sh |
Adds cert generation script for local TLS cluster testing. |
tests/tls-certs/cleanup.sh |
Adds cleanup script for generated TLS artifacts. |
tests/tls-certs/.gitignore |
Ignores generated TLS cert/key artifacts. |
docker-compose.redis-cluster.yml |
Adds local 6-node Redis Cluster compose for integration testing. |
docker-compose.redis-cluster-tls.yml |
Adds local 6-node TLS Redis Cluster compose for integration testing. |
docs/CONFIGURATION.md |
Documents Redis modes, TLS options, key prefixing, and adds expanded Kubernetes execution config docs. |
.env.example |
Adds Redis mode/cluster/sentinel/TLS/key-prefix env vars plus Kubernetes execution mode variables. |
.editorconfig |
Adds project-wide formatting rules. |
.gitignore |
Ignores .pdm-python. |
helm-deployments/kubecoderun/values.yaml |
Adds Redis mode/keyPrefix/cluster/sentinel/TLS values plus Kubernetes execution mode and GKE Sandbox values. |
helm-deployments/kubecoderun/templates/configmap.yaml |
Renders new Redis and execution-mode env vars into the ConfigMap. |
helm-deployments/kubecoderun/templates/secret.yaml |
Adds optional REDIS_PASSWORD rendering alongside REDIS_URL. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Redis deployment modes: - Three modes: standalone (default), cluster, sentinel - Full TLS/SSL configuration for managed Redis services (GCP Memorystore, AWS ElastiCache, Azure Cache) - Key prefixing for multi-tenant deployments - Pipeline transaction=False for cluster cross-slot compat - Empty password/nodes sanitization for Helm deployments Infrastructure: - Add .editorconfig for consistent coding style - Docker Compose for Redis Cluster and Cluster+TLS testing - TLS cert generation/cleanup scripts for integration tests - Update Helm chart with Redis deployment mode config - Update docs: CONFIGURATION with Redis Cluster/Sentinel/TLS Service updates: - All Redis-backed services use key-prefixed make_key() - Session, state, file, metrics, health, api_key services updated - Config validator enhanced for Redis modes
- Close Redis clients after ping in standalone mode to avoid connection leaks - Close master client after ping in sentinel mode to avoid connection leaks - Use redis_cfg.get_url() instead of settings.get_redis_url() to respect TLS scheme (rediss://) - Add comment noting Sentinel itself doesn't maintain persistent connections
c7a6631 to
4fa8e91
Compare
Contributor
Author
|
@aron-muon, I've fixed all Copilot's complaints. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces comprehensive improvements to Redis configuration support and developer tooling. The main changes add support for Redis Cluster and Sentinel deployment modes (with optional TLS/SSL), provide detailed environment variable documentation and examples, and supply Docker Compose files for local integration testing of Redis Cluster setups (both with and without TLS). Additionally, a new
.editorconfigis added to standardize code formatting across the project.Redis configuration enhancements:
docs/CONFIGURATION.mdto document support for three Redis deployment modes: standalone, cluster, and sentinel, including detailed environment variable descriptions and TLS/SSL options. Example configurations for managed Redis services (GCP Memorystore, AWS ElastiCache) are provided..env.examplewith new variables forREDIS_MODE, cluster nodes, sentinel nodes, key prefix, and TLS/SSL configuration, plus explanatory comments for each. [1] [2]Developer tooling and integration testing:
docker-compose.redis-cluster.ymlfor spinning up a 6-node Redis Cluster locally, enabling integration testing for cluster mode.docker-compose.redis-cluster-tls.ymlfor spinning up a 6-node Redis Cluster with TLS enabled, mimicking production managed Redis setups and supporting local testing of TLS connections.Formatting and code style:
.editorconfigfile to enforce consistent code formatting rules across multiple languages and file types.Kubernetes execution configuration:
.env.examplefor agent and nsenter modes, including sidecar image selection and GKE Sandbox options.