Skip to content

Commit a846d3a

Browse files
harshachpmbrullgithub-actions[bot]mohityadav766
authored
Improve Performance, Add Redis as optional cache (open-metadata#23054)
* MINOR - cache settings YAML * MINOR - cache settings YAML * Remove Redis; batch fetch all realtions in one query * Update generated TypeScript types * Add advanced configs * Fix tests * Fix tests * release 1.9.5 * fix include * Fix Indexing strategy, add HikariCP configs * add HikariCP configs to test config * Add AWS Aurora related configs * remove vacuum and relax defaults * fix includes * Use index * Add Latency breakdowns on server side * Update generated TypeScript types * Add Latency breakdowns on server side * Propagate fields properly * Add Async Search calls * Add Jetty Metrics * disable gzip * AWS JDBC Driver * add pctile * Add method to endpoint pctile * handle patch properly in metrics * tests * update metrics * bump flyway * fix jetty metric handler * default to postgres * default to postgres * ConnectionType with amazon * Update connection * Update connection * Add Redis Cache support for all entities, CacheWarmupApp * Fix aurora driver settings * Fix aurora driver settings * Fix aurora driver settings * Fix aurora driver settings * revert config * Handle ReadOnly * update config * Revert "update config" This reverts commit 9f5751c. * Revert "Handle ReadOnly" This reverts commit e0c9063. * Revert "revert config" This reverts commit e79c3d2. * Revert "Fix aurora driver settings" This reverts commit 463e6eb. * Revert "Fix aurora driver settings" This reverts commit 515d22b. * Revert "Fix aurora driver settings" This reverts commit 0a1226e. * Revert "Fix aurora driver settings" This reverts commit d959976. * Add Redis Cache support for all entities, CacheWarmupApp * Update generated TypeScript types * Redis SSL * redis auth * Fix cache warmup and lookup if cahce fails * Fix cache of relations * try search cache * fix search cache * fix cache response * Revert "fix cache response" This reverts commit 14602dc. * Revert "fix search cache" This reverts commit 8eaa76b. * Revert "try search cache" This reverts commit 0582a1d. * clean commits * clean drops * clean * clean * clean * remove hosts array for ES * Update generated TypeScript types * remove hosts array for ES * format * remove hosts array for ES * Remove Embeddings for Table Index * metrics improvements * MINOR - Report status for tests that blow up * Revert "MINOR - Report status for tests that blow up" This reverts commit e831ac0. * Fix tests * Address comments * remove unused code * fix postgres schema migration * fix tests and improve caching startegy * fix tests, making search sync * Update generated TypeScript types * Fix Failures due to merge conflicts * Fix Tag Failures * Fix Retryable Exception --------- Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com> Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
1 parent 5a7d715 commit a846d3a

File tree

107 files changed

+12154
-8508
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

107 files changed

+12154
-8508
lines changed

‎bootstrap/sql/migrations/native/1.10.0/mysql/schemaChanges.sql‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,58 @@
1+
-- Performance optimization for tag_usage prefix queries
2+
ALTER TABLE tag_usage
3+
ADD COLUMN targetfqnhash_lower VARCHAR(768) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci
4+
GENERATED ALWAYS AS (CONVERT(LOWER(targetFQNHash) USING utf8mb4) COLLATE utf8mb4_unicode_ci) STORED;
5+
6+
ALTER TABLE tag_usage
7+
ADD COLUMN tagfqn_lower VARCHAR(768) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci
8+
GENERATED ALWAYS AS (CONVERT(LOWER(tagFQN) USING utf8mb4) COLLATE utf8mb4_unicode_ci) STORED;
9+
10+
-- Create indexes on the new columns
11+
CREATE INDEX idx_targetfqnhash_lower ON tag_usage (targetfqnhash_lower(255));
12+
CREATE INDEX idx_tagfqn_lower ON tag_usage (tagfqn_lower(255));
13+
14+
-- PRIMARY INDEX: For targetFQNHash prefix searches (LIKE 'prefix%')
15+
-- Using prefix indexing for large VARCHAR columns to stay within MySQL's 3072 byte limit
16+
CREATE INDEX idx_tag_usage_target_prefix_composite
17+
ON tag_usage (source, targetfqnhash_lower(255), state, tagFQN(255), labelType);
18+
19+
-- For exact match queries
20+
CREATE INDEX idx_tag_usage_target_exact_composite
21+
ON tag_usage (source, targetFQNHash(255), state, tagFQN(255), labelType);
22+
23+
-- For tagFQN prefix searches
24+
CREATE INDEX idx_tag_usage_tagfqn_prefix_composite
25+
ON tag_usage (source, tagfqn_lower(255), state, targetFQNHash(255), labelType);
26+
27+
-- For JOIN operations
28+
CREATE INDEX idx_tag_usage_join_composite
29+
ON tag_usage (tagFQNHash(255), source, state, targetFQNHash(255), tagFQN(255), labelType);
30+
31+
CREATE FULLTEXT INDEX ft_tag_usage_targetfqn
32+
ON tag_usage(targetFQNHash);
33+
34+
-- Add index for efficient bulk term count queries
35+
-- The bulkGetTermCounts query uses: WHERE classificationHash IN (...) AND deleted = FALSE
36+
CREATE INDEX idx_tag_classification_deleted
37+
ON tag (classificationHash, deleted);
38+
39+
-- Create new composite indexes with deleted column
40+
-- MySQL doesn't support INCLUDE clause, so we put all columns in the index
41+
CREATE INDEX idx_entity_relationship_from_deleted
42+
ON entity_relationship(fromId, fromEntity, relation, deleted, toId, toEntity);
43+
44+
CREATE INDEX idx_entity_relationship_to_deleted
45+
ON entity_relationship(toId, toEntity, relation, deleted, fromId, fromEntity);
46+
47+
-- Index for queries that filter by fromEntity/toEntity
48+
CREATE INDEX idx_entity_relationship_from_typed
49+
ON entity_relationship(toId, toEntity, relation, fromEntity, deleted, fromId);
50+
51+
-- Index for bidirectional lookups
52+
CREATE INDEX idx_entity_relationship_bidirectional
53+
ON entity_relationship(fromId, toId, relation, deleted);
54+
55+
156
-- Add "Data Product Domain Validation" rule to existing entityRulesSettings configuration
257
UPDATE openmetadata_settings
358
SET json = JSON_ARRAY_APPEND(

‎bootstrap/sql/migrations/native/1.10.0/postgres/schemaChanges.sql‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,86 @@
1+
-- Performance optimization for tag_usage prefix queries
2+
ALTER TABLE tag_usage
3+
ADD COLUMN IF NOT EXISTS targetfqnhash_lower text
4+
GENERATED ALWAYS AS (lower(targetFQNHash)) STORED;
5+
6+
ALTER TABLE tag_usage
7+
ADD COLUMN IF NOT EXISTS tagfqn_lower text
8+
GENERATED ALWAYS AS (lower(tagFQN)) STORED;
9+
10+
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tag_usage_target_prefix_covering
11+
ON tag_usage (source, targetfqnhash_lower text_pattern_ops)
12+
INCLUDE (tagFQN, labelType, state)
13+
WHERE state = 1; -- Only active tags
14+
15+
-- For exact match queries on targetFQNHash
16+
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tag_usage_target_exact
17+
ON tag_usage (source, targetFQNHash, state)
18+
INCLUDE (tagFQN, labelType);
19+
20+
-- For tagFQN prefix searches if needed
21+
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tag_usage_tagfqn_prefix_covering
22+
ON tag_usage (source, tagfqn_lower text_pattern_ops)
23+
INCLUDE (targetFQNHash, labelType, state)
24+
WHERE state = 1;
25+
26+
-- For JOIN operations with classification and tag tables
27+
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tag_usage_join_source
28+
ON tag_usage (tagFQNHash, source)
29+
INCLUDE (targetFQNHash, tagFQN, labelType, state)
30+
WHERE state = 1;
31+
32+
-- Only create if you need %contains% searches
33+
CREATE EXTENSION IF NOT EXISTS pg_trgm;
34+
35+
-- GIN index for substring matches (LIKE '%foo%')
36+
CREATE INDEX CONCURRENTLY IF NOT EXISTS gin_tag_usage_targetfqn_trgm
37+
ON tag_usage USING GIN (targetFQNHash gin_trgm_ops)
38+
WHERE state = 1;
39+
40+
-- Optimize autovacuum for tag_usage (high update frequency)
41+
ALTER TABLE tag_usage SET (
42+
autovacuum_vacuum_scale_factor = 0.05, -- Vacuum at 5% dead rows (default 20%)
43+
autovacuum_analyze_scale_factor = 0.02, -- Analyze at 2% changed rows (default 10%)
44+
autovacuum_vacuum_threshold = 50, -- Minimum rows before vacuum
45+
autovacuum_analyze_threshold = 50, -- Minimum rows before analyze
46+
fillfactor = 90 -- Leave 10% free space for HOT updates
47+
);
48+
49+
-- Increase statistics target for frequently queried columns
50+
ALTER TABLE tag_usage ALTER COLUMN targetFQNHash SET STATISTICS 1000;
51+
ALTER TABLE tag_usage ALTER COLUMN targetfqnhash_lower SET STATISTICS 1000;
52+
ALTER TABLE tag_usage ALTER COLUMN tagFQN SET STATISTICS 500;
53+
ALTER TABLE tag_usage ALTER COLUMN tagfqn_lower SET STATISTICS 500;
54+
ALTER TABLE tag_usage ALTER COLUMN source SET STATISTICS 100;
55+
56+
-- Add index for efficient bulk term count queries
57+
-- The bulkGetTermCounts query uses: WHERE classificationHash IN (...) AND deleted = FALSE
58+
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_tag_classification_deleted
59+
ON tag (classificationHash, deleted);
60+
61+
-- Create new indexes with deleted column for efficient filtering
62+
-- Using partial indexes (WHERE deleted = FALSE) for even better performance
63+
CREATE INDEX IF NOT EXISTS idx_entity_relationship_from_deleted
64+
ON entity_relationship(fromId, fromEntity, relation)
65+
INCLUDE (toId, toEntity, relation)
66+
WHERE deleted = FALSE;
67+
68+
CREATE INDEX IF NOT EXISTS idx_entity_relationship_to_deleted
69+
ON entity_relationship(toId, toEntity, relation)
70+
INCLUDE (fromId, fromEntity, relation)
71+
WHERE deleted = FALSE;
72+
73+
-- Also add indexes for the specific queries that include fromEntity/toEntity filters
74+
CREATE INDEX IF NOT EXISTS idx_entity_relationship_from_typed
75+
ON entity_relationship(toId, toEntity, relation, fromEntity)
76+
INCLUDE (fromEntity, toEntity)
77+
WHERE deleted = FALSE;
78+
79+
-- Index for bidirectional lookups (used in UNION queries)
80+
CREATE INDEX IF NOT EXISTS idx_entity_relationship_bidirectional
81+
ON entity_relationship(fromId, toId, relation)
82+
WHERE deleted = FALSE;
83+
184
-- Add "Data Product Domain Validation" rule to existing entityRulesSettings configuration
285
UPDATE openmetadata_settings
386
SET json = jsonb_set(

‎conf/openmetadata.yaml‎

Lines changed: 134 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -27,20 +27,54 @@ server:
2727
- type: http
2828
bindHost: ${SERVER_HOST:-0.0.0.0}
2929
port: ${SERVER_PORT:-8585}
30-
maxRequestHeaderSize: ${MAX_REQUEST_HEADER_SIZE:-8KiB}
30+
31+
# Jetty Acceptor and Selector threads for high concurrency
32+
acceptorThreads: ${SERVER_ACCEPTOR_THREADS:-2} # 1-2 per CPU core
33+
selectorThreads: ${SERVER_SELECTOR_THREADS:-8} # 2-4 per CPU core
34+
35+
# Connection settings - relaxed for Docker/local development
36+
acceptQueueSize: ${SERVER_ACCEPT_QUEUE_SIZE:-256} # OS-level connection backlog
37+
idleTimeout: ${SERVER_IDLE_TIMEOUT:-60 seconds} # Close idle connections (increased from 30s)
38+
39+
# Buffer sizes for better throughput
40+
outputBufferSize: ${SERVER_OUTPUT_BUFFER_SIZE:-32KiB}
41+
inputBufferSize: ${SERVER_INPUT_BUFFER_SIZE:-8KiB}
42+
maxRequestHeaderSize: ${SERVER_MAX_REQUEST_HEADER_SIZE:-8KiB}
43+
maxResponseHeaderSize: ${SERVER_MAX_RESPONSE_HEADER_SIZE:-8KiB}
44+
headerCacheSize: ${SERVER_HEADER_CACHE_SIZE:-512B} # Cache parsed headers (in bytes)
45+
46+
# Performance settings
47+
useServerHeader: false # Don't send server version header
48+
useDateHeader: true
49+
useForwardedHeaders: ${SERVER_USE_FORWARDED_HEADERS:-false} # Enable if behind proxy
50+
51+
# Data rate limits (prevent slow loris attacks)
52+
minRequestDataPerSecond: ${SERVER_MIN_REQUEST_DATA_RATE:-0B} # 0B = disabled
53+
minResponseDataPerSecond: ${SERVER_MIN_RESPONSE_DATA_RATE:-0B} # 0B = disabled
54+
3155
adminConnectors:
3256
- type: http
3357
bindHost: ${SERVER_HOST:-0.0.0.0}
3458
port: ${SERVER_ADMIN_PORT:-8586}
59+
acceptorThreads: 1 # Admin endpoint needs minimal resources
60+
selectorThreads: 1
61+
62+
# Response compression disabled for maximum throughput
3563
gzip:
36-
syncFlush: true
64+
enabled: false
3765

3866
maxThreads: ${SERVER_MAX_THREADS:-150}
3967
minThreads: ${SERVER_MIN_THREADS:-100}
4068
idleThreadTimeout: ${SERVER_IDLE_THREAD_TIMEOUT:-1 minute}
4169
enableVirtualThreads: ${SERVER_ENABLE_VIRTUAL_THREAD:-false}
4270
maxQueuedRequests: ${SERVER_REQUEST_QUEUE:-1024}
4371

72+
# Request/Response logging (disable in production for performance)
73+
requestLog:
74+
appenders:
75+
- type: console
76+
threshold: ${REQUEST_LOG_LEVEL:-ERROR} # Only log errors by default
77+
4478
# Above configuration for running http is fine for dev and testing.
4579
# For production setup, where UI app will hit apis through DPS it
4680
# is strongly recommended to run https instead. Note that only
@@ -163,13 +197,59 @@ database:
163197
password: ${DB_USER_PASSWORD:-openmetadata_password}
164198
# the JDBC URL; the database is called openmetadata_db
165199
url: jdbc:${DB_SCHEME:-mysql}://${DB_HOST:-localhost}:${DB_PORT:-3306}/${OM_DATABASE:-openmetadata_db}?${DB_PARAMS:-allowPublicKeyRetrieval=true&useSSL=false&serverTimezone=UTC}
166-
maxSize: ${DB_CONNECTION_POOL_MAX_SIZE:-50}
167-
minSize: ${DB_CONNECTION_POOL_MIN_SIZE:-10}
168-
initialSize: ${DB_CONNECTION_POOL_INITIAL_SIZE:-10}
200+
201+
# HikariCP Connection Pool Settings - Optimized for Performance
202+
maxSize: ${DB_CONNECTION_POOL_MAX_SIZE:-100} # Increased from 50 for better concurrency
203+
minSize: ${DB_CONNECTION_POOL_MIN_SIZE:-20} # Increased from 10 to reduce connection creation overhead
204+
minimumIdle: ${DB_CONNECTION_POOL_MIN_IDLE:-20} # HikariCP specific minimum idle connections
205+
initialSize: ${DB_CONNECTION_POOL_INITIAL_SIZE:-20} # Start with more connections ready
169206
checkConnectionWhileIdle: ${DB_CONNECTION_CHECK_CONNECTION_WHILE_IDLE:-true}
170-
checkConnectionOnBorrow: ${DB_CONNECTION_CHECK_CONNECTION_ON_BORROW:-true}
207+
checkConnectionOnBorrow: ${DB_CONNECTION_CHECK_CONNECTION_ON_BORROW:-false} # Disable for performance
171208
evictionInterval: ${DB_CONNECTION_EVICTION_INTERVAL:-5 minutes}
172-
minIdleTime: ${DB_CONNECTION_MIN_IDLE_TIME:-1 minute}
209+
minIdleTime: ${DB_CONNECTION_MIN_IDLE_TIME:-5 minute}
210+
211+
212+
# JDBC Driver Properties - Critical for Performance
213+
# These work across both PostgreSQL and MySQL drivers
214+
properties:
215+
# HikariCP connection pool settings
216+
connectionTimeout: ${DB_CONNECTION_TIMEOUT:-300000} # 300 seconds
217+
idleTimeout: ${DB_IDLE_TIMEOUT:-600000} # 10 minutes
218+
maxLifetime: ${DB_MAX_LIFETIME:-1800000} # 30 minutes
219+
leakDetectionThreshold: ${DB_LEAK_DETECTION_THRESHOLD:-600000} # 10 minute
220+
keepaliveTime: ${DB_KEEPALIVE_TIME:-0} # 0 = disabled (set to 30000 for Aurora)
221+
validationTimeout: ${DB_VALIDATION_TIMEOUT:-300000} # 300 seconds
222+
# PostgreSQL specific - these are ignored by MySQL driver
223+
prepareThreshold: ${DB_PG_PREPARE_THRESHOLD:-1} # Use prepared statements immediately
224+
preparedStatementCacheQueries: ${DB_PG_PREP_STMT_CACHE_QUERIES:-500} # Cache more statements
225+
preparedStatementCacheSizeMiB: ${DB_PG_PREP_STMT_CACHE_SIZE_MB:-10} # Larger cache
226+
reWriteBatchedInserts: ${DB_PG_REWRITE_BATCHED_INSERTS:-true} # Critical for batch performance
227+
defaultRowFetchSize: ${DB_PG_DEFAULT_ROW_FETCH_SIZE:-1000} # Fetch more rows at once
228+
socketTimeout: ${DB_PG_SOCKET_TIMEOUT:-300}
229+
connectTimeout: ${DB_PG_CONNECT_TIMEOUT:-300}
230+
assumeMinServerVersion: ${DB_PG_ASSUME_MIN_SERVER_VERSION:-12} # Skip version checks
231+
ApplicationName: ${DB_PG_APPLICATION_NAME:-OpenMetadata}
232+
loginTimeout: ${DB_PG_LOGIN_TIMEOUT:-300} # Login timeout in seconds
233+
234+
# Aurora PostgreSQL specific optimizations
235+
loadBalanceHosts: ${DB_PG_LOAD_BALANCE_HOSTS:-false} # Set to true for Aurora reader endpoints
236+
hostRecheckSeconds: ${DB_PG_HOST_RECHECK_SECONDS:-10} # How often to check host status
237+
targetServerType: ${DB_PG_TARGET_SERVER_TYPE:-primary} # primary, secondary, any, preferSecondary
238+
239+
# MySQL specific - these are ignored by PostgreSQL driver
240+
rewriteBatchedStatements: ${DB_MYSQL_REWRITE_BATCHED_STATEMENTS:-true} # Critical for MySQL batch
241+
cachePrepStmts: ${DB_MYSQL_CACHE_PREP_STMTS:-true}
242+
prepStmtCacheSize: ${DB_MYSQL_PREP_STMT_CACHE_SIZE:-500}
243+
prepStmtCacheSqlLimit: ${DB_MYSQL_PREP_STMT_CACHE_SQL_LIMIT:-2048}
244+
useServerPrepStmts: ${DB_MYSQL_USE_SERVER_PREP_STMTS:-true}
245+
useLocalSessionState: ${DB_MYSQL_USE_LOCAL_SESSION_STATE:-true}
246+
useLocalTransactionState: ${DB_MYSQL_USE_LOCAL_TRANSACTION_STATE:-true}
247+
elideSetAutoCommits: ${DB_MYSQL_ELIDE_SET_AUTO_COMMITS:-true}
248+
maintainTimeStats: ${DB_MYSQL_MAINTAIN_TIME_STATS:-false}
249+
cacheResultSetMetadata: ${DB_MYSQL_CACHE_RESULT_SET_METADATA:-true}
250+
cacheServerConfiguration: ${DB_MYSQL_CACHE_SERVER_CONFIG:-true}
251+
tcpKeepAlive: ${DB_MYSQL_TCP_KEEP_ALIVE:-true}
252+
tcpNoDelay: ${DB_MYSQL_TCP_NO_DELAY:-true}
173253

174254
objectStorage:
175255
enabled: false
@@ -307,9 +387,12 @@ elasticsearch:
307387
clusterAlias: ${ELASTICSEARCH_CLUSTER_ALIAS:-""}
308388
truststorePath: ${ELASTICSEARCH_TRUST_STORE_PATH:-""}
309389
truststorePassword: ${ELASTICSEARCH_TRUST_STORE_PASSWORD:-""}
310-
connectionTimeoutSecs: ${ELASTICSEARCH_CONNECTION_TIMEOUT_SECS:-5}
311-
socketTimeoutSecs: ${ELASTICSEARCH_SOCKET_TIMEOUT_SECS:-60}
390+
connectionTimeoutSecs: ${ELASTICSEARCH_CONNECTION_TIMEOUT_SECS:-10} # Increased from 5s for Docker networks
391+
socketTimeoutSecs: ${ELASTICSEARCH_SOCKET_TIMEOUT_SECS:-120} # Increased from 60s for slow queries
312392
keepAliveTimeoutSecs: ${ELASTICSEARCH_KEEP_ALIVE_TIMEOUT_SECS:-600}
393+
# Connection pool settings for better load balancing and performance
394+
maxConnTotal: ${ELASTICSEARCH_MAX_CONN_TOTAL:-30} # Total connections across all hosts
395+
maxConnPerRoute: ${ELASTICSEARCH_MAX_CONN_PER_ROUTE:-10} # Max connections per host
313396
batchSize: ${ELASTICSEARCH_BATCH_SIZE:-100}
314397
payLoadSize: ${ELASTICSEARCH_PAYLOAD_BYTES_SIZE:-10485760}
315398
searchIndexMappingLanguage : ${ELASTICSEARCH_INDEX_MAPPING_LANG:-EN}
@@ -327,7 +410,6 @@ elasticsearch:
327410
secretKey: ${AWS_BEDROCK_SECRET_KEY:-""}
328411
useIamRole: ${AWS_BEDROCK_USE_IAM:-"false"}
329412

330-
331413
eventMonitoringConfiguration:
332414
eventMonitor: ${EVENT_MONITOR:-prometheus} # Possible values are "prometheus", "cloudwatch"
333415
batchSize: ${EVENT_MONITOR_BATCH_SIZE:-10}
@@ -475,3 +557,45 @@ rdf:
475557
username: ${RDF_REMOTE_USERNAME:-"admin"}
476558
password: ${RDF_REMOTE_PASSWORD:-"admin"}
477559
dataset: ${RDF_DATASET:-"openmetadata"}
560+
561+
# Cache Configuration
562+
# Caching layer for entity metadata, relationships, and tag usage to reduce database load
563+
# Default: Disabled (uses NoopCacheProvider)
564+
cache:
565+
# Cache provider: none (default) or redis
566+
provider: ${CACHE_PROVIDER:-none}
567+
568+
# TTL (Time To Live) settings in seconds
569+
entityTtlSeconds: ${CACHE_ENTITY_TTL:-172800} # 48 hour for entities
570+
relationshipTtlSeconds: ${CACHE_RELATIONSHIP_TTL:-172800} # 48 hour for relationships
571+
tagTtlSeconds: ${CACHE_TAG_TTL:-172800} # 48 hour for tags
572+
573+
# Redis configuration
574+
redis:
575+
# Redis connection URL
576+
# Standalone: redis://localhost:6379
577+
# AWS ElastiCache: redis://my-cluster.abc123.cache.amazonaws.com:6379
578+
url: ${CACHE_REDIS_URL:-redis://localhost:6379}
579+
authType: ${CACHE_REDIS_AUTH_TYPE:-NONE}
580+
database: ${CACHE_REDIS_DATABASE:-0} # Redis database index (0-15)
581+
582+
# Authentication for standalone Redis
583+
username: ${CACHE_REDIS_USERNAME:-}
584+
passwordRef: ${CACHE_REDIS_PASSWORD:-} # Reference to password in secrets manager
585+
useSSL: ${CACHE_REDIS_USE_SSL:-false}
586+
587+
# Key namespace prefix (useful for multi-tenant deployments)
588+
keyspace: ${CACHE_REDIS_KEYSPACE:-"om:prod"}
589+
590+
# Connection pool settings
591+
poolSize: ${CACHE_REDIS_POOL_SIZE:-64}
592+
connectTimeoutMs: ${CACHE_REDIS_CONNECT_TIMEOUT:-2000}
593+
594+
# AWS ElastiCache IAM Authentication (only if using ElastiCache)
595+
useIamAuth: ${CACHE_REDIS_USE_IAM:-false}
596+
awsRegion: ${CACHE_REDIS_AWS_REGION:-}
597+
awsUseInstanceProfile: ${CACHE_REDIS_AWS_INSTANCE_PROFILE:-true}
598+
# If not using instance profile, provide credentials:
599+
awsAccessKey: ${AWS_ACCESS_KEY_ID:-}
600+
awsSecretKey: ${AWS_SECRET_ACCESS_KEY:-}
601+
tokenRefreshIntervalSeconds: ${CACHE_REDIS_TOKEN_REFRESH:-900} # 15 minutes

0 commit comments

Comments
 (0)