Add comprehensive caching implementation analysis for SELECT statement query results

[Copilot]

Caching Analysis for OJP - Complete ✅

Comprehensive analysis of query result caching implementation for OJP with clear final design decision prominently displayed.

⭐ FINAL DESIGN DECISION (GO-TO APPROACH)

Query Marking: Client-Side ojp.properties Configuration

# In ojp.properties (same file as connection pool config)
postgres_prod.ojp.cache.enabled=true
postgres_prod.ojp.cache.distribute=true  # Optional: enable driver relay (default: false)
postgres_prod.ojp.cache.queries.1.pattern=SELECT .* FROM products WHERE .*
postgres_prod.ojp.cache.queries.1.ttl=600s
postgres_prod.ojp.cache.queries.1.invalidateOn=products

Why: Follows existing OJP patterns, simple, each datasource independent, no OJP restart needed.

Cache Distribution: JDBC Driver as Active Relay (Optional)

NEW: Distribution is now OPTIONAL per datasource via ojp.cache.distribute property:

• ojp.cache.distribute=true - Enable driver relay to distribute cache to other OJP servers
• ojp.cache.distribute=false - Cache only maintained locally on each server (default)

When distribution is enabled, cached data is distributed by the JDBC driver:

• Virtual threads (Java 21+) stream to other servers
• Smart policy: Only distribute < 200KB, TTL > 60s, > 1 row
• Real-time, zero database overhead

Why: Data already in driver memory, saves N-1 database queries, real-time propagation. Optional for flexibility.

Use Cases

Local-Only Caching (distribute=false - default):

• Single OJP server deployments
• Development environments
• Simple caching without coordination overhead
• Analytics datasources with unique queries

Distributed Caching (distribute=true):

• Multi-server OJP clusters
• High-traffic production environments
• Shared query patterns across servers
• Cost savings from reduced database load

Fallbacks for Special Cases

• Legacy Java (&lt;21): JDBC Notification Table
• PostgreSQL-only: LISTEN/NOTIFY
• Very large clusters (20+): Redis + JDBC

---

Deliverables

1. Comprehensive Analysis Document

File: CACHING_IMPLEMENTATION_ANALYSIS.md (3,482 lines)

Detailed analysis with Mermaid diagrams covering:

• Clear "FINAL DESIGN DECISION" section at the top with recommended approach
• 3 caching strategies with full implementation examples
• 4 query marking approaches analyzed (client-side ojp.properties recommended)
• 5 cache replication methods analyzed (JDBC driver relay recommended)
• Optional distribution control via ojp.cache.distribute property (default: false)
• Complete code examples for all major components
• Security, performance, and testing considerations
• Multi-datasource architecture support
• Phased implementation roadmap
• Other approaches considered (clearly marked as alternatives)

2. Quick Reference Guide

File: documents/analysis/CACHING_ANALYSIS_SUMMARY.md (400+ lines)

Executive summary with:

• "FINAL DESIGN DECISION" section at the very top
• Optional distribution configuration examples
• Mermaid architecture diagrams
• Configuration examples following final design
• Implementation strategy aligned with final decision
• FAQ section addressing client-side configuration
• Migration path for final design

---

Why Client-Side Configuration

Aspect	Client-Side (✅ Recommended)	Server-Side (❌ Too Complex)
Simplicity	✅ Simple	❌ Hot-reload, admin API, git-backed config
Alignment	✅ Follows existing OJP patterns	❌ New pattern
Updates	✅ Edit ojp.properties, restart app	❌ Hot-reload all servers
Impact	✅ Only affected app	❌ All apps
Decoupling	✅ Each datasource independent	❌ Centralized
Familiarity	✅ Known location	❌ New location
Complexity	✅ Minimal	❌ Significant

---

Multi-Datasource Architecture Support

Each datasource configured independently in ojp.properties:

• Single OJP server manages multiple datasources
• Each datasource can have different cache policies
• Each datasource can enable/disable distribution independently
• Cache keys include datasource name for isolation
• Production can cache with distribution, analytics local-only
• Add new datasources without code changes

---

Key Philosophy

Keep it Simple: Client-side configuration in ojp.properties is simpler than server-side with hot-reload mechanisms.

Follow Existing Patterns: OJP already does datasource config client-side - cache config should too.

Avoid Over-Engineering: Don't build complex hot-reload, admin APIs, git-backed config when a straightforward approach exists.

Optional Distribution: Start with local-only caching (default), enable distribution when needed.

Clear Communication: Final design decision prominently displayed at the top of all documents.

---

Implementation Approach

Phase 1: Local caching with client-side ojp.properties configuration (distribute=false - default)
Phase 2: Write-through invalidation (consistency)
Phase 3: Optional JDBC Driver Relay for distributed coordination (distribute=true)
Phase 4: Advanced features only if needed (Redis for very large clusters 20+)

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[rrobetti]

@copilot use mermaid for diagrams where possible.

[Copilot]

@copilot use mermaid for diagrams where possible.

Done! Converted all ASCII diagrams to Mermaid format in commit 12108ae.

Updated diagrams:

• Query flow (flowchart)
• Hybrid JDBC + Redis architecture (graph with subgraph)
• Complete architecture diagram with application layer, OJP servers, and database synchronization

[rrobetti]

@copilot add as an option and honestly ponderate on it for how to hint that a query is to be cached to configure it as a list in the ojp.properties (or ojp.yaml) file, this list gets send during connection to all ojp servers in the cluster which in turn then know which queries to cache and how frequently to refresh etc.

Also in relation to how to propagate caches add this option, again review honestly the pros and cons, in relation to how to propagate and invalidate cache use the JDBC driver as a relay mechanism where if a query is cached in ojp server once that data is read in the jdbc driver a separated thread (or virtual thread if java 21+) stream that cache to all other ojp servers. Similarly when a query cache must be invalidated a signal is returned to OJP jdbc driver which broadcast it to all other ojp servers (excluding the ojp server where it got the signal from).

At the end make an honest recomendation of what approach to implement and why.

[Copilot]

@copilot add as an option and honestly ponderate on it for how to hint that a query is to be cached to configure it as a list in the ojp.properties (or ojp.yaml) file, this list gets send during conne...

Added both approaches with comprehensive analysis in commit 7a304d1.

Client-Side Configuration (Section 3.4):

• Configuration in ojp-cache-client.yaml sent to all servers during connection
• Full implementation with connection-time propagation
• Honest assessment: ⚠️ Use sparingly - adds connection overhead and memory per session without clear advantage over SQL hints

JDBC Driver as Active Relay (Section 5.4):

• Driver uses virtual threads to push cached results to all servers
• Complete implementation with Mermaid sequence diagram
• Honest assessment: ⚠️ NOT RECOMMENDED - driver complexity, network amplification, and debugging challenges outweigh benefits

Final Recommendation (Section 13):

• RECOMMENDED: SQL hints + JDBC notification table (simple, reliable, handles 90% of use cases)
• PostgreSQL: Use LISTEN/NOTIFY instead
• High-scale: Redis + JDBC backup
• NOT recommended: Client-side config (overhead) or driver relay (complexity vs benefit)

The analysis honestly evaluates that while both approaches are technically feasible, they add complexity without sufficient benefit over simpler proven alternatives.

[rrobetti]

@copilot I think you are underestimating how complex it is to add your suggested sql hints in real-world scenarios where JDBC is hardly ever used alone, instead being wrapped by Hibernate + Spring Data or other ORMs and frameworks. In relation to the client relay, the benefit is to not have to go to the database again to get the data, the hole point is to avoid going to the database, if the data is already streamed to the JDBC driver (it is already there in memory) sending it to the other OJP servers do not seem to me so complex, can you explain further your rational and review the analysis in light of these arguments?

[Copilot]

@copilot I think you are underestimating how complex it is to add your suggested sql hints in real-world scenarios where JDBC is hardly ever used alone, instead being wrapped by Hibernate + Spring Dat...

You're absolutely right on both points. Revised in commit 97af9fe.

ORM Reality: SQL hints are impractical when most apps use Hibernate/Spring Data. Changed recommendation to server-side configuration (works with any ORM, pattern-matches generated SQL).

Data Already in Memory: You're correct - result sets are already in driver memory being returned to the app. The "serialization cost" concern was overstated - data is already serialized in gRPC format. Changed recommendation to driver relay as primary approach:

• Data already in memory (just stream it)
• Saves N-1 database queries (the whole point)
• Zero database overhead (no polling)
• Real-time propagation

Smart distribution policy mitigates network cost (skip large results >200KB, short TTL <60s, single rows).

New recommendations:

1. Query marking: Server-side config (ORM-agnostic)
2. Distribution: Driver relay (data in memory, efficient)
3. Scale to Redis only for very large clusters (20+)

Initial analysis was too conservative - revised to reflect real-world practicality.

[rrobetti]

@copilot in relation to the server side config, how do you intend to deal with the fact that a single ojp server might be managing dozens of different datasources, and datasource definition is all done in the client side?

[Copilot]

@copilot in relation to the server side config, how do you intend to deal with the fact that a single ojp server might be managing dozens of different datasources, and datasource definition is all don...

Excellent point! Addressed in commit e37737a with datasource-aware cache configuration.

Solution: Server-side config supports per-datasource rules:

cache:
  datasources:
    postgres_prod:
      rules:
        - pattern: "SELECT .* FROM products .*"
          ttl: 600s
    mysql_analytics:
      rules:
        - pattern: "SELECT .* FROM report_.*"
          ttl: 1800s

Implementation:

• Cache keys include datasource name (isolation)
• Rule matching considers both SQL + datasource
• Supports pattern-based datasource matching (postgres_.*, .*_analytics)
• Global rules as fallback

Benefits:

• Each datasource has independent cache policy
• Production can cache aggressively, analytics conservatively
• Complete isolation between datasources
• Add new datasources without code changes

Datasource name comes from session (client connection), so server naturally supports multiple datasources with different policies.

[rrobetti]

@copilot putting the configuration in the server-side means that every time it changes it requires the proxy to be restarted, that means if the same proxy serves multiple apps (which is very likely) and one app requires a cache configuration update the restart of OJP servers will affect all apps.

[Copilot]

@copilot putting the configuration in the server-side means that every time it changes it requires the proxy to be restarted, that means if the same proxy serves multiple apps (which is very likely) a...

Critical concern addressed in commit 42a8e92 with hot-reload support (zero downtime).

4 solutions provided:

1. File-watch auto-reload (simple): Polls config every 10s, atomic swap via volatile
2. Admin API (control): gRPC endpoint for validated updates, CI/CD integration
3. Version-based (safety): Keep previous config for 5min (in-flight requests)
4. Git-backed (audit): Auto-sync from git, full version control

Zero-downtime workflow:

1. Update cache-rules.yml (git or file)
2. Server detects change (auto or API)
3. Validates new config
4. Atomic swap (volatile)
5. Zero impact on other applications ✅

Benefits:

• No restart required
• One app's config change doesn't affect others
• Gradual rollout across cluster
• Quick rollback capability

Server-side config now production-ready for multi-app deployments.

[rrobetti]

@copilot no this is way too complicated, the configuration of cached queries shall be in the client within the ojp.properties under the datasource configuration. This way every datasource controls its own set of cached queries and can update them independently. This is aligned with other OJP configurations related to datasources and pooling which are already done in the client side. Update the analysis.

[Copilot]

@copilot no this is way too complicated, the configuration of cached queries shall be in the client within the ojp.properties under the datasource configuration. This way every datasource controls its...

Agreed! Simplified in commit bd2e798 to client-side configuration in ojp.properties.

Configuration location: Same file as connection pool config

postgres_prod.ojp.cache.enabled=true
postgres_prod.ojp.cache.queries.1.pattern=SELECT .* FROM products WHERE .*
postgres_prod.ojp.cache.queries.1.ttl=600s

Benefits:

• ✅ Follows existing OJP patterns (pool config, XA config all client-side)
• ✅ Simple - no server-side config, no hot-reload complexity
• ✅ Each datasource controls its own cache independently
• ✅ Update one datasource without affecting others
• ✅ Only restart affected application (not OJP server)

Removed all server-side configuration complexity (hot-reload, admin API, git-backed config). Much simpler and aligned with existing OJP architecture.

[rrobetti]

@copilot the analysis still does not clearly state the final design decision, update all documents. you can keep other considered approaches but it needs to be more clear what is the go to decision.

[Copilot]

@copilot the analysis still does not clearly state the final design decision, update all documents. you can keep other considered approaches but it needs to be more clear what is the go to decision.

Updated both documents in commit 55f1fd1 with crystal clear "⭐ FINAL DESIGN DECISION" section at the very top.

Go-to approach now clearly stated upfront:

1. Query Marking: Client-side ojp.properties configuration (same file as connection pools)
2. Cache Distribution: JDBC Driver as Active Relay (data already in memory)
3. Fallbacks: JDBC Notification Table (legacy Java), LISTEN/NOTIFY (PostgreSQL), Redis (large clusters)

Other approaches are now labeled as "Other Approaches Considered (not recommended)" with clear reasons why they're not the go-to choice. No more ambiguity about what to implement.

[rrobetti]

@copilot Make the client relay cache propagation to other ojp servers optional, if not enabled the cache is only maintained in the server itself.

[Copilot]

@copilot Make the client relay cache propagation to other ojp servers optional, if not enabled the cache is only maintained in the server itself.

Done! Added ojp.cache.distribute property in commit 3596936.

Configuration:

postgres_prod.ojp.cache.enabled=true
postgres_prod.ojp.cache.distribute=true   # Enable driver relay
postgres_prod.ojp.cache.queries.1.pattern=SELECT .* FROM products WHERE .*
postgres_prod.ojp.cache.queries.1.ttl=600s

Behavior:

• distribute=false (default): Cache maintained only on local OJP server
• distribute=true: Driver distributes cache to other OJP servers

Benefits:

• Start simple with local-only caching
• Enable distribution when needed
• Per-datasource control (production distributed, analytics local)
• No distribution overhead for single-server deployments

Updated both analysis documents with examples showing both local-only and distributed configurations.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud