Skip to content

Feature/non blocking persistence#3

Closed
ofek29 wants to merge 19 commits intomainfrom
feature/non-blocking-persistence
Closed

Feature/non blocking persistence#3
ofek29 wants to merge 19 commits intomainfrom
feature/non-blocking-persistence

Conversation

@ofek29
Copy link
Owner

@ofek29 ofek29 commented Jul 1, 2025

This pull request introduces significant enhancements to the Java-based message queue implementation, focusing on performance optimization, configurability, and testing capabilities. Key changes include adding asynchronous batch persistence, configurable settings, and a comprehensive performance testing framework. Below is a summary of the most important changes grouped by theme.

Core Features and Enhancements:

  • Async Batch Persistence: Implemented asynchronous batch persistence for high-performance file storage, reducing disk I/O overhead and improving throughput. [1] [2] [3]
  • Configurable Settings: Added options for batch size and console logging, allowing users to customize the behavior of the message queue. [1] [2] [3]

API Changes:

  • New Methods: Introduced flush() for forcing pending messages to be written to disk and shutdown() for graceful termination of the queue. [1] [2]
  • Thread-Safe Enhancements: Improved thread safety with read-write locks for queue operations. [1] [2]

Usage Documentation:

  • Updated README: Expanded documentation to include examples of basic usage, custom configurations, and detailed descriptions of async batch persistence and configuration options. [1] [2]

Performance Testing:

  • Performance Testing Framework: Added a new PerformanceTest class and run_performance_test.sh script to measure queue performance under various conditions, including single-threaded and multi-threaded scenarios. [1] [2]
  • Performance Metrics: Generated detailed reports with metrics such as enqueue/dequeue times, throughput, and memory usage.

Code Quality Improvements:

  • Refactored Persistence Logic: Replaced single-message persistence with batch-based persistence using a dedicated worker thread.
  • Error Handling: Improved error handling and logging for file operations and persistence worker threads.

@ofek29 ofek29 requested a review from avifenesh July 6, 2025 15:15
runs-on: ubuntu-latest

strategy:
matrix:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why matrix for one runner one version? just use jobs

Copy link
Collaborator

@avifenesh avifenesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Break to logical PRs, this is an overkill.
CI, performance, each feature, etc...
You dont do 2k lines pr unless no way to break it.

@ofek29 ofek29 closed this Jul 7, 2025
@ofek29 ofek29 deleted the feature/non-blocking-persistence branch July 8, 2025 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants