Skip to content

Conversation

@scunningham
Copy link
Contributor

Lazily defer writer buffer allocation until necessary. This has two advantages:

  1. Less data allocated at init time.
  2. Buffers are freed on a block write. This means a Flush will free the writer buffers, which could be useful for an application that writes/flushes in bursts.

Because write blocks are managed by a shared pool, this change should have minimal performance impact.

Lazy allocation decreases overhead at init time and will lower memory usage
when the writer has been flushed and is not actively writing data.
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements lazy buffer allocation for both synchronous and asynchronous writers, deferring memory allocation until data needs to be written. This reduces initial memory usage and allows buffers to be freed after block writes/flushes.

Key changes:

  • Removed immediate buffer allocation during writer initialization
  • Added lazy allocation when buffers are actually needed for writing
  • Enhanced state tracking to properly manage closed state

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
internal/pkg/sync/writer.go Implements lazy buffer allocation and adds explicit closed state tracking
internal/pkg/async/writer.go Implements lazy buffer allocation with a state flag system for better state management
Comments suppressed due to low confidence (1)

internal/pkg/sync/writer.go:143

  • Potential nil pointer dereference. The code returns w.srcBlk to the pool without checking if it's nil first. With lazy allocation, w.srcBlk could be nil if Close() is called before any writes occur.
	blk.ReturnBlk(w.srcBlk)

@scunningham scunningham merged commit 32e39e4 into prequel-dev:main Jul 28, 2025
3 checks passed
@scunningham scunningham deleted the lazy branch July 28, 2025 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants