Skip to content

Add buffer-native serialization hooks with dynamic write-buffer growth#319

Draft
Copilot wants to merge 3 commits into
mainfrom
copilot/add-specialized-serializers
Draft

Add buffer-native serialization hooks with dynamic write-buffer growth#319
Copilot wants to merge 3 commits into
mainfrom
copilot/add-specialized-serializers

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 27, 2026

This change adds first-class buffer serializers to Storage so callers can bypass string conversion on read/write paths. New serializeToBuffer / deserializeFromBuffer options override the legacy string serializer methods while preserving backward compatibility.

  • Serializer API extensions

    • Added config.serializeToBuffer(buffer, document, helpers) => bytesWritten for write-time buffer-native serialization.
    • Added config.deserializeFromBuffer(buffer) => object for read-time buffer-native deserialization.
    • Established precedence: buffer hooks override config.serializer.serialize / deserialize when present.
  • Buffer growth contract

    • Introduced helpers.ensureCapacity(minSize) for serializeToBuffer to request a larger working buffer.
    • serializeToBuffer writes into the provided buffer (or expanded one) and returns the exact byte count written.
    • Storage enforces return-value invariants (integer, positive, within buffer bounds).
  • Write/read pipeline integration

    • WritableStorage.write() now accepts serializer output as either string or Buffer.
    • WritablePartition write paths (write, buffered/unbuffered writes) now handle Buffer payloads directly.
    • ReadableStorage now deserializes via deserializeFromBuffer when configured, with string fallback unchanged.
  • Docs and coverage updates

    • Updated Storage API and advanced docs to describe new options and the growth helper contract.
    • Added tests for override precedence and dynamic buffer growth behavior.
const storage = new Storage('events', {
  writeBufferSize: 64,
  serializeToBuffer: (buffer, doc, { ensureCapacity }) => {
    const encoded = codec.encode(doc); // Buffer
    if (encoded.byteLength > buffer.byteLength) buffer = ensureCapacity(encoded.byteLength);
    encoded.copy(buffer, 0);
    return encoded.byteLength;
  },
  deserializeFromBuffer: (buffer) => codec.decode(buffer)
});

Copilot AI changed the title Add buffer-native serializer options for Storage Add buffer-native serialization hooks with dynamic write-buffer growth May 27, 2026
Copilot AI requested a review from albe May 27, 2026 11:44
Copilot finished work on behalf of albe May 27, 2026 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants