🐛 Bug Report — Runtime APIs: node:zlib zstdCompressSync fails with Unexpected end of file on valid input

[t-kouyama]

Summary

node:zlib zstd compression can fail with Unexpected end of file for valid, incompressible inputs.

This reproduces both on Workers production and with wrangler dev --local, so it looks like a workerd runtime bug rather than a production-only issue.

Minimal reproduction

import zlib from 'node:zlib';

function createRandomBuffer(size) {
  const output = Buffer.alloc(size);
  let state = 0x12345678;
  for (let index = 0; index < size; index += 4) {
    state ^= state << 13;
    state ^= state >>> 17;
    state ^= state << 5;
    output[index] = state & 0xff;
    if (index + 1 < size) output[index + 1] = (state >>> 8) & 0xff;
    if (index + 2 < size) output[index + 2] = (state >>> 16) & 0xff;
    if (index + 3 < size) output[index + 3] = (state >>> 24) & 0xff;
  }
  return output;
}

zlib.zstdCompressSync(createRandomBuffer(40944));
zlib.zstdCompressSync(createRandomBuffer(40960));

Observed behavior

• zlib.zstdCompressSync(createRandomBuffer(40944)) succeeds
• zlib.zstdCompressSync(createRandomBuffer(40960)) throws Unexpected end of file
• zlib.zstdCompress(createRandomBuffer(40960), cb) also fails with the same error
• zlib.gzipSync() succeeds for the same inputs

I also reproduced the same class of failure with an R2-backed input in Workers, but the generated-buffer case above is the smallest reproducer I found.

Expected behavior

Valid input should compress successfully, or return an actual zstd compression error if ZSTD_isError() reports one.

It should not be converted into Unexpected end of file while compressing valid input.

Suspected cause

In workerd/src/workerd/api/node/zlib-util.c++, ZstdEncoderContext::getError() treats any non-zero lastResult at ZSTD_e_end as EOF:

if (flush_ == ZSTD_e_end && lastResult != 0) {
  return CompressionError("Unexpected end of file"_kj, "Z_BUF_ERROR"_kj, Z_BUF_ERROR);
}

But lastResult != 0 from ZSTD_compressStream2() means there is still pending output to flush, not truncated input.

The sync loop in the same file only continues while ctx.getAvailOut() == 0:

do {
  ...
  ctx.work();
  ...
} while (ctx.getAvailOut() == 0);

So it looks possible for the loop to stop while zstd still has remaining output to emit, and then getError() turns that state into Unexpected end of file.

Source references

• workerd/src/workerd/api/node/zlib-util.c++
• workerd/src/workerd/api/node/tests/zlib-zstd-nodejs-test.js

Additional note

zstdSync() validates options.chunkSize, but constructs GrowableBuffer with ZLIB_PERFORMANT_CHUNK_SIZE instead of the provided chunkSize, so changing chunkSize does not seem to affect this path.