decodeJPEGIntoSurface leaks data and src on libjpeg longjmp from jpeg_read_scanlines

[iurisilvio]

Bug

Image::decodeJPEGIntoSurface() allocates two heap buffers via new uint8_t[] and frees them in the success path. If libjpeg encounters an error during jpeg_read_scanlines() (the most common failure for corrupt or truncated JPEGs where the header decodes successfully but the entropy-coded scan data is malformed), it calls error_exit which does longjmp back to the setjmp point in loadJPEGFromBuffer. That bypasses the cleanup code at the bottom of decodeJPEGIntoSurface, leaking both buffers.
The leak is the full decoded pixel surface (width × height × 4 bytes) plus the scanline buffer for every JPEG that hits this path.

Reproduction

'use strict';
if (typeof gc !== 'function') {
  console.error('Run with --expose-gc');
  process.exit(1);
}
const canvas = require('canvas');
const ITER = parseInt(process.argv[2] || '500', 10);
const mb = (n) => +(n / 1024 / 1024).toFixed(1);
// Build a JPEG with a complete header + start of scan, then truncate so libjpeg
// gets through jpeg_start_decompress and into jpeg_read_scanlines before hitting
// EOF and calling error_exit -> longjmp.
function makeScanTruncatedJpeg() {
  const c = canvas.createCanvas(1024, 1024);
  const ctx = c.getContext('2d');
  ctx.fillStyle = 'red';
  ctx.fillRect(0, 0, 1024, 1024);
  const full = c.toBuffer('image/jpeg', { quality: 0.9 });
  c.width = 0; c.height = 0;
  return full.slice(0, 4000);
}
(async () => {
  const buf = makeScanTruncatedJpeg();
  console.log(`truncated jpeg = ${buf.length} bytes`);
  gc(); gc();
  const baseline = mb(process.memoryUsage().rss);
  console.log(`baseline rss=${baseline} MiB`);
  for (let i = 1; i <= ITER; i++) {
    try { await canvas.loadImage(buf); } catch (_) {}
    if (i % 100 === 0) {
      gc(); gc();
      console.log(`iter=${i} rss=${mb(process.memoryUsage().rss)} MiB`);
    }
  }
  gc(); gc();
  const end = mb(process.memoryUsage().rss);
  console.log(`baseline=${baseline}M end=${end}M delta=${(end-baseline).toFixed(1)}M per-iter=${(((end-baseline)*1024)/ITER).toFixed(0)} KiB`);
})();

Observed on canvas@3.2.3 from npm (Linux arm64, Node 20)

truncated jpeg = 4000 bytes
baseline rss=47 MiB
iter=100  rss=  462 MiB
iter=500  rss= 2103 MiB
iter=1000 rss= 4154 MiB
iter=2000 rss= 8055 MiB
per-iter=4099 KiB

4099 KiB ≈ 1024 × 1024 × 4 bytes = 4 MiB — the entire decoded pixel buffer leaked on every call. Process OOMs in seconds under sustained load on malformed JPEGs.

Cause

canvas_jpeg_error_exit is registered as libjpeg's error_exit callback:

static void canvas_jpeg_error_exit(j_common_ptr cinfo) {
  canvas_jpeg_error_mgr *cjerr = static_cast<canvas_jpeg_error_mgr*>(cinfo->err);
  cjerr->output_message(cinfo);
  longjmp(cjerr->setjmp_buffer, 1);
}

When this fires from inside jpeg_read_scanlines (called by decodeJPEGIntoSurface's decode loop), the longjmp unwinds the stack back to the setjmp in loadJPEGFromBuffer. The local variables data and src in decodeJPEGIntoSurface — allocated with new uint8_t[] — are silently dropped.

Fix

Track both pointers on the canvas_jpeg_error_mgr struct and free them in canvas_jpeg_error_exit before the longjmp:

struct canvas_jpeg_error_mgr: jpeg_error_mgr {
    Image* image;
    uint8_t* data = nullptr;
    uint8_t* src = nullptr;
    jmp_buf setjmp_buffer;
};
static void canvas_jpeg_error_exit(j_common_ptr cinfo) {
  canvas_jpeg_error_mgr *cjerr = static_cast<canvas_jpeg_error_mgr*>(cinfo->err);
  cjerr->output_message(cinfo);
  delete[] cjerr->data;
  delete[] cjerr->src;
  cjerr->data = nullptr;
  cjerr->src = nullptr;
  longjmp(cjerr->setjmp_buffer, 1);
}

In decodeJPEGIntoSurface, set err->data = data; and err->src = src; after the allocations succeed, and clear them after the buffers are handed off (to _data) or freed explicitly. This way the longjmp-fired error handler always sees the correct ownership, with no double-free on the success path.