Skip to content

fix(upload): resolve stream.push() after EOF error by replacing data …#4

Open
BlazePxly wants to merge 1 commit into
zenhosta:mainfrom
BlazePxly:fix-upload-stream-error
Open

fix(upload): resolve stream.push() after EOF error by replacing data …#4
BlazePxly wants to merge 1 commit into
zenhosta:mainfrom
BlazePxly:fix-upload-stream-error

Conversation

@BlazePxly

Copy link
Copy Markdown

…listener with pass-through stream

@BlazePxly

Copy link
Copy Markdown
Author

What this PR does

This PR resolves the Error [ERR_STREAM_PUSH_AFTER_EOF]: stream.push() after EOF issue that frequently occurs when users upload files (especially large ones) using the application.

Cause of the Issue

The original implementation attached a .on('data') listener directly to the Busboy fileStream to count uploaded bytes. This put the readable stream into flowing mode. Concurrently, this same flowing stream was passed down to the S3 SDK (@aws-sdk/lib-storage) and Google Drive API (googleapis).
Because the stream was already flowing, it caused race conditions and backpressure bypasses with the SDKs' internal stream iterators. If the SDK finished or aborted parsing early, the stream state would get corrupted, causing Node.js to throw a stream.push() after EOF error when Busboy attempted to push subsequent chunks from the multipart request.

Solution

  1. Replaced the fileStream.on('data') event listener with a native Transform stream (PassThrough) that counts the bytes and pipes the output. This correctly delegates backpressure handling to Node.js's stream pipeline and keeps the stream in a state compatible with both AWS SDK and Google APIs.
  2. Added fileStream.unpipe() in the catch block to prevent stream deadlocks/hangs if the SDK throws an upload error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant