fix: buffer tar.gz stream to prevent Content-Length mismatch in file upload by baptistecolle · Pull Request #1112 · e2b-dev/E2B

baptistecolle · 2026-02-01T12:33:27Z

Summary

tarFileStreamUpload in packages/js-sdk/src/template/utils.ts calls tarFileStream twice — once to calculate Content-Length by consuming the stream, then again to create the upload body. Since gzip compression is non-deterministic (internal dictionary state, portable mode timing), the second stream can produce a different byte count than the first. This causes Node's fetch to throw:

RequestContentLengthMismatchError: Request body length does not match content-length header

which surfaces as a FileUploadError.

How to reproduce

Use .copy() with a directory containing many files (e.g. ~100 files)
The upload intermittently fails with RequestContentLengthMismatchError
Smaller single-file copies may not trigger it because the gzip variance is negligible

Fix

Buffer the tar.gz stream into memory once, then use that buffer for both the Content-Length header and the upload body. This eliminates the two-pass approach entirely.

Replaces the two calls to tarFileStream with a single call
Collects all chunks into a Uint8Array buffer
Uses buffer.length for content length and the buffer itself as the upload body

changeset-bot · 2026-02-01T12:33:48Z

⚠️ No Changeset found

Latest commit: 937c73c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ca83cc0b41

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-02-01T12:34:49Z

packages/js-sdk/src/template/utils.ts

+  for await (const chunk of stream as unknown as AsyncIterable<Uint8Array>) {
+    chunks.push(chunk)
+  }
+  const totalLength = chunks.reduce((sum, c) => sum + c.length, 0)
+  const buffer = new Uint8Array(totalLength)


Avoid buffering entire tarball in memory

This change now reads the full tar.gz stream into an in-memory Uint8Array before uploading. For large templates (e.g., multi‑GB directories), this can exhaust process memory or trigger GC thrashing, causing uploads to fail or the process to crash. Previously the data was streamed, which bounded memory usage. If users upload large directories, this is a regression in resource usage. Consider using a single stream pass with a tee (counting bytes while streaming) or buffering to a temp file instead of RAM.

Useful? React with 👍 / 👎.

I solved this with my new commit

baptistecolle · 2026-02-01T12:51:10Z

As requested in the CONTRIBUTING.md, I also started a discussion on the Discord server to bring visibility to this issue and get feedback from the community:

https://discord.com/channels/1092455714431180995/1467502046021160961

…upload tarFileStreamUpload called tarFileStream twice — once to calculate Content-Length by consuming the stream, then again to create the upload body. Since gzip compression is non-deterministic (internal dictionary state, timing), the second stream can produce a different byte count, causing fetch to throw RequestContentLengthMismatchError. Buffer the stream into memory on the first pass and reuse that buffer for both the content length and the upload body.

mishushakov · 2026-02-02T18:47:49Z

Hey there, thanks for the PR!
I believe we fixed this issue already in JS SDK 2.10.5 (#1095).

Can you please try it and let us know if the issue still persists?

baptistecolle · 2026-02-03T08:17:35Z

Thanks for taking a quick look @mishushakov !

Unfortunately, the previous PR does not resolve the issue. I am currently using e2b@2.12.0.

If you want a reproducible (though non-minimal) setup, this repository should demonstrate the problem:
https://github.com/baptistecolle/bap/tree/main/app/src/e2b-template

If I am not mistaken, even with the previous fix applied, the gzip output is still non-deterministic. The zlib/gzip algorithm keeps internal state that can change between runs, so compressing the same input twice can still result in different byte outputs.

mishushakov · 2026-02-03T09:20:52Z

Okay, can you try sending the archive without the content-length header? I think it should work. I am hesitant of using temp files

mishushakov · 2026-02-03T16:33:50Z

I have tried building your template, but on my computer it ran without any issues:

0.0s  | 05:31:18 PM INFO  Requesting build for template: bap-agent-dev
1.1s  | 05:31:19 PM INFO  Template created with ID: ibb5eo1pjpaga0jnryny, Build ID: 2d44b2e3-76b7-4afd-ae6c-f4622c08361f
2.6s  | 05:31:21 PM INFO  Uploaded 'opencode.json'
2.6s  | 05:31:21 PM INFO  Uploaded 'plugins/integration-permissions.ts'
2.7s  | 05:31:21 PM INFO  Uploaded 'cli'
2.7s  | 05:31:21 PM INFO  All file uploads completed

What operating system are you using?

noamzbr · 2026-02-04T08:52:04Z

@mishushakov as of yesterday (around 10 GMT), we started experiencing the same RequestContentLengthMismatchError: Request body length does not match content-length header error that originally triggered me to write the previous fix.

I can confirm that the this fix (@baptistecolle 🙏 ) fixes the issue for us as well. It is intermittent - without the fix (either on 2.10.5 or 2.12.0) we sometimes get the RequestContentLengthMismatchError error, and sometimes the following undici error:

TypeError: fetch failed
  [cause]: SocketError: other side closed
  code: 'UND_ERR_SOCKET'
  socket: {
    localAddress: 'XXX',
    localPort: 55335,
    remoteAddress: '142.250.75.219',
    remotePort: 443,
    remoteFamily: 'IPv4',
    bytesWritten: 0,
    bytesRead: 0
  }

Node: 22.16.0
undici: 6.21.2
MacOs

baptistecolle · 2026-02-04T08:53:34Z

What operating system are you using?

I am on Mac

baptistecolle · 2026-02-04T08:56:16Z

can you try sending the archive without the content-length header?

I am still seeing the same issue with the proposed fix. To be honest, I am currently traveling, so I have not had much time to dig into it today. It is possible that my attempt to remove the Content-Length header is not fully correct. I can try tomorrow to spend a bit more time on it

baptistecolle · 2026-02-05T13:03:22Z

So I looked into this further, and without the Content-Length header, I’m hitting request timeouts. Because of that, I wasn’t able to remove the “archive without the Content-Length header” part. From what I found, for a signed “simple upload” to GCS, Content-Length is effectively required, so I don’t think it’s possible to remove it.

@noamzbr Which operating system are you using? Also, do you have an example template that @mishushakov could use to reproduce the issue?

@mishushakov What operating system are you on? Also, could you clarify why you’re hesitant to use temporary files? I’m just trying to explore alternative solutions.

I think the main issue is the double gzip call, which is why I’m looking for a way to avoid it. Do you have any ideas or suggestions?

mishushakov · 2026-02-05T13:05:58Z

I am using Mac and we test on Linux and Windows on the CI. But I think we might be not catching anything in tests. I have a solution in mind that does use a single multiplexed stream instead of two different streams, but on sicko leave now - will implement when I feel better. Thanks

baptistecolle · 2026-02-05T13:09:14Z

will implement when I feel better.

Thanks a lot! 🔥

(FYI, I updated my previous comment: for a signed “simple upload” to GCS, Content-Length is required, which is why I was getting a timeout.)

So yes, I think the solution is to use a single multiplexed stream instead of two separate streams.

Either way, thanks for the quick responses, @mishushakov. Let me know if I can help. Otherwise, I’ll let you handle the rest of the PR.

on sick leave now

Rest well! 🤒

mishushakov · 2026-02-06T15:45:58Z

Hey both, can you try this branch?
#1118

baptistecolle requested review from ValentaTomas, jakubno and mishushakov as code owners February 1, 2026 12:33

chatgpt-codex-connector bot reviewed Feb 1, 2026

View reviewed changes

baptistecolle force-pushed the fix/gzip-stream-mismatch branch from ca83cc0 to 937c73c Compare February 1, 2026 12:52

Conversation

baptistecolle commented Feb 1, 2026

Summary

How to reproduce

Fix

Uh oh!

changeset-bot bot commented Feb 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Feb 1, 2026

Choose a reason for hiding this comment

Uh oh!

baptistecolle Feb 1, 2026

Choose a reason for hiding this comment

Uh oh!

baptistecolle commented Feb 1, 2026

Uh oh!

mishushakov commented Feb 2, 2026

Uh oh!

baptistecolle commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mishushakov commented Feb 3, 2026

Uh oh!

mishushakov commented Feb 3, 2026

Uh oh!

noamzbr commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

baptistecolle commented Feb 4, 2026

Uh oh!

baptistecolle commented Feb 4, 2026

Uh oh!

baptistecolle commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mishushakov commented Feb 5, 2026

Uh oh!

baptistecolle commented Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mishushakov commented Feb 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

changeset-bot bot commented Feb 1, 2026 •

edited

Loading

baptistecolle commented Feb 3, 2026 •

edited

Loading

noamzbr commented Feb 4, 2026 •

edited

Loading

baptistecolle commented Feb 5, 2026 •

edited

Loading

baptistecolle commented Feb 5, 2026 •

edited

Loading