Warn users about file carving size limits#47376
Open
noahtalerman wants to merge 22 commits into
Open
Conversation
fleet-release
previously approved these changes
Jun 10, 2026
noahtalerman
commented
Jun 10, 2026
noahtalerman
commented
Jun 10, 2026
noahtalerman
commented
Jun 10, 2026
noahtalerman
commented
Jun 10, 2026
noahtalerman
commented
Jun 10, 2026
rachaelshaw
reviewed
Jun 18, 2026
Co-authored-by: Rachael Shaw <r@rachael.wtf>
fleet-release
previously approved these changes
Jun 18, 2026
rachaelshaw
previously approved these changes
Jun 18, 2026
Member
Author
Hey @sgress454 can you please review this PR? |
mike-j-thomas
left a comment
mike-j-thomas
left a comment
Member
There was a problem hiding this comment.
@noahtalerman, some fixes and grammar suggestions. Plus a bunch of line breaks that need fixing 👍
| ``` | ||
|
|
||
| For the (default) MySQL Backend, the configured `carver_block_size` must be less than the value of | ||
| `max_allowed_packet` in the MySQL connection, allowing for some overhead. The default for [MySQL 8](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_max_allowed_packet) it is 64MB (`67108864`). |
Member
There was a problem hiding this comment.
@noahtalerman, is this supposed to be on a separate line? Looks like a copy/paste bug.
Member
Author
There was a problem hiding this comment.
Thanks for catching this!
| `max_allowed_packet` in the MySQL connection, allowing for some overhead. The default for [MySQL 8](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_max_allowed_packet) it is 64MB (`67108864`). | ||
|
|
||
| For the S3-compatible backend, `carver_block_size` must be set to at least 5MiB (`5242880`) due to the | ||
| [constraints of S3's multipart |
| - `FLEET_S3_FORCE_S3_PATH_STYLE=true` | ||
| - `FLEET_S3_REGION=localhost` or any non-empty string otherwise Fleet will attempt to derive the region. | ||
|
|
||
| If you're testing file carving locally with the docker-compose environment, the `--dev` flag on Fleet server will automatically point carves to the local RustFS container and write to the `carves-dev` bucket (created automatically) without needing to set additional configuration. |
Member
There was a problem hiding this comment.
Suggested change
| If you're testing file carving locally with the docker-compose environment, the `--dev` flag on Fleet server will automatically point carves to the local RustFS container and write to the `carves-dev` bucket (created automatically) without needing to set additional configuration. | |
| If you're testing file carving locally with the Docker-Compose environment, the `--dev` flag on the Fleet server will automatically point carves to the local RustFS container and write to the `carves-dev` bucket (created automatically) without needing to set additional configuration. |
Member
There was a problem hiding this comment.
@noahtalerman, should "docker-compose" be a proper noun?
Member
Author
There was a problem hiding this comment.
Just took out mention of Docker entirely.
| ### Ensure `carver_block_size` is set appropriately | ||
|
|
||
| `carver_block_size` is an option that sets the size of each part of a file carve that Fleet's agent (fleetd) | ||
| sends to the Fleet server. |
Member
There was a problem hiding this comment.
@noahtalerman, looks like another copy/paste line break.
| sends to the Fleet server. | ||
|
|
||
| When using the MySQL backend (default), this value must be less than the `max_allowed_packet` | ||
| setting in MySQL. If it is too large, MySQL will reject the writes. |
Member
There was a problem hiding this comment.
Same here, and all lines up to 115.
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
noahtalerman
dismissed stale reviews from rachaelshaw and fleet-release
via
efa3f1f
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Co-authored-by: Mike Thomas <78363703+mike-j-thomas@users.noreply.github.com>
Member
Author
|
Thanks @mike-j-thomas! I think all these line break issues are because folks are using "Prettier" in their text editor which has rules to limit line length. That makes sense for Typescript / go code but not for markdown. I opened a separate PR to update Prettier's rules: |
Member
Author
|
@rachaelshaw and @sharon-fdm one more review! Apologies. After review, please merge in the PR if you're able to. |
sharon-fdm
previously approved these changes
Jun 19, 2026
rachaelshaw
reviewed
Jun 19, 2026
fleet-release
approved these changes
Jun 19, 2026
rachaelshaw
approved these changes
Jun 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.