Skip to content

fix(backup): right-size export batch sizing#22

Open
tymrtn wants to merge 1 commit into
mainfrom
fix/issue-20-right-size-backup-export-batching-for-large-mess
Open

fix(backup): right-size export batch sizing#22
tymrtn wants to merge 1 commit into
mainfrom
fix/issue-20-right-size-backup-export-batching-for-large-mess

Conversation

@tymrtn
Copy link
Copy Markdown
Owner

@tymrtn tymrtn commented May 5, 2026

Summary

  • lowers backup export's default and maximum batch sizing for large RFC822 payloads
  • validates export batch size separately from migrate batch sizing
  • adds regression tests for conservative backup export batch limits

Fixes #20

Test Plan

  • cargo fmt --check
  • cargo test -p envelope-email-transport backup -- --nocapture
  • cargo test -p envelope-email backup -- --nocapture

@tymrtn @claude
fix(backup): right-size export batch size for large messages (#20)
89ee6a6 
Backup export materializes full RFC822 bodies in memory per batch.
Reusing migrate's DEFAULT_BATCH_SIZE (25) and MAX_BATCH_SIZE (500) is
operationally unsafe for mailboxes with large messages — 25 × 50 MB =
1.25 GB peak heap.

Introduce export-specific constants:
- DEFAULT_EXPORT_BATCH_SIZE = 5 (≈250 MB worst-case at 50 MB/msg)
- MAX_EXPORT_BATCH_SIZE = 50 (hard ceiling; operators must monitor RSS)
- validate_export_batch_size() with a clear error explaining the memory
  tradeoff

CLI --batch-size help text now documents that export holds bodies in
memory and advises keeping the value low for large-message mailboxes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Right-size backup export batching for large messages

1 participant

@tymrtn