Skip to content

fix(store): migrate redb 2.x tuple tables on open#105

Merged
Frando merged 1 commit into
n0-computer:mainfrom
cbenhagen:fix/redb-v2-tuple-migration
Jun 1, 2026
Merged

fix(store): migrate redb 2.x tuple tables on open#105
Frando merged 1 commit into
n0-computer:mainfrom
cbenhagen:fix/redb-v2-tuple-migration

Conversation

@cbenhagen
Copy link
Copy Markdown
Contributor

Description

Fixes #104.

Stores created or opened by iroh-docs 0.94 through 0.98 (redb 2.6) cannot be opened under 0.99+ (redb 4): Tables::new fails with TableTypeMismatch on records-1, records-by-key-1, and latest-by-author-1. The issue write-up has the full root cause analysis. In short, redb 3.0 changed the on-disk type tag for variable-width tuples, and #100 jumped from redb 2.6 straight to redb 4.1, skipping the redb 3.x release line where the documented Legacy<...> migration would have run. By the time a user is on 0.99+, the migration window is closed inside any release pinned to redb 4 (which dropped the Legacy type entirely).

This PR keeps redb 4.1 as the primary dependency and adds redb 3.1 alongside it for the one job that requires it. When Store::persistent sees a TableTypeMismatch from the initial Tables::new, it:

  1. Opens the file with redb 3.
  2. Reads the three affected tables through Legacy<...> wrappers (which match the redb 2.6 stamping).
  3. Copies every table into a fresh redb 3 file at a temp path in the same directory, using plain tuple types on the write side. That commit re-stamps the metadata as Internal2.
  4. Renames the original to <path>.backup-redb-v2-tuples and persists the temp file in its place.

redb 4 then opens the result normally and the existing data migrations (migrations::run_migrations) run as before.

Changes:

  • Cargo.toml: add redb_v3 = { package = "redb", version = "3.1" }.
  • src/store/fs/migrate_redb_v2_tuples.rs (new): migration logic. Uses the same migrate_table! / migrate_multimap_table! macro style as the existing migrate_v1_v2.rs.
  • src/store/fs.rs: Store::persistent catches TableError::TableTypeMismatch from new_impl, runs the migration, and retries once.
  • Tests: one synthesises a redb 2.x-shaped file by writing through Legacy<...> with redb 3, then asserts (a) redb 4 rejects it before migration, (b) Store::persistent succeeds, (c) the backup file is created, and (d) all rows survive when read back with redb 4 plain types. A second test asserts a fresh open never creates a backup file.

Breaking Changes

None for users already on 0.99+. Users coming from 0.94..=0.98 will see their original file renamed to <path>.backup-redb-v2-tuples on first open. The new file at the original path holds the same data.

Notes & open questions

  • redb 3 needs to ship as long as we want to migrate stores written by 0.94..=0.98 directly. Dropping the dependency later means those users would have to install an intermediate iroh-docs release first.
  • The fixed-width tuple tables (namespaces-2, sync-peers-1) are not affected, because variable-width tuples are the only thing redb 3 changed the tag for. The migration still copies them so the resulting file is a clean redb 3 write top to bottom.

Change checklist

  • Self-review.
  • Documentation updates following the style guide, if relevant.
  • Tests if relevant.
  • All breaking changes documented.

@n0bot n0bot Bot added this to iroh May 28, 2026
@github-project-automation github-project-automation Bot moved this to 🚑 Needs Triage in iroh May 28, 2026
Comment thread src/store/fs.rs Outdated
Comment thread Cargo.toml Outdated
@cbenhagen cbenhagen force-pushed the fix/redb-v2-tuple-migration branch from 38f027d to 6025998 Compare May 29, 2026 05:57
Stores written by iroh-docs 0.94..=0.98 (redb 2.6) carry the old
on-disk type tag for variable-width tuples on records-1,
records-by-key-1, and latest-by-author-1. Opening them under 0.99+
(redb 4) fails with TableTypeMismatch because the 2 to 4 jump in
n0-computer#100 skipped the redb 3.x Legacy<...> migration window.

When Store::persistent hits TableTypeMismatch, open the file with
redb 3 via Legacy wrappers on the three affected tables, copy
everything into a fresh redb 3 file with plain tuple types, and
swap files. The original is preserved at
<path>.backup-redb-v2-tuples. redb 4 then opens the result.

Closes n0-computer#104
@cbenhagen cbenhagen force-pushed the fix/redb-v2-tuple-migration branch from 6025998 to 7cf5269 Compare May 29, 2026 07:01
Copy link
Copy Markdown
Member

@matheus23 matheus23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, but ideally I'd like a look from @Frando once he's back :)

Copy link
Copy Markdown
Member

@Frando Frando left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks you for this PR and sorry for not catching this ourselves.

Code looks good to me. I assume you also tested this on your actual databases, and everything worked out well?

Let's get this into the next release. We can then remove the migration code in a release or two later again I'd say.

@cbenhagen
Copy link
Copy Markdown
Contributor Author

No worries. Yes I tested this with some of my databases but haven't deployed it to the whole fleet yet. I agree we could remove this later this year.

@Frando Frando merged commit fc89461 into n0-computer:main Jun 1, 2026
25 checks passed
@github-project-automation github-project-automation Bot moved this from 🚑 Needs Triage to ✅ Done in iroh Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

Stores written by 0.94 to 0.98 fail to open under 0.99+ with TableTypeMismatch

4 participants