fix(store): migrate redb 2.x tuple tables on open#105
Merged
Conversation
38f027d to
6025998
Compare
Stores written by iroh-docs 0.94..=0.98 (redb 2.6) carry the old on-disk type tag for variable-width tuples on records-1, records-by-key-1, and latest-by-author-1. Opening them under 0.99+ (redb 4) fails with TableTypeMismatch because the 2 to 4 jump in n0-computer#100 skipped the redb 3.x Legacy<...> migration window. When Store::persistent hits TableTypeMismatch, open the file with redb 3 via Legacy wrappers on the three affected tables, copy everything into a fresh redb 3 file with plain tuple types, and swap files. The original is preserved at <path>.backup-redb-v2-tuples. redb 4 then opens the result. Closes n0-computer#104
6025998 to
7cf5269
Compare
Frando
approved these changes
Jun 1, 2026
Member
Frando
left a comment
There was a problem hiding this comment.
Thanks you for this PR and sorry for not catching this ourselves.
Code looks good to me. I assume you also tested this on your actual databases, and everything worked out well?
Let's get this into the next release. We can then remove the migration code in a release or two later again I'd say.
Contributor
Author
|
No worries. Yes I tested this with some of my databases but haven't deployed it to the whole fleet yet. I agree we could remove this later this year. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #104.
Stores created or opened by iroh-docs 0.94 through 0.98 (redb 2.6) cannot be opened under 0.99+ (redb 4):
Tables::newfails withTableTypeMismatchonrecords-1,records-by-key-1, andlatest-by-author-1. The issue write-up has the full root cause analysis. In short, redb 3.0 changed the on-disk type tag for variable-width tuples, and #100 jumped from redb 2.6 straight to redb 4.1, skipping the redb 3.x release line where the documentedLegacy<...>migration would have run. By the time a user is on 0.99+, the migration window is closed inside any release pinned to redb 4 (which dropped theLegacytype entirely).This PR keeps redb 4.1 as the primary dependency and adds redb 3.1 alongside it for the one job that requires it. When
Store::persistentsees aTableTypeMismatchfrom the initialTables::new, it:Legacy<...>wrappers (which match the redb 2.6 stamping).Internal2.<path>.backup-redb-v2-tuplesand persists the temp file in its place.redb 4 then opens the result normally and the existing data migrations (
migrations::run_migrations) run as before.Changes:
Cargo.toml: addredb_v3 = { package = "redb", version = "3.1" }.src/store/fs/migrate_redb_v2_tuples.rs(new): migration logic. Uses the samemigrate_table!/migrate_multimap_table!macro style as the existingmigrate_v1_v2.rs.src/store/fs.rs:Store::persistentcatchesTableError::TableTypeMismatchfromnew_impl, runs the migration, and retries once.Legacy<...>with redb 3, then asserts (a) redb 4 rejects it before migration, (b)Store::persistentsucceeds, (c) the backup file is created, and (d) all rows survive when read back with redb 4 plain types. A second test asserts a fresh open never creates a backup file.Breaking Changes
None for users already on 0.99+. Users coming from 0.94..=0.98 will see their original file renamed to
<path>.backup-redb-v2-tupleson first open. The new file at the original path holds the same data.Notes & open questions
namespaces-2,sync-peers-1) are not affected, because variable-width tuples are the only thing redb 3 changed the tag for. The migration still copies them so the resulting file is a clean redb 3 write top to bottom.Change checklist