Skip to content

[Data Integrity] Implement atomic commit with write-ahead logging (WAL) #1499

@bbopen

Description

@bbopen

Summary

Database commits are not atomic - a crash during commit can leave the database in an inconsistent state with orphaned blocks. Implement a write-ahead log (WAL) for atomic commits with crash recovery.

Problem

In core/blockstore/commitor.ts, blocks are written individually. If a crash occurs mid-commit:

  • Some blocks may be written, others not
  • The meta record may point to non-existent blocks
  • Orphaned blocks consume storage without being reachable

Proposed Solution

  1. Design WAL entry format for commit operations
  2. Create WALService at core/blockstore/wal-service.ts
  3. Implement IndexedDB WAL storage
  4. Implement two-phase commit protocol:
    • Begin: Write WAL entry with commit intent
    • Record: Write blocks with WAL reference
    • Commit: Update meta record
    • Finalize: Remove WAL entry
  5. Implement rollback with orphan cleanup
  6. Implement crash recovery on database open
  7. Integrate WAL into commitor

Expected Outcomes

  • All blocks in commit succeed or fail together
  • Partial failures trigger complete rollback
  • Crash recovery restores consistent state
  • Optional feature flag for WAL

Dependencies

Should be implemented after the validateBlocks async fix (PR #1494).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions