Skip to content

[Data Integrity] Implement atomic commit with write-ahead logging (WAL) #1499

@bbopen

Description

@bbopen

Summary

Database commits are not atomic - a crash during commit can leave the database in an inconsistent state with orphaned blocks. Implement a write-ahead log (WAL) for atomic commits with crash recovery.

Problem

In core/blockstore/commitor.ts, blocks are written individually. If a crash occurs mid-commit:

  • Some blocks may be written, others not
  • The meta record may point to non-existent blocks
  • Orphaned blocks consume storage without being reachable

Proposed Solution

  1. Design WAL entry format for commit operations
  2. Create WALService at core/blockstore/wal-service.ts
  3. Implement IndexedDB WAL storage
  4. Implement two-phase commit protocol:
    • Begin: Write WAL entry with commit intent
    • Record: Write blocks with WAL reference
    • Commit: Update meta record
    • Finalize: Remove WAL entry
  5. Implement rollback with orphan cleanup
  6. Implement crash recovery on database open
  7. Integrate WAL into commitor

Expected Outcomes

  • All blocks in commit succeed or fail together
  • Partial failures trigger complete rollback
  • Crash recovery restores consistent state
  • Optional feature flag for WAL

Dependencies

Should be implemented after the validateBlocks async fix (PR #1494).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions