Skip to content

feat(env): FileSystem abstraction#37

Merged
alexsnaps merged 9 commits into
mainfrom
env
Mar 23, 2026
Merged

feat(env): FileSystem abstraction#37
alexsnaps merged 9 commits into
mainfrom
env

Conversation

@alexsnaps
Copy link
Copy Markdown
Collaborator

Env Abstraction — Implementation Log

Pluggable filesystem backend replacing hardcoded std::fs / libc I/O.

Steps

  • Step 1: Define traits + PosixFileSystem — Create src/env.rs with SequentialFile, RandomAccessFile,
    WritableFile, FileLock, FileSystem traits and PosixFileSystem implementation. Add Options::file_system: Arc<dyn FileSystem> defaulting to PosixFileSystem. No existing code changes beyond the new file and options field.
  • Step 2: LogWriter and LogReader — Change LogWriter to wrap Box<dyn WritableFile> (instead of
    BufWriter<File>) and LogReader to wrap Box<dyn SequentialFile> (instead of File). Update all callers
    (WAL creation, MANIFEST creation, WAL replay, repair).
  • Step 3: Table reader — Change Table::open to take Arc<dyn RandomAccessFile> instead of File.
    read_block and footer reads go through the trait. Table stores Arc<dyn RandomAccessFile>.
  • Step 4: TableBuilder — Change constructor to take Box<dyn WritableFile> instead of File.
  • Step 5: TableCacheget_or_open calls fs.open_random_access(path) instead of File::open.
    TableCache stores Arc<dyn FileSystem>.
  • Step 6: VersionSet — MANIFEST creation/reading and CURRENT file read/write go through FileSystem.
    VersionSet::create and VersionSet::recover take or store the filesystem handle.
  • Step 7: lib.rs + stopgap cleanup — Replace all remaining std::fs calls (~30 sites) in Db::open,
    begin_flush, write_flush, finish_flush, do_compaction, delete_obsolete_files, Db::destroy,
    Db::repair, and acquire_lock. Persistence stores Arc<dyn FileSystem>. libc::flock moves into
    PosixFileSystem::lock_file. Also replace all stopgap PosixFileSystem / writable_from_file /
    random_access_from_file / sequential_from_file calls in production code (marked with TODO) with
    Options::file_system — threading the Arc<dyn FileSystem> from Options through Persistence, TableCache,
    VersionSet, and all free functions that create files. Test code may remain hardcoded to POSIX.
  • Step 8: Cleanup — Remove direct libc usage from lib.rs. Verify libc is only used inside
    PosixFileSystem. Run full test suite.

Signed-off-by: Alex Snaps <alex@wcgw.dev>
Signed-off-by: Alex Snaps <alex@wcgw.dev>
Signed-off-by: Alex Snaps <alex@wcgw.dev>
Signed-off-by: Alex Snaps <alex@wcgw.dev>
Signed-off-by: Alex Snaps <alex@wcgw.dev>
Signed-off-by: Alex Snaps <alex@wcgw.dev>
Signed-off-by: Alex Snaps <alex@wcgw.dev>
Signed-off-by: Alex Snaps <alex@wcgw.dev>
Signed-off-by: Alex Snaps <alex@wcgw.dev>
@alexsnaps alexsnaps merged commit 76f80f9 into main Mar 23, 2026
1 check passed
@alexsnaps alexsnaps deleted the env branch March 23, 2026 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant