Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,16 @@ tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
ulid = "1"
uuid = { version = "1", features = ["v4"] }
aegis = { version = "0.9", features = ["raf"], optional = true }
base64 = { version = "0.22", optional = true }
hctr2-rs = { version = "0.9", optional = true }
hmac-sha256 = { version = "1", optional = true }

[features]
default = []
fuse = ["dep:fuser"]
nfs = ["dep:nfsserve"]
encrypt = ["dep:aegis", "dep:base64", "dep:hctr2-rs", "dep:hmac-sha256"]
vendored-openssl = ["openssl-sys/vendored"]

[dependencies.openssl-sys]
Expand Down
32 changes: 32 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@ cargo build --release --features fuse

# All backends
cargo build --release --features fuse,nfs

# With client-side encryption support
cargo build --release --features nfs,encrypt
```

Binaries: `target/release/hf-mount`, `target/release/hf-mount-nfs`, `target/release/hf-mount-fuse`
Expand Down Expand Up @@ -129,6 +132,29 @@ hf-mount start --hf-token $HF_TOKEN --read-only bucket myuser/my-bucket /tmp/dat
hf-mount start --hf-token $HF_TOKEN bucket myuser/my-bucket/checkpoints /tmp/ckpts
```

### Client-side encryption

Files can be encrypted locally before upload and decrypted transparently on read. The remote side only ever sees ciphertext. Requires building with `--features encrypt`.

```bash
# Generate a 256-bit master key (same key works for all algorithm variants)
head -c 32 /dev/urandom > /path/to/key.bin

# Mount with encryption (writes encrypt, reads decrypt)
hf-mount start --encryption-key-file /path/to/key.bin bucket myuser/encrypted-bucket /tmp/data

# Use a different algorithm (default: aegis-128x2)
hf-mount start --encryption-key-file /path/to/key.bin --encryption-algorithm aegis-256 bucket myuser/my-bucket /tmp/data
```

Each file records its algorithm in remote metadata, so files encrypted with different algorithms can coexist and are always read with the correct variant. Supported algorithms: `aegis-128l`, `aegis-128x2` (default), `aegis-128x4`, `aegis-256`, `aegis-256x2`, `aegis-256x4`.

Filenames are also encrypted automatically.

The master key is always 32 bytes regardless of algorithm.

Directory structure and metadata (file sizes, timestamps) are not encrypted.

### Manage mounts

```bash
Expand Down Expand Up @@ -224,6 +250,8 @@ hf-mount stop /tmp/data # daemon mounts
| `--uid` / `--gid` | current user | Override UID/GID for mounted files |
| `--fuse-owner-only` | `false` | Restrict mount access to the mounting user only (FUSE only; by default all users can access, which requires `user_allow_other` in /etc/fuse.conf) |
| `--token-file` | | Path to a token file (re-read on each request for credential rotation) |
| `--encryption-key-file` | | Path to 256-bit master key file; enables client-side encryption (requires `--features encrypt`) |
| `--encryption-algorithm` | `aegis-128x2` | Encryption algorithm (requires `--encryption-key-file`) |

### Logging

Expand All @@ -240,6 +268,7 @@ RUST_LOG=hf_mount=debug hf-mount-fuse repo gpt2 /mnt/gpt2
- **Advanced writes** (`--advanced-writes`) -- staging files on disk, random writes + seek, async debounced flush
- **Remote sync** -- background polling detects remote changes and updates the local view
- **POSIX metadata** -- chmod, chown, timestamps, symlinks (in-memory only, lost on unmount)
- **Client-side encryption** (`--features encrypt`) -- AEGIS-based authenticated encryption via the [aegis](https://crates.io/crates/aegis) crate's RAF layer; per-source HKDF-SHA256 key derivation, six algorithm variants, per-file algorithm tracking, optional filename encryption via HCTR2-128

## Consistency model

Expand Down Expand Up @@ -303,6 +332,9 @@ See the [hf-csi-driver README](https://github.com/huggingface/hf-csi-driver#read
# Unit tests (no network, no token)
cargo test --lib --features fuse,nfs

# Unit tests with encryption
cargo test --lib --features encrypt

# Integration tests (require HF_TOKEN and FUSE)
HF_TOKEN=... cargo test --release --features fuse,nfs --test fuse_ops -- --test-threads=1 --nocapture
HF_TOKEN=... cargo test --release --features fuse,nfs --test nfs_ops -- --test-threads=1 --nocapture
Expand Down
Loading