Skip to content

Spec 28: multi-device sync — opt-in client-side-encrypted BYO bucket #100

@0bserver07

Description

@0bserver07

Goal

Sync the local store across the user's own devices via their own S3 / R2 / B2 / MinIO bucket, client-side encrypted, no central service. Aggregate laptop + work machine + dev container into one view. Optional team mode: shared bucket + team keys.

Why now

Local-only is a feature, not a bug — but the user with 3 machines today has 3 disconnected stores. Solving this without breaking the no-telemetry promise is the test of whether the project can grow beyond solo use.

Schema

v023sync_state:

CREATE TABLE sync_state (
  id INTEGER PRIMARY KEY CHECK (id = 1),  -- single-row table
  device_id TEXT NOT NULL,                -- stable per-machine UUID
  bucket_url TEXT NOT NULL,
  last_sync_ts TEXT,
  last_sync_event_id INTEGER,
  encryption_key_fingerprint TEXT NOT NULL,
  remote_event_id INTEGER DEFAULT 0,
  conflict_count INTEGER DEFAULT 0
);

User-visible surface

  • CLI: stackunderflow sync init --bucket s3://my-bucket --key-from-env STACKUNDERFLOW_SYNC_KEY — generates a keypair; user keeps the private key, public-key fingerprint goes to the bucket.
  • CLI: stackunderflow sync push / stackunderflow sync pull — upload local-since-last-sync, pull remote-since-last-sync.
  • CLI: stackunderflow sync status.
  • CLI: stackunderflow sync auto --enable — daemon-thread continuous sync.
  • API: GET /api/sync/status.
  • No UI tab v1 — CLI + API only.

Implementation plan

  1. DESIGN PHASE FIRST (no agent dispatch until done):
    • Encryption: age (modern, library) or libsodium / NaCl. Pick one.
    • Conflict resolution: events are append-only, so conflict ≈ "two devices created an event with the same (provider, slug, ts)". Last-write-wins by ts? Or device-id-tiebreak? Document the policy.
    • Wire format: encrypted SQLite snapshot per device per day? Encrypted append-log? Per-event encrypted blobs? Each has tradeoffs (snapshot is simple but heavy; append-log is granular but complex). Recommend: per-device encrypted SQLite-incremental snapshots with rsync-style diff.
    • Multi-device merge: each device pulls every other device's snapshot, merges into a virtual UNION view. Doesn't write to remote on read.
    • Team mode: defer to v2. v1 is single-user-multi-device only.
  2. v023 migration.
  3. New optional dep [sync] with boto3 (S3-API compatible — works with R2, B2, MinIO) + pyrage (or pynacl) for encryption.
  4. New module stackunderflow/sync/keys.py, cipher.py, bucket.py, merge.py, runner.py.
  5. CLI commands.
  6. Background sync daemon (similar to watcher pattern).

Tests

  • Roundtrip: encrypt locally → upload to MinIO test container → pull from second "device" → decrypt → assert equality.
  • Conflict on same (provider, slug, ts) → last-write-wins.
  • Key-mismatch on pull → clean error, no data loss.
  • Bucket unreachable → graceful failure, retry queue.

Hard parts

  • Crypto. Get this wrong and the privacy promise is broken. Use a well-audited library (age or libsodium); do NOT roll our own. Document the threat model: bucket operator (S3 / R2) cannot read; only key-holders can.
  • Key management UX. Users will lose their keys. Document this loudly. Provide a "this is destructive" warning if the user runs sync init without backing up the prior key.
  • Storage cost. Encrypted incremental snapshots can grow. Prune old snapshots after merge confirmation; default keep-last-30-days.
  • Conflict resolution policy. This is product judgment. Maintainer should write the conflict-resolution doc before dispatch.

Out of scope

  • Team mode (defer to v2).
  • Cross-user discovery surfaces (someone else's sessions show up in my store).
  • Sync over peer-to-peer (no central bucket) — out of scope.

Dependencies

  • None blocking.
  • Encryption + conflict-resolution decisions need maintainer design call before dispatch.

Estimated effort

Size XL — single agent, ~4-6 hr after the design call lands. Could be split: keys+cipher / bucket+merge / CLI+daemon.

Hard rules

  • DO NOT touch versions / CHANGELOG headings.
  • Pre-assigned schema slot: v023.
  • Branch: feat/multi-device-sync off main.
  • Stays in needs-design until docs/specs/sync-protocol-v1.md lands.
  • Crypto library MUST be a well-audited dep (age, pynacl, cryptography). NO rolled-own crypto.

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-designNeeds human design call before implementationsize-xl~3-4 hr agent runspecSpec/feature for an agent to implementwave-6Wave 6: sensitive / long-tail

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions