Skip to content

DOLT.md

Latest commit

 

History

History
359 lines (258 loc) · 8.55 KB

DOLT.md

File metadata and controls

359 lines (258 loc) · 8.55 KB

Dolt Backend for Beads

Beads uses Dolt as its storage backend. Dolt provides a version-controlled SQL database with cell-level merge, native branching, and multi-writer support via server mode.

Why Dolt?

  • Native version control — cell-level diffs and merges, not line-based
  • Multi-writer support — server mode enables concurrent agents
  • Built-in history — every write creates a Dolt commit
  • Native branching — Dolt branches independent of git branches

Getting Started

New Project

# Embedded mode (single writer)
bd init

# Server mode (multi-writer)
gt dolt start           # Start the Dolt server
bd init --server        # Initialize with server mode

Migrate from SQLite (Legacy)

If upgrading from an older version that used SQLite:

# Preview the migration
bd migrate --to-dolt --dry-run

# Run the migration
bd migrate --to-dolt

# Optionally clean up SQLite files
bd migrate --to-dolt --cleanup

Migration creates backups automatically. Your original SQLite database is preserved as beads.backup-pre-dolt-*.db.

Modes of Operation

Embedded Mode (Default)

Single-process access to the Dolt database. Good for development and single-agent use.

# .beads/config.yaml (or auto-detected)
dolt:
  mode: embedded

Characteristics:

  • No server process needed
  • Single writer at a time
  • Direct database access

Server Mode (Multi-Writer)

Connects to a running dolt sql-server for multi-client access.

# Start the server (Gas Town)
gt dolt start

# Or manually
cd ~/.dolt-data/beads && dolt sql-server --port 3307
# .beads/config.yaml
dolt:
  mode: server
  host: 127.0.0.1
  port: 3307
  user: root

Server mode is required for:

  • Multiple agents writing simultaneously
  • Gas Town multi-rig setups
  • Federation with remote peers

Federation (Peer-to-Peer Sync)

Federation enables direct sync between Dolt installations without a central hub.

Architecture

┌─────────────────┐         ┌─────────────────┐
│   Gas Town A    │◄───────►│   Gas Town B    │
│  dolt sql-server│  sync   │  dolt sql-server│
│  :3306 (sql)    │         │  :3306 (sql)    │
│  :8080 (remote) │         │  :8080 (remote) │
└─────────────────┘         └─────────────────┘

In federation mode, the server exposes two ports:

  • MySQL (3306): Multi-writer SQL access
  • remotesapi (8080): Peer-to-peer push/pull

Quick Start

# Add a peer
bd federation add-peer town-beta 192.168.1.100:8080/beads

# With authentication
bd federation add-peer town-beta host:8080/beads --user sync-bot

# Sync with all peers
bd federation sync

# Handle conflicts
bd federation sync --strategy theirs  # or 'ours'

# Check status
bd federation status

Topologies

Pattern Description Use Case
Hub-spoke Central hub, satellites sync to hub Team with central coordination
Mesh All peers sync with each other Decentralized collaboration
Hierarchical Tree of hubs Multi-team organizations

Credentials

Peer credentials are AES-256 encrypted, stored locally, and used automatically during sync:

# Credentials prompted interactively
bd federation add-peer name url --user admin

# Stored in federation_peers table (encrypted)

Troubleshooting

# Check federation health
bd doctor --deep

# Verify peer connectivity
bd federation status

Contributor Onboarding (Clone Bootstrap)

When someone clones a repository that uses Dolt backend:

  1. On first bd command (e.g., bd list), bootstrap runs automatically
  2. A fresh Dolt database is created
  3. If a Dolt remote is configured, data is pulled from the remote
  4. Work continues normally

No manual steps required. The bootstrap:

  • Detects fresh clone (no Dolt database yet)
  • Acquires a lock to prevent race conditions
  • Initializes the Dolt database and pulls from configured remotes
  • Creates initial Dolt commit

Verifying Bootstrap Worked

bd list              # Should show issues
bd vc log            # Should show initial commit

Troubleshooting

Server Not Running

Symptom: Connection refused errors when using server mode.

failed to create database: dial tcp 127.0.0.1:3307: connect: connection refused

Fix:

gt dolt start        # Gas Town command
# Or
gt dolt status       # Check if running

Bootstrap Not Running

Symptom: bd list shows nothing on fresh clone.

Check:

ls .beads/dolt/            # Should NOT exist (pre-bootstrap)
BD_DEBUG=1 bd list         # See bootstrap output

Force bootstrap:

rm -rf .beads/dolt         # Remove broken state
bd list                    # Re-triggers bootstrap

Database Corruption

Symptom: Queries fail, inconsistent data.

Diagnosis:

bd doctor                  # Basic checks
bd doctor --deep           # Full validation
bd doctor --server         # Server mode checks (if applicable)

Recovery options:

  1. Repair what's fixable:

    bd doctor --fix

  2. Rebuild from remote:

    rm -rf .beads/dolt
bd list                  # Re-triggers bootstrap

Lock Contention (Embedded Mode)

Symptom: "database is locked" errors.

Embedded mode is single-writer. If you need concurrent access:

# Switch to server mode
gt dolt start
bd config set dolt.mode server

Configuration Reference

# .beads/config.yaml

# Dolt settings
dolt:
  # Auto-commit Dolt history after writes (default: on for embedded, off for server)
  auto-commit: on        # on | off

  # Server mode settings (when mode: server)
  mode: embedded         # embedded | server
  host: 127.0.0.1
  port: 3307
  user: root
  # Password via BEADS_DOLT_PASSWORD env var

Environment Variables

Variable Purpose
BEADS_DOLT_PASSWORD Server mode password
BEADS_DOLT_SERVER_MODE Enable server mode (set to "1")
BEADS_DOLT_SERVER_HOST Server host (default: 127.0.0.1)
BEADS_DOLT_SERVER_PORT Server port (default: 3307)
BEADS_DOLT_SERVER_TLS Enable TLS (set to "1" or "true")
BEADS_DOLT_SERVER_USER MySQL connection user
DOLT_REMOTE_USER Push/pull auth user
DOLT_REMOTE_PASSWORD Push/pull auth password
BD_DOLT_AUTO_COMMIT Override auto-commit setting

Dolt Version Control

Dolt maintains its own version history, separate from Git:

# View Dolt commit history
bd vc log

# Show diff between Dolt commits
bd vc diff HEAD~1 HEAD

# Create manual checkpoint
bd vc commit -m "Checkpoint before refactor"

Auto-Commit Behavior

In embedded mode (default), each bd write command creates a Dolt commit:

bd create "New issue"    # Creates issue + Dolt commit

In server mode, auto-commit defaults to OFF because the server manages its own transaction lifecycle. Firing DOLT_COMMIT after every write under concurrent load causes 'database is read only' errors.

Override for batch operations (embedded) or explicit commits (server):

bd --dolt-auto-commit off create "Issue 1"
bd --dolt-auto-commit off create "Issue 2"
bd vc commit -m "Batch: created issues"

Server Management (Gas Town)

Gas Town provides integrated Dolt server management:

gt dolt start            # Start server (background)
gt dolt stop             # Stop server
gt dolt status           # Show server status
gt dolt logs             # View server logs
gt dolt sql              # Open SQL shell

Server runs on port 3307 (avoids MySQL conflict on 3306).

Data Location

~/.dolt-data/
├── beads/               # HQ database
├── beads_rig/           # Beads rig database
└── gastown/             # Gas Town database

Migration Cleanup

After successful migration from SQLite, you may have backup files:

.beads/beads.backup-pre-dolt-20260122-213600.db
.beads/sqlite.backup-pre-dolt-20260123-192812.db

These are safe to delete once you've verified Dolt is working:

# Verify Dolt works
bd list
bd doctor

# Then clean up (after appropriate waiting period)
rm .beads/*.backup-*.db

Recommendation: Keep backups for at least a week before deleting.

See Also