[User Story] RAM-Backed Ghost Secrets Delivery

[noahwhite]

Story Summary

As a Security Architect, I want to fetch Ghost secrets from Aembit into a tmpfs RAM disk during the Flatcar boot process, so that sensitive database and mail credentials never touch the persistent SSD.

Phase: 2 - Secure Application Bootstrapping

---

✅ Acceptance Criteria

• Systemd unit ghost-secrets-bootstrap.service runs successfully before Docker
• Secrets fetched from Aembit proxy and written to /run/ghost/secrets/
• /run/ghost/secrets/ is a tmpfs mount (RAM-backed, never persisted)
• Secrets verified to exist in /run/ghost/secrets/ and nowhere else on host
• Service fails gracefully if Aembit proxy unavailable (with clear error)
• Secrets survive container restarts but not host reboots (as expected)

---

📝 Additional Context

The "Secret Pipeline"

Aembit Proxy (Auth via enrollment token)
    │
    ▼
Systemd Oneshot (Fetches via curl)
    │
    ▼
Flatcar /run/ (tmpfs) ← Storage in volatile RAM
    │
    ▼
Docker Engine (Mounts RAM file as read-only container secret)
    │
    ▼
Ghost Process (Reads secret from /run/secrets/ in container memory)

Boot Sequence

Instance Boot
    │
    ├─► systemd-sysupdate downloads aembit sysext
    ├─► systemd-sysext merges extensions
    │
    ├─► aembit-proxy.service starts
    │   └─► Enrolls with Aembit (consumes enrollment token)
    │   └─► Proxy ready on localhost:8080
    │
    ├─► ghost-secrets-bootstrap.service starts
    │   └─► Waits for aembit-proxy.service
    │   └─► Creates tmpfs at /run/ghost/secrets/
    │   └─► Fetches secrets from proxy
    │   └─► Writes to /run/ghost/secrets/*.secret (mode 0600)
    │
    └─► docker.service starts
        └─► Containers mount /run/ghost/secrets/

Butane Configuration

storage:
  files:
    - path: /opt/bin/ghost-secrets-bootstrap.sh
      mode: 0755
      contents:
        inline: |
          #!/bin/bash
          set -euo pipefail
          
          SECRETS_DIR="/run/ghost/secrets"
          PROXY_URL="http://localhost:8080"
          
          # Create tmpfs mount with hardened options
          mkdir -p "$SECRETS_DIR"
          mount -t tmpfs -o size=10M,mode=0700,nosuid,nodev,noexec tmpfs "$SECRETS_DIR"
          
          # Fetch secrets from Aembit proxy
          for secret in mysql_root_password mysql_password ghost_mail_password; do
            curl -sf "$PROXY_URL/secrets/$secret" > "$SECRETS_DIR/$secret"
            chmod 0600 "$SECRETS_DIR/$secret"
            chown root:root "$SECRETS_DIR/$secret"
          done
          
          echo "Secrets bootstrapped to $SECRETS_DIR"

systemd:
  units:
    - name: ghost-secrets-bootstrap.service
      enabled: true
      contents: |
        [Unit]
        Description=Bootstrap Ghost secrets from Aembit
        After=aembit-proxy.service
        Requires=aembit-proxy.service
        Before=docker.service
        
        [Service]
        Type=oneshot
        RemainAfterExit=yes
        ExecStart=/opt/bin/ghost-secrets-bootstrap.sh
        
        [Install]
        WantedBy=multi-user.target

---

✅ Definition of Done (Verification Tests)

1. Verifiable In-Memory Residence

The secret files must exist only in a tmpfs partition.

# Test: Verify filesystem type
findmnt /run/ghost/secrets
# FSTYPE must be "tmpfs"

# Test: Verify hardened mount options
grep /run/ghost/secrets /proc/mounts
# Must show: rw,nosuid,nodev,noexec

2. Hardened File Permissions (The 600 Rule)

Secret files must be restricted to root only.

# Test: Verify permissions
ls -la /run/ghost/secrets/
# Must show: -rw------- (600), owned by root:root

3. Automatic Lifecycle Cleanup

Secrets must be "born" and "die" with the VM session.

# Test: Create test file, reboot, verify cleanup
echo "test" > /run/ghost/secrets/test_file
sudo reboot

# After reboot:
ls /run/ghost/secrets/
# Directory should be empty or non-existent until oneshot repopulates
# Proves no data persisted to Vultr block storage

4. No Secrets on Persistent Storage

# Test: Search for secrets on persistent storage
grep -r "MYSQL_ROOT_PASSWORD_VALUE" /var/mnt/storage/ 2>/dev/null
# Must return no results

# Test: Verify block storage has no secret files
find /var/mnt/storage -name "*.secret" 2>/dev/null
# Must return no results

---

📦 Definition of Ready

• Acceptance criteria defined
• Blocked by GHO-66 (enrollment), GHO-67 (sysext)
• Story is estimated
• Team has necessary skills and access
• Priority is clear
• Business value understood

[linear]

GHO-68