Skip to content

Commit e30b9d9

Browse files
blackcoffeexbtblackcoffeexbt
committed
Add ota docs
1 parent 3bce701 commit e30b9d9

1 file changed

Lines changed: 208 additions & 0 deletions

File tree

‎docs/ota-updates.md‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# OTA System Updates
2+
3+
LNbitsBox supports over-the-air system updates so users don't need to reflash their SD card for new releases. This works by leveraging NixOS's immutable store and a Cachix binary cache — CI builds the full system closure, pushes it to Cachix, and the Pi downloads and activates it directly. Zero compilation on the Pi.
4+
5+
## How It Works
6+
7+
```
8+
CI (GitHub Actions) Pi (NixOS)
9+
┌──────────────────────┐ ┌──────────────────────┐
10+
│ Build system toplevel│ │ Admin app checks │
11+
│ Push to Cachix cache │ │ GitHub Releases API │
12+
│ Upload manifest.json │ │ for new versions │
13+
│ to GitHub Release │ │ │
14+
└──────────────────────┘ │ Download closure │
15+
│ from Cachix │
16+
│ Activate new system │
17+
└──────────────────────┘
18+
```
19+
20+
1. **CI builds** the NixOS system toplevel (`nix build .#toplevel`) — this is the entire system closure (kernel, services, config, everything).
21+
2. **CI pushes** the closure to the `lnbitsbox` Cachix binary cache so it can be downloaded without rebuilding.
22+
3. **CI uploads** a `manifest.json` to the GitHub Release containing the Nix store path and version.
23+
4. **On the Pi**, the admin dashboard checks the GitHub Releases API for new versions.
24+
5. When the user clicks "Update Now", the Pi downloads `manifest.json`, fetches the closure from Cachix via `nix copy`, and activates it with `switch-to-configuration switch`.
25+
26+
The key insight: the Pi never evaluates any Nix expressions. It just downloads pre-built binaries and switches to them.
27+
28+
## Prerequisites
29+
30+
Before OTA updates will work, you need a Cachix binary cache set up:
31+
32+
1. **Create a Cachix account** at https://cachix.org
33+
2. **Create a cache** named `lnbitsbox` (or choose another name and update the references)
34+
3. **Note the public signing key** — it looks like `lnbitsbox.cachix.org-1:XXXXXXXX...`
35+
4. **Update `nixos/configuration.nix`** — replace `<PUBLIC_KEY_HERE>` with your cache's public key:
36+
```nix
37+
trusted-public-keys = [
38+
"cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
39+
"lnbitsbox.cachix.org-1:YOUR_ACTUAL_PUBLIC_KEY_HERE"
40+
];
41+
```
42+
5. **Add the Cachix auth token as a GitHub secret** — go to your repo's Settings > Secrets and add `CACHIX_AUTH_TOKEN` with the write token from your Cachix dashboard.
43+
44+
## Releasing a New Version
45+
46+
### Step 1: Bump the version
47+
48+
Edit `flake.nix` and update the version string:
49+
50+
```nix
51+
let
52+
version = "1.1.0"; # Bump this
53+
system = "aarch64-linux";
54+
in
55+
```
56+
57+
### Step 2: Commit and tag
58+
59+
```bash
60+
git add -A
61+
git commit -m "Release v1.1.0"
62+
git tag v1.1.0
63+
git push origin main --tags
64+
```
65+
66+
### Step 3: CI does the rest
67+
68+
Pushing a `v*` tag triggers the `build-pi4-image` workflow (`.github/workflows/build-image.yml`), which:
69+
70+
1. Builds the compressed SD image (`nix build .#sdImage`)
71+
2. Builds the system toplevel (`nix build .#toplevel`)
72+
3. Pushes the toplevel closure to Cachix (`cachix push lnbitsbox result-toplevel`)
73+
4. Creates `manifest.json` with the store path and version
74+
5. Uploads the SD image, checksums, and `manifest.json` to a GitHub Release
75+
76+
After the workflow completes, any LNbitsBox running an older version will see the update available in the admin dashboard.
77+
78+
## CI Workflow Details
79+
80+
The relevant steps in `.github/workflows/build-image.yml`:
81+
82+
| Step | What it does |
83+
|------|-------------|
84+
| **Set up LNbitsBox Cachix** | Configures Cachix with the `CACHIX_AUTH_TOKEN` secret for push access |
85+
| **Build SD image** | `nix build .#sdImage -L --out-link result-sdimage` — full SD card image |
86+
| **Build and push toplevel** | `nix build .#toplevel -L --out-link result-toplevel` then `cachix push` — the OTA closure |
87+
| **Create update manifest** | Reads the store path and version, writes `manifest.json` |
88+
| **Create GitHub Release** | Uploads `.img.zst`, `SHA256SUMS.txt`, and `manifest.json` |
89+
90+
The toplevel build and Cachix push use `continue-on-error: true` so that a Cachix misconfiguration won't block the SD image release.
91+
92+
## Manual Cachix Push (for developers)
93+
94+
If you need to push a build to Cachix manually (e.g., testing OTA without a full release cycle):
95+
96+
### Install Cachix
97+
98+
```bash
99+
nix-env -iA cachix -f https://cachix.org/api/v1/install
100+
```
101+
102+
### Authenticate
103+
104+
```bash
105+
cachix authtoken <your-auth-token>
106+
```
107+
108+
You can get your auth token from https://app.cachix.org (click your cache > Settings > Auth Tokens).
109+
110+
### Build and push the toplevel
111+
112+
```bash
113+
# Build the system toplevel
114+
nix build .#toplevel -L --out-link result-toplevel
115+
116+
# Push to the cache
117+
cachix push lnbitsbox result-toplevel
118+
```
119+
120+
This pushes the entire system closure (and all its dependencies) to the binary cache. The push only uploads store paths that aren't already in the cache, so subsequent pushes of similar builds are fast.
121+
122+
### Create a manifest manually
123+
124+
If you want to test the full OTA flow without a GitHub Release, you can create a `manifest.json` manually:
125+
126+
```bash
127+
STORE_PATH=$(readlink -f result-toplevel)
128+
VERSION=$(cat "$STORE_PATH/etc/lnbitsbox-version" 2>/dev/null || echo "unknown")
129+
130+
cat > manifest.json <<EOF
131+
{
132+
"version": "$VERSION",
133+
"store_path": "$STORE_PATH",
134+
"nixos_version": "24.11"
135+
}
136+
EOF
137+
138+
cat manifest.json
139+
```
140+
141+
Then upload this `manifest.json` as a release asset (or serve it some other way for testing).
142+
143+
### Verify the push
144+
145+
You can verify the closure is available in the cache:
146+
147+
```bash
148+
# Check if a specific store path is in the cache
149+
nix path-info --store https://lnbitsbox.cachix.org $(readlink -f result-toplevel)
150+
```
151+
152+
If this returns the path info without error, it's in the cache and ready for download.
153+
154+
## What Happens on the Pi
155+
156+
The update script (`nixos/update-service.nix`) runs these steps:
157+
158+
1. **Downloads `manifest.json`** from the GitHub Release URL
159+
2. **Runs `nix copy --from https://lnbitsbox.cachix.org <store-path>`** to download the full system closure
160+
3. **Runs `nix-env -p /nix/var/nix/profiles/system --set <store-path>`** to create a new system generation
161+
4. **Runs `<store-path>/bin/switch-to-configuration switch`** to activate the new system
162+
163+
The update runs as a transient systemd unit (`lnbitsbox-update.service`) so it survives admin app restarts. Progress is written to `/var/lib/lnbitsbox-update/`:
164+
165+
| File | Contents |
166+
|------|----------|
167+
| `status` | `idle`, `downloading`, `activating`, `success`, or `failed` |
168+
| `log` | Full output log with timestamps |
169+
| `target-version` | The release tag being updated to |
170+
171+
The admin dashboard polls these files to show live progress to the user.
172+
173+
## File Overview
174+
175+
| File | Role |
176+
|------|------|
177+
| `flake.nix` | Defines `version`, exposes `.#toplevel` package |
178+
| `nixos/configuration.nix` | Writes `/etc/lnbitsbox-version`, configures Cachix substituter and nix settings |
179+
| `nixos/update-service.nix` | Update script, state directory, `lnbitsbox-update` command |
180+
| `nixos/admin-app/app.py` | `/box/api/update/check`, `/start`, `/status` endpoints |
181+
| `nixos/admin-app/templates/dashboard.html` | System Update UI card |
182+
| `.github/workflows/build-image.yml` | Builds toplevel, pushes to Cachix, uploads manifest |
183+
184+
## Garbage Collection
185+
186+
Old system generations are automatically cleaned up weekly (configured in `configuration.nix`):
187+
188+
```nix
189+
nix.gc = {
190+
automatic = true;
191+
dates = "weekly";
192+
options = "--delete-older-than 14d";
193+
};
194+
```
195+
196+
This prevents the Nix store from growing unbounded on the Pi's SD card. Generations older than 14 days are removed. The current and previous generation are always kept, allowing rollback.
197+
198+
## Troubleshooting
199+
200+
**"Cachix push failed" in CI**: Make sure `CACHIX_AUTH_TOKEN` is set as a GitHub repository secret and that the `lnbitsbox` cache exists on Cachix.
201+
202+
**Update check shows no update available**: The release must have a `manifest.json` asset. If CI's manifest step failed (e.g., Cachix wasn't configured), the admin app won't offer the update since there's no closure to download.
203+
204+
**Update fails with "Failed to download closure from Cachix"**: The public key in `configuration.nix` must match the Cachix cache's actual signing key. Also verify the Pi has internet access.
205+
206+
**Update fails during activation**: The `switch-to-configuration` step can fail if the new system has incompatible state. Check the full log at `/var/lib/lnbitsbox-update/log` on the Pi. The previous system generation is still intact — a reboot will boot into the last working generation.
207+
208+
**Testing locally with DEV_MODE**: Run the admin app with `DEV_MODE=true` and the update endpoints return mock data (update always available from v1.0.0 to v1.1.0). This lets you develop the UI without a real Cachix setup.

0 commit comments

Comments
 (0)