Skip to content

changes for V2.0.0 merge from develop #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
niwciu merged 3 commits into from
May 18, 2026
Merged

changes for V2.0.0 merge from develop #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
13267dd
feat: configurable product ID sections, mandatory keyring, full-width…
niwciu May 18, 2026
8318bf6
docs: add SecureBootloader ↔ SecureLoader compatibility table
niwciu May 18, 2026
00ca6af
fix: resolve all CI lint and type-check failures
niwciu May 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 81 additions & 4 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,88 @@ versioning follows [Semantic Versioning](https://semver.org/).

## [Unreleased]

### Planned
### Added

- **Mandatory OS keychain credential storage** — `keyring` is now a required
dependency (previously the optional `[security]` extra). The HTTP password is
always stored in macOS Keychain, Windows Credential Manager, or a D-Bus secret
store on Linux and is **never written to `config.ini`**. Existing configs that
still contain a plaintext password are silently migrated to the keychain on the
first load.

### Changed

- **Section editor — inclusive end display** — the *End* spinbox now shows the
last nibble *included* in the section (range 0–15) instead of the
Python-exclusive upper bound (range 1–16), eliminating the confusing appearance
that adjacent sections share a boundary nibble. Internal `IdSectionDef`
storage is unchanged.
- **Section editor — position-based spinbox constraints** — overlap prevention
now operates on nibble position rather than GUI row order, so a section can be
freely repositioned across the ID without being blocked by a neighbouring row.
New sections default to the first uncovered nibble.
- **Product ID visualisation** — the 16-nibble colour strip is now rendered by a
native `QPainter` widget that scales to the full dialog width. Section names
and nibble ranges are displayed in a larger font.

## [2.0.0] — 2026-05-17

This release completes the HTTP firmware download feature, introducing
fully configurable Product ID sectioning and a redesigned Server Settings dialog.

> ⚠️ **Breaking change (bootloader):** the UART wire header shrinks from 48 B to
> 44 B. Devices running **bootloader v1.0.0** will stall on `CMD_START` (they
> wait for 48 bytes and misparse the first page command). Flash the updated
> bootloader (v1.1+) to the target device before deploying this release.

> ⚠️ **Breaking change (CLI):** `sld fetch --license / --unique` are replaced by
> `sld fetch --section NAME=VALUE` (repeatable). Update any scripts that call
> `sld fetch` with the old flags.

### Added

- **Configurable Product ID sections** (`core/id_sections.py`) — the 64-bit
`productId` can now be split into any number of named sections at nibble
(half-byte) granularity (0–16). Each section has a `name`, `start`, and `end`
nibble position. The default split matches the EncryptBIN / SecureBootloader
convention: `custom_id [0:8]`, `hw_id [8:10]`, `license_id [10:12]`,
`unique_id [12:16]`.
- **`product_id.sections` config key** — serialised as `name:start:end,…`; stored
in the `[product_id]` INI section. Configurable via
`sld config set product_id.sections` or the new GUI section editor.
- **`get_sections(defs)` on `FirmwareHeader` and `DeviceInfo`** — extracts section
values from `productId` using a list of `IdSectionDef` objects.
- **Redesigned Server Settings dialog** — four groups:
- *Server* — base URL + allow-insecure checkbox (unchanged).
- *Credentials* — login / password (unchanged).
- *Product ID sections* — scrollable section editor (name + start/end spinboxes
+ delete per row, "Add section" button). A 16-nibble colour-coded hex strip
visualises the current layout dynamically.
- *URL path structure* — checklist of defined sections with Up/Down ordering and
a live URL preview.

### Changed

- **`sld fetch`** — `--license` / `--unique` flags replaced by repeatable
`--section NAME=VALUE` (e.g. `--section license_id=AB --section unique_id=C0FE`).
Any section name defined in `product_id.sections` is accepted.
- **`sld config show`** — now includes all `http.*` fields, `product_id.sections`,
and `ui.instruction_url`.
- **`FirmwareIdentifier`** — redesigned from a frozen dataclass with fixed keyword
fields to a dict-backed immutable class. Sections are accessed as attributes
(`identifier.license_id`) via `__getattr__`; all existing callers using the
default section names continue to work without changes.
- **Wire header reduced from 48 B to 44 B** — `prevAppVersion` is no longer
included in the `CMD_START` payload. The field is still read from the `.bin`
file and available for host-side rollback logic, but the bootloader
`header_t` struct no longer contains it (bootloader v1.1+ change).
`build_device_header()` now returns `file[0:16] + file[20:48]`.
- **`DEVICE_HEADER_SIZE`** constant updated from `48` to `44`.

### Fixed

- `GithubReleasesFirmwareSource` — complete integration with private
GitHub repo (see [docs/GITHUB_SOURCE_MIGRATION.md](docs/GITHUB_SOURCE_MIGRATION.md)).
- Settings dialog in GUI (firmware source selection, PAT entry).
- **CI workflow** — updated action versions for compatibility with current
GitHub Actions runner environment.

## [1.2.0] — 2026-05-09

Expand Down
36 changes: 33 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,22 @@ Related repositories: [SecureBootloader](https://github.com/niwciu/SECURE_BOOTLO

---

## 🔗 Tool Compatibility

The **SecureBootloader ↔ SecureLoader** pairing is version-locked — a protocol change in SecureLoader v2.0.0 reduced the UART wire header from 48 → 44 bytes:

| SecureLoader | SecureBootloader | Compatible |
|:-------------|:----------------|:----------:|
| < v2.0.0 | v1.0.0 | ✅ |
| ≥ v2.0.0 | ≥ v1.1 | ✅ |
| ≥ v2.0.0 | v1.0.0 | ❌ device stalls on `CMD_START` |
| < v2.0.0 | ≥ v1.1 | ❌ header size mismatch |

> ⚠️ Always match the SecureLoader version to the SecureBootloader version deployed on your device.
> Flash the updated bootloader (≥ v1.1) to the target before deploying SecureLoader v2.0.0 or later.

---

## 📦 Installation

Python 3.10+ required.
Expand Down Expand Up @@ -191,17 +207,31 @@ sld info --port COM3

SecureLoader can download firmware directly from an HTTP server before flashing — no manual file transfer needed.

The server URL path is built from configurable **Product ID sections**: named nibble-level slices of
the device's 64-bit `productId`. The default layout matches the EncryptBIN / SecureBootloader
convention (`license_id` and `unique_id` as path segments), but any split and naming scheme is supported.

```bash
# 1. Configure your server URL once (HTTPS is required by default)
sld config set http.base_url https://myserver/update

# 2. Download the latest firmware (license/unique IDs come from the device)
sld fetch --license AB --unique C0FE --output firmware.bin
# 2. Download the latest firmware using the default sections
sld fetch --section license_id=AB --section unique_id=C0FE --output firmware.bin

# 3. Flash as usual
sld flash --file firmware.bin --port /dev/ttyUSB0
```

To use custom Product ID sections matching your server layout:

```bash
# Define a custom split (once, saved to config)
sld config set product_id.sections "product:0:6,serial:6:16"

# Download using the custom section names
sld fetch --section product=AABB --section serial=C0FE0001 --output firmware.bin
```

If the server requires authentication, set credentials once:

```bash
Expand All @@ -212,7 +242,7 @@ sld config set-password # secure interactive prompt — avoids shell hi
> ⚠️ Plain HTTP (`http://`) is rejected by default. Pass `--allow-insecure` to `fetch`
> only in isolated lab environments where HTTPS is not available.

The GUI exposes the same feature via the **Fetch from server** and **Get Previous Firmware** buttons — both become active once a device is connected.
The GUI exposes the same feature via the **Fetch from server** and **Get Previous Firmware** buttons — both become active once a device is connected. The **Settings → Server settings** dialog lets you visually design the Product ID section layout and URL path structure.

See the [User Guide](https://niwciu.github.io/SecureLoader/USER_GUIDE/) for HTTP server path requirements and full configuration details.

Expand Down
95 changes: 63 additions & 32 deletions docs/ARCHITECTURE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,19 +68,32 @@ graph TB

Five modules encapsulating all business logic.

### `core/id_sections.py`

**Responsibility:** defining how the 64-bit `productId` is split into named sections.

Key objects:
- `IdSectionDef(name, start, end)` — an immutable nibble-range slice of the 16-character hex
representation of `productId`. `start` and `end` are 0-based nibble positions (0–16).
`extract(hex_id)` returns the substring `hex_id[start:end]`.
- `DEFAULT_ID_SECTIONS` — the default split matching the ecosystem convention:
`custom_id [0:8]`, `hw_id [8:10]`, `license_id [10:12]`, `unique_id [12:16]`.
- `serialize_id_sections(defs) -> str` — serialises to `name:start:end,…` for INI storage.
- `parse_id_sections(s) -> list[IdSectionDef]` — parses the above format; returns defaults if `s` is blank.

### `core/firmware.py`

**Responsibility:** parsing the `.bin` binary format.

Key objects:
- `HEADER_SIZE = 48` — total header size on disk.
- `DEVICE_HEADER_SIZE = 48` — header size sent to the device (identical to the file header).
- `FirmwareHeader` — dataclass with fields and helpers (`format_product_id`,
`license_id`, `unique_id`, `payload_size`).
- `DEVICE_HEADER_SIZE = 44` — header size sent to the device (prevAppVersion stripped).
- `FirmwareHeader` — frozen dataclass with fields and helpers (`format_product_id`,
`format_app_version`, `payload_size`, `get_sections(defs)`).
- `parse_header(bytes) -> FirmwareHeader` — parses the first 48 bytes.
- `load_firmware(path) -> (FirmwareHeader, bytes)` — parses from file.
- `build_device_header(bytes) -> bytes` — returns the 48-byte wire header
(identical to the file header).
- `build_device_header(bytes) -> bytes` — returns the 44-byte wire header
(file bytes [0:16] + [20:48]; prevAppVersion at [16:20] is omitted).
- `split_pages(payload, page_size) -> list[bytes]` — splits payload into
equal pages; an incomplete final page is dropped (matching C++ behaviour).

Expand Down Expand Up @@ -143,8 +156,16 @@ class FirmwareSource(ABC):
def fetch_previous(self, identifier, progress=None) -> bytes: ...
```

`FirmwareIdentifier` carries `license_id`, `unique_id`, optionally
`app_version` (used by `fetch_previous`).
`FirmwareIdentifier` is a dict-backed immutable class that carries named product-ID
sections extracted from the device's `productId` via `get_sections(config.id_section_defs)`.
Sections are accessed as attributes (`identifier.license_id`) or via `identifier.get("name", "")`.
`app_version` is a reserved attribute for rollback fetches (holds `prevAppVersion`).

```python
# Building an identifier from a connected device:
sections = device_info.get_sections(config.id_section_defs)
identifier = FirmwareIdentifier(sections, app_version=prev_version)
```

`FirmwareSource` is used by the network-fetch paths only: the CLI `fetch` command
and the GUI _Fetch from server_ / _Get Previous Firmware_ buttons both use
Expand Down Expand Up @@ -223,7 +244,8 @@ Files: [gui/](https://github.com/niwciu/SecureLoader/tree/main/src/secure_loader
gui/
├── app.py — QApplication bootstrap, icon, language
├── main_window.py — QMainWindow reproducing mainwindow.ui 1:1
├── server_settings_dialog.py — QDialog for server URL, credentials, and URL path structure
├── server_settings_dialog.py — QDialog for server URL, credentials, product-ID section editor,
│ and URL path structure
├── login_dialog.py — legacy QDialog (credentials only; kept for tests)
└── workers.py — ProtocolWorker, DownloadWorker (QThread wrappers)
```
Expand Down Expand Up @@ -313,6 +335,9 @@ password =
use_credentials = false
path_segments = license_id,unique_id

[product_id]
sections = custom_id:0:8,hw_id:8:10,license_id:10:12,unique_id:12:16

[ui]
language = auto
instruction_url =
Expand All @@ -333,10 +358,15 @@ Fields:
is installed; otherwise in plaintext with `chmod 0600` on Unix.
_Backward compat_: if `use_credentials` is absent in an existing file, it is inferred as
`true` when `login` is non-empty, so pre-existing credentials continue to work.
- `http.path_segments` — comma-separated list of `FirmwareIdentifier` field names that form
the URL path between `base_url` and the filename. Defaults to `license_id,unique_id`.
Available names: `custom_id`, `hw_id`, `license_id`, `unique_id`.
- `http.path_segments` — comma-separated list of section names (defined in `product_id.sections`)
that form the URL path between `base_url` and the filename. Defaults to `license_id,unique_id`.
Configured via **Settings → Server settings → URL path structure**.
- `product_id.sections` — comma-separated section definitions in `name:start:end` format, where
`start` and `end` are nibble (half-byte) positions within the 16-character hex representation
of the 64-bit `productId`. Defaults to the four-section ecosystem convention
(`custom_id:0:8,hw_id:8:10,license_id:10:12,unique_id:12:16`). Configured via
**Settings → Server settings → Product ID sections**. Can also be set with:
`sld config set product_id.sections name1:0:6,name2:6:16`
- `ui.language` — `"auto" | "en" | "de" | "fr" | "es" | "it" | "pl"`.
- `ui.instruction_url` — optional URL opened by the _Update instruction…_ GUI
menu item. When empty, the menu item is hidden.
Expand Down Expand Up @@ -414,35 +444,36 @@ offset size field
48 … encrypted pages
```

Wire header = bytes `[0:48]` = 48 B (sent with the `START` command, identical to the file header).
Wire header = bytes `[0:16]` + `[20:48]` = 44 B (sent with the `START` command; prevAppVersion at `[16:20]` is omitted).

### Product ID Convention
### Product ID Sections

The 64-bit `productId` is treated as an **opaque value** by SecureLoader — it
is compared as a whole between the firmware file and the device response.
No partial matching is performed.
The 64-bit `productId` is treated as an **opaque value** for compatibility checks
(compared as a whole between the firmware file and the device response). For HTTP
firmware routing, it is split into named sections at nibble (half-byte) granularity.

By convention, the ecosystem tools (EncryptBIN, SecureBootloader) lay out the
8 bytes as follows:
The default split matches the ecosystem convention (EncryptBIN / SecureBootloader):

```
Hex digit: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Byte: ╔══════════════════════╗ ╔══╗ ╔═══╗ ╔════════╗
║ custom (4 B) ║ ║hw║ ║lic║ ║unique ║
║ custom_id (4 B) ║ ║hw║ ║lic║ ║unique ║
╚══════════════════════╝ ╚══╝ ╚═══╝ ╚════════╝
free for product use hw_id lic unique_id
nibbles [0:8] [8:10][10:12] [12:16]
```

| Slice | Chars | Bytes | Name | Used by SecureLoader |
|-------|-------|-------|------|----------------------|
| `[0:8]` | 0–7 | 0–3 | custom | no (opaque) |
| `[8:10]` | 8–9 | 4 | `hw_id` | no (opaque) |
| `[10:12]` | 10–11 | 5 | `license_id` | HTTP fetch routing |
| `[12:16]` | 12–15 | 6–7 | `unique_id` | HTTP fetch routing |
| Section name | Nibble range | Bytes | Default purpose |
|--------------|-------------|-------|-----------------|
| `custom_id` | `[0:8]` | 0–3 | Product-specific, free for use |
| `hw_id` | `[8:10]` | 4 | Hardware revision |
| `license_id` | `[10:12]`| 5 | License tier |
| `unique_id` | `[12:16]`| 6–7 | Device serial number |

`license_id` and `unique_id` are extracted only when routing a firmware
download request to the HTTP server (`FirmwareIdentifier`). For all other
operations (compatibility check, device matching) the full 64-bit value is used.
This layout is the default — it can be changed at any time in
**Settings → Server settings → Product ID sections** or via
`sld config set product_id.sections <defs>`. Any names and nibble
boundaries (0–16) are accepted, as long as names are unique and `start < end`.
Only the sections listed in `http.path_segments` appear in download URLs.

### Serial Commands (1 byte each)

Expand All @@ -458,7 +489,7 @@ operations (compatibility check, device matching) the full 64-bit value is used.
After ACK for `GET_VERSION` the device sends 16 bytes of device info
(`u32 bootloaderVersion`, `u64 productId`, `u32 flashPageSize`).

The host sends `START` (`0x02`) immediately followed by the 48 B wire header
The host sends `START` (`0x02`) immediately followed by the 44 B wire header
as a single transmission. The device responds ACK/NAK. Then for each page the
host sends `NEXT_PAGE` (`0x03`) + `flashPageSize` bytes, and the device ACKs.

Expand All @@ -479,9 +510,9 @@ Files: [tests/](https://github.com/niwciu/SecureLoader/tree/main/tests/).
- `test_protocol.py` — state machine, tested by injecting bytes into
`_handle_byte()` (no pyserial mocking).
- `test_updater.py` — `check_device_matches_firmware`.
- `test_config.py` — config load/save and keyring integration.
- `test_config.py` — config load/save, `id_section_defs` round-trip, and keyring integration.
- `test_http_source.py` — `HttpFirmwareSource`: URL encoding, TLS enforcement,
auth passthrough, 100 MB download cap, version string validation.
auth passthrough, 100 MB download cap, version string validation, configurable path segments.
- `test_local_source.py` — `LocalFirmwareSource`.
- `test_github_source.py` — `GithubReleasesFirmwareSource` with mocked GitHub API.
- `test_cli.py` — CLI commands: `config set/show/path`, `fetch` (HTTP rejection,
Expand Down
Loading
Loading