Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
name: build

on:
push:
branches: [main, master]
tags: ['v*']
pull_request:
branches: [main, master]

permissions:
contents: read

jobs:
build-3ds:
name: Build .3dsx (devkitARM)
runs-on: ubuntu-latest
permissions:
contents: read
container:
image: devkitpro/devkitarm:latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Build
shell: bash
run: |
make
ls -la *.3dsx *.smdh

- name: Upload .3dsx artifact
uses: actions/upload-artifact@v4
with:
name: 3ds-zip-extractor-3dsx
path: |
*.3dsx
*.smdh
if-no-files-found: error

host-tests:
name: Host unit tests (ASAN/UBSAN)
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install build deps
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends build-essential

- name: Run host tests with sanitizers
run: make -C tests -f Makefile.host test-asan

release:
name: Publish GitHub Release
needs: [build-3ds, host-tests]
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Download build artifact
uses: actions/download-artifact@v4
with:
name: 3ds-zip-extractor-3dsx
path: dist/

- name: Create Release
uses: softprops/action-gh-release@v2
with:
files: |
dist/*.3dsx
dist/*.smdh
generate_release_notes: true
19 changes: 18 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
# Build directories
build/
build/*.o
build/*.d
build/*.map
build/*.lst
*.elf
# *.smdh - REMOVED: We need to commit the SMDH file for the build to work

# Build artifacts (published as Release assets, not tracked in git)
*.3dsx
*.smdh

# Editor files
*.swp
Expand All @@ -30,3 +34,16 @@ lib/

# Nested duplicated directories
_codeql_detected_source_root/

# Host test artifacts
tests/test_gdrive
tests/test_config
tests/test_archive_detect
tests/test_sha256
tests/*.gcda
tests/*.gcno
tests/coverage/
tests/fixtures/tiny.*
tests/fixtures/empty.bin
tests/fixtures/three_bytes.bin
tests/fixtures/not_an_archive.bin
Binary file removed 3ds-zip-extractor.3dsx
Binary file not shown.
Binary file removed 3ds-zip-extractor.smdh
Binary file not shown.
56 changes: 56 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,61 @@
# Changelog

## [Unreleased]

### Added
- **Modular source layout.** `source/main.c` has been split into focused
modules with public headers in `include/`:
`led`, `gdrive`, `config`, `download`, `file_browser`, `queue`,
`integrity`, plus a pure `archive_extractor_detect` module that's
reusable on the host.
- **Optional integrity check.** A URL line in `config.txt` may now end with
` sha256:<64-hex>` to verify the downloaded file. Mismatches mark the
item as failed without extracting.
- **Configurable runtime limits.** New optional config keys
`max_urls`, `download_buffer_kb`, `connect_timeout_s` (defaults: 100,
128, 30). Values out of range are clamped, never rejected.
- **Per-run unique temp paths.** Downloads now land in
`sdmc:/3ds/zip-extractor/tmp/run_<ts>_<n>.part` instead of a single
shared `temp_download.tmp`. Orphan `.part` files are cleaned up at
startup and shutdown.
- **Honest stubs.** `umask` / `getpwnam` / `getgrnam` / `LZ4_*` now log a
one-shot warning to `stderr` the first time they're hit so unexpected
use is visible.
- **Off-target unit tests.** A new `tests/` directory builds with a plain
host C compiler (no devkitARM, no 3DS) and exercises the pure modules
with vendored Unity. Run with `make -C tests -f Makefile.host test`
(or `test-asan` for ASan/UBSan, or `coverage` for an `lcov` HTML
report).
- **CI**. New `.github/workflows/build.yml` builds the `.3dsx` in the
`devkitpro/devkitarm:latest` image, runs the host test suite under
ASan/UBSan, uploads `.3dsx`/`.smdh` as build artifacts, and publishes
a GitHub Release on `v*` tags.

### Changed
- **Hardened extraction.** libarchive is now configured with
`ARCHIVE_EXTRACT_SECURE_NODOTDOT |
ARCHIVE_EXTRACT_SECURE_SYMLINKS |
ARCHIVE_EXTRACT_SECURE_NOABSOLUTEPATHS` to mitigate zip-slip and
symlink-based attacks on top of the previous protections.
- **Resume hardening.** The HTTP resume path now detects servers that
reply with `200 OK` instead of `206 Partial Content` and restarts the
download from zero rather than appending to a stale prefix.
- **Config parser.** Lines longer than `MAX_URL_LENGTH` are now drained
to the next `\n` and recorded as a single (truncated, NUL-terminated)
entry instead of splitting into bogus extra "URLs". UTF-8 BOMs on the
first line are skipped.
- **Build artifacts no longer tracked.** `.3dsx` and `.smdh` are now
ignored by git and produced fresh by every build; releases ship them
as Release assets.

### Removed
- The hand-rolled ZIP/zlib path (`extract_zip_file`, `mkdir_recursive`,
`ZIP_LOCAL_SIGNATURE`, manual `inflate`) is gone from `main.c`. All
archive types — ZIP included — now go through the libarchive
extractor.

---

## [1.0] - 2026-02-05

### First Stable Release
Expand Down
26 changes: 14 additions & 12 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
> **Stato attuale (2026-04-30):** questo documento descrive lo stato **target** del
> repository dopo il refactor pianificato (vedi prompt di lavoro nelle issue/PR).
> Alcuni moduli marcati `[post-refactor]` non esistono ancora nel codice:
> al momento `source/main.c` è monolitico e contiene tutta la logica.
> Aggiornare questo file mano a mano che il refactor procede.
> **Stato attuale (2026-04-30):** il refactor modulare è stato completato.
> Tutti i moduli elencati nella sezione 3 esistono in `source/` e in
> `include/`, la build con `make` è verde su `devkitpro/devkitarm:latest`
> e la suite host test in `tests/` passa con ASan/UBSan. Aggiornare questo
> file quando si introducono nuovi moduli o si cambiano comandi di build.
>
> # CLAUDE.md

Expand Down Expand Up @@ -48,14 +48,16 @@ L'applicazione viene distribuita come `.3dsx` da lanciare dall'**Homebrew Launch
.
├── source/ # Codice C target 3DS
│ ├── main.c # Entry point + state machine principale
│ ├── archive_extractor.c # Wrapper libarchive (multi-formato)
│ ├── archive_extractor.c # Driver libarchive (estrazione multi-formato)
│ ├── archive_extractor_detect.c # Magic-byte sniffing (puro, host-testabile)
│ ├── gui.c # Rendering citro2d (top + bottom screen)
│ ├── config.c # Parser config.txt [post-refactor]
│ ├── download.c # libcurl + retry [post-refactor]
│ ├── gdrive.c # Conversione URL Google Drive [post-refactor]
│ ├── file_browser.c # File browser SD card [post-refactor]
│ ├── led.c # Notifiche LED MCU [post-refactor]
│ └── queue.c # Gestione coda download [post-refactor]
│ ├── config.c # Parser config.txt (puro, host-testabile)
│ ├── download.c # libcurl + retry + resume
│ ├── gdrive.c # Conversione URL Google Drive (puro)
│ ├── integrity.c # SHA-256 + verify_integrity (mbedtls/host)
│ ├── file_browser.c # File browser SD card
│ ├── led.c # Notifiche LED MCU
│ └── queue.c # Rendering coda di download
├── include/ # Header pubblici corrispondenti a source/
├── tests/ # Unit test off-target (host build)
│ ├── Makefile.host
Expand Down
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ GRAPHICS := gfx
APP_TITLE := 3DS Zip Extractor
APP_DESCRIPTION := Download and extract archives
APP_AUTHOR := Marcogn
ICON := icon.png

#---------------------------------------------------------------------------------
# options for code generation
Expand Down
42 changes: 40 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,19 +50,31 @@ extract_path=sdmc:/extracted/
auto_retry=true
max_retries=3

# Optional limits (defaults shown)
# max_urls=100
# download_buffer_kb=128
# connect_timeout_s=30

# Add URLs below
https://example.com/file.zip

# Optional: integrity check — append " sha256:<64-hex>" to a URL
https://example.com/strict.zip sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
```

The format is **backwards-compatible with v1.0** config files.

---

## Controls

| Button | Action |
|--------|--------|
| A | Start/Confirm |
| A | Start / Confirm |
| B | Cancel |
| X | View queue |
| Y | (in queue view) Mark failed items as skipped |
| L / R | (in queue view) Page through URLs |
| SELECT | File browser |
| START | Exit |
| D-Pad | Navigate |
Expand All @@ -71,11 +83,36 @@ https://example.com/file.zip

## Building

### Native (devkitPRO)

```bash
# Requires devkitPRO
# Requires devkitPRO + devkitARM + portlibs 3DS
make
```

### Docker (no local devkitPRO needed)

```bash
docker run --rm -v "$PWD":/src -w /src devkitpro/devkitarm:latest make
```

This is the same image used by CI; useful for reproducible builds.

### Host unit tests

The pure modules (`config`, `gdrive`, `integrity`, archive detection)
have an off-target unit-test suite that runs on a normal Linux/macOS
PC — no devkitARM, no 3DS:

```bash
make -C tests -f Makefile.host test # plain build
make -C tests -f Makefile.host test-asan # with AddressSanitizer + UBSan
make -C tests -f Makefile.host coverage # gcov/lcov HTML report
```

The CI pipeline runs both `build-3ds` (Docker) and `host-tests`
(ASan/UBSan); both must be green for a PR to be merged.

---

## License
Expand All @@ -87,4 +124,5 @@ MIT License - see [LICENSE](LICENSE)
## Links

- [Changelog](CHANGELOG.md)
- [Architecture](docs/ARCHITECTURE.md)
- [Contributing](CONTRIBUTING.md)
93 changes: 93 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Architecture

> Snapshot of the post-refactor layout. Use this as the high-level map
> when adding a feature or tracking down a bug.

## 1. Module map

```
┌────────────────────────────────┐
│ source/main.c │
│ state machine, app lifecycle │
└───────────────┬────────────────┘
┌───────────┬───────────────┼───────────────┬─────────────┐
▼ ▼ ▼ ▼ ▼
┌─────────┐ ┌────────┐ ┌──────────────┐ ┌────────────┐ ┌─────────┐
│ config │ │ gdrive │ │ download │ │ archive_* │ │ queue │
│ (pure) │ │ (pure) │ │ (libcurl) │ │ extractor │ │ (UI) │
└────┬────┘ └────────┘ └──────┬───────┘ └────┬───────┘ └─────────┘
│ │ │
│ ▼ ▼
│ ┌────────────┐ ┌────────────────┐
│ │ libcurl │ │ libarchive │
│ │ + mbedtls │ │ + zlib/bz2/ │
│ └────────────┘ │ xz/zstd │
│ └────────────────┘
┌──────────┐ ┌────────┐ ┌──────────────┐ ┌──────┐
│integrity │ │ led │ │file_browser │ │ gui │
│(mbedtls/ │ │ (MCU │ │ (sdmc files) │ │(C2D) │
│ pure) │ │ hook) │ └──────────────┘ └──────┘
└──────────┘ └────────┘
```

## 2. Module responsibilities

| Module | File(s) | Pure? | Responsibility |
| --- | --- | --- | --- |
| `config` | `source/config.c`, `include/config.h` | ✅ | Parse `config.txt`, apply defaults, expose `AppConfig` (settings + URL list). Backwards compatible with v1.0. |
| `gdrive` | `source/gdrive.c`, `include/gdrive.h` | ✅ | Convert Google Drive sharing URLs into direct-download URLs. Pass-through for everything else. |
| `integrity` | `source/integrity.c`, `include/integrity.h` | ✅ | SHA-256 of buffers/files, optional integrity check. Uses mbedtls on target, a vendored pure-C implementation on host. |
| `archive_extractor_detect` | `source/archive_extractor_detect.c`, `include/archive_extractor.h` | ✅ | Magic-byte sniffing for ZIP/TAR/GZIP/BZIP2/XZ/ZSTD/7Z/RAR. No libarchive dependency, host-testable. |
| `archive_extractor` | `source/archive_extractor.c` | ❌ (libarchive) | Driver around `libarchive` for extraction. Ships honest stubs for `umask`/`getpwnam`/`getgrnam`/LZ4 missing on devkitARM. |
| `download` | `source/download.c`, `include/download.h` | ❌ (libcurl) | Run a `curl` transfer to a unique `*.part` file under `sdmc:/3ds/zip-extractor/tmp/`. Implements resume (Range/206) with 200 fallback, configurable buffer + connect timeout. |
| `led` | `source/led.c`, `include/led.h` | ❌ (libctru) | Thin wrapper for the MCUHWC LED notifications. |
| `file_browser` | `source/file_browser.c`, `include/file_browser.h` | ❌ (citro2d) | SD card directory browser (used to choose extraction path). |
| `queue` | `source/queue.c`, `include/queue.h` | ❌ (citro2d) | Renders the download/queue status screen on the top screen. `DownloadQueue` is just an alias for `AppConfig`. |
| `gui` | `source/gui.c`, `include/gui.h` | ❌ (citro2d/3d) | Common rendering helpers (status, menu, progress, error). Owns the dark-blue theme constants. |
| `main` | `source/main.c` | ❌ | App lifecycle: init/shutdown, state machine `MENU → BROWSER/QUEUE/RUNNING → SUMMARY → MENU/EXIT`, single-thread cooperative loop. |

## 3. Build pipeline

- Target: `make` invokes the devkitARM rules from `$DEVKITARM/3ds_rules`.
All `source/*.c` are compiled and linked against
`libcitro2d`, `libcitro3d`, `libcurl`, `libmbedtls`, `libarchive`,
`liblzma`, `libbz2`, `libzstd`, `libz`, `libctru`. Output:
`3ds-zip-extractor.{elf,3dsx,smdh}`.
- Host tests: `make -C tests -f Makefile.host` compiles only the *pure*
modules (`config`, `gdrive`, `integrity`, `archive_extractor_detect`)
plus vendored Unity, with `-DHOST_BUILD=1`. No 3DS dependency.

## 4. Data flow for a single URL

```
config.txt ──► read_config_file ──► AppConfig.items[i]
│ (url, sha256_hex, state)
download_make_temp_path
convert_gdrive_url ─► download_file ──► curl
│ (resume / 200 fallback)
verify_integrity (if hash present)
detect_archive_type
extract_archive_libarchive
led_notification_green
```

## 5. Threading model

The application is intentionally **single-thread cooperative**.
`aptMainLoop()` drives the main loop; long operations (`curl_easy_perform`,
`extract_archive_libarchive`) yield only inside their own libraries.
The progress/cancel callbacks call `hidScanInput` once per chunk to
keep the UI responsive. Adding threads would require careful work
around APT/HID/citro2d ownership and is explicitly out of scope.
Loading
Loading