diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index 8f38c179..acc650d0 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -1,5 +1,5 @@ You are a senior Kotlin Multiplatform library engineer working on "KDown", an open-source Kotlin -Multiplatform downloader library. +Multiplatform download manager library. ## Project Status @@ -8,181 +8,162 @@ Android, JVM/Desktop, iOS, and WebAssembly platforms. **Note:** The library has not been published yet, so public API breaking changes are allowed. -## Core Features (All Implemented ✅) - -1. ✅ Download with progress callbacks (bytes downloaded, total bytes, speed) -2. ✅ Pause / Resume (true resume using HTTP Range; not "restart from 0") -3. ✅ Multi-threaded segmented downloads (N segments using Range requests) -4. ✅ Robust cancellation -5. ✅ Retry with exponential backoff (configurable) -6. ✅ Persist resumable metadata (app restart can resume) -7. ✅ Comprehensive logging system with platform-specific implementations -8. ✅ Kermit integration for structured logging - -## Architecture (Implemented ✅) - -- ✅ Ktor Client as default HTTP layer via `KtorHttpEngine` implementation -- ✅ Pluggable architecture with `HttpEngine` interface for custom implementations -- ✅ Kotlin coroutines throughout; no blocking IO in common code -- ✅ `FileAccessor` expect/actual abstraction for platform-specific file operations - - Android/JVM: Uses `RandomAccessFile` with `Dispatchers.IO` - - iOS: Uses Foundation APIs (`NSFileHandle`, `NSFileManager`) - - All implementations are thread-safe with Mutex protection -- ✅ Thread-safety ensured for pause/resume/cancel operations - -## Implemented Components - -### 1. Server Capability Detection (✅ `RangeSupportDetector`) -- Performs HEAD request before downloading -- Detects: content-length, accept-ranges, ETag, Last-Modified -- Falls back to single-connection if ranges not supported - -### 2. Segmented Downloads (✅ `SegmentCalculator` + `SegmentDownloader`) -- Splits files into N segments based on total size and connections -- Each segment downloads concurrently using `Range: bytes=start-end` -- Automatic fallback to single connection if server doesn't support ranges - -### 3. File Writing (✅ `FileAccessor` expect/actual) -- Platform-specific random-access writes at offsets -- Thread-safe with Mutex protection -- Each segment writes to correct offset in same destination file - -### 4. Pause/Resume (✅ `DownloadCoordinator`) -- Pause: stops all segment jobs, persists offsets, transitions to `Paused` state -- Resume: loads metadata, validates server identity (ETag/Last-Modified), continues from last offsets -- Resume validation prevents corrupted downloads -- Local file integrity check on resume: verifies file size matches claimed progress, resets if truncated -- Duplicate download guards: `start()`, `startFromRecord()`, `resume()` check `activeDownloads` to prevent concurrent writes -- State transition guards in `KDown` action lambdas prevent invalid operations (e.g., pause on completed) - -### 5. Metadata Persistence (✅ `TaskStore` interface) -- Interface implemented with two storage backends: - - `InMemoryTaskStore`: for testing and ephemeral downloads - - `SqliteTaskStore`: SQLite-based persistence (separate module) -- Stores: url, destPath, totalBytes, acceptRanges, etag, lastModified, segment progress - -### 6. Error Handling (✅ Sealed `KDownError`) -- Types: `Network`, `Http(code)`, `Disk`, `Unsupported`, `ValidationFailed`, `Canceled`, `Unknown` -- Smart retry: only for transient errors (network issues, 5xx HTTP codes) -- Exponential backoff with configurable retry count and base delay -- I/O exceptions from `FileAccessor` (`writeAt`, `flush`, `preallocate`) classified as `KDownError.Disk` - -### 7. Progress Tracking (✅ StateFlow-based) -- Aggregates progress across all segments -- Throttled updates (default: 200ms) to prevent UI spam -- Includes: downloadedBytes, totalBytes, percent, speed - -### 8. Logging System (✅ Pluggable `Logger` interface) -- Three logging options: - 1. `Logger.None` (default): Zero-overhead, no logging - 2. `Logger.console()`: Platform-specific console output - 3. `KermitLogger`: Optional Kermit integration (separate module) -- Platform-specific console implementations: - - JVM: `println` / `System.err` - - Android: Android `Log` API with tag prefixing ("KDown.*") - - iOS: `NSLog` - - WebAssembly: `println` (browser dev tools) -- Levels: verbose, debug, info, warn, error -- Lazy evaluation via lambda parameters for zero cost when disabled -- Internal `KDownLogger` wrapper for consistent tag formatting: `[Tag] message` -- Comprehensive logging coverage: - - KDown lifecycle (init, start, pause, resume, cancel) - - Server detection and capabilities - - Segment operations (start, completion, progress) - - HTTP requests and errors - - Retry attempts -- Kermit integration available via `library/kermit` module -- See LOGGING.md for detailed usage examples - -### 9. Testing (✅ Comprehensive test suite) -- Unit tests for: SegmentCalculator, SegmentDownloader, InMemoryMetadataStore -- Model tests for all data classes -- State transition tests - -### 10. Documentation (✅ Complete) -- README.md: quickstart, features, configuration, error handling -- LOGGING.md: detailed logging guide -- CLI example: demonstrates pause/resume functionality -- Multiple platform examples: Compose Multiplatform, Desktop, Android, iOS, WebAssembly +## Module Structure -## Current Limitations - -1. ⚠️ WebAssembly file writes limited by browser APIs (basic support only) -2. ⚠️ iOS support is best-effort via expect/actual (functional but not extensively tested) - -## Roadmap - -Planned features that should be considered in architecture decisions: - -1. **Speed Limit** - Bandwidth throttling per task or globally -2. **Queue Management** - Download queue with priority, concurrency limits, and scheduling -3. **Scheduled Downloads** - Timer-based or condition-based download scheduling -4. **Web App** - Browser-based download manager UI -5. **Torrent Support** - BitTorrent protocol as a pluggable download source -6. **Media Downloads** - Download web media (like yt-dlp), with a pluggable downloader - architecture to support different media sources and extractors -7. **Daemon Server** - Run KDown as a background service/daemon with an API, supporting - switching between local and remote backends (e.g., control a remote KDown instance - from a mobile app or web UI) - -## Usage Examples - -### Basic Setup with Logging - -```kotlin -// Option 1: No logging (default, zero overhead) -val kdown = KDown(httpEngine = KtorHttpEngine()) - -// Option 2: Console logging (development/debugging) -val kdown = KDown( - httpEngine = KtorHttpEngine(), - logger = Logger.console() -) - -// Option 3: Kermit structured logging (recommended for production) -val kdown = KDown( - httpEngine = KtorHttpEngine(), - logger = KermitLogger(minSeverity = Severity.Info) -) +``` +library/ + api/ # Public API interfaces and models -- published SDK module + core/ # In-process download engine -- published SDK module + ktor/ # Ktor-based HttpEngine implementation -- published SDK module + kermit/ # Optional Kermit logging integration -- published SDK module + sqlite/ # SQLite-backed TaskStore (Android, iOS, JVM only) -- published SDK module + remote/ # Remote KDownApi client (HTTP + SSE) -- published SDK module +server/ # Ktor-based daemon server with REST API and SSE events +app/ + shared/ # Shared Compose Multiplatform UI (supports Core + Remote backends) + android/ # Android app + desktop/ # Desktop (JVM) app + web/ # Wasm browser app + ios/ # Native iOS app (Xcode project, consumes shared module) +cli/ # JVM CLI entry point ``` -See LOGGING.md for detailed logging guide with example output. +## Package Structure + +### `library:api` (public API) +- `com.linroid.kdown.api` -- `KDownApi`, `DownloadTask`, `DownloadRequest`, `DownloadState`, + `DownloadProgress`, `Segment`, `KDownError`, `SpeedLimit`, `DownloadPriority`, + `DownloadSchedule`, `DownloadCondition`, `KDownVersion` + +### `library:core` (implementation) +- `com.linroid.kdown.core` -- `KDown` (implements `KDownApi`), `DownloadConfig`, `QueueConfig` +- `com.linroid.kdown.core.engine` -- `HttpEngine`, `DownloadCoordinator`, `RangeSupportDetector`, + `ServerInfo`, `DownloadSource`, `HttpDownloadSource`, `SourceResolver`, `SourceInfo`, + `SourceResumeState`, `DownloadContext`, `DownloadScheduler`, `ScheduleManager`, + `SpeedLimiter`, `TokenBucket`, `DelegatingSpeedLimiter` +- `com.linroid.kdown.core.segment` -- `SegmentCalculator`, `SegmentDownloader` +- `com.linroid.kdown.core.file` -- `FileAccessor` (expect/actual), `FileNameResolver`, + `DefaultFileNameResolver`, `PathSerializer` +- `com.linroid.kdown.core.log` -- `Logger`, `KDownLogger` +- `com.linroid.kdown.core.task` -- `DownloadTaskImpl`, `TaskStore`, `InMemoryTaskStore`, + `TaskRecord`, `TaskState` + +### `library:remote` +- `com.linroid.kdown.remote` -- `RemoteKDown` (implements `KDownApi`), `RemoteDownloadTask`, + `ConnectionState`, `WireModels`, `WireMapper` + +## Implemented Features + +### Core Download Engine +- Multi-platform: Android (minSdk 26), JVM 11+, iOS (iosArm64, iosSimulatorArm64), WasmJs +- Segmented downloads with concurrent HTTP Range requests +- Pause / Resume with server identity validation (ETag, Last-Modified) +- File integrity check on resume (validates local file size vs. claimed progress) +- Retry with exponential backoff for transient errors +- Persistent task metadata via `TaskStore` interface +- Duplicate download guards in `start()`, `startFromRecord()`, `resume()` + +### Queue Management (`DownloadScheduler`) +- Configurable concurrent download slots (`QueueConfig.maxConcurrentDownloads`) +- Per-host connection limits (`QueueConfig.maxConnectionsPerHost`) +- Priority-based ordering (`DownloadPriority`: LOW, NORMAL, HIGH, URGENT) +- URGENT preemption: pauses lowest-priority active download to make room + +### Speed Limiting +- Global speed limit via `DownloadConfig.speedLimit` or `KDownApi.setGlobalSpeedLimit()` +- Per-task speed limit via `DownloadRequest.speedLimit` or `DownloadTask.setSpeedLimit()` +- Token-bucket algorithm (`TokenBucket`) with delegating wrapper + +### Download Scheduling (`ScheduleManager`) +- `DownloadSchedule.Immediate`, `AtTime(Instant)`, `AfterDelay(Duration)` +- `DownloadCondition` interface for user-defined conditions (e.g., WiFi-only) +- Reschedule support via `DownloadTask.reschedule()` + +### Pluggable Download Sources (`DownloadSource`) +- `SourceResolver` routes URLs to the appropriate source +- `HttpDownloadSource` is the built-in HTTP/HTTPS implementation +- Additional sources registered via `KDown(additionalSources = listOf(...))` +- Each source defines: `canHandle()`, `resolve()`, `download()`, `resume()` + +### Daemon Server (`server/`) +- Ktor-based REST API: create, list, pause, resume, cancel downloads +- SSE event stream for real-time state updates +- Remote backend (`RemoteKDown`) communicates via HTTP + SSE +- Auto-reconnection with exponential backoff + +### Logging System +- `Logger.None` (default, zero overhead), `Logger.console()`, `KermitLogger` +- Platform-specific console: Logcat (Android), NSLog (iOS), println/stderr (JVM), println (Wasm) +- Lazy lambda evaluation for zero cost when disabled + +### Error Handling (sealed `KDownError`) +- `Network` (retryable), `Http(code)` (5xx retryable), `Disk`, `Unsupported`, + `ValidationFailed`, `Canceled`, `SourceError`, `Unknown` +- I/O exceptions from `FileAccessor` classified as `KDownError.Disk` + +## Architecture Patterns + +### Dual Backend via `KDownApi` +- `KDownApi` is the service interface (in `library:api`) +- `KDown` (core) is the in-process implementation +- `RemoteKDown` (remote) communicates with a daemon server over HTTP + SSE +- UI code works identically regardless of backend + +### Pluggable Components +- `HttpEngine` interface for custom HTTP clients (default: Ktor) +- `TaskStore` interface for persistence (InMemoryTaskStore, SqliteTaskStore) +- `DownloadSource` interface for protocol-level extensibility +- `Logger` interface for logging backends + +### Expect/Actual for Platform Code +- `FileAccessor`: Android/JVM uses `RandomAccessFile`, iOS uses `NSFileHandle` +- Console logger: platform-specific implementations +- All implementations use Mutex for thread-safety + +### Coroutine-Based Concurrency +- `supervisorScope` for segment downloads (one failure doesn't cancel others) +- Structured concurrency for cleanup on cancel/pause +- `Dispatchers.IO` for file operations ## Development Guidelines -When working on this project: - ### Code Quality +- 2-space indentation, max 100 char lines (see `.editorconfig`) +- No star imports, no trailing commas - Favor simple correctness over micro-optimizations -- Keep public APIs minimal and well-documented with KDoc -- Mark internal implementation details with `internal` modifier -- Use sealed classes for closed type hierarchies - -### Architecture Patterns -- All core logic lives in `commonMain` using expect/actual for platform differences -- Avoid platform-specific APIs in common code -- Use dependency injection for pluggable components (HttpEngine, TaskStore, Logger) +- Keep public APIs minimal with KDoc +- Mark internal implementation with `internal` modifier + +### Architecture +- All core logic in `commonMain` using expect/actual for platform differences +- Dependency injection for pluggable components - Prefer composition over inheritance -- Package structure: root (KDown, DownloadConfig, DownloadRequest, DownloadProgress, DownloadState), `task/` (DownloadTask, TaskStore, InMemoryTaskStore, TaskRecord, TaskState), `segment/` (Segment, SegmentCalculator, SegmentDownloader), `engine/` (HttpEngine, DownloadCoordinator, RangeSupportDetector, ServerInfo), `file/` (FileAccessor, FileNameResolver, DefaultFileNameResolver, PathSerializer), `log/` (Logger, KDownLogger), `error/` (KDownError) +- Public API types live in `library:api`, implementations in `library:core` ### Testing -- Add unit tests for new features in `commonTest` -- Test segment calculation edge cases (file sizes, connection counts) -- Test metadata serialization/deserialization -- Test state transitions (Idle → Downloading → Paused → Downloading → Completed) -- Mock HTTP responses for testing resume logic +- Unit tests in `commonTest` for segment math, state transitions, serialization +- Mock `HttpEngine` for testing without network +- Test edge cases: 0-byte files, 1-byte files, uneven segment splits ### Logging -- Use `KDownLogger` for all internal logging (automatically prefixes tags) -- Pass component tag as first parameter: `KDownLogger.i("KDown") { "message" }` -- Log at appropriate levels: - - `verbose`: Detailed flow information (segment-level progress, byte details) - - `debug`: Important state changes (server detection, metadata operations, segment start/completion) - - `info`: User-facing events (download started/paused/resumed/completed, server capabilities) - - `warn`: Recoverable errors, retries, validation warnings - - `error`: Fatal errors (download failures, network errors) -- Include context in log messages (task ID, segment index, URLs, byte ranges) -- Use lazy lambdas for message construction: `logger.d { "expensive $computation" }` - - Messages are only computed if logging is enabled - - This ensures zero overhead when using `Logger.None` (default) -- Common component tags: "KDown", "Coordinator", "SegmentDownloader", "RangeDetector", "KtorHttpEngine" +- Use `KDownLogger` for all internal logging +- Tags: "KDown", "Coordinator", "SegmentDownloader", "RangeDetector", "KtorHttpEngine", + "Scheduler", "ScheduleManager", "SourceResolver" +- Levels: verbose (segment detail), debug (state changes), info (user events), + warn (retries), error (fatal) +- Use lazy lambdas: `logger.d { "expensive $computation" }` + +## Current Limitations + +1. WasmJs: Local file I/O not supported (`FileAccessor` is a stub that throws + `UnsupportedOperationException`). Use `RemoteKDown` for browser-based downloads. +2. iOS support is best-effort via expect/actual (iosArm64 + iosSimulatorArm64) +3. `library:sqlite` does not support WasmJs -- use `InMemoryTaskStore` on that platform + +## Roadmap + +Planned features not yet implemented: + +1. **Web App** - Browser-based download manager UI +2. **Torrent Support** - BitTorrent protocol as a pluggable `DownloadSource` +3. **Media Downloads** - Web media extraction (like yt-dlp) as a pluggable `DownloadSource` diff --git a/README.md b/README.md index 9e7abd17..4449b17b 100644 --- a/README.md +++ b/README.md @@ -1,68 +1,46 @@ # KDown -[![Kotlin](https://img.shields.io/badge/Kotlin-2.3.0-7F52FF.svg?logo=kotlin&logoColor=white)](https://kotlinlang.org) +[![Kotlin](https://img.shields.io/badge/Kotlin-2.3.10-7F52FF.svg?logo=kotlin&logoColor=white)](https://kotlinlang.org) [![Kotlin Multiplatform](https://img.shields.io/badge/Kotlin-Multiplatform-4c8dec?logo=kotlin&logoColor=white)](https://kotlinlang.org/docs/multiplatform.html) [![Ktor](https://img.shields.io/badge/Ktor-3.4.0-087CFA.svg?logo=ktor&logoColor=white)](https://ktor.io) [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) -[![Android](https://img.shields.io/badge/Android-24+-3DDC84.svg?logo=android&logoColor=white)](https://developer.android.com) +[![Android](https://img.shields.io/badge/Android-26+-3DDC84.svg?logo=android&logoColor=white)](https://developer.android.com) [![iOS](https://img.shields.io/badge/iOS-supported-000000.svg?logo=apple&logoColor=white)](https://developer.apple.com) [![JVM](https://img.shields.io/badge/JVM-11+-DB380E.svg?logo=openjdk&logoColor=white)](https://openjdk.org) [![Built with Claude Code](https://img.shields.io/badge/Built_with-Claude_Code-6b48ff.svg?logo=anthropic&logoColor=white)](https://claude.ai/claude-code) -A Kotlin Multiplatform download library with pause/resume, multi-threaded segmented downloads, and progress tracking. +A Kotlin Multiplatform download manager with segmented downloads, pause/resume, queue management, speed limiting, and scheduling -- for Android, JVM, iOS, and WebAssembly. -> **WIP:** This project is under active development. APIs may change and some features are not yet complete. Contributions and feedback are welcome! +> **WIP:** This project is under active development. APIs may change. Contributions and feedback are welcome! ## Features -- **Multi-platform** - Android, iOS, JVM/Desktop, and WebAssembly (Wasm) -- **Segmented downloads** - Split files into N concurrent segments using HTTP Range requests for faster downloads -- **Pause / Resume** - True resume using HTTP Range headers, not restart-from-zero -- **Persistent resume** - Metadata is persisted so downloads survive app restarts -- **Resume validation** - ETag / Last-Modified checks ensure the remote file hasn't changed -- **Progress tracking** - Aggregated progress across all segments via `StateFlow`, with download speed -- **Retry with backoff** - Configurable exponential backoff for transient network errors and 5xx responses -- **Cancellation** - Robust coroutine-based cancellation -- **Pluggable HTTP engine** - Ships with a Ktor adapter; bring your own `HttpEngine` if needed - -## Architecture - -``` -library/ - core/ # Platform-agnostic download engine (commonMain) - ktor/ # Ktor-based HttpEngine implementation - kermit/ # Optional Kermit logging integration -examples/ - cli/ # JVM CLI sample - app/ # Compose Multiplatform app - desktopApp/ # Desktop (JVM) app - androidApp/ # Android app - webApp/ # Wasm browser app -``` - -Key internal components: - -| Class | Role | -|---|---| -| `KDown` | Main entry point | -| `DownloadCoordinator` | Orchestrates segment jobs, manages state | -| `SegmentDownloader` | Downloads a single byte-range segment | -| `RangeSupportDetector` | Probes server for Range/ETag/content-length | -| `FileAccessor` | expect/actual abstraction for random-access file writes | -| `SqliteTaskStore` | SQLite-backed task record persistence (includes segments) | +- **Multi-platform** -- Android, iOS, JVM/Desktop, and WebAssembly (WasmJs) +- **Segmented downloads** -- Split files into N concurrent segments using HTTP Range requests +- **Pause / Resume** -- True resume using byte ranges, with ETag/Last-Modified validation +- **Queue management** -- Priority-based queue with configurable concurrency limits and per-host throttling +- **Speed limiting** -- Global and per-task bandwidth throttling via token-bucket algorithm +- **Scheduling** -- Start downloads at a specific time, after a delay, or when conditions are met +- **Download conditions** -- User-defined conditions (e.g., WiFi-only) that gate download start +- **Pluggable sources** -- Extensible `DownloadSource` interface for custom protocols (HTTP built-in) +- **Persistent resume** -- Task metadata survives app restarts via pluggable `TaskStore` +- **Progress tracking** -- Aggregated progress across segments via `StateFlow`, with download speed +- **Retry with backoff** -- Configurable exponential backoff for transient errors +- **Daemon server** -- Run KDown as a background service with REST API and SSE events +- **Remote control** -- Control a daemon server from any client via `RemoteKDown` +- **Pluggable HTTP engine** -- Ships with Ktor; bring your own `HttpEngine` if needed ## Quick Start ```kotlin // 1. Create a KDown instance -val taskStore = createSqliteTaskStore(driverFactory) val kdown = KDown( httpEngine = KtorHttpEngine(), - taskStore = taskStore, + taskStore = createSqliteTaskStore(driverFactory), config = DownloadConfig( maxConnections = 4, retryCount = 3, - retryDelayMs = 1000 + queueConfig = QueueConfig(maxConcurrentDownloads = 3) ) ) @@ -70,7 +48,7 @@ val kdown = KDown( val task = kdown.download( DownloadRequest( url = "https://example.com/large-file.zip", - destPath = "/path/to/output.zip", + directory = "/path/to/downloads", connections = 4 ) ) @@ -90,116 +68,112 @@ launch { } } -// 4. Pause / Resume / Cancel +// 4. Control the download task.pause() task.resume() task.cancel() // 5. Or just await the result -val result = task.await() // Result - -// 6. Observe all tasks (includes persisted tasks after loadTasks()) -kdown.loadTasks() -kdown.tasks.collect { tasks -> - tasks.forEach { println("${it.taskId}: ${it.state.value}") } -} -``` - -## Logging - -KDown provides pluggable logging support. By default, logging is disabled for zero overhead. - -### Built-in Console Logger - -Use the platform-specific console logger for quick debugging: - -```kotlin -val kdown = KDown( - httpEngine = KtorHttpEngine(), - logger = Logger.console() // Logs to Logcat (Android), NSLog (iOS), stdout/stderr (JVM) -) +val result: Result = task.await() ``` -### Structured Logging with Kermit +## Modules -For production use, integrate with [Kermit](https://github.com/touchlab/Kermit) for structured, multi-platform logging: +KDown is split into published SDK modules that you add as dependencies: -```kotlin -// Add dependency: implementation("com.linroid.kdown:kermit:1.0.0") +| Module | Description | Platforms | +|---|---|---| +| `library:api` | Public API interfaces and models (`KDownApi`, `DownloadTask`, `DownloadState`, etc.) | All | +| `library:core` | In-process download engine -- embed downloads directly in your app | All | +| `library:ktor` | Ktor-based `HttpEngine` implementation (required by `core`) | All | +| `library:sqlite` | SQLite-backed `TaskStore` for persistent resume | Android, iOS, JVM | +| `library:kermit` | Optional [Kermit](https://github.com/touchlab/Kermit) logging integration | All | +| `library:remote` | Remote client -- control a KDown daemon server from any platform | All | +| `server` | Daemon server with REST API and SSE events (not an SDK; standalone service) | JVM | -val kdown = KDown( - httpEngine = KtorHttpEngine(), - logger = KermitLogger( - minSeverity = Severity.Debug, - tag = "MyApp" - ) -) -``` +Choose your backend: use **`core`** for in-process downloads, or **`remote`** to control a daemon server. Both implement the same `KDownApi` interface, so your UI code works identically. -### Custom Logger +### `library:api` -Implement the `Logger` interface to integrate with your own logging framework: +The public API surface. Both `library:core` and `library:remote` implement the `KDownApi` interface, +so UI code works identically regardless of backend: ```kotlin -class MyLogger : Logger { - override fun v(message: () -> String) { /* verbose log */ } - override fun d(message: () -> String) { /* debug log */ } - override fun i(message: () -> String) { /* info log */ } - override fun w(message: () -> String, throwable: Throwable?) { /* warning log */ } - override fun e(message: () -> String, throwable: Throwable?) { /* error log */ } +interface KDownApi { + val tasks: StateFlow> + suspend fun download(request: DownloadRequest): DownloadTask + suspend fun setGlobalSpeedLimit(limit: SpeedLimit) + fun close() + // ... plus backendLabel, version } - -val kdown = KDown(httpEngine = KtorHttpEngine(), logger = MyLogger()) ``` -**Log Levels:** -- **Verbose**: Detailed diagnostics (segment-level progress) -- **Debug**: Internal operations (server detection, metadata save/load) -- **Info**: User-facing events (download start/complete, server capabilities) -- **Warn**: Recoverable errors (retry attempts, validation warnings) -- **Error**: Fatal failures (download failures, network errors) - -See [LOGGING.md](LOGGING.md) for detailed logging documentation. - -## Modules - ### `library:core` -The platform-agnostic download engine. No HTTP client dependency -- just the `HttpEngine` interface: +The in-process download engine. Depends on an `HttpEngine` interface (no HTTP client dependency): ```kotlin interface HttpEngine { - suspend fun head(url: String): ServerInfo - suspend fun download(url: String, range: LongRange?, onData: suspend (ByteArray) -> Unit) + suspend fun head(url: String, headers: Map = emptyMap()): ServerInfo + suspend fun download(url: String, range: LongRange?, headers: Map = emptyMap(), onData: suspend (ByteArray) -> Unit) fun close() } ``` ### `library:ktor` -A ready-made `HttpEngine` backed by Ktor Client with per-platform engines: +Ready-made `HttpEngine` backed by Ktor Client with per-platform engines: | Platform | Ktor Engine | |---|---| | Android | OkHttp | | iOS | Darwin | | JVM | CIO | -| Wasm/JS | Js | - -### `library:kermit` - -Optional [Kermit](https://github.com/touchlab/Kermit) integration for production-grade structured logging across all platforms. +| WasmJs | Js | ## Configuration ```kotlin DownloadConfig( - maxConnections = 4, // max concurrent segments - retryCount = 3, // retries per segment - retryDelayMs = 1000, // base delay (exponential backoff) + maxConnections = 4, // max concurrent segments per task + retryCount = 3, // retries per segment + retryDelayMs = 1000, // base delay (exponential backoff) progressUpdateIntervalMs = 200, // progress throttle - bufferSize = 8192 // read buffer size + bufferSize = 8192, // read buffer size + speedLimit = SpeedLimit.kbps(500), // global speed limit + queueConfig = QueueConfig( + maxConcurrentDownloads = 3, // max simultaneous downloads + maxConnectionsPerHost = 4, // per-host limit + autoStart = true // auto-start queued tasks + ) +) +``` + +### Priority & Scheduling + +```kotlin +// High-priority download +kdown.download( + DownloadRequest( + url = "https://example.com/urgent.zip", + directory = "/downloads", + priority = DownloadPriority.URGENT // preempts lower-priority tasks + ) +) + +// Scheduled download +kdown.download( + DownloadRequest( + url = "https://example.com/file.zip", + directory = "/downloads", + schedule = DownloadSchedule.AtTime(startAt), + conditions = listOf(wifiOnlyCondition) + ) ) + +// Speed limiting +task.setSpeedLimit(SpeedLimit.mbps(1)) // per-task +kdown.setGlobalSpeedLimit(SpeedLimit.kbps(500)) // global ``` ## Error Handling @@ -213,49 +187,87 @@ All errors are modeled as a sealed class `KDownError`: | `Disk` | No | File I/O failures | | `Unsupported` | No | Server doesn't support required features | | `ValidationFailed` | No | ETag / Last-Modified mismatch on resume | +| `SourceError` | No | Error from a pluggable download source | | `Canceled` | No | Download was canceled | | `Unknown` | No | Unexpected errors | +## Logging + +KDown provides pluggable logging with zero overhead when disabled (default). + +```kotlin +// No logging (default) +KDown(httpEngine = KtorHttpEngine()) + +// Console logging (development) +KDown(httpEngine = KtorHttpEngine(), logger = Logger.console()) + +// Kermit structured logging (production) +KDown(httpEngine = KtorHttpEngine(), logger = KermitLogger(minSeverity = Severity.Debug)) +``` + +See [LOGGING.md](LOGGING.md) for detailed documentation. + ## How It Works -1. **Probe** -- HEAD request to get `Content-Length`, `Accept-Ranges`, `ETag`, `Last-Modified` -2. **Plan** -- If ranges are supported, split the file into N segments; otherwise fall back to a single connection -3. **Download** -- Each segment downloads its byte range concurrently and writes to the correct file offset -4. **Persist** -- Segment progress is saved to `TaskStore` so pause/resume works across restarts -5. **Resume** -- On resume, validates server identity (ETag/Last-Modified), then continues from last offsets +1. **Resolve** -- Query the download source (HEAD request for HTTP) to get size, range support, identity headers +2. **Plan** -- If ranges are supported, split the file into N segments; otherwise use a single connection +3. **Queue** -- If max concurrent downloads reached, queue with priority ordering +4. **Download** -- Each segment downloads its byte range concurrently and writes to the correct file offset +5. **Throttle** -- Token-bucket speed limiter controls bandwidth per task and globally +6. **Persist** -- Segment progress is saved to `TaskStore` so pause/resume works across restarts +7. **Resume** -- On resume, validates server identity (ETag/Last-Modified) and file integrity, then continues + +## Daemon Server + +Run KDown as a background service and control it remotely: -## Current Limitations +```kotlin +// Server side (JVM) +val kdown = KDown(httpEngine = KtorHttpEngine()) +val server = KDownServer(kdown) +server.start() // REST API + SSE on port 8642 + +// Client side (any platform) +val remote = RemoteKDown(baseUrl = "http://localhost:8642") +val task = remote.download(DownloadRequest(url = "...", directory = "...")) +task.state.collect { /* real-time updates via SSE */ } +``` -- WebAssembly file writes are limited by browser APIs -- iOS support is best-effort via expect/actual +## Platform Support -## Roadmap +| Feature | Android | JVM | iOS | WasmJs | +|---|---|---|---|---| +| Segmented downloads | Yes | Yes | Yes | Remote only* | +| Pause / Resume | Yes | Yes | Yes | Remote only* | +| SQLite persistence | Yes | Yes | Yes | No | +| Console logging | Logcat | stdout/stderr | NSLog | println | +| Daemon server | -- | Yes | -- | -- | -- **Speed Limit** - Bandwidth throttling per task or globally -- **Queue Management** - Download queue with priority and concurrency limits -- **Scheduled Downloads** - Timer-based or condition-based download scheduling -- **Web App** - Browser-based download manager UI -- **Torrent Support** - BitTorrent protocol as a pluggable download source -- **Media Downloads** - Download web media (like yt-dlp) with pluggable extractors -- **Daemon Server** - Background service with API, supporting local and remote backends +\*WasmJs: Local file I/O is not supported. Use `RemoteKDown` to control a daemon server from the browser. ## Building ```bash -# Build all +# Build all modules ./gradlew build -# Run CLI example -./gradlew :examples:cli:run --args="https://example.com/file.zip" +# Run CLI +./gradlew :cli:run --args="https://example.com/file.zip" -# Run desktop example -./gradlew :examples:desktopApp:run +# Run desktop app +./gradlew :app:desktop:run ``` +## Contributing + +Contributions are welcome! Please open an issue to discuss your idea before submitting a PR. +See the [code style rules](.claude/rules/code-style.md) for formatting guidelines. + ## License Apache-2.0 --- -*This project was built with [Claude Code](https://claude.ai/claude-code) by Anthropic.* +*Built with [Claude Code](https://claude.ai/claude-code) by Anthropic.* diff --git a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadProgress.kt b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadProgress.kt index 7ddb309f..b12b5def 100644 --- a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadProgress.kt +++ b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadProgress.kt @@ -1,13 +1,22 @@ package com.linroid.kdown.api +/** + * Snapshot of download progress aggregated across all segments. + * + * @property downloadedBytes total number of bytes received so far + * @property totalBytes expected file size in bytes, or 0 if unknown + * @property bytesPerSecond current download speed in bytes per second + */ data class DownloadProgress( val downloadedBytes: Long, val totalBytes: Long, val bytesPerSecond: Long = 0 ) { + /** Fraction complete in the range `0f..1f`, or `0f` if [totalBytes] is unknown. */ val percent: Float get() = if (totalBytes > 0) downloadedBytes.toFloat() / totalBytes else 0f + /** `true` when [downloadedBytes] has reached or exceeded [totalBytes]. */ val isComplete: Boolean get() = totalBytes > 0 && downloadedBytes >= totalBytes } diff --git a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadState.kt b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadState.kt index bb22ee3a..aa19f099 100644 --- a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadState.kt +++ b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadState.kt @@ -1,19 +1,54 @@ package com.linroid.kdown.api +/** + * Represents the lifecycle state of a download task. + * + * State transitions follow this general flow: + * ``` + * Idle -> Scheduled -> Queued -> Pending -> Downloading -> Completed + * | | + * v v + * Canceled Paused -> Downloading + * | + * v + * Failed + * ``` + * + * @see DownloadTask.state + */ sealed class DownloadState { + /** Initial state before the task has been submitted. */ data object Idle : DownloadState() + + /** Waiting for a [DownloadSchedule] trigger or [DownloadCondition]s. */ data class Scheduled(val schedule: DownloadSchedule) : DownloadState() + + /** Waiting in the download queue for an available slot. */ data object Queued : DownloadState() + + /** Slot acquired; download is about to start. */ data object Pending : DownloadState() + + /** Actively downloading. [progress] is updated periodically. */ data class Downloading(val progress: DownloadProgress) : DownloadState() + + /** Download paused by the user or preempted by the scheduler. */ data class Paused(val progress: DownloadProgress) : DownloadState() + + /** Download finished successfully. [filePath] is the output file. */ data class Completed(val filePath: String) : DownloadState() + + /** Download failed with [error]. May be retried if the error is retryable. */ data class Failed(val error: KDownError) : DownloadState() + + /** Download was explicitly canceled. */ data object Canceled : DownloadState() + /** `true` when the task has reached a final state and cannot be resumed. */ val isTerminal: Boolean get() = this is Completed || this is Failed || this is Canceled + /** `true` when the task is actively using a download slot. */ val isActive: Boolean get() = this is Pending || this is Downloading } diff --git a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadTask.kt b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadTask.kt index 3e76fe98..0a910362 100644 --- a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadTask.kt +++ b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/DownloadTask.kt @@ -20,8 +20,13 @@ interface DownloadTask { val state: StateFlow val segments: StateFlow> + /** Pauses the download, preserving segment progress for later resume. */ suspend fun pause() + + /** Resumes a paused or failed download from where it left off. */ suspend fun resume() + + /** Cancels the download. This is a terminal action. */ suspend fun cancel() /** @@ -59,6 +64,12 @@ interface DownloadTask { */ suspend fun remove() + /** + * Suspends until the download reaches a terminal state. + * + * @return [Result.success] with the output file path on completion, + * or [Result.failure] with a [KDownError] on failure or cancellation + */ suspend fun await(): Result } diff --git a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/KDownError.kt b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/KDownError.kt index ee587a31..06e2000d 100644 --- a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/KDownError.kt +++ b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/KDownError.kt @@ -1,40 +1,76 @@ package com.linroid.kdown.api +/** + * Sealed hierarchy of all errors that KDown can produce. + * + * Use [isRetryable] to determine whether the operation should be + * retried automatically. Only transient failures ([Network] and + * server-side [Http] 5xx) are considered retryable. + * + * @property message human-readable error description + * @property cause underlying exception, if any + */ sealed class KDownError( override val message: String?, override val cause: Throwable? = null ) : Exception(message, cause) { + /** Connection or timeout failure. Always retryable. */ data class Network( override val cause: Throwable? = null ) : KDownError("Network error occurred", cause) + /** + * Non-success HTTP status code. + * Retryable only for server errors (5xx). + * + * @property code the HTTP status code + * @property statusMessage optional reason phrase from the server + */ data class Http( val code: Int, val statusMessage: String? = null ) : KDownError("HTTP error $code: $statusMessage") + /** File I/O failure (write, flush, preallocate). Not retryable. */ data class Disk( override val cause: Throwable? = null ) : KDownError("Disk I/O error", cause) + /** Server does not support a required feature (e.g., byte ranges). */ data object Unsupported : KDownError("Operation not supported by server") + /** + * Resume validation failed (ETag or Last-Modified mismatch). + * + * @property reason description of what failed validation + */ data class ValidationFailed( val reason: String ) : KDownError("Validation failed: $reason") + /** Download was explicitly canceled by the user. */ data object Canceled : KDownError("Download was canceled") + /** + * Error originating from a pluggable [DownloadSource][com.linroid.kdown.core.engine.DownloadSource]. + * + * @property sourceType identifier of the source that failed + */ data class SourceError( val sourceType: String, override val cause: Throwable? = null ) : KDownError("Source '$sourceType' error", cause) + /** Catch-all for unexpected errors. Not retryable. */ data class Unknown( override val cause: Throwable? = null ) : KDownError("Unknown error occurred", cause) + /** + * Whether this error is transient and the download should be retried. + * Only [Network] and [Http] with a 5xx status code are retryable. + */ val isRetryable: Boolean get() = when (this) { is Network -> true diff --git a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/KDownVersion.kt b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/KDownVersion.kt index 9bf1436e..9ce31359 100644 --- a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/KDownVersion.kt +++ b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/KDownVersion.kt @@ -2,6 +2,12 @@ package com.linroid.kdown.api import kotlinx.serialization.Serializable +/** + * Version information for a KDown client-backend pair. + * + * @property client version of the client library (e.g., the app) + * @property backend version of the backend (Core or Remote server) + */ @Serializable data class KDownVersion( val client: String, diff --git a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/Segment.kt b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/Segment.kt index 501ec207..b654ad4a 100644 --- a/library/api/src/commonMain/kotlin/com/linroid/kdown/api/Segment.kt +++ b/library/api/src/commonMain/kotlin/com/linroid/kdown/api/Segment.kt @@ -2,6 +2,17 @@ package com.linroid.kdown.api import kotlinx.serialization.Serializable +/** + * Represents a byte-range segment of a download. + * + * A file is split into one or more segments that download concurrently. + * Each segment tracks its own progress independently. + * + * @property index zero-based segment index + * @property start inclusive start byte offset in the file + * @property end inclusive end byte offset in the file + * @property downloadedBytes number of bytes downloaded so far in this segment + */ @Serializable data class Segment( val index: Int, @@ -9,15 +20,19 @@ data class Segment( val end: Long, val downloadedBytes: Long = 0 ) { + /** Total number of bytes this segment is responsible for. */ val totalBytes: Long get() = end - start + 1 + /** The next byte offset to write at (`start + downloadedBytes`). */ val currentOffset: Long get() = start + downloadedBytes + /** `true` when [downloadedBytes] has reached [totalBytes]. */ val isComplete: Boolean get() = downloadedBytes >= totalBytes + /** Number of bytes still to be downloaded. */ val remainingBytes: Long get() = totalBytes - downloadedBytes } diff --git a/library/core/src/commonMain/kotlin/com/linroid/kdown/core/engine/HttpEngine.kt b/library/core/src/commonMain/kotlin/com/linroid/kdown/core/engine/HttpEngine.kt index 41a41273..a13a39a6 100644 --- a/library/core/src/commonMain/kotlin/com/linroid/kdown/core/engine/HttpEngine.kt +++ b/library/core/src/commonMain/kotlin/com/linroid/kdown/core/engine/HttpEngine.kt @@ -1,9 +1,33 @@ package com.linroid.kdown.core.engine - +/** + * Abstraction over the HTTP layer used by KDown. + * + * The default implementation is backed by Ktor (`library:ktor` module). + * Implement this interface to plug in a different HTTP client. + */ interface HttpEngine { + /** + * Performs an HTTP HEAD request to retrieve server metadata. + * + * @param url the resource URL + * @param headers additional request headers + * @return server metadata including content length and range support + * @throws com.linroid.kdown.api.KDownError.Network on connection failure + * @throws com.linroid.kdown.api.KDownError.Http on non-success status + */ suspend fun head(url: String, headers: Map = emptyMap()): ServerInfo + /** + * Downloads data from [url] and delivers chunks via [onData]. + * + * @param url the resource URL + * @param range byte range to request, or `null` for the entire resource + * @param headers additional request headers + * @param onData callback invoked for each chunk of received bytes + * @throws com.linroid.kdown.api.KDownError.Network on connection failure + * @throws com.linroid.kdown.api.KDownError.Http on non-success status + */ suspend fun download( url: String, range: LongRange?, @@ -11,5 +35,6 @@ interface HttpEngine { onData: suspend (ByteArray) -> Unit ) + /** Releases underlying resources (e.g., the HTTP client). */ fun close() } diff --git a/library/core/src/commonMain/kotlin/com/linroid/kdown/core/engine/ServerInfo.kt b/library/core/src/commonMain/kotlin/com/linroid/kdown/core/engine/ServerInfo.kt index 80236eb9..af3a9070 100644 --- a/library/core/src/commonMain/kotlin/com/linroid/kdown/core/engine/ServerInfo.kt +++ b/library/core/src/commonMain/kotlin/com/linroid/kdown/core/engine/ServerInfo.kt @@ -1,5 +1,17 @@ package com.linroid.kdown.core.engine +/** + * Metadata returned by an HTTP HEAD request. + * + * Used to determine download strategy (segmented vs. single) and + * to validate resume integrity. + * + * @property contentLength total content size in bytes, or `null` if unknown + * @property acceptRanges `true` if the server advertises `Accept-Ranges: bytes` + * @property etag the `ETag` header value, used for resume validation + * @property lastModified the `Last-Modified` header value, used for resume validation + * @property contentDisposition the `Content-Disposition` header, used for file name resolution + */ data class ServerInfo( val contentLength: Long?, val acceptRanges: Boolean, @@ -7,6 +19,7 @@ data class ServerInfo( val lastModified: String?, val contentDisposition: String? = null ) { + /** `true` when the server supports byte-range requests and reports a content length. */ val supportsResume: Boolean get() = acceptRanges && contentLength != null && contentLength > 0 } diff --git a/library/core/src/commonMain/kotlin/com/linroid/kdown/core/file/FileAccessor.kt b/library/core/src/commonMain/kotlin/com/linroid/kdown/core/file/FileAccessor.kt index bc3cdb56..d8eda6ba 100644 --- a/library/core/src/commonMain/kotlin/com/linroid/kdown/core/file/FileAccessor.kt +++ b/library/core/src/commonMain/kotlin/com/linroid/kdown/core/file/FileAccessor.kt @@ -2,11 +2,34 @@ package com.linroid.kdown.core.file import kotlinx.io.files.Path +/** + * Platform-specific random-access file writer. + * + * Each platform provides an actual implementation: + * - **Android/JVM**: `RandomAccessFile` with `Dispatchers.IO` + * - **iOS**: Foundation `NSFileHandle` / `NSFileManager` with `Dispatchers.IO` + * - **WasmJs**: Stub that throws `UnsupportedOperationException` (no file I/O) + * + * Android, JVM, and iOS implementations are thread-safe (protected by a `Mutex`). + * + * @param path the file system path to write to + */ expect class FileAccessor(path: Path) { + /** Writes [data] starting at the given byte [offset]. */ suspend fun writeAt(offset: Long, data: ByteArray) + + /** Flushes buffered writes to disk. */ suspend fun flush() + + /** Closes the underlying file handle. */ fun close() + + /** Deletes the file from disk. */ suspend fun delete() + + /** Returns the current file size in bytes. */ suspend fun size(): Long + + /** Pre-allocates [size] bytes on disk to avoid fragmentation. */ suspend fun preallocate(size: Long) } diff --git a/library/ktor/src/commonMain/kotlin/com/linroid/kdown/engine/KtorHttpEngine.kt b/library/ktor/src/commonMain/kotlin/com/linroid/kdown/engine/KtorHttpEngine.kt index af557c2d..35c09917 100644 --- a/library/ktor/src/commonMain/kotlin/com/linroid/kdown/engine/KtorHttpEngine.kt +++ b/library/ktor/src/commonMain/kotlin/com/linroid/kdown/engine/KtorHttpEngine.kt @@ -16,6 +16,15 @@ import io.ktor.http.isSuccess import io.ktor.utils.io.readAvailable import kotlin.coroutines.cancellation.CancellationException +/** + * [HttpEngine] implementation backed by a Ktor [HttpClient]. + * + * Uses platform-specific Ktor engines: OkHttp (Android), Darwin (iOS), + * CIO (JVM), and Js (WasmJs/JS). + * + * @param client the Ktor HTTP client to use, or a default client + * with infinite timeouts (suitable for large downloads) + */ class KtorHttpEngine( private val client: HttpClient = defaultClient() ) : HttpEngine {