diff --git a/.actrc b/.actrc deleted file mode 100644 index a078f6406..000000000 --- a/.actrc +++ /dev/null @@ -1 +0,0 @@ --e contrib/act/event.json diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index 497bace54..000000000 --- a/AGENTS.md +++ /dev/null @@ -1,40 +0,0 @@ -# Repository Guidelines - -## Project Structure & Module Organization -- Workspace with crates: `dash`, `hashes`, `internals`, `dash-network`, `dash-spv`, `key-wallet`, `rpc-*`, utilities (`fuzz`, `test-utils`), and FFI crates (`*-ffi`). -- Each crate keeps sources in `src/`; unit tests live alongside code with `#[cfg(test)]`. Integration tests use `tests/` (e.g., `rpc-integration-test`). -- Mobile SDK lives in `swift-dash-core-sdk/`; FFI bindings are in `*-ffi`. Shared helpers in `internals/` and `test-utils/`. - -## Build, Test, and Development Commands -- MSRV: 1.89. Build all: `cargo build --workspace --all-features` -- Test all: `cargo test --workspace --all-features` or `./contrib/test.sh` (set `DO_COV=true`, `DO_LINT=true`, `DO_FMT=true` as needed) -- Targeted tests: `cargo test -p dash-spv --all-features` -- Integration RPC: `cd dash-spv && DASH_SPV_IP= cargo test --test integration_real_node_test -- --nocapture` -- FFI iOS builds: `cd key-wallet-ffi && ./build-ios.sh` or `cd swift-dash-core-sdk && ./build-ios.sh` -- Lint/format: `cargo clippy --workspace --all-targets -- -D warnings` and `cargo fmt --all` -- Docs: `cargo doc --workspace` (add `--open` locally) - -## Coding Style & Naming Conventions -- Mixed editions (2021/2024); follow crate idioms. Prefer async via `tokio` where applicable. -- Format with `rustfmt` (see `rustfmt.toml`); run `cargo fmt --all` before commits. -- Lint with `clippy`; some crates deny warnings in CI. Avoid `unwrap()/expect()` in library code; use error types (e.g., `thiserror`). -- Naming: `snake_case` (funcs/vars), `UpperCamelCase` (types/traits), `SCREAMING_SNAKE_CASE` (consts). Keep modules focused. - -## Testing Guidelines -- Unit tests near code; integration tests under `tests/`. Use descriptive names (e.g., `test_parse_address_mainnet`). -- Run targeted suites: `cargo test -p key-wallet --all-features`. Network‑dependent or long tests may be `#[ignore]`; run with `-- --ignored`. -- Cover critical parsing, networking, SPV, and wallet flows. Add regression tests for fixes; consider property tests (e.g., `proptest`) where valuable. - -## Commit & Pull Request Guidelines -- Prefer Conventional Commits: `feat:`, `fix:`, `refactor:`, `chore:`, `docs:`. Keep subject ≤72 chars with clear scope and rationale. -- Target branches: feature work to `v**-dev` (development), hotfixes/docs to `master` unless directed otherwise. -- Pre‑PR checks: `cargo fmt`, `cargo clippy`, `cargo test` (workspace). Update docs/CHANGELOG if user-facing. -- Include in PRs: description, linked issues, test evidence (commands/output), and notes on features/FFI impacts. - -## Security & Configuration Tips -- Not for consensus‑critical validation; do not rely on exact Dash Core consensus behavior. -- Never commit secrets or real keys; avoid logging sensitive data. Keep test vectors deterministic. -- Mirror strict CI locally if helpful: `export RUSTFLAGS="-D warnings"`. - -## References -- General workflow lives in `CONTRIBUTING.md`. For workspace specifics and current tooling, prefer this guide and `CLAUDE.md` when they differ. diff --git a/TEST_SUMMARY.md b/TEST_SUMMARY.md deleted file mode 100644 index 49b4289f1..000000000 --- a/TEST_SUMMARY.md +++ /dev/null @@ -1,112 +0,0 @@ -# Test Coverage Enhancement Summary - -## Overview -I have successfully implemented comprehensive unit tests for several critical dash-spv modules. Here's the current status: - -## Successfully Implemented and Passing Tests - -### 1. Bloom Filter Module (✅ 40 tests - ALL PASSING) -- **Location**: `dash-spv/src/bloom/tests.rs` -- **Coverage**: - - BloomFilterBuilder construction and configuration - - BloomFilterManager lifecycle and operations - - BloomFilterStats tracking and reporting - - Utility functions for pubkey hash extraction and outpoint serialization - - Thread safety and concurrent operations - - Edge cases and error handling - -### 2. Validation Module (✅ 54 tests - ALL PASSING) -- **Location**: - - `dash-spv/src/validation/headers_test.rs` - - `dash-spv/src/validation/headers_edge_test.rs` - - `dash-spv/src/validation/manager_test.rs` -- **Coverage**: - - HeaderValidator with all ValidationModes (None, Basic, Full) - - Chain continuity validation - - PoW verification (when enabled) - - Edge cases: empty chains, large chains, boundary conditions - - ValidationManager mode switching - - InstantLock and Quorum validation - -### 3. Chain Module (✅ 69 tests - ALL PASSING) -- **Location**: - - `dash-spv/src/chain/fork_detector_test.rs` - - `dash-spv/src/chain/orphan_pool_test.rs` - - `dash-spv/src/chain/checkpoint_test.rs` -- **Coverage**: - - Fork detection with checkpoint sync - - Multiple concurrent forks handling - - Orphan expiration and chain reactions - - Checkpoint validation and selection - - Thread safety for concurrent operations - - Chain reorganization scenarios - -## Tests Implemented but Not Compiling - -### 4. Client Module (⚠️ Tests written but API mismatch) -- **Location**: - - `dash-spv/src/client/config_test.rs` - - `dash-spv/src/client/watch_manager_test.rs` - - `dash-spv/src/client/block_processor_test.rs` - - `dash-spv/src/client/consistency_test.rs` - - `dash-spv/src/client/message_handler_test.rs` -- **Issue**: Tests were written against an incorrect API and need adjustment -- **Status**: Commented out in mod.rs to avoid blocking compilation - -### 5. Wallet Module (⚠️ Tests written but API mismatch) -- **Location**: - - `dash-spv/src/wallet/transaction_processor_test.rs` - - `dash-spv/src/wallet/utxo_test.rs` - - `dash-spv/src/wallet/wallet_state_test.rs` - - `dash-spv/src/wallet/utxo_rollback_test.rs` -- **Issue**: Some methods used are not part of the public API -- **Status**: Commented out in mod.rs to avoid blocking compilation - -### 6. Error Handling Tests (⚠️ Integration tests with compilation issues) -- **Location**: `dash-spv/tests/error_handling_test.rs` -- **Issue**: StorageManager trait methods don't match implementation -- **Status**: Part of integration tests that have compilation errors - -## Test Statistics - -- **Total Tests Written**: ~250+ tests -- **Currently Passing**: 163 tests (40 bloom + 54 validation + 69 chain) -- **Blocked by API Issues**: ~90+ tests (client and wallet modules) - -## Key Achievements - -1. **Comprehensive Coverage**: The implemented tests cover critical functionality including: - - Data structure construction and validation - - State management and persistence - - Concurrent operations and thread safety - - Edge cases and error scenarios - - Performance considerations - -2. **Test Quality**: All tests follow best practices: - - Clear test names describing what is being tested - - Proper setup/teardown - - Both positive and negative test cases - - Edge case coverage - - Thread safety verification where applicable - -3. **Module Coverage**: - - ✅ Bloom Filters: Complete coverage - - ✅ Validation: Complete coverage of existing functionality - - ✅ Chain Management: Comprehensive fork and orphan handling tests - - ⚠️ Client: Tests written but need API adjustment - - ⚠️ Wallet: Tests written but need API adjustment - -## Recommendations - -1. **Fix API Mismatches**: The client and wallet module tests need to be updated to match the actual API -2. **Integration Test Fixes**: The integration tests have trait method mismatches that need resolution -3. **Enable Commented Tests**: Once API issues are resolved, uncomment the test modules in mod.rs files -4. **Add Missing Coverage**: Still need tests for: - - Filters module (BIP157) - - Network module (additional edge cases) - - Storage module (error scenarios) - - Sync module components - -## Conclusion - -The test enhancement effort has significantly improved test coverage for dash-spv, with 163 tests currently passing in critical modules. The bloom filter, validation, and chain modules now have comprehensive test suites that verify functionality, handle edge cases, and ensure thread safety. The remaining work involves fixing API mismatches in client and wallet tests and resolving integration test compilation issues. diff --git a/TODOS.md b/TODOS.md deleted file mode 100644 index 187910662..000000000 --- a/TODOS.md +++ /dev/null @@ -1,61 +0,0 @@ -# TODOs for Rust Dashcore Key Wallet System - -## Critical Issues (Prevent Compilation) - -### dashcore crate compilation errors -The underlying `dashcore` crate has pre-existing compilation errors that prevent `key-wallet-manager` from building: - -1. **Missing imports in crypto/sighash.rs**: Two unresolved imports are causing E0432 errors -2. **65 warnings in dashcore**: Various deprecated method usage and unused variables - -**Impact**: key-wallet-manager cannot compile until dashcore is fixed. -**Priority**: Critical - blocks all high-level wallet functionality. - -## Remaining Features (Optional) - -### Serialization support -The last pending feature from the original plan: - -1. **Create wallet serialization**: Add serde support for saving/loading wallets from disk -2. **Encrypted wallet storage**: Add password protection for saved wallets -3. **Backup and restore**: Implement mnemonic and xprv/xpub backup functionality - -**Impact**: Wallets cannot be persisted between application runs. -**Priority**: Medium - useful for production applications. - -### Testing improvements -1. **Multi-language mnemonic tests**: Currently marked as `#[ignore]` - need actual multi-language support -2. **Integration tests**: More comprehensive testing of key-wallet + key-wallet-manager integration -3. **Transaction building tests**: Test actual transaction creation and signing - -## Known Limitations - -### Watch-only wallet derivation -The current watch-only wallet implementation creates its own derivation paths rather than using the exact same addresses as the original wallet. This is due to the separation between account-level xpubs and the AddressPool API requirements. - -### dashcore dependency issues -The architecture assumes dashcore will eventually compile. If dashcore continues to have issues, key-wallet-manager may need to: -1. Use a different transaction library -2. Implement transaction types internally -3. Wait for dashcore fixes - -## Status Summary - -✅ **Completed Successfully:** -- Restructured crate architecture (key-wallet + key-wallet-manager) -- Fixed all key-wallet compilation issues -- Added comprehensive tests for mnemonics and address management -- Created watch-only wallet functionality -- Enhanced derivation module with builder pattern -- Separated low-level primitives from high-level operations - -❌ **Blocked by External Issues:** -- key-wallet-manager compilation (blocked by dashcore) -- Transaction building functionality (blocked by dashcore) -- Integration tests (blocked by dashcore) - -✅ **Architecture Goals Met:** -- Clean separation of concerns -- No circular dependencies -- Proper use of existing dashcore types -- Extensible design for future features diff --git a/dash-spv/ARCHITECTURE.md b/dash-spv/ARCHITECTURE.md deleted file mode 100644 index b6260eb1f..000000000 --- a/dash-spv/ARCHITECTURE.md +++ /dev/null @@ -1,1617 +0,0 @@ -# Dash SPV Client - Comprehensive Code Guide - -**Version:** 0.40.0 -**Last Updated:** 2025-01-21 -**Total Lines of Code:** ~40,000 -**Total Files:** 110+ -**Overall Grade:** A+ (96/100) - -## Table of Contents - -1. [Executive Summary](#executive-summary) -2. [Architecture Overview](#architecture-overview) -3. [Module Analysis](#module-analysis) -4. [Critical Assessment](#critical-assessment) -5. [Recommendations](#recommendations) -6. [Complexity Metrics](#complexity-metrics) -7. [Security Considerations](#security-considerations) - ---- - -## Executive Summary - -### What is dash-spv? - -`dash-spv` is a professionally-architected Rust implementation of a Dash SPV (Simplified Payment Verification) client library. It provides: -- **Blockchain synchronization** via header chains and BIP157 compact block filters -- **Dash-specific features**: ChainLocks, InstantLocks, Masternode list tracking, Quorum management -- **Wallet integration** through clean WalletInterface trait -- **Modular architecture** with well-organized, focused modules -- **Async/await** throughout using Tokio runtime -- **Robust error handling** with comprehensive error types - -### Current State: Production-Ready Structure ✅ - -**Code Organization: EXCELLENT (A+)** -- ✅ All major modules refactored into focused components -- ✅ sync/filters/: 10 modules (4,281 lines) -- ✅ sync/sequential/: 11 modules (4,785 lines) -- ✅ client/: 8 modules (2,895 lines) -- ✅ storage/disk/: 7 modules (2,458 lines) -- ✅ All files under 1,500 lines (most under 500) - -**Critical Remaining Work:** -- 🚨 **Security**: BLS signature validation (ChainLocks + InstantLocks) - 1-2 weeks effort - -### Key Architectural Strengths - -**EXCELLENT DESIGN:** -- ✅ **Trait-based abstractions** (NetworkManager, StorageManager, WalletInterface) -- ✅ **Sequential sync manager** with clear phase transitions -- ✅ **Modular organization** with focused responsibilities -- ✅ **Comprehensive error types** with clear categorization -- ✅ **External wallet integration** with clean interface boundaries -- ✅ **Lock ordering documented** to prevent deadlocks -- ✅ **Performance optimizations** (cached headers, segmented storage, flow control) -- ✅ **Strong test coverage** (242/243 tests passing) - -**AREAS FOR IMPROVEMENT:** -- ⚠️ **BLS validation** required for mainnet security -- ⚠️ **Integration tests** could be more comprehensive -- ⚠️ **Resource limits** not yet enforced (connections, bandwidth) -- ℹ️ **Type aliases** could improve ergonomics (optional - generic design is intentional and beneficial) - -### Statistics - -| Category | Count | Notes | -|----------|-------|-------| -| Total Files | 110+ | Well-organized module structure | -| Total Lines | ~40,000 | All files appropriately sized | -| Largest File | network/manager.rs | 1,322 lines - Acceptable complexity | -| Module Count | 10+ | Well-separated concerns | -| Test Coverage | 242/243 passing | 99.6% pass rate | -| Major Modules Refactored | 4 | sync/filters/, sync/sequential/, client/, storage/disk/ | - ---- - -## Architecture Overview - -### High-Level Architecture - -``` -┌─────────────────────────────────────────────────────────────┐ -│ DashSpvClient │ -│ (Main Orchestrator - 2,819 lines) │ -└─────────────────────────────────────────────────────────────┘ - │ │ │ - ▼ ▼ ▼ - ┌───────────┐ ┌───────────┐ ┌───────────┐ - │ Network │ │ Storage │ │ Wallet │ - │ (Trait N) │ │ (Trait S) │ │ (Trait W) │ - └───────────┘ └───────────┘ └───────────┘ - │ - ▼ - ┌─────────────────────────────────────────┐ - │ SyncManager │ - │ - HeadersSync │ - │ - MasternodeSync │ - │ - FilterSync (4,027 lines - TOO BIG) │ - └─────────────────────────────────────────┘ - │ - ▼ - ┌──────────────┬──────────────┬──────────────┐ - │ Validation │ ChainLock │ Bloom │ - │ Manager │ Manager │ Manager │ - └──────────────┴──────────────┴──────────────┘ -``` - -### Data Flow - -``` -Network Messages → MessageHandler → SyncManager - │ - ▼ - ┌─────────────────────┐ - │ Validation Manager │ - └─────────────────────┘ - │ - ▼ - ┌─────────────────────┐ - │ Storage Manager │ - └─────────────────────┘ - │ - ▼ - ┌─────────────────────┐ - │ ChainState Update │ - └─────────────────────┘ - │ - ▼ - ┌─────────────────────┐ - │ Event Emission │ - └─────────────────────┘ -``` - ---- - -## Module Analysis - -### 1. ROOT LEVEL FILES - -#### `src/lib.rs` (120 lines) ✅ EXCELLENT - -**Purpose**: Library entry point and public API surface. - -**What it does**: -- Declares all public modules -- Re-exports key types for convenience -- Provides VERSION constant and logging initialization -- Feature-gates terminal UI module - -**Analysis**: -- **GOOD**: Clean public API, well-documented -- **GOOD**: Proper feature gating for optional dependencies -- **GOOD**: Re-exports reduce boilerplate for users -- **EXCELLENT**: Comprehensive module documentation - -**Refactoring needed**: ❌ None - this file is well-structured - -#### `src/error.rs` (303 lines) ✅ EXCELLENT - -**Purpose**: Centralized error handling with domain-specific error types. - -**Complex Types Used**: -- **`thiserror::Error`**: Automatic Display/Error impl - **JUSTIFIED** for ergonomics -- **Manual Clone for StorageError**: Required because `io::Error` doesn't impl Clone - **NECESSARY** - -**What it does**: -- Defines 6 error categories: SpvError, NetworkError, StorageError, ValidationError, SyncError, WalletError -- Provides type aliases for Results -- Implements error categorization via `SyncError::category()` - -**Analysis**: -- **EXCELLENT**: Clear error hierarchy -- **EXCELLENT**: Deprecated variant properly marked (#[deprecated]) -- **EXCELLENT**: Test coverage for error categorization -- **GOOD**: Detailed error messages -- **ISSUE**: `SyncError::SyncFailed` is deprecated but still used - should migrate callers - -**Refactoring needed**: -- ⚠️ **MINOR**: Migrate remaining uses of deprecated `SyncError::SyncFailed` -- ✅ **OPTIONAL**: Consider adding error codes for programmatic handling - -#### `src/types.rs` (1,065 lines) ⚠️ LARGE - -**Purpose**: Common type definitions shared across modules. - -**Complex Types Used**: - -1. **`CachedHeader`** (lines 16-77) - - **WHY**: Dash uses X11 hashing which is 4-6x slower than Bitcoin's SHA256 - - **COMPLEXITY**: Uses `Arc>` for thread-safe lazy caching - - **JUSTIFIED**: Massive performance improvement during header validation - - **EXCELLENT DESIGN**: Implements Deref to make it transparent - -2. **`SharedFilterHeights`** (line 14) - - Type alias: `Arc>>` - - **WHY**: Needs to be shared between stats tracking and filter sync - - **COULD BE SIMPLER**: Consider using `Arc` for better read concurrency - -3. **`ChainState`** (lines 216-456) - - **CRITICAL TYPE**: Holds entire SPV state - - **COMPLEXITY**: Manages headers, filters, chainlocks, masternode engine - - **ISSUE**: Methods like `tip_height()` have complex logic mixing `sync_base_height` - - **GOOD**: Checkpoint sync support - - **BAD**: No documentation on thread-safety assumptions - -4. **`DetailedSyncProgress`** (lines 138-213) - - Performance metrics and ETA calculation - - **GOOD**: Useful for UX - - **ISSUE**: Tight coupling to specific sync stages - -5. **Custom serde for `AddressBalance`** (lines 707-804) - - **WHY**: dashcore::Amount doesn't derive Serialize - - **COMPLEXITY**: Manual Visitor pattern - - **JUSTIFIED**: Necessary for persistence - - **ISSUE**: Verbose - consider upstream fix - -**Analysis**: -- **GOOD**: Comprehensive type coverage -- **ISSUE**: File is becoming a dumping ground (1,065 lines) -- **ISSUE**: Mixing sync logic (SyncStage) with storage types (ChainState) -- **EXCELLENT**: CachedHeader optimization is well-documented - -**Refactoring needed**: -- ⚠️ **HIGH PRIORITY**: Split into multiple files: - - `types/chain.rs` - ChainState, CachedHeader - - `types/sync.rs` - SyncProgress, SyncStage, DetailedSyncProgress - - `types/events.rs` - SpvEvent, MempoolRemovalReason - - `types/stats.rs` - SpvStats, PeerInfo - - `types/balances.rs` - AddressBalance, MempoolBalance, UnconfirmedTransaction -- ⚠️ **MEDIUM**: Add documentation on thread-safety for ChainState -- ✅ **LOW**: Consider using Arc for SharedFilterHeights - -#### `src/main.rs` (654 lines) ⚠️ COMPLEX - -**Purpose**: CLI binary for running SPV client. - -**What it does**: -- Parses command-line arguments -- Initializes wallet, network, storage -- Runs the SPV client with event logging -- Handles graceful shutdown - -**Complex Types Used**: -- Generic client instantiation with concrete types -- Arc> for shared wallet access - -**Analysis**: -- **GOOD**: Comprehensive CLI argument parsing -- **GOOD**: Graceful Ctrl-C handling -- **ISSUE**: 654 lines is too long for a binary -- **ISSUE**: Business logic (wallet balance logging) mixed with CLI concerns -- **ISSUE**: Event handling code is verbose (lines 374-468) -- **GOOD**: Terminal UI properly feature-gated - -**Refactoring needed**: -- ⚠️ **HIGH PRIORITY**: Extract event handler to separate module -- ⚠️ **MEDIUM**: Extract wallet initialization logic -- ⚠️ **MEDIUM**: Consider using clap's derive API more extensively -- ✅ **LOW**: Add more structured logging configuration - ---- - -### 2. BLOOM MODULE (6 files, ~2,000 lines) - -#### Overview -The bloom module manages BIP37 bloom filters for SPV transaction filtering. - -#### `src/bloom/mod.rs` (104 lines) ✅ GOOD - -**Purpose**: Module exports and main BloomFilter type. - -**What it does**: -- Re-exports bloom filter types -- Provides BloomFilter wrapper around dashcore's implementation - -**Analysis**: -- **GOOD**: Clean module organization -- **GOOD**: Simple wrapper pattern -- **EXCELLENT**: Delegates to upstream dashcore implementation - -#### `src/bloom/manager.rs` (157 lines) ✅ GOOD - -**Purpose**: Manages bloom filter lifecycle and updates. - -**Complex Types Used**: -- `Arc>` - **JUSTIFIED**: Shared between sync and update tasks - -**What it does**: -- Creates and updates bloom filters -- Recalculates filters when addresses added -- Sends filter updates to network - -**Analysis**: -- **GOOD**: Clear separation of concerns -- **GOOD**: Proper async/await usage -- **ISSUE**: No rate limiting on filter updates -- **GOOD**: Integrates with wallet interface - -**Refactoring needed**: -- ⚠️ **MEDIUM**: Add rate limiting/debouncing for filter updates -- ✅ **LOW**: Add metrics for filter update frequency - -#### `src/bloom/builder.rs` (98 lines) ✅ EXCELLENT - -**Purpose**: Builds bloom filters from wallet addresses. - -**What it does**: -- Takes addresses and scripts -- Configures false-positive rate -- Creates optimally-sized bloom filter - -**Analysis**: -- **EXCELLENT**: Well-focused module -- **EXCELLENT**: Proper FPR configuration -- **GOOD**: Clear documentation - -**Refactoring needed**: ❌ None - -#### `src/bloom/stats.rs` (71 lines) ✅ GOOD - -**Purpose**: Statistics tracking for bloom filter performance. - -**Analysis**: -- **GOOD**: Useful metrics -- **ISSUE**: Not actually used anywhere in codebase (dead code?) - -**Refactoring needed**: -- ⚠️ **HIGH**: Either integrate stats or remove module - -#### `src/bloom/utils.rs` (87 lines) ✅ GOOD - -**Purpose**: Utility functions for bloom filter operations. - -**What it does**: -- Calculates optimal filter parameters -- Provides helper functions - -**Analysis**: -- **GOOD**: Math is correct -- **GOOD**: Well-tested - -#### `src/bloom/tests.rs` (799 lines) ✅ EXCELLENT - -**Purpose**: Comprehensive bloom filter tests. - -**Analysis**: -- **EXCELLENT**: Very thorough test coverage -- **EXCELLENT**: Tests edge cases -- **EXCELLENT**: Property-based testing would be valuable addition - -**Refactoring needed**: -- ✅ **ENHANCEMENT**: Add proptest for filter properties - ---- - -### 3. CHAIN MODULE (10 files, ~3,500 lines) - -#### Overview -The chain module handles blockchain structure, reorgs, checkpoints, and chain locks. - -#### `src/chain/mod.rs` (116 lines) ✅ GOOD - -**Purpose**: Module exports and initialization. - -**Analysis**: -- **GOOD**: Clean exports -- **GOOD**: Well-organized - -#### `src/chain/checkpoints.rs` (605 lines) ⚠️ COMPLEX - -**Purpose**: Hardcoded checkpoint management for fast-sync. - -**What it does**: -- Stores hardcoded checkpoints for mainnet/testnet -- Validates checkpoint consistency -- Provides checkpoint selection logic - -**Complex Types Used**: -- `CheckpointManager` with BTreeMap for efficient range queries - **JUSTIFIED** - -**Analysis**: -- **GOOD**: Checkpoints enable fast sync (don't need to validate from genesis) -- **ISSUE**: Hardcoded checkpoint data is 400+ lines -- **ISSUE**: Confusing dual-checkpoint design (sync vs terminal chains) -- **EXCELLENT**: Comprehensive validation of checkpoint consistency -- **ISSUE**: Comment at line 67 references "terminal chain" - unclear terminology - -**Refactoring needed**: -- ⚠️ **HIGH PRIORITY**: Move checkpoint data to separate JSON/TOML file -- ⚠️ **HIGH PRIORITY**: Clarify "terminal chain" terminology (or remove concept) -- ⚠️ **MEDIUM**: Add checkpoint update process documentation -- ✅ **LOW**: Add build script to validate checkpoints against actual blocks - -#### `src/chain/chain_work.rs` (136 lines) ✅ EXCELLENT - -**Purpose**: Calculates cumulative proof-of-work for chain comparison. - -**What it does**: -- Implements arbitrary-precision arithmetic for chain work -- Used to determine best chain during reorgs - -**Complex Types Used**: -- `U256` (256-bit unsigned integer) - **JUSTIFIED**: Required for PoW calculations - -**Analysis**: -- **EXCELLENT**: Correct implementation of Bitcoin-style chain work -- **EXCELLENT**: Well-tested -- **GOOD**: Clear documentation - -**Refactoring needed**: ❌ None - this is a well-crafted module - -#### `src/chain/chainlock_manager.rs` (271 lines) ✅ GOOD - -**Purpose**: Manages Dash ChainLock verification and storage. - -**What it does**: -- Validates ChainLock BLS signatures -- Maintains latest ChainLock state -- Provides finality guarantees - -**Complex Types Used**: -- BLS signature verification - **NECESSARY**: Dash-specific consensus feature - -**Analysis**: -- **GOOD**: Core Dash functionality well-implemented -- **GOOD**: Proper signature validation -- **ISSUE**: TODO comment on line 127: "Implement actual signature validation" -- **CRITICAL BUG**: Signature validation is stubbed out! - -**Refactoring needed**: -- 🚨 **CRITICAL PRIORITY**: Implement actual BLS signature validation -- ⚠️ **HIGH**: Add integration tests with real ChainLock messages -- ⚠️ **MEDIUM**: Add metrics for ChainLock validation timing - -#### `src/chain/fork_detector.rs` (215 lines) ✅ EXCELLENT - -**Purpose**: Detects chain reorganizations. - -**What it does**: -- Monitors for competing chain tips -- Identifies fork points -- Triggers reorg handling - -**Analysis**: -- **EXCELLENT**: Clean state machine -- **EXCELLENT**: Well-tested (fork_detector_test.rs) -- **GOOD**: Clear documentation - -**Refactoring needed**: ❌ None - -#### `src/chain/orphan_pool.rs` (194 lines) ✅ EXCELLENT - -**Purpose**: Stores headers received out-of-order. - -**What it does**: -- Temporarily holds orphan blocks -- Attempts to connect orphans when parent arrives -- Prevents memory bloat with size limits - -**Analysis**: -- **EXCELLENT**: Essential for robust P2P handling -- **EXCELLENT**: Proper size limits to prevent DoS -- **EXCELLENT**: Well-tested - -**Refactoring needed**: ❌ None - -#### `src/chain/reorg.rs` (248 lines) ✅ GOOD - -**Purpose**: Handles blockchain reorganizations. - -**What it does**: -- Finds fork point -- Rolls back to common ancestor -- Applies new chain - -**Analysis**: -- **GOOD**: Correct reorg logic -- **GOOD**: ChainLock protection (won't reorg past chainlock) -- **ISSUE**: Could be more defensive about deep reorgs -- **GOOD**: Well-tested - -**Refactoring needed**: -- ⚠️ **MEDIUM**: Add configurable max reorg depth -- ✅ **LOW**: Add reorg event emission - -#### `src/chain/chain_tip.rs` (51 lines) ✅ GOOD - -**Purpose**: Simple wrapper for chain tip tracking. - -**Analysis**: -- **GOOD**: Clear single responsibility -- **QUESTION**: Is this file necessary? Could be folded into ChainState - -**Refactoring needed**: -- ✅ **LOW PRIORITY**: Consider merging into types.rs::ChainState - ---- - -### 4. CLIENT MODULE (17 files, ~6,500 lines) ✅ **REFACTORED** - -#### Overview -The client module provides the high-level API and orchestrates all subsystems. - -#### `src/client/` (Module - Refactored) ✅ **COMPLETE** - -**REFACTORING STATUS**: Complete (2025-01-21) -- ✅ Converted from single 2,851-line file to 8 focused modules -- ✅ All 243 tests passing (1 pre-existing test failure unrelated to refactoring) -- ✅ Compilation successful -- ✅ Production ready - -**Previous state**: Single file with 2,851 lines - GOD OBJECT -**Current state**: 8 well-organized modules (2,895 lines total) - MAINTAINABLE - -#### `src/client/mod.rs` (221 lines) ✅ **REFACTORED** - -**Purpose**: Module coordinator that re-exports DashSpvClient and declares submodules. - -**Current Structure**: -``` -client/ -├── mod.rs (221 lines) - Module declarations and re-exports -├── client.rs (252 lines) - Core struct and simple methods -├── lifecycle.rs (519 lines) - start/stop/initialization -├── sync_coordinator.rs (1,255 lines) - Sync orchestration -├── progress.rs (115 lines) - Progress tracking -├── mempool.rs (164 lines) - Mempool coordination -├── events.rs (46 lines) - Event handling -├── queries.rs (173 lines) - Peer/masternode/balance queries -├── chainlock.rs (150 lines) - ChainLock processing -├── block_processor.rs (649 lines) - Block processing -├── config.rs (484 lines) - Configuration -├── filter_sync.rs (171 lines) - Filter coordination -├── message_handler.rs (585 lines) - Message routing -└── status_display.rs (242 lines) - Status display -``` - -**Analysis**: -- ✅ **COMPLETE**: Successfully refactored from monolithic file -- ✅ **MAINTAINABLE**: Clear module boundaries -- ✅ **TESTABLE**: Each module can be tested independently -- ✅ **DOCUMENTED**: Lock ordering preserved in mod.rs -- ✅ **PRODUCTION READY**: All tests passing - -#### `src/client/config.rs` (253 lines) ✅ EXCELLENT - -**Purpose**: Client configuration with builder pattern. - -**What it does**: -- Network selection (mainnet/testnet/regtest) -- Storage path configuration -- Validation mode selection -- Feature toggles (filters, masternodes) -- Peer configuration - -**Analysis**: -- **EXCELLENT**: Clean builder pattern -- **EXCELLENT**: Sensible defaults -- **EXCELLENT**: Validation in `validate()` method -- **GOOD**: Well-documented fields - -**Refactoring needed**: ❌ None - this is exemplary - -#### `src/client/block_processor.rs` (649 lines) ⚠️ COMPLEX - -**Purpose**: Processes full blocks downloaded after filter matches. - -**What it does**: -- Downloads full blocks for filter matches -- Extracts relevant transactions -- Updates wallet state -- Emits transaction events - -**Complex Types Used**: -- `mpsc::UnboundedSender` - **JUSTIFIED**: Task queue pattern -- Async task spawning - **JUSTIFIED**: Parallel block processing - -**Analysis**: -- **GOOD**: Proper separation from main client -- **GOOD**: Async task management -- **ISSUE**: Could benefit from retry logic for failed downloads -- **ISSUE**: No priority queue (all blocks treated equally) - -**Refactoring needed**: -- ⚠️ **MEDIUM**: Add retry logic with exponential backoff -- ⚠️ **MEDIUM**: Add priority queue (recent blocks first) -- ✅ **LOW**: Add timeout configuration - -#### `src/client/filter_sync.rs` (289 lines) ✅ GOOD - -**Purpose**: Coordinates compact filter synchronization. - -**What it does**: -- Manages filter header download -- Coordinates filter download -- Detects filter matches -- Triggers block downloads - -**Analysis**: -- **GOOD**: Clear responsibility -- **GOOD**: Integrates well with sync manager -- **ISSUE**: Some duplication with sync/filters.rs - -**Refactoring needed**: -- ⚠️ **MEDIUM**: Clarify relationship with sync/filters.rs -- ⚠️ **LOW**: Reduce duplication - -#### `src/client/message_handler.rs` (243 lines) ✅ GOOD - -**Purpose**: Routes network messages to appropriate handlers. - -**What it does**: -- Receives messages from network layer -- Dispatches to sync/validation/mempool handlers -- Handles unknown message types - -**Analysis**: -- **GOOD**: Clean routing logic -- **GOOD**: Extensible design -- **EXCELLENT**: Well-tested (message_handler_test.rs) - -**Refactoring needed**: ❌ None - -#### `src/client/status_display.rs` (215 lines) ✅ GOOD - -**Purpose**: Calculates and displays sync progress. - -**What it does**: -- Computes header height from storage -- Handles checkpoint sync display -- Updates terminal UI (if enabled) -- Logs progress - -**Analysis**: -- **GOOD**: Clean separation of display logic -- **GOOD**: Proper feature gating for terminal UI -- **EXCELLENT**: Handles both checkpoint and genesis sync correctly -- **GOOD**: Comprehensive logging - -**Refactoring needed**: ❌ None - ---- - -### 5. NETWORK MODULE (14 files, ~5,000 lines) - -#### Overview -The network module handles all P2P communication with the Dash network. - -#### `src/network/mod.rs` (190 lines) ✅ EXCELLENT - -**Purpose**: Defines NetworkManager trait and module structure. - -**Complex Types Used**: -- **`NetworkManager` trait** - **JUSTIFIED**: Enables testing with mock network -- **Async trait** - **NECESSARY**: All network operations are async - -**What it does**: -- Defines trait for network implementations -- Requires: send_message, broadcast_message, get_peer_count, shutdown - -**Analysis**: -- **EXCELLENT**: Clean abstraction -- **EXCELLENT**: Trait design enables dependency injection -- **GOOD**: Well-documented trait methods - -**Refactoring needed**: ❌ None - exemplary trait design - -#### `src/network/manager.rs` (1,322 lines) 🚨 **TOO LARGE** - -**Purpose**: Peer network manager implementation. - -**What it does** (TOO MUCH): -- Peer discovery via DNS seeds -- Connection management -- Message routing to peers -- Peer health monitoring -- Reputation tracking -- Request/response correlation -- Statistics tracking -- Graceful shutdown - -**Complex Types Used**: -- `HashMap>` - **JUSTIFIED**: Efficient peer lookup -- Multiple tokio::sync primitives - **JUSTIFIED**: Complex concurrent operations - -**Critical Issues**: - -1. **File is 1,322 lines** - Should be split -2. **Too many responsibilities** - Violates SRP -3. **Complex state machine** - Peer states not explicitly modeled -4. **Lock contention potential** - Multiple Mutex/RwLock without ordering docs - -**Analysis**: -- **GOOD**: Robust peer management -- **GOOD**: DNS discovery implementation -- **ISSUE**: No connection pooling limits -- **ISSUE**: No bandwidth throttling -- **EXCELLENT**: Proper async shutdown - -**Refactoring needed**: -- 🚨 **CRITICAL**: Split into: - - `network/peer/manager.rs` - Main PeerNetworkManager - - `network/peer/discovery.rs` - DNS and peer discovery - - `network/peer/routing.rs` - Message routing - - `network/peer/health.rs` - Health monitoring -- ⚠️ **HIGH**: Add connection limit configuration -- ⚠️ **HIGH**: Add bandwidth throttling -- ⚠️ **MEDIUM**: Document lock ordering - -#### `src/network/connection.rs` (726 lines) ⚠️ LARGE - -**Purpose**: TCP connection to a single peer. - -**What it does**: -- Establishes TCP connection -- Performs handshake -- Message framing and parsing -- Keepalive/ping handling -- Connection timeout detection - -**Complex Types Used**: -- `TcpStream` with `BufReader`/`BufWriter` - **JUSTIFIED**: Standard pattern -- `Arc` for shutdown - **JUSTIFIED**: Signal across threads - -**Analysis**: -- **GOOD**: Robust connection handling -- **GOOD**: Proper framing -- **ISSUE**: No connection pooling -- **ISSUE**: No automatic reconnection - -**Refactoring needed**: -- ⚠️ **MEDIUM**: Add automatic reconnection with backoff -- ⚠️ **MEDIUM**: Add connection pooling -- ✅ **LOW**: Add per-connection statistics - -#### `src/network/handshake.rs` (212 lines) ✅ EXCELLENT - -**Purpose**: Dash P2P protocol handshake. - -**What it does**: -- Sends VERSION message -- Receives VERACK -- Exchanges service flags -- Validates protocol compatibility - -**Analysis**: -- **EXCELLENT**: Correct P2P handshake -- **EXCELLENT**: Proper error handling -- **GOOD**: Version negotiation - -**Refactoring needed**: ❌ None - -#### `src/network/manager.rs` (188 lines) ✅ GOOD - -**Purpose**: Peer metadata and state tracking. - -**What it does**: -- Stores peer information -- Tracks last seen time -- Service flags -- Version information - -**Analysis**: -- **GOOD**: Clean data structure -- **GOOD**: Useful helper methods - -**Refactoring needed**: ❌ None - -#### `src/network/reputation.rs` (142 lines) ✅ GOOD - -**Purpose**: Peer reputation and banning. - -**What it does**: -- Scores peer behavior -- Bans misbehaving peers -- Tracks ban durations - -**Analysis**: -- **GOOD**: Essential for P2P robustness -- **GOOD**: Configurable ban durations -- **ISSUE**: Ban list persists only in memory - -**Refactoring needed**: -- ⚠️ **MEDIUM**: Persist ban list to storage -- ✅ **LOW**: Add reputation decay over time - -#### `src/network/discovery.rs` (168 lines) ✅ GOOD - -**Purpose**: DNS seed peer discovery. - -**What it does**: -- Queries DNS seeds -- Resolves peer addresses -- Filters by network - -**Analysis**: -- **GOOD**: Standard DNS discovery -- **GOOD**: Proper error handling -- **ISSUE**: No fallback if all DNS seeds fail - -**Refactoring needed**: -- ⚠️ **LOW**: Add hardcoded fallback peers - -#### `src/network/mock.rs` (312 lines) ✅ EXCELLENT - -**Purpose**: Mock network implementation for testing. - -**What it does**: -- Implements NetworkManager trait -- Simulates peer responses -- Enables unit testing without real network - -**Analysis**: -- **EXCELLENT**: Essential for testing -- **EXCELLENT**: Well-implemented -- **GOOD**: Covers main use cases - -**Refactoring needed**: ❌ None - -#### Other network files: - -- `addrv2.rs` (128 lines) ✅ **GOOD** - Address serialization -- `constants.rs` (45 lines) ✅ **EXCELLENT** - Network constants -- `message_handler.rs` (94 lines) ✅ **GOOD** - Message dispatching -- `persist.rs` (87 lines) ✅ **GOOD** - Peer persistence -- `pool.rs` (143 lines) ✅ **GOOD** - Peer pool management - -**Overall Network Module Assessment**: -- ⚠️ NEEDS: Breaking up large files (peer.rs, connection.rs) -- ✅ GOOD: Strong abstractions -- ⚠️ NEEDS: Better documentation of concurrent access patterns -- ✅ GOOD: Comprehensive mock support - ---- - -### 6. STORAGE MODULE (12 files, ~4,100 lines) ✅ **REFACTORED** - -#### Overview -Storage module provides persistence abstraction with disk and memory implementations. - -#### `src/storage/disk/` (Module - Refactored) ✅ **COMPLETE** - -**REFACTORING STATUS**: Complete (2025-01-21) -- ✅ Converted from single 2,247-line file to 7 focused modules -- ✅ All 3 storage tests passing -- ✅ All 243 tests passing -- ✅ Compilation successful -- ✅ Production ready - -**Previous state**: Single file with 2,247 lines - MONOLITHIC -**Current state**: 7 well-organized modules (2,458 lines total) - MAINTAINABLE - -**Module Structure**: -``` -storage/disk/ -├── mod.rs (35 lines) - Module coordinator -├── manager.rs (383 lines) - Core struct & worker -├── segments.rs (313 lines) - Segment caching/eviction -├── headers.rs (437 lines) - Header storage -├── filters.rs (223 lines) - Filter storage -├── state.rs (896 lines) - State persistence & trait impl -└── io.rs (171 lines) - Low-level I/O -``` - -#### `src/storage/mod.rs` (229 lines) ✅ EXCELLENT - -**Purpose**: StorageManager trait definition. - -**Complex Types Used**: -- **`async_trait`** - **NECESSARY**: Async trait methods -- **Trait object compatibility** - **GOOD**: Enables dynamic dispatch - -**What it does**: -- Defines storage interface -- Methods for headers, filters, chainlocks, sync state -- Clear separation of concerns - -**Analysis**: -- **EXCELLENT**: Well-designed trait -- **EXCELLENT**: Comprehensive coverage of storage needs -- **GOOD**: Enables both memory and disk implementations - -**Refactoring needed**: ❌ None - exemplary trait design - -#### `src/storage/disk.rs` → `src/storage/disk/` ✅ **REFACTORED** - -**Previous Purpose**: Monolithic disk-based storage implementation. - -**Refactoring Complete (2025-01-21)**: -- ✅ Split from 2,247 lines into 7 focused modules -- ✅ Clear separation of concerns -- ✅ All storage tests passing -- ✅ Production ready - -**Current Module Responsibilities**: - -1. **manager.rs** (383 lines) - Core infrastructure - - DiskStorageManager struct with `pub(super)` fields - - Background worker for async I/O - - Constructor and worker management - - Segment ID/offset helpers - -2. **segments.rs** (313 lines) - Segment management - - SegmentCache and SegmentState - - Segment loading and eviction - - LRU cache management - - Dirty segment tracking - -3. **headers.rs** (437 lines) - Header operations - - Store/load headers with segment coordination - - Checkpoint sync support - - Header queries and batch operations - - Tip height tracking - -4. **filters.rs** (223 lines) - Filter operations - - Store/load filter headers - - Compact filter storage - - Filter tip height tracking - -5. **state.rs** (896 lines) - State persistence - - Chain state, masternode state, sync state - - ChainLocks and InstantLocks - - Mempool transaction persistence - - Complete StorageManager trait implementation - - All unit tests - -6. **io.rs** (171 lines) - Low-level I/O - - File loading/saving with encoding - - Atomic write operations - - Index file management - -**Analysis**: -- ✅ **COMPLETE**: Successfully modularized -- ✅ **MAINTAINABLE**: Clear module boundaries -- ✅ **TESTABLE**: Tests isolated in state.rs -- ✅ **SEGMENTED DESIGN**: Smart 50K-header segments preserved -- ⚠️ **FUTURE**: Could still benefit from checksums, compression, embedded DB - -#### `src/storage/memory.rs` (636 lines) ✅ GOOD - -**Purpose**: In-memory storage for testing. - -**What it does**: -- Implements StorageManager with HashMaps -- No persistence -- Fast for tests - -**Analysis**: -- **EXCELLENT**: Essential for fast tests -- **GOOD**: Clean implementation -- **GOOD**: Matches disk storage interface - -**Refactoring needed**: -- ✅ **ENHANCEMENT**: Consider using this for ephemeral nodes - -#### `src/storage/sync_state.rs` (178 lines) ✅ GOOD - -**Purpose**: Sync state serialization. - -**What it does**: -- Serializes/deserializes sync progress -- Enables resuming sync after restart -- Versioned format - -**Analysis**: -- **GOOD**: Enables resume functionality -- **GOOD**: Version tracking -- **ISSUE**: No backward compatibility handling - -**Refactoring needed**: -- ⚠️ **MEDIUM**: Add migration support for format changes - -#### `src/storage/sync_storage.rs` (85 lines) ✅ GOOD - -**Purpose**: Wrapper for sync-specific storage operations. - -**Analysis**: -- **GOOD**: Clean abstraction - -#### `src/storage/types.rs` (92 lines) ✅ GOOD - -**Purpose**: Storage-specific types. - -**Analysis**: -- **GOOD**: Clear types - ---- - -### 7. SYNC MODULE (16 files, ~12,000 lines) 🚨 **NEEDS MAJOR REFACTORING** - -#### Overview -The sync module coordinates all blockchain synchronization. This is the most complex part of the codebase. - -#### `src/sync/mod.rs` (167 lines) ✅ GOOD - -**Purpose**: Module exports and common sync utilities. - -**Analysis**: -- **GOOD**: Clean module organization - -#### `src/sync/sequential/` (Module - Refactored) ✅ **COMPLETE** - -**Purpose**: Sequential synchronization manager - coordinates all sync phases. - -**REFACTORING STATUS**: Complete (2025-01-21) -- ✅ Converted from single 2,246-line file to 11 focused modules -- ✅ All 242 tests passing -- ✅ Production ready - -**Module Structure**: -``` -sync/sequential/ (4,785 lines total across 11 modules) -├── mod.rs (52 lines) - Module coordinator and re-exports -├── manager.rs (234 lines) - Core SyncManager struct and accessors -├── lifecycle.rs (225 lines) - Initialization, startup, and shutdown -├── phase_execution.rs (519 lines) - Phase execution, transitions, timeout handling -├── message_handlers.rs (808 lines) - Handlers for sync phase messages -├── post_sync.rs (530 lines) - Handlers for post-sync messages (after initial sync) -├── phases.rs (621 lines) - SyncPhase enum and phase-related types -├── progress.rs (369 lines) - Progress tracking utilities -├── recovery.rs (559 lines) - Recovery and error handling logic -├── request_control.rs (410 lines) - Request flow control -└── transitions.rs (458 lines) - Phase transition management -``` - -**What it does**: -- Coordinates header sync (via `HeaderSyncManagerWithReorg`) -- Coordinates masternode list sync (via `MasternodeSyncManager`) -- Coordinates filter sync (via `FilterSyncManager`) -- Manages sync state machine through SyncPhase enum -- Handles phase transitions with validation -- Implements error recovery and retry logic -- Tracks progress across all sync phases -- Routes network messages to appropriate handlers -- Handles post-sync maintenance (new blocks, filters, etc.) - -**Complex Types Used**: -- **Generic constraints**: `` -- **State machine**: SyncPhase enum with strict sequential transitions -- **Shared state**: Arc> for wallet and stats -- **Sub-managers**: Delegates to specialized sync managers - -**Strengths**: -- ✅ **EXCELLENT**: Clean module separation by responsibility -- ✅ **EXCELLENT**: Sequential approach simplifies reasoning -- ✅ **GOOD**: Clear phase boundaries and transitions -- ✅ **GOOD**: Comprehensive error recovery -- ✅ **GOOD**: All phases well-documented -- ✅ **GOOD**: Lock ordering documented to prevent deadlocks - -#### `src/sync/filters/` (Module - Phase 1 Complete) ✅ **REFACTORED** - -**Purpose**: Compact filter synchronization logic. - -**REFACTORING STATUS**: Phase 1 Complete (2025-01-XX) -- ✅ Converted from single 4,060-line file to module directory -- ✅ Extracted types and constants to `types.rs` (89 lines) -- ✅ Main logic in `manager_full.rs` (4,027 lines - awaiting Phase 2) -- ✅ All 243 tests passing - -**Previous state**: Single file with 4,027 lines - UNACCEPTABLE -**Current state**: Module structure established - Phase 2 extraction needed - -**What it does**: -- Filter header sync (CFHeaders) -- Compact filter download (CFilters) -- Filter matching against wallet addresses -- Gap detection and recovery -- Request batching and flow control -- Timeout and retry logic -- Progress tracking and statistics -- Peer selection and routing - -**Phase 2 Accomplishment (2025-01-21)**: -- ✅ All 8 modules successfully extracted -- ✅ `manager.rs` - Core coordinator (342 lines) -- ✅ `headers.rs` - CFHeaders sync (1,345 lines) -- ✅ `download.rs` - CFilter download (659 lines) -- ✅ `matching.rs` - Filter matching (454 lines) -- ✅ `gaps.rs` - Gap detection (490 lines) -- ✅ `retry.rs` - Retry logic (381 lines) -- ✅ `stats.rs` - Statistics (234 lines) -- ✅ `requests.rs` - Request management (248 lines) -- ✅ `types.rs` - Type definitions (86 lines) -- ✅ `mod.rs` - Module coordinator (42 lines) -- ✅ `manager_full.rs` deleted -- ✅ All 243 tests passing -- ✅ Compilation successful - -**Final Module Structure:** -``` -sync/filters/ -├── mod.rs (42 lines) - Module coordinator -├── types.rs (86 lines) - Type definitions -├── manager.rs (342 lines) - Core coordinator -├── stats.rs (234 lines) - Statistics tracking -├── retry.rs (381 lines) - Timeout/retry logic -├── requests.rs (248 lines) - Request queues -├── gaps.rs (490 lines) - Gap detection -├── headers.rs (1,345 lines) - CFHeaders sync -├── download.rs (659 lines) - CFilter download -└── matching.rs (454 lines) - Filter matching -``` - -**Analysis**: -- ✅ **COMPLETE**: All refactoring objectives met -- ✅ **MAINTAINABLE**: Clear module boundaries and responsibilities -- ✅ **TESTABLE**: Each module can be tested independently -- ✅ **DOCUMENTED**: Each module has focused documentation -- ✅ **PRODUCTION READY**: All tests passing, no regressions - -#### `src/sync/headers.rs` (705 lines) ⚠️ LARGE - -**Purpose**: Header synchronization logic. - -**What it does**: -- Downloads headers from peers -- Validates header chain -- Handles headers2 compression -- Detects reorgs - -**Analysis**: -- **GOOD**: Comprehensive header sync -- **GOOD**: Headers2 support -- **ISSUE**: Could be split into headers1 and headers2 modules - -**Refactoring needed**: -- ⚠️ **MEDIUM**: Split headers1 and headers2 into separate files -- ⚠️ **LOW**: Add more documentation - -#### `src/sync/headers_with_reorg.rs` (1,148 lines) 🚨 **TOO LARGE** - -**Purpose**: Header sync with reorganization detection. - -**Analysis**: -- **ISSUE**: 1,148 lines is too large -- **GOOD**: Handles complex reorg scenarios -- **ISSUE**: Overlaps with sync/headers.rs - -**Refactoring needed**: -- ⚠️ **HIGH**: Merge with headers.rs or clearly separate concerns -- ⚠️ **HIGH**: Split into smaller modules - -#### `src/sync/masternodes.rs` (775 lines) ⚠️ LARGE - -**Purpose**: Masternode list synchronization. - -**What it does**: -- Downloads masternode diffs -- Updates masternode list engine -- Validates quorums - -**Analysis**: -- **GOOD**: Dash-specific functionality -- **GOOD**: Proper validation -- **ISSUE**: Could be split - -**Refactoring needed**: -- ⚠️ **MEDIUM**: Split diff download and validation - -#### Other sync files: -- `chainlock_validation.rs` (231 lines) ✅ **GOOD** -- `discovery.rs` (98 lines) ✅ **GOOD** -- `embedded_data.rs` (118 lines) ✅ **GOOD** -- `state.rs` (157 lines) ✅ **GOOD** -- `validation.rs` (283 lines) ✅ **GOOD** - -**Overall Sync Module Assessment**: -- ✅ **EXCELLENT**: sync/filters/ fully refactored (10 modules, 4,281 lines) -- ✅ **EXCELLENT**: sync/sequential/ fully refactored (11 modules, 4,785 lines) -- ✅ **EXCELLENT**: State machine clearly modeled in phases.rs -- ✅ **EXCELLENT**: Error recovery consolidated in recovery.rs -- ✅ **GOOD**: Sequential approach is sound -- ✅ **GOOD**: Individual algorithms appear correct - ---- - -### 8. VALIDATION MODULE (6 files, ~2,000 lines) - -#### Overview -Validation module handles header validation, ChainLock verification, and InstantLock verification. - -#### `src/validation/mod.rs` (264 lines) ✅ GOOD - -**Purpose**: ValidationManager orchestration. - -**What it does**: -- Coordinates header validation -- Coordinates ChainLock validation -- Coordinates InstantLock validation -- Configurable validation modes - -**Analysis**: -- **GOOD**: Clean orchestration -- **GOOD**: Mode-based validation -- **EXCELLENT**: Well-tested - -**Refactoring needed**: ❌ None - -#### `src/validation/headers.rs` (418 lines) ✅ GOOD - -**Purpose**: Header chain validation. - -**What it does**: -- Validates PoW -- Validates timestamps -- Validates difficulty transitions -- Validates block linking - -**Analysis**: -- **GOOD**: Correct validation rules -- **GOOD**: Proper Dash-specific rules -- **EXCELLENT**: Comprehensive tests (headers_test.rs, headers_edge_test.rs) - -**Refactoring needed**: ❌ None - well-crafted - -#### `src/validation/quorum.rs` (248 lines) ✅ GOOD - -**Purpose**: Quorum validation for ChainLocks and InstantLocks. - -**What it does**: -- Validates quorum membership -- Validates BLS signatures -- Tracks active quorums - -**Analysis**: -- **GOOD**: Dash-specific functionality -- **ISSUE**: TODO comments indicate incomplete implementation - -**Refactoring needed**: -- ⚠️ **HIGH**: Complete TODO items for signature validation - -#### `src/validation/instantlock.rs` (87 lines) ⚠️ INCOMPLETE - -**Purpose**: InstantLock validation. - -**Analysis**: -- **ISSUE**: Contains TODO for actual signature validation -- **CRITICAL**: Validation is stubbed out - -**Refactoring needed**: -- 🚨 **CRITICAL**: Implement actual InstantLock signature validation - -**Overall Validation Module Assessment**: -- ✅ **GOOD**: Header validation is solid -- 🚨 **CRITICAL**: BLS signature validation incomplete (security risk) -- ✅ **EXCELLENT**: Test coverage for headers -- ⚠️ **HIGH PRIORITY**: Complete Dash-specific validation features - ---- - -### 9. MEMPOOL_FILTER.RS (793 lines) ✅ GOOD - -**Purpose**: Filters mempool transactions based on wallet addresses. - -**What it does**: -- Receives mempool transactions -- Checks against watched addresses -- Emits events for relevant txns -- Manages mempool state - -**Complex Types Used**: -- `Arc>` - **JUSTIFIED**: Shared between sync and mempool tasks - -**Analysis**: -- **GOOD**: Clean implementation -- **GOOD**: Proper async handling -- **GOOD**: Event emission - -**Refactoring needed**: -- ✅ **LOW**: Could extract to mempool/ module directory - ---- - -### 10. TERMINAL.RS (223 lines) ✅ EXCELLENT - -**Purpose**: Terminal UI for CLI binary (optional feature). - -**What it does**: -- Renders status bar -- Updates sync progress -- Displays peer count - -**Analysis**: -- **EXCELLENT**: Properly feature-gated -- **EXCELLENT**: Clean implementation -- **GOOD**: Uses crossterm effectively - -**Refactoring needed**: ❌ None - this is well-done - ---- - -## Critical Assessment - -### 🏆 STRENGTHS - -1. **Excellent Architecture Principles** - - Trait-based abstraction (NetworkManager, StorageManager) - - Dependency injection enables testing - - Clear module boundaries - -2. **Comprehensive Functionality** - - Full SPV implementation - - Dash-specific features (ChainLocks, InstantLocks, Masternodes) - - BIP157 compact filters - - Robust reorg handling - -3. **Good Testing Culture** - - Mock network implementation - - Comprehensive header validation tests - - Unit tests for critical components - -4. **Modern Rust** - - Async/await throughout - - Proper error handling with thiserror - - Good use of type system - -5. **Performance Optimizations** - - CachedHeader for X11 hash caching - - Segmented storage for efficient I/O - - Bloom filters for transaction filtering - -### 🚨 CRITICAL PROBLEMS - -1. **INCOMPLETE SECURITY FEATURES** 🔥🔥 - - ChainLock signature validation stubbed (chainlock_manager.rs:127) - - InstantLock signature validation incomplete - - **SECURITY RISK**: Could accept invalid ChainLocks/InstantLocks - - **PRIORITY**: Must be completed before mainnet production use - - **EFFORT**: 1-2 weeks - -### ⚠️ AREAS FOR IMPROVEMENT - -1. **Testing Coverage** - - Network layer could use more integration tests - - End-to-end sync cycle testing would increase confidence - - Property-based testing could validate invariants - -2. **Resource Management** - - Connection limits not enforced - - No bandwidth throttling - - Peer ban list not persisted across restarts - -3. **Code Duplication** - - Some overlap between headers.rs and headers_with_reorg.rs - - Validation logic could be further consolidated - -5. **Error Recovery** - - Retry strategies could be more consistent - - Some edge cases may lack retry logic - -### ✅ MINOR ISSUES - -1. **Dead Code** - - bloom/stats.rs not used - - Some deprecated error variants still present - -2. **Hardcoded Values** - - Checkpoints in code rather than data file - - Timeout values not configurable - -3. **Missing Features** - - No compression in storage - - No checksums for corruption detection - - Peer ban list not persisted - ---- - -## Recommendations - -### 🚨 CRITICAL PRIORITY (Do First) - -1. **Implement BLS Signature Validation** - - **Why**: Security vulnerability - could accept invalid ChainLocks/InstantLocks - - **Impact**: 🔥🔥🔥 CRITICAL SECURITY - - **Effort**: 1-2 weeks (requires BLS library integration) - - **Benefit**: Production-ready security for mainnet - -### ⚠️ HIGH PRIORITY (Do Soon) - -2. **Add Comprehensive Integration Tests** - - **Why**: Increase confidence in network layer and sync pipeline - - **Impact**: 🔥🔥 HIGH - - **Effort**: 1 week - - **Benefit**: Catch regressions, validate end-to-end behavior - -3. **Document Lock Ordering More Prominently** - - **Why**: Prevent deadlocks - - **Impact**: 🔥🔥 HIGH (correctness) - - **Effort**: 1 day - - **Benefit**: Correctness, debugging - -7. **Add Comprehensive Integration Tests** - - **Why**: Network layer undertested - - **Impact**: 🔥🔥 HIGH - - **Effort**: 1 week - - **Benefit**: Confidence, regression prevention - -### ✅ MEDIUM PRIORITY (Plan For) - -8. **Extract Checkpoint Data to Config File** - - **Impact**: 🔥 MEDIUM - - **Effort**: 1 day - -9. **Add Resource Limits** - - Connection limits - - Bandwidth throttling - - Memory limits - - **Impact**: 🔥 MEDIUM (DoS protection) - - **Effort**: 3-4 days - -10. **Improve Error Recovery** - - Consolidate retry logic - - Consistent backoff strategies - - **Impact**: 🔥 MEDIUM - - **Effort**: 1 week - -11. **Add Property-Based Tests** - - Use proptest for filter properties - - Test reorg handling - - **Impact**: 🔥 MEDIUM - - **Effort**: 1 week - -### ✅ LOW PRIORITY (Nice to Have) - -12. **Type Aliases for Common Configurations** (Ergonomics Only) - - Generic design is intentional and excellent for library flexibility - - Type aliases just provide convenience without losing flexibility - ```rust - type StandardSpvClient = DashSpvClient< - WalletManager, - PeerNetworkManager, - DiskStorageManager - >; - ``` - -13. **Consider Embedded DB for Storage** - - RocksDB or Sled - - Better concurrency - - Compression built-in - -14. **Add Compression to Storage** - - Filters compress well - - Save disk space - -15. **Persist Peer Ban List** - - Survives restarts - ---- - -## Complexity Metrics - -### File Complexity (Largest Files) - -| File | Lines | Complexity | Notes | -|------|-------|------------|-------| -| sync/filters/ | 10 modules (4,281 total) | ✅ EXCELLENT | Well-organized filter sync modules | -| sync/sequential/ | 11 modules (4,785 total) | ✅ EXCELLENT | Sequential sync pipeline modules | -| client/ | 8 modules (2,895 total) | ✅ EXCELLENT | Client functionality modules | -| storage/disk/ | 7 modules (2,458 total) | ✅ EXCELLENT | Persistent storage modules | -| network/manager.rs | 1,322 | ✅ ACCEPTABLE | Complex peer management logic | -| sync/headers_with_reorg.rs | 1,148 | ✅ ACCEPTABLE | Reorg handling complexity justified | -| types.rs | 1,064 | ✅ ACCEPTABLE | Core type definitions | -| mempool_filter.rs | 793 | ✅ GOOD | Mempool management | -| bloom/tests.rs | 799 | ✅ GOOD | Comprehensive bloom tests | -| sync/masternodes.rs | 775 | ✅ GOOD | Masternode sync logic | - -**Note:** All files are now at acceptable complexity levels. The 1,000-1,500 line files contain inherently complex logic that justifies their size. - -### Module Health - -| Module | Files | Lines | Health | Characteristics | -|--------|-------|-------|--------|-----------------| -| sync/ | 37 | ~12,000 | ✅ EXCELLENT | Filters and sequential both fully modularized | -| client/ | 8 | ~2,895 | ✅ EXCELLENT | Clean separation: lifecycle, sync, progress, mempool, events | -| storage/ | 13 | ~3,500 | ✅ EXCELLENT | Disk storage split into focused modules | -| network/ | 14 | ~5,000 | ✅ GOOD | Handles peer management, connections, message routing | -| chain/ | 10 | ~3,500 | ✅ GOOD | ChainLock, checkpoint, orphan pool management | -| bloom/ | 6 | ~2,000 | ✅ GOOD | Bloom filter implementation for transaction filtering | -| validation/ | 6 | ~2,000 | ⚠️ FAIR | Needs BLS validation implementation (security) | -| error/ | 1 | 303 | ✅ EXCELLENT | Clean error hierarchy with thiserror | -| types/ | 1 | 1,065 | ✅ ACCEPTABLE | Core type definitions, reasonable size | - ---- - -## Security Considerations - -### 🚨 CRITICAL SECURITY ISSUES - -1. **Incomplete ChainLock Validation** - - File: `chain/chainlock_manager.rs:127` - - Issue: Signature validation stubbed out - - Risk: Could accept invalid ChainLocks - - Fix: Implement BLS signature verification - -2. **Incomplete InstantLock Validation** - - File: `validation/instantlock.rs` - - Issue: Validation incomplete - - Risk: Could accept invalid InstantLocks - - Fix: Complete InstantLock validation - -### ⚠️ POTENTIAL RISKS - -3. **No Checksums on Stored Data** - - File: `storage/disk.rs` - - Risk: Silent corruption - - Fix: Add checksums - -4. **No Connection Limits** - - File: `network/manager.rs` - - Risk: DoS via connection exhaustion - - Fix: Add configurable limits - -5. **Peer Ban List Not Persisted** - - File: `network/reputation.rs` - - Risk: Misbehaving peers reconnect after restart - - Fix: Persist ban list - ---- - -## Performance Considerations - -### ✅ OPTIMIZATIONS PRESENT - -1. **CachedHeader** - Excellent X11 hash caching -2. **Segmented Storage** - Good I/O patterns -3. **Bloom Filters** - Efficient transaction filtering -4. **Async/Await** - Non-blocking operations - -### 🔧 POTENTIAL IMPROVEMENTS - -1. **Add Compression** - Filters compress ~70% -2. **Connection Pooling** - Reuse TCP connections -3. **Batch Storage Writes** - Reduce fsync calls -4. **RocksDB** - Better than file-based storage - ---- - -## Maintainability Score - -### By Module - -| Module | Maintainability | Reasoning | -|--------|----------------|-----------| -| error | 95/100 ✅ | Perfect design | -| terminal | 90/100 ✅ | Small, focused | -| bloom | 85/100 ✅ | Well-organized | -| chain | 80/100 ✅ | Good structure | -| validation | 70/100 ⚠️ | Incomplete features | -| network | 65/100 ⚠️ | Large files | -| storage | 60/100 ⚠️ | disk.rs too large | -| client | 45/100 🔥 | God object | -| sync | 30/100 🔥🔥🔥 | Massive files | - -### Overall: **55/100** ⚠️ NEEDS IMPROVEMENT - -**Primary Blockers**: -1. File size issues (sync/filters.rs especially) -2. God objects -3. Missing security features - -**After Refactoring Estimate**: **75-80/100** ✅ - ---- - -## Conclusion - -### The Good - -This is a **comprehensive, feature-rich SPV client** with: -- Excellent architectural foundations -- Good use of Rust's type system -- Comprehensive Dash-specific features -- Solid testing culture - -### The Bad - -The codebase suffers from **maintainability crisis**: -- Several files exceed 2,000 lines (one is 4,027!) -- God objects violate Single Responsibility Principle -- Critical security features incomplete - -### The Path Forward - -**Phase 1 (2-3 weeks)**: Critical refactoring -1. Split sync/filters.rs -2. Implement BLS signature validation -3. Split client/mod.rs - -**Phase 2 (2-3 weeks)**: High-priority improvements -4. Split remaining large files -5. Document lock ordering -6. Add integration tests - -**Phase 3 (Ongoing)**: Incremental improvements -7. Resource limits -8. Enhanced error recovery -9. Performance optimizations - -### Final Verdict - -**Rating**: ⚠️ **B- (Good but Needs Work)** - -- **Architecture**: A- (excellent design) -- **Functionality**: A (comprehensive features) -- **Code Quality**: C+ (too many large files) -- **Security**: C (critical features incomplete) -- **Testing**: B- (good but gaps) -- **Documentation**: C+ (incomplete) - -**Recommendation**: This codebase is **production-capable** for its current feature set, but **REQUIRES IMMEDIATE REFACTORING** before adding major new features. The file size issues will cause serious problems for collaboration and maintenance. The incomplete signature validation is a security concern that must be addressed before production use on mainnet. - -**With the recommended refactorings**, this could easily become an **A-grade codebase** - the foundations are solid. - ---- - -*End of Architectural Analysis* diff --git a/dash-spv/CODE_ANALYSIS_SUMMARY.md b/dash-spv/CODE_ANALYSIS_SUMMARY.md deleted file mode 100644 index 8c761612b..000000000 --- a/dash-spv/CODE_ANALYSIS_SUMMARY.md +++ /dev/null @@ -1,349 +0,0 @@ -# Dash SPV Codebase Analysis - Executive Summary - -**Date:** 2025-01-21 -**Analyzer:** Claude (Anthropic AI) -**Codebase Version:** 0.40.0 -**Total Files Analyzed:** 110+ files -**Total Lines of Code:** ~40,000 - ---- - -## 📊 Analysis Overview - -✅ **Full codebase analyzed** - All files reviewed and refactored -✅ **Architecture guide created** - See `ARCHITECTURE.md` for comprehensive documentation -✅ **Major refactoring complete** - All critical file size issues resolved -✅ **Production-ready structure** - Clean module boundaries and focused components - ---- - -## 🎯 Overall Assessment - -### Current Grade: **A+ (96/100)** - -| Aspect | Grade | Comment | -|--------|-------|---------| -| Architecture | A+ | Excellent trait-based design with clear module boundaries | -| Functionality | A | Comprehensive Dash SPV features | -| Code Organization | A+ | All modules properly sized and focused | -| Security | C | BLS signature validation incomplete (only remaining critical issue) | -| Testing | B+ | Good coverage with 242/243 tests passing | -| Documentation | B+ | Well-documented modules with clear structure | - ---- - -## 🔥 CRITICAL ISSUES (Must Fix) - -### Incomplete Security Features 🚨 - -**Problem:** BLS signature validation is stubbed out - -**Affected Files:** -- `chain/chainlock_manager.rs:127` - ChainLock validation incomplete -- `validation/instantlock.rs` - InstantLock validation incomplete - -**Risk:** Could accept invalid ChainLocks/InstantLocks, breaking Dash's security model - -**Priority:** HIGH - Must be completed before mainnet production use - -**Estimated Effort:** 1-2 weeks - ---- - -## ✅ STRENGTHS - -### 1. Excellent Architecture -- Clean trait-based abstractions (NetworkManager, StorageManager, WalletInterface) -- Dependency injection enables comprehensive testing -- Clear module boundaries with focused responsibilities -- All modules under 1,000 lines (most under 500) - -### 2. Comprehensive Features -- Full SPV implementation with checkpoint support -- Dash-specific: ChainLocks, InstantLocks, Masternodes -- BIP157 compact block filters -- Robust reorg handling with recovery logic -- Sequential sync pipeline for reliable synchronization - -### 3. Well-Organized Modules -- **sync/filters/** - 10 focused modules (4,281 lines) for filter synchronization -- **sync/sequential/** - 11 focused modules (4,785 lines) for sequential sync coordination -- **client/** - 8 focused modules (2,895 lines) for client functionality -- **storage/disk/** - 7 focused modules (2,458 lines) for persistent storage - -### 4. Performance Optimizations -- CachedHeader for X11 hash caching (4-6x speedup) -- Segmented storage for efficient I/O -- Flow control for parallel filter downloads -- Async/await throughout for non-blocking operations - -### 5. Strong Testing Culture -- 242/243 tests passing (99.6% pass rate) -- Mock implementations for testing (MockNetworkManager) -- Comprehensive validation tests -- Integration test suite - ---- - -## ⚠️ AREAS FOR IMPROVEMENT - -### High Priority - -1. **Complete BLS Signature Validation** 🚨 - - Required for mainnet security - - ChainLock and InstantLock validation - - Estimated effort: 1-2 weeks - -2. **Document Lock Ordering** - - Critical for preventing deadlocks - - Lock acquisition order documented but could be more prominent - - Estimated effort: 1 day - -3. **Add Comprehensive Integration Tests** - - Network layer needs more end-to-end testing - - Full sync cycle testing - - Estimated effort: 1 week - -### Medium Priority - -4. **Resource Management** - - Add connection limits - - Implement bandwidth throttling - - Persist peer ban list - -5. **Error Recovery Consistency** - - Standardize retry strategies across modules - - Add more detailed error context - -6. **Type Aliases for Common Configurations** (Optional Convenience) - - Add type aliases like `StandardSpvClient` for common use cases - - Improves ergonomics while keeping generic flexibility - - Note: The generic design itself is excellent for library flexibility - -### Low Priority - -7. **Extract Checkpoint Data to Config File** - - Currently hardcoded in source - - Would enable easier updates - -8. **Consider Embedded Database** - - Alternative to current file-based storage - - Could improve query performance - ---- - -## 📈 METRICS - -### Module Health Scorecard - -| Module | Files | Health | Main Characteristics | -|--------|-------|--------|----------------------| -| sync/ | 37 | ✅ EXCELLENT | Well-organized with filters/ and sequential/ fully modularized | -| client/ | 8 | ✅ EXCELLENT | Clean separation: lifecycle, sync, progress, mempool, events | -| storage/ | 13 | ✅ EXCELLENT | disk/ module with focused components (headers, filters, state) | -| network/ | 14 | ✅ GOOD | Handles peer management, connections, message routing | -| validation/ | 6 | ⚠️ FAIR | Missing BLS validation (security concern) | -| chain/ | 10 | ✅ GOOD | ChainLock, checkpoint, orphan pool management | -| bloom/ | 6 | ✅ GOOD | Bloom filter implementation for transaction filtering | -| error/ | 1 | ✅ EXCELLENT | Clean error type hierarchy with thiserror | -| types/ | 1 | ✅ GOOD | Core type definitions (could be split further) | - -### File Size Distribution - -``` -2000+ lines: 0 files ✅ (all large files refactored) -1000-2000: 4 files ✅ (acceptable complexity) -500-1000: 12 files ✅ (good module size) -<500 lines: 95+ files ✅ (excellent - focused modules) -``` - -**Largest Remaining Files:** -- `network/manager.rs` (1,322 lines) - Acceptable for complex peer management -- `sync/headers_with_reorg.rs` (1,148 lines) - Acceptable for reorg handling -- `types.rs` (1,064 lines) - Could be split but acceptable - ---- - -## 🎓 DEVELOPMENT GUIDELINES - -### Adding New Features - -**Before adding code:** -1. Check target file size (prefer <500 lines) -2. Identify appropriate module or create new one -3. Add comprehensive tests -4. Document complex logic -5. Update ARCHITECTURE.md if adding major features - -### Working with Locks - -**Critical lock ordering (to prevent deadlocks):** -1. `running` (client state) -2. `state` (ChainState) -3. `stats` (SpvStats) -4. `mempool_state` (MempoolState) -5. `storage` (StorageManager operations) - -**Never acquire locks in reverse order!** - -### Module Organization Principles - -**Key design principles followed:** -- **Single Responsibility**: Each module has one clear purpose -- **Focused Files**: Target 200-500 lines per file -- **Clear Boundaries**: Public API vs internal implementation -- **`pub(super)` for Cross-Module Access**: Sibling modules can share helpers -- **Comprehensive Tests**: Tests live with the code they test - -### Complex Types Explained - -**`Arc>`** - Shared state with concurrent reads -- Used for: state, stats, mempool_state -- Pattern: Many readers OR one writer - -**`Arc>`** - Shared state with exclusive access -- Used for: storage operations -- Simpler than RwLock when writes are common - -**`CachedHeader`** - Performance optimization -- Caches X11 hash (expensive to compute) -- 4-6x speedup during header validation -- Uses Arc for thread-safe lazy initialization - -### Testing Strategy - -**Test Types:** -- **Unit Tests**: Individual functions/modules (in-file with `#[cfg(test)]`) -- **Integration Tests**: Cross-module interactions (`tests/` directory) -- **Mock Tests**: Use MockNetworkManager, DiskStorageManager::new_tmp -- **Property Tests**: Invariant testing (could add more with proptest) - ---- - -## 📚 MODULE DOCUMENTATION - -### Comprehensive Module Guides - -Each major module has detailed documentation: - -1. **`sync/filters/`** - Compact filter synchronization - - 10 modules: types, manager, stats, retry, requests, gaps, headers, download, matching - - Handles BIP157 filter headers and filter download - - Flow control for parallel downloads - -2. **`sync/sequential/`** - Sequential sync coordination - - 11 modules: manager, lifecycle, phase_execution, message_handlers, post_sync, phases, progress, recovery, request_control, transitions - - Strict sequential pipeline: Headers → MnList → CFHeaders → Filters → Blocks - - Clear state machine with phase transitions - -3. **`client/`** - High-level SPV client - - 8 modules: client, lifecycle, sync_coordinator, progress, mempool, events, queries, chainlock - - Main entry point: DashSpvClient - - Coordinates all subsystems - -4. **`storage/disk/`** - Persistent storage - - 7 modules: manager, segments, headers, filters, state, io - - Segmented storage: 50,000 headers per segment - - Background I/O worker for non-blocking operations - ---- - -## 🚀 PATH TO PRODUCTION - -### Current Status: **Development-Ready** (A+) - -✅ **Completed:** -- Excellent code organization -- Comprehensive feature set -- Good test coverage (242/243 passing) -- Well-documented architecture -- Robust error handling -- Performance optimizations - -⚠️ **Before Mainnet Use:** -- 🚨 **MUST** implement BLS signature validation (ChainLocks + InstantLocks) -- ⚠️ **SHOULD** add comprehensive integration tests -- ⚠️ **SHOULD** add resource limits (connections, bandwidth) - -### For Testnet Use: -✅ **Ready** - Current state is suitable for testnet development and testing - -### For Mainnet Use: -🚨 **Complete BLS validation first** - This is the only blocking security issue - ---- - -## 💡 FINAL ASSESSMENT - -### The Excellent 🌟 - -This codebase demonstrates **professional-grade Rust development**: -- Exceptional module organization with clear boundaries -- Solid architectural foundations using traits and dependency injection -- Comprehensive Dash-specific features (ChainLocks, InstantLocks, Masternodes) -- Strong testing culture with high test coverage -- Modern async/await patterns throughout -- Well-documented code with clear intent - -### The Remaining Work ⚠️ - -Only **one critical issue** remains: -- BLS signature validation for ChainLocks and InstantLocks - -This is a **security feature** required for production use but does not affect the overall code quality, organization, or architecture. - -### The Verdict 🎯 - -**Rating: A+ (96/100)** ✨ - -**Strengths:** -- Outstanding code organization (100% of large files refactored) -- Excellent architecture and design patterns -- Comprehensive feature set -- Strong test coverage - -**Remaining:** -- BLS signature validation (security, not organization) - -**Assessment:** This codebase has transformed from "good but needs work" to **"excellent and production-ready structure"**. Only security features remain before full mainnet deployment. - -The organizational refactoring work is **complete and successful**. The codebase is now: -- ✅ Easy to maintain -- ✅ Easy to contribute to -- ✅ Well-tested -- ✅ Well-documented -- ✅ Performance-optimized -- ⚠️ Secure (pending BLS validation) - ---- - -## 📞 NEXT STEPS - -### Immediate Priority: Security - -1. **Implement BLS Signature Validation** 🚨 **CRITICAL** - - ChainLock validation (chain/chainlock_manager.rs:127) - - InstantLock validation (validation/instantlock.rs) - - **Effort**: 1-2 weeks - - **Benefit**: Production-ready security for mainnet - -### Recommended Improvements - -2. **Add Comprehensive Integration Tests** - - End-to-end sync testing - - Network layer testing - - **Effort**: 1 week - -3. **Document Lock Ordering More Prominently** - - Add visual diagrams - - Include in developer documentation - - **Effort**: 1 day - -4. **Add Resource Limits** - - Connection limits - - Bandwidth throttling - - **Effort**: 3-5 days - ---- - -*This analysis reflects the current state of the codebase after comprehensive organizational refactoring completed on 2025-01-21. For architectural details, see `ARCHITECTURE.md`.* diff --git a/dash-spv/run_integration_tests.md b/dash-spv/run_integration_tests.md deleted file mode 100644 index 0bbfd8a6a..000000000 --- a/dash-spv/run_integration_tests.md +++ /dev/null @@ -1,192 +0,0 @@ -# Running Integration Tests with Real Dash Core Node - -This document explains how to run the integration tests that connect to a real Dash Core node. - -## Prerequisites - -1. **Dash Core Node**: You need a Dash Core node running and accessible at `127.0.0.1:9999` -2. **Network**: The node should be connected to Dash mainnet -3. **Sync Status**: The node should be synced (for testing header sync up to 10k headers) - -## Setting Up Dash Core Node - -### Option 1: Local Dash Core Node - -1. Download and install Dash Core from https://github.com/dashpay/dash/releases -2. Configure `dash.conf`: - ``` - # dash.conf - testnet=0 # Use mainnet - rpcuser=dashrpc - rpcpassword=your_password - server=1 - listen=1 - ``` -3. Start Dash Core: `dashd` or use the GUI -4. Wait for initial sync (this can take several hours for mainnet) - -### Option 2: Docker Dash Core Node - -```bash -# Run Dash Core in Docker -docker run -d \ - --name dash-node \ - -p 9999:9999 \ - -p 9998:9998 \ - dashpay/dashd:latest \ - dashd -server=1 -listen=1 -discover=1 -``` - -## Running the Integration Tests - -### Check Node Availability - -First, verify your node is accessible: -```bash -# Test basic connectivity -nc -zv 127.0.0.1 9999 -``` - -### Run Individual Integration Tests - -```bash -cd dash-spv - -# Test basic connectivity -cargo test --test integration_real_node_test test_real_node_connectivity -- --nocapture - -# Test header sync up to 1000 headers -cargo test --test integration_real_node_test test_real_header_sync_genesis_to_1000 -- --nocapture - -# Test header sync up to 10k headers (requires synced node) -cargo test --test integration_real_node_test test_real_header_sync_up_to_10k -- --nocapture - -# Test header validation with real data -cargo test --test integration_real_node_test test_real_header_validation_with_node -- --nocapture - -# Test header chain continuity -cargo test --test integration_real_node_test test_real_header_chain_continuity -- --nocapture - -# Test sync resumption -cargo test --test integration_real_node_test test_real_node_sync_resumption -- --nocapture - -# Run performance benchmarks -cargo test --test integration_real_node_test test_real_node_performance_benchmarks -- --nocapture -``` - -### Run All Integration Tests - -```bash -# Run all integration tests -cargo test --test integration_real_node_test -- --nocapture -``` - -## Expected Test Behavior - -### With Node Available - -When a Dash Core node is running at 127.0.0.1:9999, the tests will: - -1. **Connect and handshake** with the real node -2. **Download actual headers** from the Dash mainnet blockchain -3. **Validate real blockchain data** using the SPV client -4. **Measure performance** of header synchronization -5. **Test chain continuity** with real header linkage -6. **Benchmark sync rates** (typically 50-200+ headers/second) - -Sample output: -``` -Running 6 tests -test test_real_node_connectivity ... ok -test test_real_header_sync_genesis_to_1000 ... ok -test test_real_header_sync_up_to_10k ... ok -test test_real_header_validation_with_node ... ok -test test_real_header_chain_continuity ... ok -test test_real_node_sync_resumption ... ok -test test_real_node_performance_benchmarks ... ok - -test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out -``` - -### Without Node Available - -When no node is running, the tests will: - -1. **Detect unavailability** and log a warning -2. **Skip gracefully** without failing -3. **Return immediately** with success - -Sample output: -``` -test test_real_node_connectivity ... ok -Dash Core node not available at 127.0.0.1:9999: Connection refused -Skipping integration test - ensure Dash Core is running on mainnet -``` - -## Performance Expectations - -With a properly synced Dash Core node, you can expect: - -### Header Sync Performance -- **Connection time**: < 5 seconds -- **Handshake time**: < 2 seconds -- **Sync rate**: 50-200+ headers/second (depends on node and network) -- **10k headers**: 30-120 seconds (full sync from genesis) - -### Memory Usage -- **10k headers**: ~2-5 MB RAM -- **Storage efficiency**: Headers stored in compressed format -- **Retrieval speed**: < 100ms for 1000 header ranges - -### Test Timeouts -- **Basic connectivity**: 15 seconds -- **Header sync (1k)**: 2 minutes -- **Header sync (10k)**: 5 minutes -- **Chain validation**: 3 minutes - -## Troubleshooting - -### Connection Issues - -**Error**: "Connection refused" -- Check if Dash Core is running: `ps aux | grep dash` -- Verify port 9999 is open: `netstat -an | grep 9999` -- Check firewall settings - -**Error**: "Connection timeout" -- Node may be starting up - wait a few minutes -- Check if node is still syncing initial blockchain -- Verify network connectivity - -### Sync Issues - -**Error**: "Sync timeout" -- Node may be under heavy load -- Check node sync status: `dash-cli getblockchaininfo` -- Increase timeout values in test configuration - -**Error**: "Header validation failed" -- Node may have corrupted data -- Try restarting Dash Core -- Check node logs for errors - -### Performance Issues - -**Slow sync rates** (< 10 headers/second): -- Node may be under load or syncing -- Check system resources (CPU, memory, disk I/O) -- Consider using SSD storage for the node - -## Test Coverage Summary - -The integration tests provide comprehensive coverage of: - -✅ **Network Layer**: Real TCP connections and Dash protocol handshakes -✅ **Header Sync**: Actual blockchain header downloading and validation -✅ **Storage Layer**: Real data storage and retrieval with large datasets -✅ **Performance**: Real-world sync rates and memory efficiency -✅ **Validation**: Full blockchain header validation with real data -✅ **Error Handling**: Network timeouts and connection recovery -✅ **Chain Continuity**: Real blockchain linkage and consistency checks - -These tests prove the SPV client works correctly with the actual Dash network and can handle real-world data loads and network conditions. diff --git a/dash/contrib/act/event.json b/dash/contrib/act/event.json deleted file mode 100644 index 43c2f0ebf..000000000 --- a/dash/contrib/act/event.json +++ /dev/null @@ -1,3 +0,0 @@ -{ - "act": true -} diff --git a/docs/implementation-notes/BLOOM_FILTER_SPEC.md b/docs/implementation-notes/BLOOM_FILTER_SPEC.md deleted file mode 100644 index 1021392a7..000000000 --- a/docs/implementation-notes/BLOOM_FILTER_SPEC.md +++ /dev/null @@ -1,726 +0,0 @@ -# Bloom Filter Implementation Specification for rust-dashcore - -## Executive Summary - -This specification defines the implementation of full BIP37 bloom filter support in rust-dashcore and dash-spv. While the codebase currently includes bloom filter message types, there is no actual bloom filter implementation. This spec outlines a complete implementation that will enable SPV clients to use bloom filters for transaction filtering, providing an alternative to BIP157/158 compact filters. - -## Background - -### Current State -- **Message Types**: BIP37 bloom filter messages (filterload, filteradd, filterclear) are defined in `dash/src/network/message_bloom.rs` -- **Configuration**: `MempoolStrategy::BloomFilter` exists but is not implemented -- **Alternative**: The SPV client currently uses BIP157/158 compact filters exclusively -- **Gap**: No actual bloom filter data structure or filtering logic exists - -### Motivation -1. **Compatibility**: Many Dash nodes support BIP37 bloom filters -2. **Real-time Filtering**: Unlike compact filters, bloom filters allow dynamic updates -3. **Resource Efficiency**: Lower bandwidth for wallets monitoring few addresses -4. **User Choice**: Provide flexibility between privacy (BIP158) and efficiency (BIP37) - -## Architecture Overview - -### Core Components - -``` -┌─────────────────────────────────────────────────────────────┐ -│ dash crate │ -├─────────────────────────────────────────────────────────────┤ -│ bloom/ │ -│ ├── filter.rs - Core BloomFilter implementation │ -│ ├── hash.rs - Murmur3 hash implementation │ -│ ├── error.rs - Bloom filter specific errors │ -│ └── mod.rs - Module exports │ -├─────────────────────────────────────────────────────────────┤ -│ network/ │ -│ └── message_bloom.rs - [EXISTING] BIP37 messages │ -└─────────────────────────────────────────────────────────────┘ - -┌─────────────────────────────────────────────────────────────┐ -│ dash-spv crate │ -├─────────────────────────────────────────────────────────────┤ -│ bloom/ │ -│ ├── manager.rs - Bloom filter lifecycle manager │ -│ ├── builder.rs - Filter construction utilities │ -│ └── mod.rs - Module exports │ -├─────────────────────────────────────────────────────────────┤ -│ mempool_filter.rs - [MODIFY] Integrate bloom filtering│ -│ network/ │ -│ └── peer.rs - [MODIFY] Handle bloom messages │ -└─────────────────────────────────────────────────────────────┘ -``` - -## Detailed Implementation - -### 1. Core Bloom Filter (`dash/src/bloom/filter.rs`) - -```rust -use crate::consensus::encode::{Decodable, Encodable}; -use crate::bloom::hash::murmur3; - -/// A BIP37 bloom filter -#[derive(Clone, Debug, PartialEq)] -pub struct BloomFilter { - /// Filter bit field - data: Vec, - /// Number of hash functions to use - n_hash_funcs: u32, - /// Seed value for hash functions - n_tweak: u32, - /// Bloom filter update flags - flags: BloomFlags, -} - -impl BloomFilter { - /// Create a new bloom filter - /// - /// # Parameters - /// - `elements`: Expected number of elements - /// - `fp_rate`: Desired false positive rate (0.0 - 1.0) - /// - `tweak`: Random seed for hash functions - /// - `flags`: Filter update behavior - pub fn new(elements: usize, fp_rate: f64, tweak: u32, flags: BloomFlags) -> Result { - // Validate parameters - if fp_rate <= 0.0 || fp_rate >= 1.0 { - return Err(BloomError::InvalidFalsePositiveRate); - } - - // Calculate optimal filter size (BIP37 formula) - let filter_size = (-1.0 * elements as f64 * fp_rate.ln() / (2.0_f64.ln().powi(2))).ceil() as usize; - let filter_size = filter_size.max(1).min(MAX_BLOOM_FILTER_SIZE); - - // Calculate optimal number of hash functions - let n_hash_funcs = ((filter_size * 8) as f64 / elements as f64 * 2.0_f64.ln()).round() as u32; - let n_hash_funcs = n_hash_funcs.max(1).min(MAX_HASH_FUNCS); - - Ok(BloomFilter { - data: vec![0u8; (filter_size + 7) / 8], - n_hash_funcs, - n_tweak: tweak, - flags, - }) - } - - /// Insert data into the filter - pub fn insert(&mut self, data: &[u8]) { - for i in 0..self.n_hash_funcs { - let hash = self.hash(i, data); - let index = (hash as usize) % (self.data.len() * 8); - self.data[index / 8] |= 1 << (index & 7); - } - } - - /// Check if data might be in the filter - pub fn contains(&self, data: &[u8]) -> bool { - if self.is_full() { - return true; - } - - for i in 0..self.n_hash_funcs { - let hash = self.hash(i, data); - let index = (hash as usize) % (self.data.len() * 8); - if self.data[index / 8] & (1 << (index & 7)) == 0 { - return false; - } - } - true - } - - /// Calculate hash for given data and function index - fn hash(&self, n_hash_num: u32, data: &[u8]) -> u32 { - murmur3(data, n_hash_num.wrapping_mul(0xFBA4C795).wrapping_add(self.n_tweak)) - } - - /// Check if filter matches everything (all bits set) - pub fn is_full(&self) -> bool { - self.data.iter().all(|&byte| byte == 0xFF) - } - - /// Clear the filter - pub fn clear(&mut self) { - self.data.fill(0); - } - - /// Update filter based on flags when transaction is matched - pub fn update_from_tx(&mut self, tx: &Transaction) { - match self.flags { - BloomFlags::None => {}, - BloomFlags::All => { - // Add all outputs - for (index, output) in tx.output.iter().enumerate() { - let outpoint = OutPoint::new(tx.compute_txid(), index as u32); - self.insert(&consensus::encode::serialize(&outpoint)); - } - }, - BloomFlags::PubkeyOnly => { - // Add only outputs that are pay-to-pubkey or pay-to-multisig - for (index, output) in tx.output.iter().enumerate() { - if output.script_pubkey.is_p2pk() || output.script_pubkey.is_multisig() { - let outpoint = OutPoint::new(tx.compute_txid(), index as u32); - self.insert(&consensus::encode::serialize(&outpoint)); - } - } - }, - } - } -} - -/// Constants from BIP37 -const MAX_BLOOM_FILTER_SIZE: usize = 36_000; // 36KB -const MAX_HASH_FUNCS: u32 = 50; -``` - -### 2. Murmur3 Hash Implementation (`dash/src/bloom/hash.rs`) - -```rust -/// MurmurHash3 as specified in BIP37 -pub fn murmur3(data: &[u8], seed: u32) -> u32 { - const C1: u32 = 0xcc9e2d51; - const C2: u32 = 0x1b873593; - const R1: u32 = 15; - const R2: u32 = 13; - const M: u32 = 5; - const N: u32 = 0xe6546b64; - - let mut hash = seed; - let mut chunks = data.chunks_exact(4); - - // Process 4-byte chunks - for chunk in &mut chunks { - let mut k = u32::from_le_bytes([chunk[0], chunk[1], chunk[2], chunk[3]]); - k = k.wrapping_mul(C1); - k = k.rotate_left(R1); - k = k.wrapping_mul(C2); - - hash ^= k; - hash = hash.rotate_left(R2); - hash = hash.wrapping_mul(M).wrapping_add(N); - } - - // Process remaining bytes - let remainder = chunks.remainder(); - if !remainder.is_empty() { - let mut k = 0u32; - for (i, &byte) in remainder.iter().enumerate() { - k |= (byte as u32) << (i * 8); - } - k = k.wrapping_mul(C1); - k = k.rotate_left(R1); - k = k.wrapping_mul(C2); - hash ^= k; - } - - // Finalization - hash ^= data.len() as u32; - hash ^= hash >> 16; - hash = hash.wrapping_mul(0x85ebca6b); - hash ^= hash >> 13; - hash = hash.wrapping_mul(0xc2b2ae35); - hash ^= hash >> 16; - - hash -} -``` - -### 3. SPV Bloom Filter Manager (`dash-spv/src/bloom/manager.rs`) - -```rust -use dash::bloom::{BloomFilter, BloomFlags}; -use dash::network::message_bloom::{FilterLoad, FilterAdd}; -use crate::wallet::Wallet; - -/// Manages bloom filter lifecycle for SPV client -pub struct BloomFilterManager { - /// Current bloom filter - filter: Option, - /// False positive rate - fp_rate: f64, - /// Filter update flags - flags: BloomFlags, - /// Elements added since last filter load - elements_added: usize, - /// Maximum elements before filter reload - max_elements: usize, -} - -impl BloomFilterManager { - pub fn new(fp_rate: f64, flags: BloomFlags) -> Self { - Self { - filter: None, - fp_rate, - flags, - elements_added: 0, - max_elements: 1000, // Reload filter after 1000 additions - } - } - - /// Build initial bloom filter from wallet - pub fn build_from_wallet(&mut self, wallet: &Wallet) -> Result { - let addresses = wallet.get_all_addresses(); - let utxos = wallet.get_unspent_outputs(); - - // Calculate total elements - let total_elements = addresses.len() + utxos.len() + 100; // Extra capacity - - // Generate random tweak - let tweak = rand::thread_rng().gen::(); - - // Create filter - let mut filter = BloomFilter::new(total_elements, self.fp_rate, tweak, self.flags)?; - - // Add addresses - for address in &addresses { - filter.insert(&address.to_script_pubkey().as_bytes()); - } - - // Add UTXOs - for utxo in &utxos { - filter.insert(&consensus::encode::serialize(&utxo.outpoint)); - } - - // Create FilterLoad message - let filter_load = FilterLoad { - filter: filter.clone(), - }; - - self.filter = Some(filter); - self.elements_added = 0; - - Ok(filter_load) - } - - /// Add element to filter - pub fn add_element(&mut self, data: &[u8]) -> Option { - if let Some(ref mut filter) = self.filter { - filter.insert(data); - self.elements_added += 1; - - // Return FilterAdd message - Some(FilterAdd { - data: data.to_vec(), - }) - } else { - None - } - } - - /// Check if filter needs reload - pub fn needs_reload(&self) -> bool { - self.elements_added >= self.max_elements || - self.filter.as_ref().map_or(false, |f| f.is_full()) - } - - /// Test if transaction matches filter - pub fn matches_transaction(&self, tx: &Transaction) -> bool { - if let Some(ref filter) = self.filter { - // Check each output - for output in &tx.output { - if filter.contains(&output.script_pubkey.as_bytes()) { - return true; - } - } - - // Check each input's previous output - for input in &tx.input { - if filter.contains(&consensus::encode::serialize(&input.previous_output)) { - return true; - } - } - - false - } else { - // No filter means accept everything - true - } - } - - /// Update filter after matching transaction - pub fn update_from_transaction(&mut self, tx: &Transaction) { - if let Some(ref mut filter) = self.filter { - filter.update_from_tx(tx); - } - } -} -``` - -### 4. Integration with Mempool Filter (`dash-spv/src/mempool_filter.rs` modifications) - -```rust -// Add to existing MempoolFilter implementation -impl MempoolFilter { - pub fn should_fetch_transaction( - &self, - txid: &Txid, - bloom_manager: Option<&BloomFilterManager> - ) -> bool { - match self.strategy { - MempoolStrategy::FetchAll => true, - MempoolStrategy::BloomFilter => { - // Use bloom filter if available - bloom_manager.map_or(false, |manager| { - // We can't check txid directly, need the full transaction - // Return true to fetch, then filter after receiving - true - }) - }, - MempoolStrategy::Selective => { - self.is_recently_sent(txid) || self.watching_addresses_involved(txid) - }, - } - } - - pub fn process_received_transaction( - &mut self, - tx: &Transaction, - bloom_manager: Option<&mut BloomFilterManager> - ) -> bool { - match self.strategy { - MempoolStrategy::BloomFilter => { - if let Some(manager) = bloom_manager { - if manager.matches_transaction(tx) { - manager.update_from_transaction(tx); - true - } else { - false - } - } else { - false - } - }, - _ => { - // Existing logic for other strategies - true - } - } - } -} -``` - -### 5. Network Integration (`dash-spv/src/network/manager.rs` modifications) - -```rust -// Add to Peer struct -pub struct Peer { - // ... existing fields ... - /// Bloom filter manager for this peer - bloom_manager: Option, -} - -// Add bloom filter message handling -impl Peer { - /// Initialize bloom filter for this peer - pub async fn setup_bloom_filter(&mut self, wallet: &Wallet) -> Result<(), Error> { - if let MempoolStrategy::BloomFilter = self.config.mempool_strategy { - let mut manager = BloomFilterManager::new(0.001, BloomFlags::All); - let filter_load = manager.build_from_wallet(wallet)?; - - // Send filterload message - self.send_message(NetworkMessage::FilterLoad(filter_load)).await?; - - self.bloom_manager = Some(manager); - } - Ok(()) - } - - /// Update bloom filter with new element - pub async fn add_to_bloom_filter(&mut self, data: &[u8]) -> Result<(), Error> { - if let Some(ref mut manager) = self.bloom_manager { - if let Some(filter_add) = manager.add_element(data) { - self.send_message(NetworkMessage::FilterAdd(filter_add)).await?; - } - - // Check if filter needs reload - if manager.needs_reload() { - self.reload_bloom_filter().await?; - } - } - Ok(()) - } - - /// Reload bloom filter - async fn reload_bloom_filter(&mut self) -> Result<(), Error> { - if let Some(ref mut manager) = self.bloom_manager { - // Clear current filter - self.send_message(NetworkMessage::FilterClear).await?; - - // Build and send new filter - let filter_load = manager.build_from_wallet(&self.wallet)?; - self.send_message(NetworkMessage::FilterLoad(filter_load)).await?; - } - Ok(()) - } -} -``` - -## Testing Strategy - -### 1. Unit Tests - -```rust -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn test_bloom_filter_basic() { - let mut filter = BloomFilter::new(10, 0.001, 0, BloomFlags::None).unwrap(); - - // Insert and check - let data = b"hello world"; - assert!(!filter.contains(data)); - filter.insert(data); - assert!(filter.contains(data)); - - // False positive rate - let mut false_positives = 0; - for i in 0..10000 { - let test_data = format!("test{}", i); - if filter.contains(test_data.as_bytes()) { - false_positives += 1; - } - } - assert!(false_positives < 20); // Should be ~0.1% - } - - #[test] - fn test_murmur3_vectors() { - // Test vectors from BIP37 - assert_eq!(murmur3(b"", 0), 0); - assert_eq!(murmur3(b"", 0xFBA4C795), 0x6a396f08); - assert_eq!(murmur3(b"\x00", 0x00000000), 0x514e28b7); - assert_eq!(murmur3(b"\x00\x00\x00\x00", 0x00000000), 0x2362f9de); - } - - #[test] - fn test_filter_update_flags() { - let tx = create_test_transaction(); - - // Test None flag - let mut filter = BloomFilter::new(10, 0.01, 0, BloomFlags::None).unwrap(); - let initial = filter.clone(); - filter.update_from_tx(&tx); - assert_eq!(filter, initial); // No change - - // Test All flag - let mut filter = BloomFilter::new(100, 0.01, 0, BloomFlags::All).unwrap(); - filter.update_from_tx(&tx); - // Should contain all outputs - for (i, _) in tx.output.iter().enumerate() { - let outpoint = OutPoint::new(tx.compute_txid(), i as u32); - assert!(filter.contains(&consensus::encode::serialize(&outpoint))); - } - } -} -``` - -### 2. Integration Tests - -```rust -#[tokio::test] -async fn test_bloom_filter_with_peer() { - let mut peer = create_test_peer(); - let wallet = create_test_wallet(); - - // Setup bloom filter - peer.setup_bloom_filter(&wallet).await.unwrap(); - - // Verify filter contains wallet addresses - let manager = peer.bloom_manager.as_ref().unwrap(); - for addr in wallet.get_all_addresses() { - assert!(manager.filter.as_ref().unwrap() - .contains(&addr.to_script_pubkey().as_bytes())); - } - - // Test adding new address - let new_addr = wallet.get_new_address(); - peer.add_to_bloom_filter(&new_addr.to_script_pubkey().as_bytes()) - .await.unwrap(); -} - -#[tokio::test] -async fn test_bloom_filter_transaction_matching() { - let manager = BloomFilterManager::new(0.001, BloomFlags::All); - let wallet = create_test_wallet(); - - // Build filter from wallet - manager.build_from_wallet(&wallet).unwrap(); - - // Create transaction to wallet address - let tx = create_transaction_to_address(wallet.get_address()); - assert!(manager.matches_transaction(&tx)); - - // Create transaction to unknown address - let tx = create_transaction_to_address(random_address()); - assert!(!manager.matches_transaction(&tx)); -} -``` - -### 3. Performance Tests - -```rust -#[bench] -fn bench_bloom_filter_insert(b: &mut Bencher) { - let mut filter = BloomFilter::new(10000, 0.001, 0, BloomFlags::None).unwrap(); - let data: Vec> = (0..1000) - .map(|i| format!("test{}", i).into_bytes()) - .collect(); - - b.iter(|| { - for d in &data { - filter.insert(d); - } - }); -} - -#[bench] -fn bench_bloom_filter_contains(b: &mut Bencher) { - let mut filter = BloomFilter::new(10000, 0.001, 0, BloomFlags::None).unwrap(); - for i in 0..1000 { - filter.insert(&format!("test{}", i).into_bytes()); - } - - b.iter(|| { - for i in 0..1000 { - filter.contains(&format!("test{}", i).into_bytes()); - } - }); -} -``` - -## Security Considerations - -### 1. Privacy Implications -- Bloom filters reveal approximate wallet contents to peers -- False positive rate should be tuned to balance privacy vs bandwidth -- Consider warning users about privacy trade-offs - -### 2. DoS Protection -- Limit filter size to MAX_BLOOM_FILTER_SIZE (36KB) -- Limit hash functions to MAX_HASH_FUNCS (50) -- Implement rate limiting for filter updates -- Monitor for peers sending excessive filteradd messages - -### 3. Validation -- Validate all parameters before creating filters -- Check for malformed filter data in network messages -- Ensure filters don't consume excessive memory - -## Migration Plan - -### Phase 1: Core Implementation -1. Implement BloomFilter in dash crate -2. Add comprehensive unit tests -3. Ensure compatibility with existing message types - -### Phase 2: SPV Integration -1. Implement BloomFilterManager -2. Integrate with MempoolFilter -3. Update Peer to handle bloom filters -4. Add integration tests - -### Phase 3: FFI Updates -1. Expose bloom filter configuration in FFI -2. Add callbacks for filter events -3. Update Swift SDK bindings - -### Phase 4: Documentation -1. Update API documentation -2. Add usage examples -3. Document privacy implications - -## Configuration - -### SPV Client Configuration -```rust -pub struct BloomFilterConfig { - /// False positive rate (0.0001 - 0.01 recommended) - pub false_positive_rate: f64, - /// Filter update behavior - pub flags: BloomFlags, - /// Maximum elements before filter reload - pub max_elements_before_reload: usize, - /// Enable automatic filter updates - pub auto_update: bool, -} - -impl Default for BloomFilterConfig { - fn default() -> Self { - Self { - false_positive_rate: 0.001, - flags: BloomFlags::All, - max_elements_before_reload: 1000, - auto_update: true, - } - } -} -``` - -## API Examples - -### Basic Usage -```rust -// Create SPV client with bloom filter -let config = SPVClientConfig { - mempool_strategy: MempoolStrategy::BloomFilter, - bloom_config: Some(BloomFilterConfig { - false_positive_rate: 0.001, - flags: BloomFlags::All, - ..Default::default() - }), - ..Default::default() -}; - -let client = SPVClient::new(config); -client.connect().await?; - -// Filter will be automatically managed -// Transactions matching wallet addresses will be received -``` - -### Manual Filter Management -```rust -// Create bloom filter manually -let mut filter = BloomFilter::new(100, 0.001, rand::random(), BloomFlags::PubkeyOnly)?; - -// Add addresses -for addr in wallet.get_addresses() { - filter.insert(&addr.to_script_pubkey().as_bytes()); -} - -// Send to peer -peer.send_filter_load(filter).await?; - -// Add new element -peer.send_filter_add(new_address.to_script_pubkey().as_bytes()).await?; -``` - -## Performance Metrics - -### Expected Performance -- Filter creation: < 1ms for 1000 elements -- Insert operation: O(k) where k = number of hash functions -- Contains check: O(k) -- Memory usage: ~4.5KB for 0.1% false positive rate with 1000 elements - -### Bandwidth Savings -- Full blocks: ~1-2MB per block -- With bloom filters: ~10-100KB per block (depending on wallet activity) -- Vs compact filters: More efficient for active wallets, less private - -## Future Enhancements - -1. **Adaptive Filter Sizing**: Automatically adjust filter size based on false positive rate -2. **Multi-peer Filters**: Different filters for different peers to improve privacy -3. **Filter Compression**: Compress filter data for network transmission -4. **Hybrid Mode**: Use bloom filters for recent blocks, compact filters for historical data -5. **Metrics**: Track filter performance and false positive rates - -## Conclusion - -This specification provides a complete blueprint for implementing BIP37 bloom filters in rust-dashcore. The implementation prioritizes: -- Compatibility with existing Dash network nodes -- Performance for resource-constrained devices -- Flexibility in privacy/efficiency trade-offs -- Robust error handling and security - -The modular design allows gradual rollout and easy testing of each component independently. diff --git a/docs/implementation-notes/CHAINLOCK_IMPLEMENTATION.md b/docs/implementation-notes/CHAINLOCK_IMPLEMENTATION.md deleted file mode 100644 index d017dacb5..000000000 --- a/docs/implementation-notes/CHAINLOCK_IMPLEMENTATION.md +++ /dev/null @@ -1,107 +0,0 @@ -# ChainLock (DIP8) Implementation for dash-spv - -This document describes the implementation of ChainLock validation (DIP8) for the dash-spv Rust client, providing protection against 51% attacks and securing InstantSend transactions. - -## Overview - -ChainLocks use Long Living Masternode Quorums (LLMQs) to sign and lock blocks, preventing chain reorganizations past locked blocks. When a quorum of masternodes (240 out of 400) agrees on a block as the first seen at a specific height, they create a ChainLock signature that all nodes must respect. - -## Key Components - -### 1. ChainLockManager (`src/chain/chainlock_manager.rs`) -- Manages ChainLock validation and storage -- Maintains in-memory cache of chain locks by height and hash -- Integrates with storage layer for persistence -- Provides methods to check if blocks are chain-locked -- Enforces chain lock rules during validation - -### 2. ChainLockValidator (`src/validation/chainlock.rs`) -- Performs structural validation of ChainLock messages -- Validates timing constraints (not too far in future/past) -- Constructs signing messages according to DIP8 spec -- Handles quorum signature validation (when masternode list available) - -### 3. QuorumManager (`src/validation/quorum.rs`) -- Manages LLMQ quorum information for validation -- Tracks active quorums by type (ChainLock vs InstantSend) -- Validates BLS threshold signatures -- Ensures quorum age requirements are met - -### 4. ReorgManager Integration (`src/chain/reorg.rs`) -- Enhanced to respect chain locks during reorganization -- Prevents reorganizations past chain-locked blocks -- Can be configured to enable/disable chain lock enforcement - -### 5. Storage Layer -- Added chain lock storage methods to StorageManager trait -- Implemented in DiskStorageManager -- Persistent storage of chain locks by height - -### 6. ChainState Updates (`src/types.rs`) -- Added chain lock tracking to ChainState -- Methods to update and query chain lock status -- Track last chain-locked height and hash - -## Security Features - -1. **51% Attack Prevention**: Once a block is chain-locked, it cannot be reorganized even with majority hashpower -2. **InstantSend Security**: Chain locks provide finality for InstantSend transactions -3. **Quorum Validation**: Requires 60% threshold (240/400) signatures from masternode quorum -4. **Timing Validation**: Prevents acceptance of far-future chain locks - -## Usage Example - -```rust -use dash_spv::chain::ChainLockManager; -use dash_spv::validation::QuorumManager; - -// Create managers -let chain_lock_mgr = Arc::new(ChainLockManager::new(true)); -let quorum_mgr = QuorumManager::new(); - -// Process incoming chain lock -let chain_lock = ChainLock { - block_height: 1000, - block_hash: block_hash, - signature: bls_signature, -}; - -chain_lock_mgr.process_chain_lock( - chain_lock, - &chain_state, - &mut storage -).await?; - -// Check if block is chain-locked -if chain_lock_mgr.is_block_chain_locked(&block_hash, height) { - println!("Block is chain-locked and cannot be reorganized"); -} -``` - -## Testing - -Comprehensive tests are provided in `tests/chainlock_test.rs` covering: -- Basic chain lock validation -- Storage and retrieval -- Reorg prevention -- Timing constraints -- Quorum management - -## Future Enhancements - -1. **BLS Signature Verification**: Currently stubbed out, needs full BLS library integration -2. **Masternode List Integration**: Automatic quorum extraction from masternode list -3. **Network Message Handling**: Full CLSig message processing from P2P network -4. **Performance Optimization**: Batch validation of multiple chain locks - -## Configuration - -Chain lock enforcement can be configured when creating the ChainLockManager: -- `ChainLockManager::new(true)` - Enforce chain locks (production) -- `ChainLockManager::new(false)` - Disable enforcement (testing only) - -## References - -- [DIP8: ChainLocks](https://github.com/dashpay/dips/blob/master/dip-0008.md) -- [Dash Core Implementation](https://github.com/dashpay/dash/pull/2643) -- [Long Living Masternode Quorums](https://www.dash.org/blog/long-living-masternode-quorums/) diff --git a/docs/implementation-notes/CHECKPOINT_IMPLEMENTATION.md b/docs/implementation-notes/CHECKPOINT_IMPLEMENTATION.md deleted file mode 100644 index b13bd553e..000000000 --- a/docs/implementation-notes/CHECKPOINT_IMPLEMENTATION.md +++ /dev/null @@ -1,72 +0,0 @@ -# Checkpoint System Implementation - -## Overview -Successfully implemented a comprehensive checkpoint system for dash-spv based on the iOS implementation. This adds critical security and optimization features for blockchain synchronization. - -## Implementation Details - -### 1. Core Data Structures -- **Checkpoint**: Represents a known valid block at a specific height - - Fields: height, block_hash, timestamp, target, merkle_root, chain_work, masternode_list_name - - Protocol version extraction from masternode list names (e.g., "ML1088640__70218") - -- **CheckpointManager**: Manages checkpoints for a specific network - - Indexed by height for O(1) lookup - - Sorted heights for efficient range queries - - Methods for validation, finding checkpoints before a height, etc. - -### 2. Checkpoint Data -Ported checkpoint data from iOS: -- **Mainnet**: 5 checkpoints from genesis to height 1,720,000 -- **Testnet**: 2 checkpoints including genesis and height 760,000 -- Each checkpoint includes full block data for validation - -### 3. Integration with Header Sync -Enhanced `HeaderSyncManagerWithReorg` with checkpoint support: -- **Validation**: Blocks at checkpoint heights must match the expected hash -- **Fork Protection**: Prevents reorganizations past checkpoints -- **Sync Optimization**: Can start sync from last checkpoint -- **Skip Ahead**: Can jump to future checkpoints during initial sync - -### 4. Security Features -- **Deep Reorg Protection**: Enforces checkpoints to prevent deep chain reorganizations -- **Fork Rejection**: Rejects forks that would reorganize past a checkpoint -- **Configurable Enforcement**: `enforce_checkpoints` flag in ReorgConfig - -### 5. Test Coverage -- Unit tests for checkpoint validation and queries -- Integration tests for checkpoint enforcement during sync -- Protocol version extraction tests - -## Usage Example - -```rust -// Create checkpoint manager for mainnet -let checkpoints = mainnet_checkpoints(); -let manager = CheckpointManager::new(checkpoints); - -// Validate a block at a checkpoint height -let valid = manager.validate_block(height, &block_hash); - -// Find checkpoint before a height -let checkpoint = manager.last_checkpoint_before_height(current_height); - -// Use in header sync with reorg protection -let reorg_config = ReorgConfig { - enforce_checkpoints: true, - ..Default::default() -}; -let sync_manager = HeaderSyncManagerWithReorg::new(&config, reorg_config); -``` - -## Benefits -1. **Security**: Prevents acceptance of alternate chains that don't match checkpoints -2. **Performance**: Enables faster initial sync by starting from recent checkpoints -3. **Recovery**: Provides known-good points for chain recovery -4. **Masternode Support**: Includes masternode list identifiers for DIP3 sync - -## Future Enhancements -- Add more checkpoints for recent blocks -- Implement checkpoint-based fast sync -- Add checkpoint consensus rules for different protocol versions -- Support for downloading checkpoint data from trusted sources diff --git a/docs/implementation-notes/IMPLEMENTATION_STATUS.md b/docs/implementation-notes/IMPLEMENTATION_STATUS.md deleted file mode 100644 index 6eadac036..000000000 --- a/docs/implementation-notes/IMPLEMENTATION_STATUS.md +++ /dev/null @@ -1,141 +0,0 @@ -# dash-spv Implementation Status Report - -## Current Status Overview - -### ✅ Completed Features -1. **Reorg Handling System** (CRITICAL ✓) - - Fork detection with `ForkDetector` - - Chain reorganization with `ReorgManager` - - Chain work calculation - - Multiple chain tip tracking - - Comprehensive test coverage (8/8 tests passing) - -2. **Checkpoint System** (HIGH ✓) - - Checkpoint data structures - - Mainnet/testnet checkpoint data - - Checkpoint validation during sync - - Fork protection past checkpoints - - Unit tests passing (3/3) - -### ⚠️ Partially Integrated Features -1. **HeaderSyncManagerWithReorg** - - ✅ Implemented with checkpoint support - - ❌ Not integrated into main sync flow - - ❌ Still using basic HeaderSyncManager without reorg protection - -### ❌ Missing Critical Features (from iOS) -1. **Persistent Sync State** (IN PROGRESS) - - Need to save/restore sync progress - - Chain state persistence - - Masternode list persistence - -2. **Chain Lock Validation (DIP8)** (PENDING) - - Instant finality protection - - 51% attack prevention - - Required for production use - -3. **Peer Reputation System** (PENDING) - - Misbehavior tracking - - Peer scoring - - Ban management - -4. **UTXO Rollback Mechanism** (PENDING) - - Transaction status updates during reorg - - Wallet state recovery - -5. **Terminal Blocks Support** (PENDING) - - Masternode list synchronization - - Deterministic masternode lists - -6. **Enhanced Testing** (PENDING) - - InstantSend validation tests - - ChainLock validation tests - - Network failure scenarios - - Malicious peer tests - -## Integration Gaps - -### 1. Main Sync Flow Not Using Reorg Manager -```rust -// Current: Basic HeaderSyncManager without reorg protection -pub struct SyncManager { - header_sync: HeaderSyncManager, // ❌ No reorg support - ... -} - -// Should be: HeaderSyncManagerWithReorg -pub struct SyncManager { - header_sync: HeaderSyncManagerWithReorg, // ✅ With reorg + checkpoints - ... -} -``` - -### 2. Storage Layer Missing Persistence -- Headers stored but not chain state -- No recovery after restart -- Masternode lists not persisted - -### 3. Network Layer Missing Features -- No peer reputation tracking -- No misbehavior detection -- No automatic peer banning - -## Test Status - -### Unit Tests: ✅ 49/49 passing -- Chain work calculation -- Fork detection -- Reorg logic -- Checkpoint validation - -### Integration Tests: ⚠️ Partial -- Reorg tests: ✅ 8/8 passing -- Checkpoint integration: ❌ 2 compilation errors -- Real node tests: ✅ Working but limited - -### Missing Test Coverage -- Chain lock validation -- InstantSend validation -- Network failure recovery -- Malicious peer scenarios -- Persistent state recovery - -## Production Readiness: ❌ NOT READY - -### Critical Missing for Production: -1. **Chain Lock Support** - Without this, vulnerable to 51% attacks -2. **Persistent State** - Loses all progress on restart -3. **Reorg Integration** - Reorg protection not active in main sync -4. **Peer Management** - No protection against malicious peers -5. **UTXO Rollback** - Wallet can show incorrect balances after reorg - -### Security Vulnerabilities: -1. No chain lock validation = 51% attack vulnerable -2. No peer reputation = DoS vulnerable -3. Basic HeaderSyncManager = reorg attack vulnerable (even though we implemented protection) - -## Recommended Next Steps - -### 1. Immediate Integration (HIGH PRIORITY) -- Replace HeaderSyncManager with HeaderSyncManagerWithReorg in SyncManager -- Test the integrated reorg + checkpoint system -- Ensure all existing tests still pass - -### 2. Critical Security Features -- Implement chain lock validation (DIP8) -- Add persistent state storage -- Implement peer reputation system - -### 3. Production Features -- UTXO rollback mechanism -- Terminal blocks support -- Enhanced error recovery - -### 4. Comprehensive Testing -- Integration tests with malicious scenarios -- Performance benchmarks -- Long-running stability tests - -## Conclusion - -While significant progress has been made with reorg handling and checkpoints, **dash-spv is NOT production-ready**. The implemented features are not fully integrated, and critical security features like chain locks are missing. The library remains vulnerable to several attack vectors that the iOS implementation protects against. diff --git a/docs/implementation-notes/MEMPOOL_IMPLEMENTATION_SUMMARY.md b/docs/implementation-notes/MEMPOOL_IMPLEMENTATION_SUMMARY.md deleted file mode 100644 index 8ab840ae3..000000000 --- a/docs/implementation-notes/MEMPOOL_IMPLEMENTATION_SUMMARY.md +++ /dev/null @@ -1,149 +0,0 @@ -# Mempool Transaction Support Implementation Summary - -## Overview - -This document summarizes the implementation of unconfirmed transaction (mempool) support for the dash-spv Rust SPV client, including FFI bindings and Swift SDK integration. - -## Implementation Phases Completed - -### Phase 1: Core Infrastructure (Rust) -✅ **Configuration** -- Added `MempoolStrategy` enum (FetchAll, BloomFilter, Selective) -- Added mempool configuration fields to `ClientConfig` -- Default strategy: Selective (privacy-preserving) - -✅ **Types** -- Created `UnconfirmedTransaction` struct with metadata -- Created `MempoolState` for tracking mempool transactions -- Added `MempoolRemovalReason` enum -- Extended `SpvEvent` with mempool variants - -✅ **Storage** -- Added mempool methods to `StorageManager` trait -- Implemented in `DiskStorageManager` -- Support for optional persistence - -### Phase 2: Transaction Processing (Rust) -✅ **Filtering** -- Created `MempoolFilter` module for transaction filtering -- Implements three strategies with different privacy/efficiency tradeoffs -- Selective strategy tracks recent sends - -✅ **Message Handling** -- Updated `MessageHandler` to process `Inv` and `Tx` messages -- Integrated mempool filter for relevance checking -- Automatic transaction fetching based on strategy - -✅ **Wallet Integration** -- Added mempool-aware balance calculation -- New methods: `has_utxo`, `calculate_net_amount`, `is_transaction_relevant` -- Extended `Balance` struct with mempool fields - -### Phase 3: FFI Integration (C/Rust) -✅ **FFI Types** -- Added `FFIMempoolStrategy`, `FFIMempoolRemovalReason` -- Extended `FFIBalance` with mempool fields -- Created `FFIUnconfirmedTransaction` for C compatibility - -✅ **Callbacks** -- Added mempool-specific callbacks for transaction lifecycle -- Integrated into existing event callback system -- Proper memory management for C strings - -✅ **Client Methods** -- `dash_spv_ffi_client_enable_mempool_tracking` -- `dash_spv_ffi_client_get_balance_with_mempool` -- `dash_spv_ffi_client_get_mempool_transaction_count` -- `dash_spv_ffi_client_record_send` - -### Phase 4: iOS Integration (Swift) -✅ **Swift Types** -- Created `MempoolStrategy` enum matching Rust -- Created `MempoolRemovalReason` enum -- Extended `Balance` model with mempool properties - -✅ **SPVClient Extensions** -- `enableMempoolTracking(strategy:)` -- `getBalanceWithMempool()` -- `getMempoolTransactionCount()` -- `recordSend(txid:)` - -✅ **Event Handling** -- Added mempool events to `SPVEvent` enum -- Implemented C callbacks for mempool events -- Proper event routing through Combine publishers - -✅ **Example App Updates** -- Updated `WalletService` to handle mempool events -- Balance calculations now include mempool -- Transaction lifecycle tracking (mempool → confirmed) - -## Key Design Decisions - -1. **Privacy-First Default**: Selective strategy minimizes information leakage -2. **Backward Compatible**: Feature is opt-in, doesn't break existing code -3. **Event-Driven**: Real-time updates via callbacks -4. **Efficient Filtering**: Limits on transaction count and timeouts -5. **Flexible Persistence**: Optional mempool state persistence - -## API Usage Examples - -### Rust -```rust -// Enable mempool tracking -let config = ClientConfig::mainnet() - .with_mempool_tracking(MempoolStrategy::Selective) - .with_max_mempool_transactions(1000); -``` - -### Swift -```swift -// Enable mempool tracking -try await spvClient.enableMempoolTracking(strategy: .selective) - -// Get balance including mempool -let balance = try await spvClient.getBalanceWithMempool() -print("Total including mempool: \(balance.total)") - -// Record a sent transaction -try await spvClient.recordSend(txid: "abc123...") -``` - -## Testing Recommendations - -1. **Unit Tests**: Test each component in isolation -2. **Integration Tests**: Test full transaction flow -3. **Network Tests**: Test with real Dash nodes -4. **Memory Tests**: Verify no leaks in FFI boundaries -5. **Performance Tests**: Measure impact on sync speed - -## Future Enhancements - -1. **Bloom Filter Implementation**: Currently a placeholder -2. **Fee Estimation**: Calculate actual fees from inputs -3. **InstantSend Detection**: Identify IS transactions -4. **Replace-by-Fee**: Handle transaction replacement -5. **Mempool Persistence**: Optimize storage format - -## Migration Guide - -Existing users need no changes - mempool tracking is opt-in. To enable: - -1. Update configuration to enable mempool tracking -2. Replace `getBalance()` with `getBalanceWithMempool()` if needed -3. Subscribe to new mempool events for real-time updates -4. Call `recordSend()` after broadcasting transactions - -## Performance Impact - -- Minimal when disabled (default) -- Selective strategy: Low overhead, tracks only relevant transactions -- FetchAll strategy: High bandwidth usage, not recommended -- Memory usage: Limited by max_mempool_transactions - -## Security Considerations - -- Selective strategy reveals minimal information -- Bloom filters have known privacy weaknesses -- FetchAll strategy reveals interest in all transactions -- No private keys or sensitive data in mempool storage diff --git a/docs/implementation-notes/PEER_REPUTATION_SYSTEM.md b/docs/implementation-notes/PEER_REPUTATION_SYSTEM.md deleted file mode 100644 index c149a8c6d..000000000 --- a/docs/implementation-notes/PEER_REPUTATION_SYSTEM.md +++ /dev/null @@ -1,245 +0,0 @@ -# Peer Reputation System - -## Overview - -The Dash SPV client implements a comprehensive peer reputation system to protect against malicious peers and improve network reliability. This system tracks both positive and negative peer behaviors, automatically bans misbehaving peers, and implements reputation decay over time for recovery. - -## Architecture - -### Core Components - -1. **PeerReputationManager** (`src/network/reputation.rs`) - - Central component managing all peer reputations - - Thread-safe implementation using Arc - - Handles reputation updates, banning logic, and persistence - -2. **PeerReputation** - - Individual peer reputation data structure - - Tracks score, ban status, connection history, and behavior counts - -3. **Integration with PeerNetworkManager** - - Reputation checks before connecting to peers - - Automatic reputation updates based on peer behavior - - Reputation-based peer selection for connections - -## Reputation Scoring System - -### Misbehavior Scores (Positive Points = Bad) - -| Behavior | Score | Description | -|----------|-------|-------------| -| `INVALID_MESSAGE` | +10 | Invalid message format or protocol violation | -| `INVALID_HEADER` | +50 | Invalid block header | -| `INVALID_FILTER` | +25 | Invalid compact filter | -| `TIMEOUT` | +5 | Timeout or slow response | -| `UNSOLICITED_DATA` | +15 | Sending unsolicited data | -| `INVALID_TRANSACTION` | +20 | Invalid transaction | -| `INVALID_MASTERNODE_DIFF` | +30 | Invalid masternode list diff | -| `INVALID_CHAINLOCK` | +40 | Invalid ChainLock | -| `DUPLICATE_MESSAGE` | +5 | Duplicate message | -| `CONNECTION_FLOOD` | +20 | Connection flood attempt | - -### Positive Behavior Scores (Negative Points = Good) - -| Behavior | Score | Description | -|----------|-------|-------------| -| `VALID_HEADERS` | -5 | Successfully provided valid headers | -| `VALID_FILTERS` | -3 | Successfully provided valid filters | -| `VALID_BLOCK` | -10 | Successfully provided valid block | -| `FAST_RESPONSE` | -2 | Fast response time | -| `LONG_UPTIME` | -5 | Long uptime connection | - -### Thresholds and Limits - -- **Ban Threshold**: 100 points (MAX_MISBEHAVIOR_SCORE) -- **Minimum Score**: -50 points (MIN_SCORE) -- **Ban Duration**: 24 hours -- **Decay Interval**: 1 hour -- **Decay Amount**: 5 points per interval - -## Features - -### 1. Automatic Behavior Tracking - -The system automatically tracks peer behavior during normal operations: - -```rust -// Example: Headers received -match &msg { - NetworkMessage::Headers(headers) => { - if !headers.is_empty() { - reputation_manager.update_reputation( - peer_addr, - positive_scores::VALID_HEADERS, - "Provided valid headers", - ).await; - } - } - // ... other message types -} -``` - -### 2. Peer Banning - -Peers are automatically banned when their score reaches 100: - -```rust -// Automatic ban on threshold -if reputation.score >= MAX_MISBEHAVIOR_SCORE { - reputation.banned_until = Some(Instant::now() + BAN_DURATION); - reputation.ban_count += 1; -} -``` - -### 3. Reputation Decay - -Reputation scores decay over time, allowing peers to recover: - -```rust -// Applied every hour -let decay = (intervals as i32) * DECAY_AMOUNT; -self.score = (self.score - decay).max(MIN_SCORE); -``` - -### 4. Connection Management - -The system prevents connections to banned peers: - -```rust -// Check before connecting -if !self.reputation_manager.should_connect_to_peer(&addr).await { - log::warn!("Not connecting to {} due to bad reputation", addr); - return; -} -``` - -### 5. Reputation-Based Peer Selection - -When selecting peers for connections, the system prioritizes peers with better reputations: - -```rust -// Select best peers based on reputation -let best_peers = reputation_manager.select_best_peers(known_addresses, needed).await; -``` - -### 6. Persistent Storage - -Reputation data is saved to disk and persists across restarts: - -```rust -// Save path: /peer_reputation.json -reputation_manager.save_to_storage(&reputation_path).await?; -``` - -## Usage Examples - -### Manual Peer Management - -```rust -// Ban a peer manually -network_manager.ban_peer(&peer_addr, "Reason for ban").await?; - -// Unban a peer -network_manager.unban_peer(&peer_addr).await; - -// Get all peer reputations -let reputations = network_manager.get_peer_reputations().await; -for (addr, (score, banned)) in reputations { - println!("{}: score={}, banned={}", addr, score, banned); -} -``` - -### Monitoring Reputation Events - -```rust -// Get recent reputation changes -let events = reputation_manager.get_recent_events().await; -for event in events { - println!("{}: {} points - {}", event.peer, event.change, event.reason); -} -``` - -## Integration Points - -### 1. Connection Establishment -- Reputation checked before connecting -- Connection attempts recorded -- Successful connections tracked - -### 2. Message Processing -- Valid messages improve reputation -- Invalid messages penalize reputation -- Timeouts and errors tracked - -### 3. Peer Discovery -- Known peers sorted by reputation -- Banned peers excluded from selection -- DNS peers start with neutral reputation - -### 4. Maintenance Loop -- Periodic reputation data persistence -- Failed pings penalize reputation -- Long-lived connections rewarded - -## Testing - -The reputation system includes comprehensive tests: - -1. **Unit Tests** (`src/network/reputation.rs`) - - Basic scoring logic - - Ban/unban functionality - - Reputation decay - -2. **Integration Tests** (`tests/reputation_test.rs`) - - Concurrent updates - - Persistence across restarts - - Event tracking - -3. **Network Integration** (`tests/reputation_integration_test.rs`) - - Integration with PeerNetworkManager - - Real network scenarios - -## Future Enhancements - -1. **Configurable Thresholds** - - Allow users to adjust ban thresholds - - Customizable decay rates - -2. **Advanced Metrics** - - Track bandwidth usage per peer - - Monitor response times - - Success rate statistics - -3. **Reputation Sharing** - - Share reputation data between nodes - - Collaborative filtering of bad peers - -4. **Machine Learning** - - Detect patterns in misbehavior - - Predictive peer selection - -## Configuration - -Currently, the reputation system uses hardcoded values. Future versions may support configuration via: - -```toml -[reputation] -max_misbehavior_score = 100 -ban_duration_hours = 24 -decay_interval_hours = 1 -decay_amount = 5 -min_score = -50 -``` - -## Logging - -The reputation system logs important events: - -- `INFO`: Significant reputation changes, bans -- `WARN`: Connection rejections, manual bans -- `DEBUG`: All reputation updates - -Enable detailed logging with: -```bash -RUST_LOG=dash_spv::network::reputation=debug cargo run -``` diff --git a/docs/implementation-notes/REORG_INTEGRATION_STATUS.md b/docs/implementation-notes/REORG_INTEGRATION_STATUS.md deleted file mode 100644 index 0ce3b6c81..000000000 --- a/docs/implementation-notes/REORG_INTEGRATION_STATUS.md +++ /dev/null @@ -1,65 +0,0 @@ -# Reorg and Checkpoint Integration Status - -## ✅ Successfully Integrated - -### 1. HeaderSyncManagerWithReorg Fully Integrated -- Replaced basic `HeaderSyncManager` with `HeaderSyncManagerWithReorg` throughout the codebase -- Updated both `SyncManager` and `SequentialSyncManager` to use the new implementation -- All existing APIs maintained for backward compatibility - -### 2. Key Integration Points -- **SyncManager**: Now uses `HeaderSyncManagerWithReorg` with default `ReorgConfig` -- **SequentialSyncManager**: Updated to use reorg-aware header sync -- **SyncAdapter**: Updated type signatures to expose `HeaderSyncManagerWithReorg` -- **MessageHandler**: Works seamlessly with the new implementation - -### 3. New Features Active -- **Fork Detection**: Automatically detects competing chains during sync -- **Reorg Handling**: Can perform chain reorganizations when a stronger fork is found -- **Checkpoint Validation**: Blocks at checkpoint heights are validated against known hashes -- **Checkpoint-based Sync**: Can start sync from last checkpoint for faster initial sync -- **Deep Reorg Protection**: Prevents reorganizations past checkpoint heights - -### 4. Configuration -Default `ReorgConfig` settings: -```rust -ReorgConfig { - max_reorg_depth: 1000, // Maximum 1000 block reorg - respect_chain_locks: true, // Honor chain locks (when implemented) - max_forks: 10, // Track up to 10 competing forks - enforce_checkpoints: true, // Enforce checkpoint validation -} -``` - -### 5. Test Results -- ✅ All 49 library tests passing -- ✅ Reorg tests (8/8) passing -- ✅ Checkpoint unit tests (3/3) passing -- ✅ Compilation successful with full integration - -## What This Means - -### Security Improvements -1. **Protection Against Deep Reorgs**: The library now rejects attempts to reorganize the chain past checkpoints -2. **Fork Awareness**: Multiple competing chains are tracked and evaluated -3. **Best Chain Selection**: Automatically switches to the chain with most work - -### Performance Improvements -1. **Checkpoint-based Fast Sync**: Can start from recent checkpoints instead of genesis -2. **Optimized Fork Handling**: Efficient tracking of multiple chain tips - -### Compatibility -- All existing code continues to work without modification -- The integration is transparent to users of the library -- Additional methods available for advanced use cases - -## Next Steps - -While reorg handling and checkpoints are now fully integrated, several critical features remain: - -1. **Chain Lock Validation** - Still needed for InstantSend security -2. **Persistent State** - Sync progress is lost on restart -3. **Peer Reputation** - No protection against malicious peers -4. **UTXO Rollback** - Wallet state not updated during reorgs - -The library is now significantly more secure against reorganization attacks, but still requires the remaining features for production use. diff --git a/docs/implementation-notes/SEQUENTIAL_SYNC_DESIGN.md b/docs/implementation-notes/SEQUENTIAL_SYNC_DESIGN.md deleted file mode 100644 index deb39fb35..000000000 --- a/docs/implementation-notes/SEQUENTIAL_SYNC_DESIGN.md +++ /dev/null @@ -1,440 +0,0 @@ -# Sequential Sync Design Document - -## Overview - -This document outlines the design for transforming dash-spv from an interleaved sync approach to a strict sequential sync pipeline. - -## State Machine Design - -### Core State Enum - -```rust -#[derive(Debug, Clone, PartialEq)] -pub enum SyncPhase { - /// Not syncing, waiting to start - Idle, - - /// Phase 1: Downloading headers - DownloadingHeaders { - start_time: Instant, - start_height: u32, - current_height: u32, - target_height: Option, - last_progress: Instant, - headers_per_second: f64, - }, - - /// Phase 2: Downloading masternode lists - DownloadingMnList { - start_time: Instant, - start_height: u32, - current_height: u32, - target_height: u32, - last_progress: Instant, - }, - - /// Phase 3: Downloading compact filter headers - DownloadingCFHeaders { - start_time: Instant, - start_height: u32, - current_height: u32, - target_height: u32, - last_progress: Instant, - cfheaders_per_second: f64, - }, - - /// Phase 4: Downloading compact filters - DownloadingFilters { - start_time: Instant, - requested_ranges: HashMap<(u32, u32), Instant>, - completed_heights: HashSet, - total_filters: u32, - last_progress: Instant, - }, - - /// Phase 5: Downloading full blocks - DownloadingBlocks { - start_time: Instant, - pending_blocks: VecDeque<(BlockHash, u32)>, - downloading: HashMap, - completed: Vec, - last_progress: Instant, - }, - - /// Fully synchronized - FullySynced { - sync_completed_at: Instant, - total_sync_time: Duration, - }, -} -``` - -### Phase Manager - -```rust -pub struct SequentialSyncManager { - /// Current sync phase - current_phase: SyncPhase, - - /// Phase-specific managers (existing, but controlled) - header_sync: HeaderSyncManager, - filter_sync: FilterSyncManager, - masternode_sync: MasternodeSyncManager, - - /// Configuration - config: ClientConfig, - - /// Phase transition history - phase_history: Vec, - - /// Phase-specific request queue - pending_requests: VecDeque, - - /// Active request tracking - active_requests: HashMap, -} - -#[derive(Debug)] -struct PhaseTransition { - from_phase: SyncPhase, - to_phase: SyncPhase, - timestamp: Instant, - reason: String, -} -``` - -## Phase Lifecycle - -### 1. Phase Entry -Each phase has strict entry conditions: - -```rust -impl SequentialSyncManager { - fn can_enter_phase(&self, phase: &SyncPhase) -> Result { - match phase { - SyncPhase::DownloadingHeaders { .. } => Ok(true), // Always can start - - SyncPhase::DownloadingMnList { .. } => { - // Headers must be 100% complete - self.are_headers_complete() - } - - SyncPhase::DownloadingCFHeaders { .. } => { - // Headers complete AND MnList complete (or disabled) - Ok(self.are_headers_complete()? && - (self.are_masternodes_complete()? || !self.config.enable_masternodes)) - } - - SyncPhase::DownloadingFilters { .. } => { - // CFHeaders must be 100% complete - self.are_cfheaders_complete() - } - - SyncPhase::DownloadingBlocks { .. } => { - // Filters complete (or no blocks needed) - Ok(self.are_filters_complete()? || self.no_blocks_needed()) - } - - _ => Ok(false), - } - } -} -``` - -### 2. Phase Execution -Each phase follows a standard pattern: - -```rust -async fn execute_current_phase(&mut self, network: &mut dyn NetworkManager, storage: &mut dyn StorageManager) -> Result { - match &self.current_phase { - SyncPhase::DownloadingHeaders { .. } => { - self.execute_headers_phase(network, storage).await - } - SyncPhase::DownloadingMnList { .. } => { - self.execute_mnlist_phase(network, storage).await - } - // ... etc - } -} - -enum PhaseAction { - Continue, // Keep working on current phase - TransitionTo(SyncPhase), // Move to next phase - Error(SyncError), // Handle error - Complete, // Sync fully complete -} -``` - -### 3. Phase Completion -Strict completion criteria for each phase: - -```rust -impl SequentialSyncManager { - async fn is_phase_complete(&self, storage: &dyn StorageManager) -> Result { - match &self.current_phase { - SyncPhase::DownloadingHeaders { current_height, .. } => { - // Headers complete when we receive empty headers response - // AND we've verified chain continuity - let tip = storage.get_tip_height().await?; - let peer_height = self.get_peer_reported_height().await?; - Ok(tip == Some(peer_height) && self.last_headers_response_was_empty()) - } - - SyncPhase::DownloadingCFHeaders { current_height, target_height, .. } => { - // Complete when current matches target exactly - Ok(current_height >= target_height) - } - - // ... etc - } - } -} -``` - -### 4. Phase Transition -Clean handoff between phases: - -```rust -async fn transition_to_next_phase(&mut self, storage: &mut dyn StorageManager) -> Result<()> { - let next_phase = match &self.current_phase { - SyncPhase::Idle => SyncPhase::DownloadingHeaders { /* ... */ }, - - SyncPhase::DownloadingHeaders { .. } => { - if self.config.enable_masternodes { - SyncPhase::DownloadingMnList { /* ... */ } - } else if self.config.enable_filters { - SyncPhase::DownloadingCFHeaders { /* ... */ } - } else { - SyncPhase::FullySynced { /* ... */ } - } - } - - // ... etc - }; - - // Log transition - info!("📊 Phase transition: {:?} -> {:?}", self.current_phase, next_phase); - - // Record history - self.phase_history.push(PhaseTransition { - from_phase: self.current_phase.clone(), - to_phase: next_phase.clone(), - timestamp: Instant::now(), - reason: "Phase completed successfully".to_string(), - }); - - // Clean up current phase - self.cleanup_current_phase().await?; - - // Initialize next phase - self.current_phase = next_phase; - self.initialize_current_phase().await?; - - Ok(()) -} -``` - -## Request Management - -### Request Control Flow - -```rust -impl SequentialSyncManager { - /// All requests must go through this method - pub async fn request(&mut self, request_type: RequestType, network: &mut dyn NetworkManager) -> Result<()> { - // Phase validation - if !self.is_request_allowed_in_phase(&request_type) { - debug!("Rejecting {:?} request in phase {:?}", request_type, self.current_phase); - return Err(SyncError::InvalidPhase); - } - - // Rate limiting - if !self.can_send_request(&request_type) { - self.pending_requests.push_back(NetworkRequest { - request_type, - queued_at: Instant::now(), - }); - return Ok(()); - } - - // Send request - self.send_request(request_type, network).await - } - - fn is_request_allowed_in_phase(&self, request_type: &RequestType) -> bool { - match (&self.current_phase, request_type) { - (SyncPhase::DownloadingHeaders { .. }, RequestType::GetHeaders(_)) => true, - (SyncPhase::DownloadingMnList { .. }, RequestType::GetMnListDiff(_)) => true, - (SyncPhase::DownloadingCFHeaders { .. }, RequestType::GetCFHeaders(_)) => true, - (SyncPhase::DownloadingFilters { .. }, RequestType::GetCFilters(_)) => true, - (SyncPhase::DownloadingBlocks { .. }, RequestType::GetBlock(_)) => true, - _ => false, - } - } -} -``` - -### Message Filtering - -```rust -impl SequentialSyncManager { - /// Filter incoming messages based on current phase - pub async fn handle_message(&mut self, msg: NetworkMessage, network: &mut dyn NetworkManager, storage: &mut dyn StorageManager) -> Result<()> { - // Check if message is expected in current phase - if !self.is_message_expected(&msg) { - debug!("Ignoring unexpected {:?} message in phase {:?}", msg, self.current_phase); - return Ok(()); - } - - // Route to appropriate handler - match (&mut self.current_phase, msg) { - (SyncPhase::DownloadingHeaders { .. }, NetworkMessage::Headers(headers)) => { - self.handle_headers_in_phase(headers, network, storage).await - } - (SyncPhase::DownloadingCFHeaders { .. }, NetworkMessage::CFHeaders(cfheaders)) => { - self.handle_cfheaders_in_phase(cfheaders, network, storage).await - } - // ... etc - _ => Ok(()), // Ignore messages for other phases - } - } -} -``` - -## Progress Tracking - -### Per-Phase Progress - -```rust -impl SyncPhase { - pub fn progress(&self) -> PhaseProgress { - match self { - SyncPhase::DownloadingHeaders { start_height, current_height, target_height, .. } => { - PhaseProgress { - phase_name: "Headers", - items_completed: current_height - start_height, - items_total: target_height.map(|t| t - start_height), - percentage: calculate_percentage(*start_height, *current_height, *target_height), - rate: self.calculate_rate(), - eta: self.calculate_eta(), - } - } - // ... etc - } - } -} -``` - -### Overall Progress - -```rust -pub struct OverallSyncProgress { - pub current_phase: String, - pub phase_progress: PhaseProgress, - pub phases_completed: Vec, - pub phases_remaining: Vec, - pub total_elapsed: Duration, - pub estimated_total_time: Option, -} -``` - -## Error Recovery - -### Phase-Specific Recovery - -```rust -impl SequentialSyncManager { - async fn handle_phase_error(&mut self, error: SyncError, network: &mut dyn NetworkManager, storage: &mut dyn StorageManager) -> Result<()> { - match &self.current_phase { - SyncPhase::DownloadingHeaders { .. } => { - // Retry from last known good header - let last_good = storage.get_tip_height().await?.unwrap_or(0); - self.restart_headers_from(last_good).await - } - - SyncPhase::DownloadingCFHeaders { current_height, .. } => { - // Retry from current_height (already validated) - self.restart_cfheaders_from(*current_height).await - } - - // ... etc - } - } -} -``` - -## Implementation Strategy - -### Step 1: Create New Module Structure -``` -src/sync/ -├── mod.rs # Keep existing -├── sequential/ -│ ├── mod.rs # New SequentialSyncManager -│ ├── phases.rs # Phase definitions and state machine -│ ├── transitions.rs # Phase transition logic -│ ├── progress.rs # Progress tracking -│ └── recovery.rs # Error recovery -``` - -### Step 2: Refactor Existing Managers -- Keep existing sync managers but make them phase-aware -- Add phase validation to their request methods -- Remove automatic interleaving behavior - -### Step 3: Integration Points -- Modify `client/mod.rs` to use SequentialSyncManager -- Update `client/message_handler.rs` to route through sequential manager -- Add phase information to monitoring and logging - -### Step 4: Migration Path -1. Add feature flag for sequential sync -2. Run both implementations in parallel for testing -3. Gradually migrate to sequential as default -4. Remove old interleaved code - -## Testing Strategy - -### Unit Tests -- Test each phase in isolation -- Test phase transitions -- Test error recovery -- Test progress calculation - -### Integration Tests -- Full sync from genesis with phase verification -- Interruption and resume testing -- Network failure recovery -- Performance benchmarks - -### Phase Boundary Tests -```rust -#[test] -async fn test_headers_must_complete_before_cfheaders() { - // Setup - let mut sync = create_test_sync_manager(); - - // Start headers sync - sync.start_sync().await.unwrap(); - assert_eq!(sync.current_phase(), SyncPhase::DownloadingHeaders { .. }); - - // Try to request cfheaders - should fail - let result = sync.request(RequestType::GetCFHeaders(..), network).await; - assert!(matches!(result, Err(SyncError::InvalidPhase))); - - // Complete headers - complete_headers_phase(&mut sync).await; - - // Now cfheaders should be allowed - let result = sync.request(RequestType::GetCFHeaders(..), network).await; - assert!(result.is_ok()); -} -``` - -## Benefits - -1. **Clarity**: Single active phase, clear state machine -2. **Reliability**: No race conditions or dependency issues -3. **Debuggability**: Phase transitions clearly logged -4. **Performance**: Better request batching within phases -5. **Maintainability**: Easier to reason about and extend diff --git a/docs/implementation-notes/SEQUENTIAL_SYNC_SUMMARY.md b/docs/implementation-notes/SEQUENTIAL_SYNC_SUMMARY.md deleted file mode 100644 index fb76634c4..000000000 --- a/docs/implementation-notes/SEQUENTIAL_SYNC_SUMMARY.md +++ /dev/null @@ -1,180 +0,0 @@ -# Sequential Sync Implementation Summary - -## Overview - -I have successfully implemented a sequential synchronization manager for dash-spv that enforces strict phase ordering, preventing the race conditions and complexity issues caused by interleaved downloads. - -## What Was Implemented - -### 1. Core Architecture (`src/sync/sequential/`) - -#### Phase State Machine (`phases.rs`) -- **SyncPhase enum**: Defines all synchronization phases with detailed state tracking - - Idle - - DownloadingHeaders - - DownloadingMnList - - DownloadingCFHeaders - - DownloadingFilters - - DownloadingBlocks - - FullySynced - -- Each phase tracks: - - Start time and last progress time - - Current progress metrics (items completed, rates) - - Phase-specific state (e.g., received_empty_response for headers) - -#### Sequential Sync Manager (`mod.rs`) -- **SequentialSyncManager**: Main coordinator that ensures phases complete sequentially -- Wraps existing sync managers (HeaderSyncManager, FilterSyncManager, MasternodeSyncManager) -- Key features: - - Phase-aware message routing - - Automatic phase transitions on completion - - Timeout detection and recovery - - Progress tracking across all phases - -#### Phase Transitions (`transitions.rs`) -- **TransitionManager**: Validates and manages phase transitions -- Enforces strict dependencies: - - Headers must complete before MnList/CFHeaders - - MnList must complete before CFHeaders (if enabled) - - CFHeaders must complete before Filters - - Filters must complete before Blocks -- Creates detailed transition history for debugging - -#### Request Control (`request_control.rs`) -- **RequestController**: Phase-aware request management -- Features: - - Validates requests match current phase - - Rate limiting per phase - - Request queuing and batching - - Concurrent request limits -- Prevents out-of-phase requests from being sent - -#### Progress Tracking (`progress.rs`) -- **ProgressTracker**: Comprehensive progress monitoring -- Tracks: - - Per-phase progress (items, percentage, rate, ETA) - - Overall sync progress across all phases - - Phase completion history - - Time estimates - -#### Error Recovery (`recovery.rs`) -- **RecoveryManager**: Smart error recovery strategies -- Recovery strategies: - - Retry with exponential backoff - - Restart phase from checkpoint - - Switch to different peer - - Wait for network connectivity -- Phase-specific recovery logic - -## Key Benefits - -### 1. **No Race Conditions** -- Each phase completes 100% before the next begins -- No interleaving of different data types -- Clear dependencies are enforced - -### 2. **Simplified State Management** -- Single active phase at any time -- Clear state machine with well-defined transitions -- Easy to reason about system state - -### 3. **Better Error Recovery** -- Phase-specific recovery strategies -- Can restart from last known good state -- Prevents cascading failures - -### 4. **Improved Debugging** -- Phase transition logging -- Detailed progress tracking -- Clear error messages with phase context - -### 5. **Performance Optimization** -- Better request batching within phases -- Reduced network overhead -- More efficient resource usage - -## Current Status - -✅ **Implemented**: -- Complete phase state machine -- Sequential sync manager with phase enforcement -- Phase transition logic with validation -- Request filtering and control -- Progress tracking and reporting -- Error recovery framework -- Integration with existing sync managers - -⚠️ **TODO**: -- Integration with DashSpvClient -- Comprehensive test suite -- Performance benchmarking -- Documentation updates - -## Usage Example - -```rust -// Create sequential sync manager -let mut seq_sync = SequentialSyncManager::new(&config, received_filter_heights); - -// Start sync process -seq_sync.start_sync(&mut network, &mut storage).await?; - -// Handle incoming messages -match message { - NetworkMessage::Headers(headers) => { - seq_sync.handle_message(message, &mut network, &mut storage).await?; - } - // ... other message types -} - -// Check for timeouts periodically -seq_sync.check_timeout(&mut network, &mut storage).await?; - -// Get progress -let progress = seq_sync.get_progress(); -println!("Current phase: {}", progress.current_phase); -``` - -## Phase Flow Example - -``` -[Idle] - ↓ -[Downloading Headers] - - Request headers from genesis/checkpoint - - Process batches of 2000 headers - - Complete when empty response received - ↓ -[Downloading MnList] (if enabled) - - Request masternode list diffs - - Process incrementally - - Complete when caught up to header tip - ↓ -[Downloading CFHeaders] (if filters enabled) - - Request filter headers in batches - - Validate against block headers - - Complete when caught up to header tip - ↓ -[Downloading Filters] - - Request filters for watched addresses - - Check for matches - - Complete when all needed filters downloaded - ↓ -[Downloading Blocks] - - Request full blocks for filter matches - - Process transactions - - Complete when all blocks downloaded - ↓ -[Fully Synced] -``` - -## Next Steps - -1. **Integration**: Wire up SequentialSyncManager in DashSpvClient -2. **Testing**: Create comprehensive test suite for phase transitions -3. **Migration**: Add feature flag to switch between interleaved and sequential -4. **Optimization**: Fine-tune batch sizes and timeouts per phase -5. **Documentation**: Update API docs and examples - -The sequential sync implementation provides a solid foundation for reliable, predictable synchronization in dash-spv. diff --git a/docs/implementation-notes/WALLET_SPV_INTEGRATION.md b/docs/implementation-notes/WALLET_SPV_INTEGRATION.md deleted file mode 100644 index 14f85cb2a..000000000 --- a/docs/implementation-notes/WALLET_SPV_INTEGRATION.md +++ /dev/null @@ -1,78 +0,0 @@ -# Wallet Address to SPV Client Integration - -## Summary - -This document describes how wallet addresses are connected to the SPV client in the Swift SDK. - -## Architecture - -### 1. SPVClient Methods - -Added two new public methods to `SPVClient`: -- `addWatchItem(type: WatchItemType, data: String)` - Adds address/script/outpoint to watch list -- `removeWatchItem(type: WatchItemType, data: String)` - Removes from watch list - -These methods: -- Check if client is connected -- Create appropriate FFI watch item based on type -- Call the FFI function with the client's internal pointer -- Clean up memory appropriately - -### 2. WalletManager Integration - -Updated `WalletManager` to use the new SPVClient methods: -- `watchAddress()` now calls `client.addWatchItem(.address, data: address)` -- `unwatchAddress()` now calls `client.removeWatchItem(.address, data: address)` -- `watchScript()` converts script data to hex and calls `client.addWatchItem(.script, data: scriptHex)` - -### 3. Persistence Integration - -Updated `PersistentWalletManager`: -- When loading persisted addresses, it re-watches them in the SPV client if connected -- This ensures addresses are tracked after app restart - -### 4. Connection Flow - -Updated `DashSDK.connect()`: -- After starting SPV client, calls `syncPersistedAddresses()` -- This triggers reload of watched addresses from storage - -## Address Watching Flow - -1. **New Address Generation**: - - Wallet generates new address - - Calls `watchAddress(address)` - - WalletManager calls `client.addWatchItem(.address, data: address)` - - SPVClient creates FFI watch item and registers with Rust SPV client - - Address is now tracked for balance/transaction updates - -2. **App Restart**: - - DashSDK.connect() is called - - SPV client starts - - PersistentWalletManager loads addresses from storage - - Each address is re-watched via `client.addWatchItem()` - - SPV client resumes tracking all addresses - -3. **Balance/Transaction Updates**: - - SPV client detects changes for watched addresses - - Events are sent through the event callback system - - WalletManager handles events and updates balances - -## Key Design Decisions - -1. **Encapsulation**: WalletManager doesn't need direct FFI access - SPVClient handles all FFI interactions -2. **Type Safety**: Using `WatchItemType` enum to ensure correct watch item creation -3. **Memory Management**: Proper cleanup of FFI watch items using defer blocks -4. **Error Handling**: Proper error propagation with meaningful error messages - -## FFI Functions Used - -## Testing - -To test the integration: - -1. Generate a new address in the wallet -2. Verify it's watched via SPV client logs -3. Send funds to the address -4. Verify balance updates are received -5. Restart app and verify addresses are re-watched diff --git a/key-wallet-ffi/include/key_wallet_ffi_test.h b/key-wallet-ffi/include/key_wallet_ffi_test.h deleted file mode 100644 index 2f7994c6d..000000000 --- a/key-wallet-ffi/include/key_wallet_ffi_test.h +++ /dev/null @@ -1,3449 +0,0 @@ -/** - * Key Wallet FFI - C Header File - * - * This header provides C-compatible function declarations for the key-wallet - * Rust library FFI bindings. - * - * AUTO-GENERATED FILE - DO NOT EDIT - * Generated using cbindgen - */ - -#ifndef KEY_WALLET_FFI_H -#define KEY_WALLET_FFI_H - -/* Generated with cbindgen:0.29.0 */ - -/* Warning: This file is auto-generated by cbindgen. Do not modify manually. */ - -#include -#include -#include -#include -#include -#include -#include -#include - -/* - FFI Account Creation Option Type - */ -typedef enum { - /* - Create default accounts (BIP44 account 0, CoinJoin account 0, and special accounts) - */ - DEFAULT = 0, - /* - Create all specified accounts plus all special purpose accounts - */ - ALL_ACCOUNTS = 1, - /* - Create only BIP44 accounts (no CoinJoin or special accounts) - */ - BIP44_ACCOUNTS_ONLY = 2, - /* - Create specific accounts with full control - */ - SPECIFIC_ACCOUNTS = 3, - /* - Create no accounts at all - */ - NO_ACCOUNTS = 4, -} FFIAccountCreationOptionType; - -/* - Account type enumeration matching all key_wallet AccountType variants - - This enum provides a complete FFI representation of all account types - supported by the key_wallet library: - - - Standard accounts: BIP44 and BIP32 variants for regular transactions - - CoinJoin: Privacy-enhanced transactions - - Identity accounts: Registration, top-up, and invitation funding - - Provider accounts: Various masternode provider key types (voting, owner, operator, platform) - */ -typedef enum { - /* - Standard BIP44 account (m/44'/coin_type'/account'/x/x) - */ - STANDARD_BIP44 = 0, - /* - Standard BIP32 account (m/account'/x/x) - */ - STANDARD_BIP32 = 1, - /* - CoinJoin account for private transactions - */ - COIN_JOIN = 2, - /* - Identity registration funding - */ - IDENTITY_REGISTRATION = 3, - /* - Identity top-up funding (requires registration_index) - */ - IDENTITY_TOP_UP = 4, - /* - Identity top-up funding not bound to a specific identity - */ - IDENTITY_TOP_UP_NOT_BOUND_TO_IDENTITY = 5, - /* - Identity invitation funding - */ - IDENTITY_INVITATION = 6, - /* - Provider voting keys (DIP-3) - Path: m/9'/5'/3'/1'/[key_index] - */ - PROVIDER_VOTING_KEYS = 7, - /* - Provider owner keys (DIP-3) - Path: m/9'/5'/3'/2'/[key_index] - */ - PROVIDER_OWNER_KEYS = 8, - /* - Provider operator keys (DIP-3) - Path: m/9'/5'/3'/3'/[key_index] - */ - PROVIDER_OPERATOR_KEYS = 9, - /* - Provider platform P2P keys (DIP-3, ED25519) - Path: m/9'/5'/3'/4'/[key_index] - */ - PROVIDER_PLATFORM_KEYS = 10, -} FFIAccountType; - -/* - Address pool type - */ -typedef enum { - /* - External (receive) addresses - */ - EXTERNAL = 0, - /* - Internal (change) addresses - */ - INTERNAL = 1, - /* - Single pool (for non-standard accounts) - */ - SINGLE = 2, -} FFIAddressPoolType; - -/* - Derivation path type for DIP9 - */ -typedef enum { - PATH_UNKNOWN = 0, - PATH_BIP32 = 1, - PATH_BIP44 = 2, - PATH_BLOCKCHAIN_IDENTITIES = 3, - PATH_PROVIDER_FUNDS = 4, - PATH_PROVIDER_VOTING_KEYS = 5, - PATH_PROVIDER_OPERATOR_KEYS = 6, - PATH_PROVIDER_OWNER_KEYS = 7, - PATH_CONTACT_BASED_FUNDS = 8, - PATH_CONTACT_BASED_FUNDS_ROOT = 9, - PATH_CONTACT_BASED_FUNDS_EXTERNAL = 10, - PATH_BLOCKCHAIN_IDENTITY_CREDIT_REGISTRATION_FUNDING = 11, - PATH_BLOCKCHAIN_IDENTITY_CREDIT_TOPUP_FUNDING = 12, - PATH_BLOCKCHAIN_IDENTITY_CREDIT_INVITATION_FUNDING = 13, - PATH_PROVIDER_PLATFORM_NODE_KEYS = 14, - PATH_COIN_JOIN = 15, - PATH_ROOT = 255, -} FFIDerivationPathType; - -/* - FFI Error code - */ -typedef enum { - SUCCESS = 0, - INVALID_INPUT = 1, - ALLOCATION_FAILED = 2, - INVALID_MNEMONIC = 3, - INVALID_DERIVATION_PATH = 4, - INVALID_NETWORK = 5, - INVALID_ADDRESS = 6, - INVALID_TRANSACTION = 7, - WALLET_ERROR = 8, - SERIALIZATION_ERROR = 9, - NOT_FOUND = 10, - INVALID_STATE = 11, - INTERNAL_ERROR = 12, -} FFIErrorCode; - -/* - Language enumeration for mnemonic generation - - This enum must be kept in sync with key_wallet::mnemonic::Language. - When adding new languages to the key_wallet crate, remember to update - this FFI enum and both From implementations below. - */ -typedef enum { - ENGLISH = 0, - CHINESE_SIMPLIFIED = 1, - CHINESE_TRADITIONAL = 2, - CZECH = 3, - FRENCH = 4, - ITALIAN = 5, - JAPANESE = 6, - KOREAN = 7, - PORTUGUESE = 8, - SPANISH = 9, -} FFILanguage; - -/* - FFI Network type (bit flags for multiple networks) - */ -typedef enum { - NO_NETWORKS = 0, - DASH = 1, - TESTNET = 2, - REGTEST = 4, - DEVNET = 8, - ALL_NETWORKS = 15, -} FFINetworks; - -/* - Provider key type - */ -typedef enum { - /* - BLS voting keys (m/9'/5'/3'/1'/[key_index]) - */ - VOTING_KEYS = 0, - /* - BLS owner keys (m/9'/5'/3'/2'/[key_index]) - */ - OWNER_KEYS = 1, - /* - BLS operator keys (m/9'/5'/3'/3'/[key_index]) - */ - OPERATOR_KEYS = 2, - /* - EdDSA platform P2P keys (m/9'/5'/3'/4'/[key_index]) - */ - PLATFORM_KEYS = 3, -} FFIProviderKeyType; - -/* - Transaction context for checking - */ -typedef enum { - /* - Transaction is in mempool (unconfirmed) - */ - MEMPOOL = 0, - /* - Transaction is in a block - */ - IN_BLOCK = 1, - /* - Transaction is in a chain-locked block - */ - IN_CHAIN_LOCKED_BLOCK = 2, -} FFITransactionContext; - -/* - Opaque account handle - */ -typedef struct FFIAccount FFIAccount; - -/* - Opaque handle to an account collection - */ -typedef struct FFIAccountCollection FFIAccountCollection; - -/* - FFI wrapper for an AddressPool from a ManagedAccount - - This is a lightweight wrapper that holds a reference to an AddressPool - from within a ManagedAccount. It allows querying addresses and pool information. - */ -typedef struct FFIAddressPool FFIAddressPool; - -/* - Opaque BLS account handle - */ -typedef struct FFIBLSAccount FFIBLSAccount; - -/* - Opaque EdDSA account handle - */ -typedef struct FFIEdDSAAccount FFIEdDSAAccount; - -/* - Extended private key structure - */ -typedef struct FFIExtendedPrivKey FFIExtendedPrivKey; - -/* - Opaque type for an extended private key - */ -typedef struct FFIExtendedPrivateKey FFIExtendedPrivateKey; - -/* - Extended public key structure - */ -typedef struct FFIExtendedPubKey FFIExtendedPubKey; - -/* - Opaque type for an extended public key - */ -typedef struct FFIExtendedPublicKey FFIExtendedPublicKey; - -/* - Opaque managed account handle that wraps ManagedAccount - */ -typedef struct FFIManagedAccount FFIManagedAccount; - -/* - Opaque handle to a managed account collection - */ -typedef struct FFIManagedAccountCollection FFIManagedAccountCollection; - -/* - FFI wrapper for ManagedWalletInfo - */ -typedef struct FFIManagedWalletInfo FFIManagedWalletInfo; - -/* - Opaque type for a private key (SecretKey) - */ -typedef struct FFIPrivateKey FFIPrivateKey; - -/* - Opaque type for a public key - */ -typedef struct FFIPublicKey FFIPublicKey; - -/* - Opaque wallet handle - */ -typedef struct FFIWallet FFIWallet; - -/* - FFI wrapper for WalletManager - - This struct holds a cloned Arc reference to the WalletManager, - allowing FFI code to interact with it directly without going through - the SPV client. - */ -typedef struct FFIWalletManager FFIWalletManager; - -/* - FFI Result type for Account operations - */ -typedef struct { - /* - The account handle if successful, NULL if error - */ - FFIAccount *account; - /* - Error code (0 = success) - */ - int32_t error_code; - /* - Error message (NULL if success, must be freed by caller if not NULL) - */ - char *error_message; -} FFIAccountResult; - -/* - FFI Error structure - */ -typedef struct { - FFIErrorCode code; - char *message; -} FFIError; - -/* - C-compatible summary of all accounts in a collection - - This struct provides Swift with structured data about all accounts - that exist in the collection, allowing programmatic access to account - indices and presence information. - */ -typedef struct { - /* - Array of BIP44 account indices - */ - unsigned int *bip44_indices; - /* - Number of BIP44 accounts - */ - size_t bip44_count; - /* - Array of BIP32 account indices - */ - unsigned int *bip32_indices; - /* - Number of BIP32 accounts - */ - size_t bip32_count; - /* - Array of CoinJoin account indices - */ - unsigned int *coinjoin_indices; - /* - Number of CoinJoin accounts - */ - size_t coinjoin_count; - /* - Array of identity top-up registration indices - */ - unsigned int *identity_topup_indices; - /* - Number of identity top-up accounts - */ - size_t identity_topup_count; - /* - Whether identity registration account exists - */ - bool has_identity_registration; - /* - Whether identity invitation account exists - */ - bool has_identity_invitation; - /* - Whether identity top-up not bound account exists - */ - bool has_identity_topup_not_bound; - /* - Whether provider voting keys account exists - */ - bool has_provider_voting_keys; - /* - Whether provider owner keys account exists - */ - bool has_provider_owner_keys; - /* - Whether provider operator keys account exists - */ - bool has_provider_operator_keys; - /* - Whether provider platform keys account exists - */ - bool has_provider_platform_keys; -} FFIAccountCollectionSummary; - -/* - Deprecated alias: historically a separate wrapper was used for transaction - checking. It now aliases the canonical FFIManagedWalletInfo. - */ -typedef FFIManagedWalletInfo FFIManagedWallet; - -/* - Address pool info - */ -typedef struct { - /* - Pool type - */ - FFIAddressPoolType pool_type; - /* - Number of generated addresses - */ - unsigned int generated_count; - /* - Number of used addresses - */ - unsigned int used_count; - /* - Current gap (unused addresses at the end) - */ - unsigned int current_gap; - /* - Gap limit setting - */ - unsigned int gap_limit; - /* - Highest used index (-1 if none used) - */ - int32_t highest_used_index; -} FFIAddressPoolInfo; - -/* - FFI-compatible version of AddressInfo - */ -typedef struct { - /* - Address as string - */ - char *address; - /* - Script pubkey bytes - */ - uint8_t *script_pubkey; - /* - Length of script pubkey - */ - size_t script_pubkey_len; - /* - Public key bytes (nullable) - */ - uint8_t *public_key; - /* - Length of public key - */ - size_t public_key_len; - /* - Derivation index - */ - uint32_t index; - /* - Derivation path as string - */ - char *path; - /* - Whether address has been used - */ - bool used; - /* - When generated (timestamp) - */ - uint64_t generated_at; - /* - When first used (0 if never) - */ - uint64_t used_at; - /* - Transaction count - */ - uint32_t tx_count; - /* - Total received - */ - uint64_t total_received; - /* - Total sent - */ - uint64_t total_sent; - /* - Current balance - */ - uint64_t balance; - /* - Custom label (nullable) - */ - char *label; -} FFIAddressInfo; - -/* - FFI Result type for ManagedAccount operations - */ -typedef struct { - /* - The managed account handle if successful, NULL if error - */ - FFIManagedAccount *account; - /* - Error code (0 = success) - */ - int32_t error_code; - /* - Error message (NULL if success, must be freed by caller if not NULL) - */ - char *error_message; -} FFIManagedAccountResult; - -/* - FFI Balance type for representing wallet balances - */ -typedef struct { - /* - Confirmed balance in duffs - */ - uint64_t confirmed; - /* - Unconfirmed balance in duffs - */ - uint64_t unconfirmed; - /* - Immature balance in duffs (e.g., mining rewards) - */ - uint64_t immature; - /* - Total balance (confirmed + unconfirmed) in duffs - */ - uint64_t total; -} FFIBalance; - -/* - C-compatible summary of all accounts in a managed collection - - This struct provides Swift with structured data about all accounts - that exist in the managed collection, allowing programmatic access to account - indices and presence information. - */ -typedef struct { - /* - Array of BIP44 account indices - */ - unsigned int *bip44_indices; - /* - Number of BIP44 accounts - */ - size_t bip44_count; - /* - Array of BIP32 account indices - */ - unsigned int *bip32_indices; - /* - Number of BIP32 accounts - */ - size_t bip32_count; - /* - Array of CoinJoin account indices - */ - unsigned int *coinjoin_indices; - /* - Number of CoinJoin accounts - */ - size_t coinjoin_count; - /* - Array of identity top-up registration indices - */ - unsigned int *identity_topup_indices; - /* - Number of identity top-up accounts - */ - size_t identity_topup_count; - /* - Whether identity registration account exists - */ - bool has_identity_registration; - /* - Whether identity invitation account exists - */ - bool has_identity_invitation; - /* - Whether identity top-up not bound account exists - */ - bool has_identity_topup_not_bound; - /* - Whether provider voting keys account exists - */ - bool has_provider_voting_keys; - /* - Whether provider owner keys account exists - */ - bool has_provider_owner_keys; - /* - Whether provider operator keys account exists - */ - bool has_provider_operator_keys; - /* - Whether provider platform keys account exists - */ - bool has_provider_platform_keys; -} FFIManagedAccountCollectionSummary; - -/* - Provider key info - */ -typedef struct { - /* - Key index - */ - unsigned int key_index; - /* - Public key bytes (48 bytes for BLS, 32 bytes for EdDSA) - */ - uint8_t *public_key; - /* - Public key length - */ - size_t public_key_len; - /* - Private key bytes (32 bytes, only if available) - */ - uint8_t *private_key; - /* - Private key length (0 if not available) - */ - size_t private_key_len; - /* - Derivation path as string - */ - char *derivation_path; -} FFIProviderKeyInfo; - -/* - Transaction output for building - */ -typedef struct { - const char *address; - uint64_t amount; -} FFITxOutput; - -/* - Transaction check result - */ -typedef struct { - /* - Whether the transaction belongs to the wallet - */ - bool is_relevant; - /* - Total amount received - */ - uint64_t total_received; - /* - Total amount sent - */ - uint64_t total_sent; - /* - Number of affected accounts - */ - uint32_t affected_accounts_count; -} FFITransactionCheckResult; - -/* - UTXO structure for FFI - */ -typedef struct { - uint8_t txid[32]; - uint32_t vout; - uint64_t amount; - char *address; - uint8_t *script_pubkey; - size_t script_len; - uint32_t height; - uint32_t confirmations; -} FFIUTXO; - -/* - FFI structure for wallet account creation options - This single struct represents all possible account creation configurations - */ -typedef struct { - /* - The type of account creation option - */ - FFIAccountCreationOptionType option_type; - /* - Array of BIP44 account indices to create - */ - const uint32_t *bip44_indices; - size_t bip44_count; - /* - Array of BIP32 account indices to create - */ - const uint32_t *bip32_indices; - size_t bip32_count; - /* - Array of CoinJoin account indices to create - */ - const uint32_t *coinjoin_indices; - size_t coinjoin_count; - /* - Array of identity top-up registration indices to create - */ - const uint32_t *topup_indices; - size_t topup_count; - /* - For SpecificAccounts: Additional special account types to create - (e.g., IdentityRegistration, ProviderKeys, etc.) - This is an array of FFIAccountType values - */ - const FFIAccountType *special_account_types; - size_t special_account_types_count; -} FFIWalletAccountCreationOptions; - -/* - FFI-compatible transaction context details - */ -typedef struct { - /* - The context type - */ - FFITransactionContext context_type; - /* - Block height (0 for mempool) - */ - unsigned int height; - /* - Block hash (32 bytes, null for mempool or if unknown) - */ - const uint8_t *block_hash; - /* - Timestamp (0 if unknown) - */ - unsigned int timestamp; -} FFITransactionContextDetails; - -#ifdef __cplusplus -extern "C" { -#endif // __cplusplus - -/* - Initialize the library - */ - bool key_wallet_ffi_initialize(void) ; - -/* - Get library version - - Returns a static string that should NOT be freed by the caller - */ - const char *key_wallet_ffi_version(void) ; - -/* - Get an account handle for a specific account type - Returns a result containing either the account handle or an error - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet instance - - The caller must ensure the wallet pointer remains valid for the duration of this call - */ - -FFIAccountResult wallet_get_account(const FFIWallet *wallet, - FFINetworks network, - unsigned int account_index, - FFIAccountType account_type) -; - -/* - Get an IdentityTopUp account handle with a specific registration index - This is used for top-up accounts that are bound to a specific identity - Returns a result containing either the account handle or an error - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet instance - - The caller must ensure the wallet pointer remains valid for the duration of this call - */ - -FFIAccountResult wallet_get_top_up_account_with_registration_index(const FFIWallet *wallet, - FFINetworks network, - unsigned int registration_index) -; - -/* - Free an account handle - - # Safety - - - `account` must be a valid pointer to an FFIAccount that was allocated by this library - - The pointer must not be used after calling this function - - This function must only be called once per allocation - */ - void account_free(FFIAccount *account) ; - -/* - Free a BLS account handle - - # Safety - - - `account` must be a valid pointer to an FFIBLSAccount - - The pointer must not be used after calling this function - - This function must only be called once per allocation - */ - void bls_account_free(FFIBLSAccount *account) ; - -/* - Free an EdDSA account handle - - # Safety - - - `account` must be a valid pointer to an FFIEdDSAAccount - - The pointer must not be used after calling this function - - This function must only be called once per allocation - */ - void eddsa_account_free(FFIEdDSAAccount *account) ; - -/* - Free an account result's error message (if any) - Note: This does NOT free the account handle itself - use account_free for that - - # Safety - - - `result` must be a valid pointer to an FFIAccountResult - - The error_message field must be either null or a valid CString allocated by this library - - The caller must ensure the result pointer remains valid for the duration of this call - */ - void account_result_free_error(FFIAccountResult *result) ; - -/* - Get the extended public key of an account as a string - - # Safety - - - `account` must be a valid pointer to an FFIAccount instance - - The returned string must be freed by the caller using `string_free` - - Returns NULL if the account is null - */ - char *account_get_extended_public_key_as_string(const FFIAccount *account) ; - -/* - Get the network of an account - - # Safety - - - `account` must be a valid pointer to an FFIAccount instance - - Returns FFINetwork::NoNetworks if the account is null - */ - FFINetworks account_get_network(const FFIAccount *account) ; - -/* - Get the parent wallet ID of an account - - # Safety - - - `account` must be a valid pointer to an FFIAccount instance - - Returns a pointer to the 32-byte wallet ID, or NULL if not set or account is null - - The returned pointer is valid only as long as the account exists - - The caller should copy the data if needed for longer use - */ - const uint8_t *account_get_parent_wallet_id(const FFIAccount *account) ; - -/* - Get the account type of an account - - # Safety - - - `account` must be a valid pointer to an FFIAccount instance - - `out_index` must be a valid pointer to a c_uint where the index will be stored - - Returns FFIAccountType::StandardBIP44 with index 0 if the account is null - */ - FFIAccountType account_get_account_type(const FFIAccount *account, unsigned int *out_index) ; - -/* - Check if an account is watch-only - - # Safety - - - `account` must be a valid pointer to an FFIAccount instance - - Returns false if the account is null - */ - bool account_get_is_watch_only(const FFIAccount *account) ; - -/* - Get the extended public key of a BLS account as a string - - # Safety - - - `account` must be a valid pointer to an FFIBLSAccount instance - - The returned string must be freed by the caller using `string_free` - - Returns NULL if the account is null - */ - char *bls_account_get_extended_public_key_as_string(const FFIBLSAccount *account) ; - -/* - Get the network of a BLS account - - # Safety - - - `account` must be a valid pointer to an FFIBLSAccount instance - - Returns FFINetwork::NoNetworks if the account is null - */ - FFINetworks bls_account_get_network(const FFIBLSAccount *account) ; - -/* - Get the parent wallet ID of a BLS account - - # Safety - - - `account` must be a valid pointer to an FFIBLSAccount instance - - Returns a pointer to the 32-byte wallet ID, or NULL if not set or account is null - - The returned pointer is valid only as long as the account exists - - The caller should copy the data if needed for longer use - */ - const uint8_t *bls_account_get_parent_wallet_id(const FFIBLSAccount *account) ; - -/* - Get the account type of a BLS account - - # Safety - - - `account` must be a valid pointer to an FFIBLSAccount instance - - `out_index` must be a valid pointer to a c_uint where the index will be stored - - Returns FFIAccountType::StandardBIP44 with index 0 if the account is null - */ - -FFIAccountType bls_account_get_account_type(const FFIBLSAccount *account, - unsigned int *out_index) -; - -/* - Check if a BLS account is watch-only - - # Safety - - - `account` must be a valid pointer to an FFIBLSAccount instance - - Returns false if the account is null - */ - bool bls_account_get_is_watch_only(const FFIBLSAccount *account) ; - -/* - Get the extended public key of an EdDSA account as a string - - # Safety - - - `account` must be a valid pointer to an FFIEdDSAAccount instance - - The returned string must be freed by the caller using `string_free` - - Returns NULL if the account is null - */ - char *eddsa_account_get_extended_public_key_as_string(const FFIEdDSAAccount *account) ; - -/* - Get the network of an EdDSA account - - # Safety - - - `account` must be a valid pointer to an FFIEdDSAAccount instance - - Returns FFINetwork::NoNetworks if the account is null - */ - FFINetworks eddsa_account_get_network(const FFIEdDSAAccount *account) ; - -/* - Get the parent wallet ID of an EdDSA account - - # Safety - - - `account` must be a valid pointer to an FFIEdDSAAccount instance - - Returns a pointer to the 32-byte wallet ID, or NULL if not set or account is null - - The returned pointer is valid only as long as the account exists - - The caller should copy the data if needed for longer use - */ - const uint8_t *eddsa_account_get_parent_wallet_id(const FFIEdDSAAccount *account) ; - -/* - Get the account type of an EdDSA account - - # Safety - - - `account` must be a valid pointer to an FFIEdDSAAccount instance - - `out_index` must be a valid pointer to a c_uint where the index will be stored - - Returns FFIAccountType::StandardBIP44 with index 0 if the account is null - */ - -FFIAccountType eddsa_account_get_account_type(const FFIEdDSAAccount *account, - unsigned int *out_index) -; - -/* - Check if an EdDSA account is watch-only - - # Safety - - - `account` must be a valid pointer to an FFIEdDSAAccount instance - - Returns false if the account is null - */ - bool eddsa_account_get_is_watch_only(const FFIEdDSAAccount *account) ; - -/* - Get number of accounts - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet instance - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure both pointers remain valid for the duration of this call - */ - -unsigned int wallet_get_account_count(const FFIWallet *wallet, - FFINetworks network, - FFIError *error) -; - -/* - Get account collection for a specific network from wallet - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet instance - - `error` must be a valid pointer to an FFIError structure or null - - The returned pointer must be freed with `account_collection_free` when no longer needed - */ - -FFIAccountCollection *wallet_get_account_collection(const FFIWallet *wallet, - FFINetworks network, - FFIError *error) -; - -/* - Free an account collection handle - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection created by this library - - `collection` must not be used after calling this function - */ - void account_collection_free(FFIAccountCollection *collection) ; - -/* - Get a BIP44 account by index from the collection - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `account_free` when no longer needed - */ - -FFIAccount *account_collection_get_bip44_account(const FFIAccountCollection *collection, - unsigned int index) -; - -/* - Get all BIP44 account indices - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - `out_indices` must be a valid pointer to store the indices array - - `out_count` must be a valid pointer to store the count - - The returned array must be freed with `free_u32_array` when no longer needed - */ - -bool account_collection_get_bip44_indices(const FFIAccountCollection *collection, - unsigned int **out_indices, - size_t *out_count) -; - -/* - Get a BIP32 account by index from the collection - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `account_free` when no longer needed - */ - -FFIAccount *account_collection_get_bip32_account(const FFIAccountCollection *collection, - unsigned int index) -; - -/* - Get all BIP32 account indices - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - `out_indices` must be a valid pointer to store the indices array - - `out_count` must be a valid pointer to store the count - - The returned array must be freed with `free_u32_array` when no longer needed - */ - -bool account_collection_get_bip32_indices(const FFIAccountCollection *collection, - unsigned int **out_indices, - size_t *out_count) -; - -/* - Get a CoinJoin account by index from the collection - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `account_free` when no longer needed - */ - -FFIAccount *account_collection_get_coinjoin_account(const FFIAccountCollection *collection, - unsigned int index) -; - -/* - Get all CoinJoin account indices - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - `out_indices` must be a valid pointer to store the indices array - - `out_count` must be a valid pointer to store the count - - The returned array must be freed with `free_u32_array` when no longer needed - */ - -bool account_collection_get_coinjoin_indices(const FFIAccountCollection *collection, - unsigned int **out_indices, - size_t *out_count) -; - -/* - Get the identity registration account if it exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `account_free` when no longer needed - */ - FFIAccount *account_collection_get_identity_registration(const FFIAccountCollection *collection) ; - -/* - Check if identity registration account exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - */ - bool account_collection_has_identity_registration(const FFIAccountCollection *collection) ; - -/* - Get an identity topup account by registration index - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `account_free` when no longer needed - */ - -FFIAccount *account_collection_get_identity_topup(const FFIAccountCollection *collection, - unsigned int registration_index) -; - -/* - Get all identity topup registration indices - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - `out_indices` must be a valid pointer to store the indices array - - `out_count` must be a valid pointer to store the count - - The returned array must be freed with `free_u32_array` when no longer needed - */ - -bool account_collection_get_identity_topup_indices(const FFIAccountCollection *collection, - unsigned int **out_indices, - size_t *out_count) -; - -/* - Get the identity topup not bound account if it exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `account_free` when no longer needed - */ - -FFIAccount *account_collection_get_identity_topup_not_bound(const FFIAccountCollection *collection) -; - -/* - Check if identity topup not bound account exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - */ - bool account_collection_has_identity_topup_not_bound(const FFIAccountCollection *collection) ; - -/* - Get the identity invitation account if it exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `account_free` when no longer needed - */ - FFIAccount *account_collection_get_identity_invitation(const FFIAccountCollection *collection) ; - -/* - Check if identity invitation account exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - */ - bool account_collection_has_identity_invitation(const FFIAccountCollection *collection) ; - -/* - Get the provider voting keys account if it exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `account_free` when no longer needed - */ - FFIAccount *account_collection_get_provider_voting_keys(const FFIAccountCollection *collection) ; - -/* - Check if provider voting keys account exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - */ - bool account_collection_has_provider_voting_keys(const FFIAccountCollection *collection) ; - -/* - Get the provider owner keys account if it exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `account_free` when no longer needed - */ - FFIAccount *account_collection_get_provider_owner_keys(const FFIAccountCollection *collection) ; - -/* - Check if provider owner keys account exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - */ - bool account_collection_has_provider_owner_keys(const FFIAccountCollection *collection) ; - -/* - Get the provider operator keys account if it exists - Note: Returns null if the `bls` feature is not enabled - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `bls_account_free` when no longer needed (when BLS is enabled) - */ - -void *account_collection_get_provider_operator_keys(const FFIAccountCollection *collection) -; - -/* - Check if provider operator keys account exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - */ - bool account_collection_has_provider_operator_keys(const FFIAccountCollection *collection) ; - -/* - Get the provider platform keys account if it exists - Note: Returns null if the `eddsa` feature is not enabled - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `eddsa_account_free` when no longer needed (when EdDSA is enabled) - */ - -void *account_collection_get_provider_platform_keys(const FFIAccountCollection *collection) -; - -/* - Check if provider platform keys account exists - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - */ - bool account_collection_has_provider_platform_keys(const FFIAccountCollection *collection) ; - -/* - Free a u32 array allocated by this library - - # Safety - - - `array` must be a valid pointer to an array allocated by this library - - `array` must not be used after calling this function - */ - void free_u32_array(unsigned int *array, size_t count) ; - -/* - Get the total number of accounts in the collection - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - */ - unsigned int account_collection_count(const FFIAccountCollection *collection) ; - -/* - Get a human-readable summary of all accounts in the collection - - Returns a formatted string showing all account types and their indices. - The format is designed to be clear and readable for end users. - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned string must be freed with `string_free` when no longer needed - - Returns null if the collection pointer is null - */ - char *account_collection_summary(const FFIAccountCollection *collection) ; - -/* - Get structured account collection summary data - - Returns a struct containing arrays of indices for each account type and boolean - flags for special accounts. This provides Swift with programmatic access to - account information. - - # Safety - - - `collection` must be a valid pointer to an FFIAccountCollection - - The returned pointer must be freed with `account_collection_summary_free` when no longer needed - - Returns null if the collection pointer is null - */ - -FFIAccountCollectionSummary *account_collection_summary_data(const FFIAccountCollection *collection) -; - -/* - Free an account collection summary and all its allocated memory - - # Safety - - - `summary` must be a valid pointer to an FFIAccountCollectionSummary created by `account_collection_summary_data` - - `summary` must not be used after calling this function - */ - -void account_collection_summary_free(FFIAccountCollectionSummary *summary) -; - -/* - Free address string - - # Safety - - - `address` must be a valid pointer created by address functions or null - - After calling this function, the pointer becomes invalid - */ - void address_free(char *address) ; - -/* - Free address array - - # Safety - - - `addresses` must be a valid pointer to an array of address strings or null - - Each address in the array must be a valid C string pointer - - `count` must be the correct number of addresses in the array - - After calling this function, all pointers become invalid - */ - void address_array_free(char **addresses, size_t count) ; - -/* - Validate an address - - # Safety - - - `address` must be a valid null-terminated C string - - `error` must be a valid pointer to an FFIError - */ - bool address_validate(const char *address, FFINetworks network, FFIError *error) ; - -/* - Get address type - - Returns: - - 0: P2PKH address - - 1: P2SH address - - 2: Other address type - - u8::MAX (255): Error occurred - - # Safety - - - `address` must be a valid null-terminated C string - - `error` must be a valid pointer to an FFIError - */ - unsigned char address_get_type(const char *address, FFINetworks network, FFIError *error) ; - -/* - Free an address pool handle - - # Safety - - - `pool` must be a valid pointer to an FFIAddressPool that was allocated by this library - - The pointer must not be used after calling this function - - This function must only be called once per allocation - */ - void address_pool_free(FFIAddressPool *pool) ; - -/* - Get address pool information for an account - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWallet - - `info_out` must be a valid pointer to store the pool info - - `error` must be a valid pointer to an FFIError or null - */ - -bool managed_wallet_get_address_pool_info(const FFIManagedWalletInfo *managed_wallet, - FFINetworks network, - FFIAccountType account_type, - unsigned int account_index, - FFIAddressPoolType pool_type, - FFIAddressPoolInfo *info_out, - FFIError *error) -; - -/* - Set the gap limit for an address pool - - The gap limit determines how many unused addresses to maintain at the end - of the pool. This is important for wallet recovery and address discovery. - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWallet - - `error` must be a valid pointer to an FFIError or null - */ - -bool managed_wallet_set_gap_limit(FFIManagedWalletInfo *managed_wallet, - FFINetworks network, - FFIAccountType account_type, - unsigned int account_index, - FFIAddressPoolType pool_type, - unsigned int gap_limit, - FFIError *error) -; - -/* - Generate addresses up to a specific index in a pool - - This ensures that addresses up to and including the specified index exist - in the pool. This is useful for wallet recovery or when specific indices - are needed. - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWallet - - `wallet` must be a valid pointer to an FFIWallet (for key derivation) - - `error` must be a valid pointer to an FFIError or null - */ - -bool managed_wallet_generate_addresses_to_index(FFIManagedWalletInfo *managed_wallet, - const FFIWallet *wallet, - FFINetworks network, - FFIAccountType account_type, - unsigned int account_index, - FFIAddressPoolType pool_type, - unsigned int target_index, - FFIError *error) -; - -/* - Mark an address as used in the pool - - This updates the pool's tracking of which addresses have been used, - which is important for gap limit management and wallet recovery. - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWallet - - `address` must be a valid C string - - `error` must be a valid pointer to an FFIError or null - */ - -bool managed_wallet_mark_address_used(FFIManagedWalletInfo *managed_wallet, - FFINetworks network, - const char *address, - FFIError *error) -; - -/* - Get a single address info at a specific index from the pool - - Returns detailed information about the address at the given index, or NULL - if the index is out of bounds or not generated yet. - - # Safety - - - `pool` must be a valid pointer to an FFIAddressPool - - `error` must be a valid pointer to an FFIError or null - - The returned FFIAddressInfo must be freed using `address_info_free` - */ - -FFIAddressInfo *address_pool_get_address_at_index(const FFIAddressPool *pool, - uint32_t index, - FFIError *error) -; - -/* - Get a range of addresses from the pool - - Returns an array of FFIAddressInfo structures for addresses in the range [start_index, end_index). - The count_out parameter will be set to the actual number of addresses returned. - - Note: This function only reads existing addresses from the pool. It does not generate new addresses. - Use managed_wallet_generate_addresses_to_index if you need to generate addresses first. - - # Safety - - - `pool` must be a valid pointer to an FFIAddressPool - - `count_out` must be a valid pointer to store the count - - `error` must be a valid pointer to an FFIError or null - - The returned array must be freed using `address_info_array_free` - */ - -FFIAddressInfo **address_pool_get_addresses_in_range(const FFIAddressPool *pool, - uint32_t start_index, - uint32_t end_index, - size_t *count_out, - FFIError *error) -; - -/* - Free a single FFIAddressInfo structure - - # Safety - - - `info` must be a valid pointer to an FFIAddressInfo allocated by this library or null - - The pointer must not be used after calling this function - */ - void address_info_free(FFIAddressInfo *info) ; - -/* - Free an array of FFIAddressInfo structures - - # Safety - - - `infos` must be a valid pointer to an array of FFIAddressInfo pointers allocated by this library or null - - `count` must be the exact number of elements in the array - - The pointers must not be used after calling this function - */ - -void address_info_array_free(FFIAddressInfo **infos, - size_t count) -; - -/* - Create a new master extended private key from seed - - # Safety - - - `seed` must be a valid pointer to a byte array of `seed_len` length - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure the seed pointer remains valid for the duration of this call - */ - -FFIExtendedPrivKey *derivation_new_master_key(const uint8_t *seed, - size_t seed_len, - FFINetworks network, - FFIError *error) -; - -/* - Derive a BIP44 account path (m/44'/5'/account') - */ - -bool derivation_bip44_account_path(FFINetworks network, - unsigned int account_index, - char *path_out, - size_t path_max_len, - FFIError *error) -; - -/* - Derive a BIP44 payment path (m/44'/5'/account'/change/index) - */ - -bool derivation_bip44_payment_path(FFINetworks network, - unsigned int account_index, - bool is_change, - unsigned int address_index, - char *path_out, - size_t path_max_len, - FFIError *error) -; - -/* - Derive CoinJoin path (m/9'/5'/4'/account') - */ - -bool derivation_coinjoin_path(FFINetworks network, - unsigned int account_index, - char *path_out, - size_t path_max_len, - FFIError *error) -; - -/* - Derive identity registration path (m/9'/5'/5'/1'/index') - */ - -bool derivation_identity_registration_path(FFINetworks network, - unsigned int identity_index, - char *path_out, - size_t path_max_len, - FFIError *error) -; - -/* - Derive identity top-up path (m/9'/5'/5'/2'/identity_index'/top_up_index') - */ - -bool derivation_identity_topup_path(FFINetworks network, - unsigned int identity_index, - unsigned int topup_index, - char *path_out, - size_t path_max_len, - FFIError *error) -; - -/* - Derive identity authentication path (m/9'/5'/5'/0'/identity_index'/key_index') - */ - -bool derivation_identity_authentication_path(FFINetworks network, - unsigned int identity_index, - unsigned int key_index, - char *path_out, - size_t path_max_len, - FFIError *error) -; - -/* - Derive private key for a specific path from seed - - # Safety - - - `seed` must be a valid pointer to a byte array of `seed_len` length - - `path` must be a valid pointer to a null-terminated C string - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - */ - -FFIExtendedPrivKey *derivation_derive_private_key_from_seed(const uint8_t *seed, - size_t seed_len, - const char *path, - FFINetworks network, - FFIError *error) -; - -/* - Derive public key from extended private key - - # Safety - - - `xpriv` must be a valid pointer to an FFIExtendedPrivKey - - `error` must be a valid pointer to an FFIError - - The returned pointer must be freed with `extended_public_key_free` - */ - FFIExtendedPubKey *derivation_xpriv_to_xpub(const FFIExtendedPrivKey *xpriv, FFIError *error) ; - -/* - Get extended private key as string - - # Safety - - - `xpriv` must be a valid pointer to an FFIExtendedPrivKey - - `error` must be a valid pointer to an FFIError - - The returned string must be freed with `string_free` - */ - char *derivation_xpriv_to_string(const FFIExtendedPrivKey *xpriv, FFIError *error) ; - -/* - Get extended public key as string - - # Safety - - - `xpub` must be a valid pointer to an FFIExtendedPubKey - - `error` must be a valid pointer to an FFIError - - The returned string must be freed with `string_free` - */ - char *derivation_xpub_to_string(const FFIExtendedPubKey *xpub, FFIError *error) ; - -/* - Get fingerprint from extended public key (4 bytes) - - # Safety - - - `xpub` must be a valid pointer to an FFIExtendedPubKey - - `fingerprint_out` must be a valid pointer to a buffer of at least 4 bytes - - `error` must be a valid pointer to an FFIError - */ - -bool derivation_xpub_fingerprint(const FFIExtendedPubKey *xpub, - uint8_t *fingerprint_out, - FFIError *error) -; - -/* - Free extended private key - - # Safety - - - `xpriv` must be a valid pointer to an FFIExtendedPrivKey that was allocated by this library - - The pointer must not be used after calling this function - - This function must only be called once per allocation - */ - void derivation_xpriv_free(FFIExtendedPrivKey *xpriv) ; - -/* - Free extended public key - - # Safety - - - `xpub` must be a valid pointer to an FFIExtendedPubKey that was allocated by this library - - The pointer must not be used after calling this function - - This function must only be called once per allocation - */ - void derivation_xpub_free(FFIExtendedPubKey *xpub) ; - -/* - Free derivation path string - - # Safety - - - `s` must be a valid pointer to a C string that was allocated by this library - - The pointer must not be used after calling this function - - This function must only be called once per allocation - */ - void derivation_string_free(char *s) ; - -/* - Derive key using DIP9 path constants for identity - - # Safety - - - `seed` must be a valid pointer to a byte array of `seed_len` length - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure the seed pointer remains valid for the duration of this call - */ - -FFIExtendedPrivKey *dip9_derive_identity_key(const uint8_t *seed, - size_t seed_len, - FFINetworks network, - unsigned int identity_index, - unsigned int key_index, - FFIDerivationPathType key_type, - FFIError *error) -; - -/* - Free an error message - - # Safety - - - `message` must be a valid pointer to a C string that was allocated by this library - - The pointer must not be used after calling this function - - This function must only be called once per allocation - */ - void error_message_free(char *message) ; - -/* - Get extended private key for account - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `error` must be a valid pointer to an FFIError - - The returned string must be freed with `string_free` - */ - -char *wallet_get_account_xpriv(const FFIWallet *wallet, - FFINetworks network, - unsigned int account_index, - FFIError *error) -; - -/* - Get extended public key for account - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `error` must be a valid pointer to an FFIError - - The returned string must be freed with `string_free` - */ - -char *wallet_get_account_xpub(const FFIWallet *wallet, - FFINetworks network, - unsigned int account_index, - FFIError *error) -; - -/* - Derive private key at a specific path - Returns an opaque FFIPrivateKey pointer that must be freed with private_key_free - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `derivation_path` must be a valid null-terminated C string - - `error` must be a valid pointer to an FFIError - - The returned pointer must be freed with `private_key_free` - */ - -FFIPrivateKey *wallet_derive_private_key(const FFIWallet *wallet, - FFINetworks network, - const char *derivation_path, - FFIError *error) -; - -/* - Derive extended private key at a specific path - Returns an opaque FFIExtendedPrivateKey pointer that must be freed with extended_private_key_free - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `derivation_path` must be a valid null-terminated C string - - `error` must be a valid pointer to an FFIError - - The returned pointer must be freed with `extended_private_key_free` - */ - -FFIExtendedPrivateKey *wallet_derive_extended_private_key(const FFIWallet *wallet, - FFINetworks network, - const char *derivation_path, - FFIError *error) -; - -/* - Derive private key at a specific path and return as WIF string - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `derivation_path` must be a valid null-terminated C string - - `error` must be a valid pointer to an FFIError - - The returned string must be freed with `string_free` - */ - -char *wallet_derive_private_key_as_wif(const FFIWallet *wallet, - FFINetworks network, - const char *derivation_path, - FFIError *error) -; - -/* - Free a private key - - # Safety - - - `key` must be a valid pointer created by private key functions or null - - After calling this function, the pointer becomes invalid - */ - void private_key_free(FFIPrivateKey *key) ; - -/* - Free an extended private key - - # Safety - - - `key` must be a valid pointer created by extended private key functions or null - - After calling this function, the pointer becomes invalid - */ - void extended_private_key_free(FFIExtendedPrivateKey *key) ; - -/* - Get extended private key as string (xprv format) - - Returns the extended private key in base58 format (xprv... for mainnet, tprv... for testnet) - - # Safety - - - `key` must be a valid pointer to an FFIExtendedPrivateKey - - `network` is ignored; the network is encoded in the extended key - - `error` must be a valid pointer to an FFIError - - The returned string must be freed with `string_free` - */ - -char *extended_private_key_to_string(const FFIExtendedPrivateKey *key, - FFINetworks network, - FFIError *error) -; - -/* - Get the private key from an extended private key - - Extracts the non-extended private key from an extended private key. - - # Safety - - - `extended_key` must be a valid pointer to an FFIExtendedPrivateKey - - `error` must be a valid pointer to an FFIError - - The returned FFIPrivateKey must be freed with `private_key_free` - */ - -FFIPrivateKey *extended_private_key_get_private_key(const FFIExtendedPrivateKey *extended_key, - FFIError *error) -; - -/* - Get private key as WIF string from FFIPrivateKey - - # Safety - - - `key` must be a valid pointer to an FFIPrivateKey - - `error` must be a valid pointer to an FFIError - - The returned string must be freed with `string_free` - */ - char *private_key_to_wif(const FFIPrivateKey *key, FFINetworks network, FFIError *error) ; - -/* - Derive public key at a specific path - Returns an opaque FFIPublicKey pointer that must be freed with public_key_free - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `derivation_path` must be a valid null-terminated C string - - `error` must be a valid pointer to an FFIError - - The returned pointer must be freed with `public_key_free` - */ - -FFIPublicKey *wallet_derive_public_key(const FFIWallet *wallet, - FFINetworks network, - const char *derivation_path, - FFIError *error) -; - -/* - Derive extended public key at a specific path - Returns an opaque FFIExtendedPublicKey pointer that must be freed with extended_public_key_free - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `derivation_path` must be a valid null-terminated C string - - `error` must be a valid pointer to an FFIError - - The returned pointer must be freed with `extended_public_key_free` - */ - -FFIExtendedPublicKey *wallet_derive_extended_public_key(const FFIWallet *wallet, - FFINetworks network, - const char *derivation_path, - FFIError *error) -; - -/* - Derive public key at a specific path and return as hex string - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `derivation_path` must be a valid null-terminated C string - - `error` must be a valid pointer to an FFIError - - The returned string must be freed with `string_free` - */ - -char *wallet_derive_public_key_as_hex(const FFIWallet *wallet, - FFINetworks network, - const char *derivation_path, - FFIError *error) -; - -/* - Free a public key - - # Safety - - - `key` must be a valid pointer created by public key functions or null - - After calling this function, the pointer becomes invalid - */ - void public_key_free(FFIPublicKey *key) ; - -/* - Free an extended public key - - # Safety - - - `key` must be a valid pointer created by extended public key functions or null - - After calling this function, the pointer becomes invalid - */ - void extended_public_key_free(FFIExtendedPublicKey *key) ; - -/* - Get extended public key as string (xpub format) - - Returns the extended public key in base58 format (xpub... for mainnet, tpub... for testnet) - - # Safety - - - `key` must be a valid pointer to an FFIExtendedPublicKey - - `network` is ignored; the network is encoded in the extended key - - `error` must be a valid pointer to an FFIError - - The returned string must be freed with `string_free` - */ - -char *extended_public_key_to_string(const FFIExtendedPublicKey *key, - FFINetworks network, - FFIError *error) -; - -/* - Get the public key from an extended public key - - Extracts the non-extended public key from an extended public key. - - # Safety - - - `extended_key` must be a valid pointer to an FFIExtendedPublicKey - - `error` must be a valid pointer to an FFIError - - The returned FFIPublicKey must be freed with `public_key_free` - */ - -FFIPublicKey *extended_public_key_get_public_key(const FFIExtendedPublicKey *extended_key, - FFIError *error) -; - -/* - Get public key as hex string from FFIPublicKey - - # Safety - - - `key` must be a valid pointer to an FFIPublicKey - - `error` must be a valid pointer to an FFIError - - The returned string must be freed with `string_free` - */ - char *public_key_to_hex(const FFIPublicKey *key, FFIError *error) ; - -/* - Convert derivation path string to indices - - # Safety - - - `path` must be a valid null-terminated C string or null - - `indices_out` must be a valid pointer to store the indices array pointer - - `hardened_out` must be a valid pointer to store the hardened flags array pointer - - `count_out` must be a valid pointer to store the count - - `error` must be a valid pointer to an FFIError - - The returned arrays must be freed with `derivation_path_free` - */ - -bool derivation_path_parse(const char *path, - uint32_t **indices_out, - bool **hardened_out, - size_t *count_out, - FFIError *error) -; - -/* - Free derivation path arrays - Note: This function expects the count to properly free the slices - - # Safety - - - `indices` must be a valid pointer created by `derivation_path_parse` or null - - `hardened` must be a valid pointer created by `derivation_path_parse` or null - - `count` must match the count from `derivation_path_parse` - - After calling this function, the pointers become invalid - */ - void derivation_path_free(uint32_t *indices, bool *hardened, size_t count) ; - -/* - Get a managed account from a managed wallet - - This function gets a ManagedAccount from the wallet manager's managed wallet info, - returning a managed account handle that wraps the ManagedAccount. - - # Safety - - - `manager` must be a valid pointer to an FFIWalletManager instance - - `wallet_id` must be a valid pointer to a 32-byte wallet ID - - `network` must specify exactly one network - - The caller must ensure all pointers remain valid for the duration of this call - - The returned account must be freed with `managed_account_free` when no longer needed - */ - -FFIManagedAccountResult managed_wallet_get_account(const FFIWalletManager *manager, - const uint8_t *wallet_id, - FFINetworks network, - unsigned int account_index, - FFIAccountType account_type) -; - -/* - Get a managed IdentityTopUp account with a specific registration index - - This is used for top-up accounts that are bound to a specific identity. - Returns a managed account handle that wraps the ManagedAccount. - - # Safety - - - `manager` must be a valid pointer to an FFIWalletManager instance - - `wallet_id` must be a valid pointer to a 32-byte wallet ID - - `network` must specify exactly one network - - The caller must ensure all pointers remain valid for the duration of this call - - The returned account must be freed with `managed_account_free` when no longer needed - */ - -FFIManagedAccountResult managed_wallet_get_top_up_account_with_registration_index(const FFIWalletManager *manager, - const uint8_t *wallet_id, - FFINetworks network, - unsigned int registration_index) -; - -/* - Get the network of a managed account - - # Safety - - - `account` must be a valid pointer to an FFIManagedAccount instance - */ - FFINetworks managed_account_get_network(const FFIManagedAccount *account) ; - -/* - Get the parent wallet ID of a managed account - - Note: ManagedAccount doesn't store the parent wallet ID directly. - The wallet ID is typically known from the context (e.g., when getting the account from a managed wallet). - - # Safety - - - `wallet_id` must be a valid pointer to a 32-byte wallet ID buffer that was provided by the caller - - The returned pointer is the same as the input pointer for convenience - - The caller must not free the returned pointer as it's the same as the input - */ - -const uint8_t *managed_account_get_parent_wallet_id(const uint8_t *wallet_id) -; - -/* - Get the account type of a managed account - - # Safety - - - `account` must be a valid pointer to an FFIManagedAccount instance - - `index_out` must be a valid pointer to receive the account index (or null) - */ - -FFIAccountType managed_account_get_account_type(const FFIManagedAccount *account, - unsigned int *index_out) -; - -/* - Check if a managed account is watch-only - - # Safety - - - `account` must be a valid pointer to an FFIManagedAccount instance - */ - bool managed_account_get_is_watch_only(const FFIManagedAccount *account) ; - -/* - Get the balance of a managed account - - # Safety - - - `account` must be a valid pointer to an FFIManagedAccount instance - - `balance_out` must be a valid pointer to an FFIBalance structure - */ - bool managed_account_get_balance(const FFIManagedAccount *account, FFIBalance *balance_out) ; - -/* - Get the number of transactions in a managed account - - # Safety - - - `account` must be a valid pointer to an FFIManagedAccount instance - */ - unsigned int managed_account_get_transaction_count(const FFIManagedAccount *account) ; - -/* - Get the number of UTXOs in a managed account - - # Safety - - - `account` must be a valid pointer to an FFIManagedAccount instance - */ - unsigned int managed_account_get_utxo_count(const FFIManagedAccount *account) ; - -/* - Free a managed account handle - - # Safety - - - `account` must be a valid pointer to an FFIManagedAccount that was allocated by this library - - The pointer must not be used after calling this function - - This function must only be called once per allocation - */ - void managed_account_free(FFIManagedAccount *account) ; - -/* - Free a managed account result's error message (if any) - Note: This does NOT free the account handle itself - use managed_account_free for that - - # Safety - - - `result` must be a valid pointer to an FFIManagedAccountResult - - The error_message field must be either null or a valid CString allocated by this library - - The caller must ensure the result pointer remains valid for the duration of this call - */ - void managed_account_result_free_error(FFIManagedAccountResult *result) ; - -/* - Get number of accounts in a managed wallet - - # Safety - - - `manager` must be a valid pointer to an FFIWalletManager instance - - `wallet_id` must be a valid pointer to a 32-byte wallet ID - - `network` must specify exactly one network - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - */ - -unsigned int managed_wallet_get_account_count(const FFIWalletManager *manager, - const uint8_t *wallet_id, - FFINetworks network, - FFIError *error) -; - -/* - Get the account index from a managed account - - Returns the primary account index for Standard and CoinJoin accounts. - Returns 0 for account types that don't have an index (like Identity or Provider accounts). - - # Safety - - - `account` must be a valid pointer to an FFIManagedAccount instance - */ - unsigned int managed_account_get_index(const FFIManagedAccount *account) ; - -/* - Get the external address pool from a managed account - - This function returns the external (receive) address pool for Standard accounts. - Returns NULL for account types that don't have separate external/internal pools. - - # Safety - - - `account` must be a valid pointer to an FFIManagedAccount instance - - The returned pool must be freed with `address_pool_free` when no longer needed - */ - FFIAddressPool *managed_account_get_external_address_pool(const FFIManagedAccount *account) ; - -/* - Get the internal address pool from a managed account - - This function returns the internal (change) address pool for Standard accounts. - Returns NULL for account types that don't have separate external/internal pools. - - # Safety - - - `account` must be a valid pointer to an FFIManagedAccount instance - - The returned pool must be freed with `address_pool_free` when no longer needed - */ - FFIAddressPool *managed_account_get_internal_address_pool(const FFIManagedAccount *account) ; - -/* - Get an address pool from a managed account by type - - This function returns the appropriate address pool based on the pool type parameter. - For Standard accounts with External/Internal pool types, returns the corresponding pool. - For non-standard accounts with Single pool type, returns their single address pool. - - # Safety - - - `manager` must be a valid pointer to an FFIWalletManager instance - - `account` must be a valid pointer to an FFIManagedAccount instance - - `wallet_id` must be a valid pointer to a 32-byte wallet ID - - The returned pool must be freed with `address_pool_free` when no longer needed - */ - -FFIAddressPool *managed_account_get_address_pool(const FFIManagedAccount *account, - FFIAddressPoolType pool_type) -; - -/* - Get managed account collection for a specific network from wallet manager - - # Safety - - - `manager` must be a valid pointer to an FFIWalletManager instance - - `wallet_id` must be a valid pointer to a 32-byte wallet ID - - `error` must be a valid pointer to an FFIError structure or null - - The returned pointer must be freed with `managed_account_collection_free` when no longer needed - */ - -FFIManagedAccountCollection *managed_wallet_get_account_collection(const FFIWalletManager *manager, - const uint8_t *wallet_id, - FFINetworks network, - FFIError *error) -; - -/* - Free a managed account collection handle - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection created by this library - - `collection` must not be used after calling this function - */ - void managed_account_collection_free(FFIManagedAccountCollection *collection) ; - -/* - Get a BIP44 account by index from the managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned pointer must be freed with `managed_account_free` when no longer needed - */ - -FFIManagedAccount *managed_account_collection_get_bip44_account(const FFIManagedAccountCollection *collection, - unsigned int index) -; - -/* - Get all BIP44 account indices from managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - `out_indices` must be a valid pointer to store the indices array - - `out_count` must be a valid pointer to store the count - - The returned array must be freed with `free_u32_array` when no longer needed - */ - -bool managed_account_collection_get_bip44_indices(const FFIManagedAccountCollection *collection, - unsigned int **out_indices, - size_t *out_count) -; - -/* - Get a BIP32 account by index from the managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned pointer must be freed with `managed_account_free` when no longer needed - */ - -FFIManagedAccount *managed_account_collection_get_bip32_account(const FFIManagedAccountCollection *collection, - unsigned int index) -; - -/* - Get all BIP32 account indices from managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - `out_indices` must be a valid pointer to store the indices array - - `out_count` must be a valid pointer to store the count - - The returned array must be freed with `free_u32_array` when no longer needed - */ - -bool managed_account_collection_get_bip32_indices(const FFIManagedAccountCollection *collection, - unsigned int **out_indices, - size_t *out_count) -; - -/* - Get a CoinJoin account by index from the managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned pointer must be freed with `managed_account_free` when no longer needed - */ - -FFIManagedAccount *managed_account_collection_get_coinjoin_account(const FFIManagedAccountCollection *collection, - unsigned int index) -; - -/* - Get all CoinJoin account indices from managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - `out_indices` must be a valid pointer to store the indices array - - `out_count` must be a valid pointer to store the count - - The returned array must be freed with `free_u32_array` when no longer needed - */ - -bool managed_account_collection_get_coinjoin_indices(const FFIManagedAccountCollection *collection, - unsigned int **out_indices, - size_t *out_count) -; - -/* - Get the identity registration account if it exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned pointer must be freed with `managed_account_free` when no longer needed - */ - -FFIManagedAccount *managed_account_collection_get_identity_registration(const FFIManagedAccountCollection *collection) -; - -/* - Check if identity registration account exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - */ - -bool managed_account_collection_has_identity_registration(const FFIManagedAccountCollection *collection) -; - -/* - Get an identity topup account by registration index from managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned pointer must be freed with `managed_account_free` when no longer needed - */ - -FFIManagedAccount *managed_account_collection_get_identity_topup(const FFIManagedAccountCollection *collection, - unsigned int registration_index) -; - -/* - Get all identity topup registration indices from managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - `out_indices` must be a valid pointer to store the indices array - - `out_count` must be a valid pointer to store the count - - The returned array must be freed with `free_u32_array` when no longer needed - */ - -bool managed_account_collection_get_identity_topup_indices(const FFIManagedAccountCollection *collection, - unsigned int **out_indices, - size_t *out_count) -; - -/* - Get the identity topup not bound account if it exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - `manager` must be a valid pointer to an FFIWalletManager - - The returned pointer must be freed with `managed_account_free` when no longer needed - */ - -FFIManagedAccount *managed_account_collection_get_identity_topup_not_bound(const FFIManagedAccountCollection *collection) -; - -/* - Check if identity topup not bound account exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - */ - -bool managed_account_collection_has_identity_topup_not_bound(const FFIManagedAccountCollection *collection) -; - -/* - Get the identity invitation account if it exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned pointer must be freed with `managed_account_free` when no longer needed - */ - -FFIManagedAccount *managed_account_collection_get_identity_invitation(const FFIManagedAccountCollection *collection) -; - -/* - Check if identity invitation account exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - */ - -bool managed_account_collection_has_identity_invitation(const FFIManagedAccountCollection *collection) -; - -/* - Get the provider voting keys account if it exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned pointer must be freed with `managed_account_free` when no longer needed - */ - -FFIManagedAccount *managed_account_collection_get_provider_voting_keys(const FFIManagedAccountCollection *collection) -; - -/* - Check if provider voting keys account exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - */ - -bool managed_account_collection_has_provider_voting_keys(const FFIManagedAccountCollection *collection) -; - -/* - Get the provider owner keys account if it exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned pointer must be freed with `managed_account_free` when no longer needed - */ - -FFIManagedAccount *managed_account_collection_get_provider_owner_keys(const FFIManagedAccountCollection *collection) -; - -/* - Check if provider owner keys account exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - */ - -bool managed_account_collection_has_provider_owner_keys(const FFIManagedAccountCollection *collection) -; - -/* - Get the provider operator keys account if it exists in managed collection - Note: Returns null if the `bls` feature is not enabled - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned pointer must be freed with `managed_account_free` when no longer needed (when BLS is enabled) - */ - -void *managed_account_collection_get_provider_operator_keys(const FFIManagedAccountCollection *collection) -; - -/* - Check if provider operator keys account exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - */ - -bool managed_account_collection_has_provider_operator_keys(const FFIManagedAccountCollection *collection) -; - -/* - Get the provider platform keys account if it exists in managed collection - Note: Returns null if the `eddsa` feature is not enabled - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned pointer must be freed with `managed_account_free` when no longer needed (when EdDSA is enabled) - */ - -void *managed_account_collection_get_provider_platform_keys(const FFIManagedAccountCollection *collection) -; - -/* - Check if provider platform keys account exists in managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - */ - -bool managed_account_collection_has_provider_platform_keys(const FFIManagedAccountCollection *collection) -; - -/* - Get the total number of accounts in the managed collection - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - */ - unsigned int managed_account_collection_count(const FFIManagedAccountCollection *collection) ; - -/* - Get a human-readable summary of all accounts in the managed collection - - Returns a formatted string showing all account types and their indices. - The format is designed to be clear and readable for end users. - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned string must be freed with `string_free` when no longer needed - - Returns null if the collection pointer is null - */ - char *managed_account_collection_summary(const FFIManagedAccountCollection *collection) ; - -/* - Get structured account collection summary data for managed collection - - Returns a struct containing arrays of indices for each account type and boolean - flags for special accounts. This provides Swift with programmatic access to - account information. - - # Safety - - - `collection` must be a valid pointer to an FFIManagedAccountCollection - - The returned pointer must be freed with `managed_account_collection_summary_free` when no longer needed - - Returns null if the collection pointer is null - */ - -FFIManagedAccountCollectionSummary *managed_account_collection_summary_data(const FFIManagedAccountCollection *collection) -; - -/* - Free a managed account collection summary and all its allocated memory - - # Safety - - - `summary` must be a valid pointer to an FFIManagedAccountCollectionSummary created by `managed_account_collection_summary_data` - - `summary` must not be used after calling this function - */ - -void managed_account_collection_summary_free(FFIManagedAccountCollectionSummary *summary) -; - -/* - Get the next unused receive address - - Generates the next unused receive address for the specified account. - This properly manages address gaps and updates the managed wallet state. - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWalletInfo - - `wallet` must be a valid pointer to an FFIWallet - - `error` must be a valid pointer to an FFIError - - The returned string must be freed by the caller - */ - -char *managed_wallet_get_next_bip44_receive_address(FFIManagedWalletInfo *managed_wallet, - const FFIWallet *wallet, - FFINetworks network, - unsigned int account_index, - FFIError *error) -; - -/* - Get the next unused change address - - Generates the next unused change address for the specified account. - This properly manages address gaps and updates the managed wallet state. - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWalletInfo - - `wallet` must be a valid pointer to an FFIWallet - - `error` must be a valid pointer to an FFIError - - The returned string must be freed by the caller - */ - -char *managed_wallet_get_next_bip44_change_address(FFIManagedWalletInfo *managed_wallet, - const FFIWallet *wallet, - FFINetworks network, - unsigned int account_index, - FFIError *error) -; - -/* - Get BIP44 external (receive) addresses in the specified range - - Returns external addresses from start_index (inclusive) to end_index (exclusive). - If addresses in the range haven't been generated yet, they will be generated. - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWalletInfo - - `wallet` must be a valid pointer to an FFIWallet - - `addresses_out` must be a valid pointer to store the address array pointer - - `count_out` must be a valid pointer to store the count - - `error` must be a valid pointer to an FFIError - - Free the result with address_array_free(addresses_out, count_out) - */ - -bool managed_wallet_get_bip_44_external_address_range(FFIManagedWalletInfo *managed_wallet, - const FFIWallet *wallet, - FFINetworks network, - unsigned int account_index, - unsigned int start_index, - unsigned int end_index, - char ***addresses_out, - size_t *count_out, - FFIError *error) -; - -/* - Get BIP44 internal (change) addresses in the specified range - - Returns internal addresses from start_index (inclusive) to end_index (exclusive). - If addresses in the range haven't been generated yet, they will be generated. - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWalletInfo - - `wallet` must be a valid pointer to an FFIWallet - - `addresses_out` must be a valid pointer to store the address array pointer - - `count_out` must be a valid pointer to store the count - - `error` must be a valid pointer to an FFIError - - Free the result with address_array_free(addresses_out, count_out) - */ - -bool managed_wallet_get_bip_44_internal_address_range(FFIManagedWalletInfo *managed_wallet, - const FFIWallet *wallet, - FFINetworks network, - unsigned int account_index, - unsigned int start_index, - unsigned int end_index, - char ***addresses_out, - size_t *count_out, - FFIError *error) -; - -/* - Get wallet balance from managed wallet info - - Returns the balance breakdown including confirmed, unconfirmed, locked, and total amounts. - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWalletInfo - - `confirmed_out` must be a valid pointer to store the confirmed balance - - `unconfirmed_out` must be a valid pointer to store the unconfirmed balance - - `locked_out` must be a valid pointer to store the locked balance - - `total_out` must be a valid pointer to store the total balance - - `error` must be a valid pointer to an FFIError - */ - -bool managed_wallet_get_balance(const FFIManagedWalletInfo *managed_wallet, - uint64_t *confirmed_out, - uint64_t *unconfirmed_out, - uint64_t *locked_out, - uint64_t *total_out, - FFIError *error) -; - -/* - Free managed wallet info - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWalletInfo or null - - After calling this function, the pointer becomes invalid and must not be used - */ - void managed_wallet_free(FFIManagedWalletInfo *managed_wallet) ; - -/* - Free managed wallet info returned by wallet_manager_get_managed_wallet_info - - # Safety - - - `wallet_info` must be a valid pointer returned by wallet_manager_get_managed_wallet_info or null - - After calling this function, the pointer becomes invalid and must not be used - */ - void managed_wallet_info_free(FFIManagedWalletInfo *wallet_info) ; - -/* - Generate a new mnemonic with specified word count (12, 15, 18, 21, or 24) - */ - char *mnemonic_generate(unsigned int word_count, FFIError *error) ; - -/* - Generate a new mnemonic with specified language and word count - */ - -char *mnemonic_generate_with_language(unsigned int word_count, - FFILanguage language, - FFIError *error) -; - -/* - Validate a mnemonic phrase - - # Safety - - - `mnemonic` must be a valid null-terminated C string or null - - `error` must be a valid pointer to an FFIError - */ - bool mnemonic_validate(const char *mnemonic, FFIError *error) ; - -/* - Convert mnemonic to seed with optional passphrase - - # Safety - - - `mnemonic` must be a valid null-terminated C string - - `passphrase` must be a valid null-terminated C string or null - - `seed_out` must be a valid pointer to a buffer of at least 64 bytes - - `seed_len` must be a valid pointer to store the seed length - - `error` must be a valid pointer to an FFIError - */ - -bool mnemonic_to_seed(const char *mnemonic, - const char *passphrase, - uint8_t *seed_out, - size_t *seed_len, - FFIError *error) -; - -/* - Get word count from mnemonic - - # Safety - - - `mnemonic` must be a valid null-terminated C string or null - - `error` must be a valid pointer to an FFIError - */ - unsigned int mnemonic_word_count(const char *mnemonic, FFIError *error) ; - -/* - Free a mnemonic string - - # Safety - - - `mnemonic` must be a valid pointer created by mnemonic generation functions or null - - After calling this function, the pointer becomes invalid - */ - void mnemonic_free(char *mnemonic) ; - -/* - Generate a provider key at a specific index - - This generates a provider key (BLS or EdDSA) at the specified index. - For voting, owner, and operator keys, this generates BLS keys. - For platform keys, this generates EdDSA keys. - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `info_out` must be a valid pointer to store the key info - - `error` must be a valid pointer to an FFIError or null - - The returned public_key, private_key, and derivation_path must be freed by the caller - */ - -bool wallet_generate_provider_key(const FFIWallet *wallet, - FFINetworks network, - FFIProviderKeyType key_type, - unsigned int key_index, - bool include_private, - FFIProviderKeyInfo *info_out, - FFIError *error) -; - -/* - Free provider key info - - # Safety - - - `info` must be a valid pointer to an FFIProviderKeyInfo - - This function must only be called once per info structure - */ - void provider_key_info_free(FFIProviderKeyInfo *info) ; - -/* - Sign data with a provider key - - This signs arbitrary data with the provider key at the specified index. - For BLS keys, this produces a BLS signature. - For EdDSA keys, this produces an Ed25519 signature. - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `data` must be a valid pointer to data with at least `data_len` bytes - - `signature_out` must be a valid pointer to store the signature pointer - - `signature_len_out` must be a valid pointer to store the signature length - - `error` must be a valid pointer to an FFIError or null - - The returned signature must be freed with `libc::free` - */ - -bool wallet_sign_with_provider_key(const FFIWallet *wallet, - FFINetworks network, - FFIProviderKeyType key_type, - unsigned int _key_index, - const uint8_t *data, - size_t data_len, - uint8_t **signature_out, - size_t *signature_len_out, - FFIError *error) -; - -/* - Build a transaction - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `outputs` must be a valid pointer to an array of FFITxOutput with at least `outputs_count` elements - - `tx_bytes_out` must be a valid pointer to store the transaction bytes pointer - - `tx_len_out` must be a valid pointer to store the transaction length - - `error` must be a valid pointer to an FFIError - - The returned transaction bytes must be freed with `transaction_bytes_free` - */ - -bool wallet_build_transaction(FFIWallet *wallet, - FFINetworks _network, - unsigned int account_index, - const FFITxOutput *outputs, - size_t outputs_count, - uint64_t fee_per_kb, - uint8_t **tx_bytes_out, - size_t *tx_len_out, - FFIError *error) -; - -/* - Sign a transaction - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `tx_bytes` must be a valid pointer to transaction bytes with at least `tx_len` bytes - - `signed_tx_out` must be a valid pointer to store the signed transaction bytes pointer - - `signed_len_out` must be a valid pointer to store the signed transaction length - - `error` must be a valid pointer to an FFIError - - The returned signed transaction bytes must be freed with `transaction_bytes_free` - */ - -bool wallet_sign_transaction(const FFIWallet *wallet, - FFINetworks _network, - const uint8_t *tx_bytes, - size_t tx_len, - uint8_t **signed_tx_out, - size_t *signed_len_out, - FFIError *error) -; - -/* - Check if a transaction belongs to the wallet using ManagedWalletInfo - - # Safety - - - `wallet` must be a valid mutable pointer to an FFIWallet - - `tx_bytes` must be a valid pointer to transaction bytes with at least `tx_len` bytes - - `inputs_spent_out` must be a valid pointer to store the spent inputs count - - `addresses_used_out` must be a valid pointer to store the used addresses count - - `new_balance_out` must be a valid pointer to store the new balance - - `new_address_out` must be a valid pointer to store the address array pointer - - `new_address_count_out` must be a valid pointer to store the address count - - `error` must be a valid pointer to an FFIError - */ - -bool wallet_check_transaction(FFIWallet *wallet, - FFINetworks network, - const uint8_t *tx_bytes, - size_t tx_len, - FFITransactionContext context_type, - uint32_t block_height, - const uint8_t *block_hash, - uint64_t timestamp, - bool update_state, - FFITransactionCheckResult *result_out, - FFIError *error) -; - -/* - Free transaction bytes - - # Safety - - - `tx_bytes` must be a valid pointer created by transaction functions or null - - After calling this function, the pointer becomes invalid - */ - void transaction_bytes_free(uint8_t *tx_bytes) ; - -/* - Create a managed wallet from a regular wallet - - This creates a ManagedWalletInfo instance from a Wallet, which includes - address pools and transaction checking capabilities. - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `error` must be a valid pointer to an FFIError or null - - The returned pointer must be freed with `ffi_managed_wallet_free` - */ -FFIManagedWalletInfo *wallet_create_managed_wallet(const FFIWallet *wallet, FFIError *error) ; - -/* - Check if a transaction belongs to the wallet - - This function checks a transaction against all relevant account types in the wallet - and returns detailed information about which accounts are affected. - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWallet - - `wallet` must be a valid pointer to an FFIWallet (needed for address generation) - - `tx_bytes` must be a valid pointer to transaction bytes with at least `tx_len` bytes - - `result_out` must be a valid pointer to store the result - - `error` must be a valid pointer to an FFIError - - The affected_accounts array in the result must be freed with `transaction_check_result_free` - */ - -bool managed_wallet_check_transaction(FFIManagedWalletInfo *managed_wallet, - const FFIWallet *wallet, - FFINetworks network, - const uint8_t *tx_bytes, - size_t tx_len, - FFITransactionContext context_type, - unsigned int block_height, - const uint8_t *block_hash, - uint64_t timestamp, - bool update_state, - FFITransactionCheckResult *result_out, - FFIError *error) -; - -/* - Free a transaction check result - - # Safety - - - `result` must be a valid pointer to an FFITransactionCheckResult - - This function must only be called once per result - */ - void transaction_check_result_free(FFITransactionCheckResult *result) ; - -/* - Free a managed wallet (FFIManagedWallet type) - - # Safety - - - `managed_wallet` must be a valid pointer to an FFIManagedWallet - - This function must only be called once per managed wallet - */ -void ffi_managed_wallet_free(FFIManagedWalletInfo *managed_wallet) ; - -/* - Get the transaction classification for routing - - Returns a string describing the transaction type (e.g., "Standard", "CoinJoin", - "AssetLock", "AssetUnlock", "ProviderRegistration", etc.) - - # Safety - - - `tx_bytes` must be a valid pointer to transaction bytes with at least `tx_len` bytes - - `error` must be a valid pointer to an FFIError or null - - The returned string must be freed by the caller - */ - char *transaction_classify(const uint8_t *tx_bytes, size_t tx_len, FFIError *error) ; - -/* - Free a string - - # Safety - - - `s` must be a valid pointer created by C string creation functions or null - - After calling this function, the pointer becomes invalid - */ - void string_free(char *s) ; - -/* - Get all UTXOs from managed wallet info - - # Safety - - - `managed_info` must be a valid pointer to an FFIManagedWalletInfo instance - - `utxos_out` must be a valid pointer to store the UTXO array pointer - - `count_out` must be a valid pointer to store the UTXO count - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - - The returned UTXO array must be freed with `utxo_array_free` when no longer needed - */ - -bool managed_wallet_get_utxos(const FFIManagedWalletInfo *managed_info, - FFINetworks network, - FFIUTXO **utxos_out, - size_t *count_out, - FFIError *error) -; - -/* - Get all UTXOs (deprecated - use managed_wallet_get_utxos instead) - - # Safety - - This function is deprecated and returns an empty list. - Use `managed_wallet_get_utxos` with a ManagedWalletInfo instead. - */ - -bool wallet_get_utxos(const FFIWallet *_wallet, - FFINetworks _network, - FFIUTXO **utxos_out, - size_t *count_out, - FFIError *error) -; - -/* - Free UTXO array - - # Safety - - - `utxos` must be a valid pointer to an array of FFIUTXO structs allocated by this library - - `count` must match the number of UTXOs in the array - - The pointer must not be used after calling this function - - This function must only be called once per array - */ - void utxo_array_free(FFIUTXO *utxos, size_t count) ; - -/* - Create a new wallet from mnemonic with options - - # Safety - - - `mnemonic` must be a valid pointer to a null-terminated C string - - `passphrase` must be a valid pointer to a null-terminated C string or null - - `account_options` must be a valid pointer to FFIWalletAccountCreationOptions or null - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - - The returned pointer must be freed with `wallet_free` when no longer needed - */ - -FFIWallet *wallet_create_from_mnemonic_with_options(const char *mnemonic, - const char *passphrase, - FFINetworks networks, - const FFIWalletAccountCreationOptions *account_options, - FFIError *error) -; - -/* - Create a new wallet from mnemonic (backward compatibility - single network) - - # Safety - - - `mnemonic` must be a valid pointer to a null-terminated C string - - `passphrase` must be a valid pointer to a null-terminated C string or null - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - - The returned pointer must be freed with `wallet_free` when no longer needed - */ - -FFIWallet *wallet_create_from_mnemonic(const char *mnemonic, - const char *passphrase, - FFINetworks network, - FFIError *error) -; - -/* - Create a new wallet from seed with options - - # Safety - - - `seed` must be a valid pointer to a byte array of `seed_len` length - - `account_options` must be a valid pointer to FFIWalletAccountCreationOptions or null - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - */ - -FFIWallet *wallet_create_from_seed_with_options(const uint8_t *seed, - size_t seed_len, - FFINetworks networks, - const FFIWalletAccountCreationOptions *account_options, - FFIError *error) -; - -/* - Create a new wallet from seed (backward compatibility) - - # Safety - - - `seed` must be a valid pointer to a byte array of `seed_len` length - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - */ - -FFIWallet *wallet_create_from_seed(const uint8_t *seed, - size_t seed_len, - FFINetworks network, - FFIError *error) -; - -/* - Create a new random wallet with options - - # Safety - - - `account_options` must be a valid pointer to FFIWalletAccountCreationOptions or null - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - */ - -FFIWallet *wallet_create_random_with_options(FFINetworks networks, - const FFIWalletAccountCreationOptions *account_options, - FFIError *error) -; - -/* - Create a new random wallet (backward compatibility) - - # Safety - - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure the pointer remains valid for the duration of this call - */ - FFIWallet *wallet_create_random(FFINetworks network, FFIError *error) ; - -/* - Get wallet ID (32-byte hash) - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet - - `id_out` must be a valid pointer to a 32-byte buffer - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - */ - bool wallet_get_id(const FFIWallet *wallet, uint8_t *id_out, FFIError *error) ; - -/* - Check if wallet has mnemonic - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet instance - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - */ - bool wallet_has_mnemonic(const FFIWallet *wallet, FFIError *error) ; - -/* - Check if wallet is watch-only - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet instance - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - */ - bool wallet_is_watch_only(const FFIWallet *wallet, FFIError *error) ; - -/* - Get extended public key for account - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet instance - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - - The returned C string must be freed by the caller when no longer needed - */ - -char *wallet_get_xpub(const FFIWallet *wallet, - FFINetworks network, - unsigned int account_index, - FFIError *error) -; - -/* - Free a wallet - - # Safety - - - `wallet` must be a valid pointer to an FFIWallet that was created by this library - - The pointer must not be used after calling this function - - This function must only be called once per wallet - */ - void wallet_free(FFIWallet *wallet) ; - -/* - Free a const wallet handle - - This is a const-safe wrapper for wallet_free() that accepts a const pointer. - Use this function when you have a *const FFIWallet that needs to be freed, - such as wallets returned from wallet_manager_get_wallet(). - - # Safety - - - `wallet` must be a valid pointer created by wallet creation functions or null - - After calling this function, the pointer becomes invalid - - This function must only be called once per wallet - - The wallet must have been allocated by this library (not stack or static memory) - */ - void wallet_free_const(const FFIWallet *wallet) ; - -/* - Add an account to the wallet without xpub - - # Safety - - This function dereferences a raw pointer to FFIWallet. - The caller must ensure that: - - The wallet pointer is either null or points to a valid FFIWallet - - The FFIWallet remains valid for the duration of this call - */ - -FFIAccountResult wallet_add_account(FFIWallet *wallet, - FFINetworks network, - FFIAccountType account_type, - unsigned int account_index) -; - -/* - Add an account to the wallet with xpub as byte array - - # Safety - - This function dereferences raw pointers. - The caller must ensure that: - - The wallet pointer is either null or points to a valid FFIWallet - - The xpub_bytes pointer is either null or points to at least xpub_len bytes - - The FFIWallet remains valid for the duration of this call - */ - -FFIAccountResult wallet_add_account_with_xpub_bytes(FFIWallet *wallet, - FFINetworks network, - FFIAccountType account_type, - unsigned int account_index, - const uint8_t *xpub_bytes, - size_t xpub_len) -; - -/* - Add an account to the wallet with xpub as string - - # Safety - - This function dereferences raw pointers. - The caller must ensure that: - - The wallet pointer is either null or points to a valid FFIWallet - - The xpub_string pointer is either null or points to a valid null-terminated C string - - The FFIWallet remains valid for the duration of this call - */ - -FFIAccountResult wallet_add_account_with_string_xpub(FFIWallet *wallet, - FFINetworks network, - FFIAccountType account_type, - unsigned int account_index, - const char *xpub_string) -; - -/* - Create a new wallet manager - */ - FFIWalletManager *wallet_manager_create(FFIError *error) ; - -/* - Add a wallet from mnemonic to the manager with options - - # Safety - - - `manager` must be a valid pointer to an FFIWalletManager instance - - `mnemonic` must be a valid pointer to a null-terminated C string - - `passphrase` must be a valid pointer to a null-terminated C string or null - - `account_options` must be a valid pointer to FFIWalletAccountCreationOptions or null - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - */ - -bool wallet_manager_add_wallet_from_mnemonic_with_options(FFIWalletManager *manager, - const char *mnemonic, - const char *passphrase, - FFINetworks network, - const FFIWalletAccountCreationOptions *account_options, - FFIError *error) -; - -/* - Add a wallet from mnemonic to the manager (backward compatibility) - - # Safety - - - `manager` must be a valid pointer to an FFIWalletManager instance - - `mnemonic` must be a valid pointer to a null-terminated C string - - `passphrase` must be a valid pointer to a null-terminated C string or null - - `error` must be a valid pointer to an FFIError structure or null - - The caller must ensure all pointers remain valid for the duration of this call - */ - -bool wallet_manager_add_wallet_from_mnemonic(FFIWalletManager *manager, - const char *mnemonic, - const char *passphrase, - FFINetworks network, - FFIError *error) -; diff --git a/key-wallet-manager/SPV_WALLET_GUIDE.md b/key-wallet-manager/SPV_WALLET_GUIDE.md deleted file mode 100644 index ebe5eddce..000000000 --- a/key-wallet-manager/SPV_WALLET_GUIDE.md +++ /dev/null @@ -1,231 +0,0 @@ -# SPV Wallet with Compact Filters (BIP 157/158) - -This guide explains how the filter-based SPV wallet implementation works and how to use it. - -## Overview - -The system implements a lightweight SPV (Simplified Payment Verification) wallet using compact block filters as specified in BIP 157 and BIP 158. This approach provides: - -- **95% bandwidth savings** compared to downloading full blocks -- **Privacy**: Servers don't learn which addresses belong to the wallet -- **Efficiency**: Only download blocks containing relevant transactions -- **Security**: Full SPV validation with merkle proofs - -## Architecture - -``` -┌─────────────────────────────────────────────────────┐ -│ FilterSPVClient │ -│ │ -│ ┌─────────────────┐ ┌──────────────────┐ │ -│ │ FilterClient │◄──────►│ WalletManager │ │ -│ │ │ │ │ │ -│ │ - Check filters │ │ - Manage wallets │ │ -│ │ - Fetch blocks │ │ - Track UTXOs │ │ -│ │ - Process txs │ │ - Update balances │ │ -│ └────────┬────────┘ └──────────────────┘ │ -│ │ │ -│ ▼ │ -│ ┌─────────────────┐ │ -│ │ Network Layer │ │ -│ │ │ │ -│ │ - P2P Protocol │ │ -│ │ - Fetch filters │ │ -│ │ - Fetch blocks │ │ -│ └─────────────────┘ │ -└─────────────────────────────────────────────────────┘ -``` - -## Workflow - -### 1. Initial Setup - -```rust -use key_wallet_manager::{FilterSPVClient, Network}; - -// Create SPV client -let mut spv_client = FilterSPVClient::new(Network::Testnet); - -// Add wallet from mnemonic -spv_client.add_wallet( - "main_wallet".to_string(), - "My Wallet".to_string(), - mnemonic, - passphrase, - Some(birth_height), // Start scanning from this height -)?; -``` - -### 2. Filter Processing Flow - -``` -For each new block: - 1. Receive compact filter from network - 2. Check if filter matches any of: - - Our addresses (watched scripts) - - Our UTXOs (watched outpoints) - 3. If match found: - - Fetch full block - - Process transactions - - Update wallet state - 4. If no match: - - Skip block (save bandwidth) -``` - -### 3. Filter Matching - -The system watches two types of data: - -#### Watched Scripts (Addresses) -- All addresses generated for the wallet -- Automatically updated when new addresses are created -- Matched against transaction outputs - -#### Watched Outpoints (UTXOs) -- All unspent transaction outputs owned by the wallet -- Automatically updated when receiving/spending -- Matched against transaction inputs (spending detection) - -### 4. Processing Matched Blocks - -When a filter matches, the system: - -1. **Fetches the full block** from the network -2. **Processes each transaction**: - - Check outputs for payments to our addresses - - Check inputs for spending of our UTXOs -3. **Updates wallet state**: - - Add new UTXOs - - Remove spent UTXOs - - Update balances - - Record transaction history - -## Implementation Details - -### Compact Filters (BIP 158) - -Compact filters use Golomb-Rice coding to create a probabilistic data structure: - -- **Size**: ~1/20th of the full block -- **False positive rate**: 1 in 784,931 -- **No false negatives**: If your transaction is in the block, the filter will match - -### Filter Chain Validation - -The system maintains a chain of filter headers for validation: - -```rust -FilterHeader { - filter_type: FilterType::Basic, - block_hash: [u8; 32], - prev_header: [u8; 32], // Hash of previous filter header - filter_hash: [u8; 32], // Hash of this block's filter -} -``` - -### Address Gap Limit - -The wallet implements BIP 44 gap limit handling: - -- Default gap limit: 20 addresses -- Automatically generates new addresses when used -- Tracks both receive and change addresses separately - -## Usage Example - -```rust -// Process incoming filter -let filter = receive_filter_from_network(); -let block_hash = BlockHash::from_slice(&filter.block_hash)?; - -// Check if we need this block -match spv_client.process_new_filter(height, block_hash, filter)? { - Some(result) => { - println!("Found {} relevant transactions", result.relevant_txs.len()); - println!("New UTXOs: {}", result.new_outpoints.len()); - println!("Spent UTXOs: {}", result.spent_outpoints.len()); - } - None => { - println!("Block not relevant, skipping"); - } -} - -// Check balance -let (confirmed, unconfirmed) = spv_client.get_balance("main_wallet")?; -println!("Balance: {} confirmed, {} unconfirmed", confirmed, unconfirmed); -``` - -## Network Integration - -To integrate with a P2P network, implement the trait interfaces: - -```rust -impl BlockFetcher for YourNetworkClient { - fn fetch_block(&mut self, block_hash: &BlockHash) -> Result { - // Send getdata message - // Wait for block response - // Return parsed block - } -} - -impl FilterFetcher for YourNetworkClient { - fn fetch_filter(&mut self, block_hash: &BlockHash) -> Result { - // Send getcfilters message - // Wait for cfilter response - // Return parsed filter - } -} -``` - -## Performance Characteristics - -### Bandwidth Usage - -| Method | Data Downloaded | Privacy | Speed | -|--------|----------------|---------|-------| -| Full Node | 100% of blocks | Full | Slow | -| Traditional SPV | 100% of blocks with txs | Low | Medium | -| **Compact Filters** | ~5% of blocks | High | Fast | - -### Storage Requirements - -- **Headers**: ~4 MB per year -- **Filters**: ~50 MB per year -- **Relevant blocks**: Only blocks with your transactions -- **Total**: <100 MB for typical wallet - -## Security Considerations - -1. **SPV Security**: Validates proof-of-work and merkle proofs -2. **Privacy**: Server doesn't know which addresses are yours -3. **Filter Validation**: Validates filter chain to prevent omission attacks -4. **Multiple Peers**: Should connect to multiple peers for security - -## Testing - -Run the example: - -```bash -cargo run --example spv_wallet -``` - -Run tests: - -```bash -cargo test -p key-wallet-manager -``` - -## Future Enhancements - -- [ ] Batch filter requests for efficiency -- [ ] Filter caching and persistence -- [ ] Peer rotation for privacy -- [ ] Tor/proxy support -- [ ] Lightning Network integration -- [ ] Hardware wallet support - -## References - -- [BIP 157: Client Side Block Filtering](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki) -- [BIP 158: Compact Block Filters](https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki) -- [Neutrino Protocol](https://github.com/lightninglabs/neutrino) diff --git a/test_checksum.rs b/test_checksum.rs deleted file mode 100644 index b6c2f3c19..000000000 --- a/test_checksum.rs +++ /dev/null @@ -1,13 +0,0 @@ -use dashcore::hashes::{Hash, sha256d}; - -fn sha2_checksum(data: &[u8]) -> [u8; 4] { - let checksum = ::hash(data); - [checksum[0], checksum[1], checksum[2], checksum[3]] -} - -fn main() { - let empty_data = &[]; - let checksum = sha2_checksum(empty_data); - println\!("SHA256D checksum for empty data: {:02x?}", checksum); -} -EOF < /dev/null diff --git a/test_smart_algo.sh b/test_smart_algo.sh deleted file mode 100755 index e58cb4f92..000000000 --- a/test_smart_algo.sh +++ /dev/null @@ -1,14 +0,0 @@ -#!/bin/bash -# Test the smart algorithm with debug logging enabled - -# Enable debug logging for the relevant modules -export RUST_LOG=dash_spv::sync::masternodes=debug,dash_spv::sync::sequential=debug - -# Run with start height at 1100000 to trigger the smart algorithm for the range 1260302-1290302 -./target/debug/dash-spv \ - --network testnet \ - --data-dir ./test-smart-algo \ - --start-height 1100000 \ - 2>&1 | tee smart_algo_debug.log - -echo "Debug log saved to smart_algo_debug.log"