Skip to content

DKG 1.0 Enhancement #45

Open
cocoyoon wants to merge 64 commits intodevfrom
feat/work-raf
Open

DKG 1.0 Enhancement #45
cocoyoon wants to merge 64 commits intodevfrom
feat/work-raf

Conversation

@cocoyoon
Copy link

@cocoyoon cocoyoon commented May 21, 2025

Issues Tracking:

Description:

This pull request implements a fundamental architectural refactoring, transitioning the entire project from a single, monolithic crate into a structured, multi-crate Cargo workspace. This strategic overhaul is aimed at enhancing modularity, enforcing clearer API boundaries, and improving the overall maintainability and scalability of the codebase.

I. High-Level Summary of Changes

The core of this refactoring is the decomposition of the original src directory into several independent, yet interconnected, crates. Each crate is designed around a specific domain of responsibility, creating a clean separation of concerns. The new architecture provides a robust foundation for future development, parallelizes build processes, and simplifies dependency management.

II. Detailed Architectural Breakdown
The new workspace is comprised of the following crates:

  1. primitives crate:
    This crate serves as the foundational layer of the workspace, containing the core data structures, traits, and consensus logic that are shared across all other crates.
    types module: Defines universal data models such as Error, Event, Hash, and other core model structures.
    consensus module: Encapsulates consensus-related logic, including Commitment and Payload structures.
    traits.rs: Contains shared traits that define common behavior for different components in the system.

  2. node crate (Logical Grouping):
    This is the heart of the node's functionality and is physically divided into two crates for better separation between data definition and service implementation.

node/primitives Crate: Defines the essential building blocks and configurations specific to a node's operation.

  • config.rs: Manages node configuration and setup.
  • auth.rs: Handles authentication logic.
  • key_service/skde.rs: Contains definitions for the SKDE key service.

node/service Crate: Implements the node's runtime logic and orchestrates its various roles through background tasks.

  • builder.rs: A service builder responsible for constructing and initializing a node instance with all its dependencies.
  • task module: The core of the node's concurrent operations, containing role-specific workers:
    • committee/worker.rs: Logic for committee members.
    • solver/worker.rs: Logic for the solver role.
    • verifier/worker.rs: Logic for the verifier.
  1. cli crate:
    This crate is the main entry point for users, providing a command-line interface to configure and launch a node.
    It parses command-line arguments to determine the node's role, data directory, and other settings.
    It utilizes the node/service crate to build and run the node instance according to the provided configuration.

III. Key Benefits and Motivation

  • Strict Separation of Concerns: RPC logic is now completely decoupled from core service implementation, and data structures are separated from the logic that operates on them.
  • Improved Maintainability: With smaller, focused crates, it is easier to navigate, understand, and modify the code without unintended side effects.
  • Enhanced Scalability: The new architecture makes it straightforward to add new functionality or node roles by simply adding new crates to the workspace, minimizing disruption to the existing codebase.
  • Clearer Dependency Graph: The workspace structure makes dependencies between components explicit in their respective Cargo.toml files, providing a clear overview of the project's architecture.

@cocoyoon cocoyoon self-assigned this Jun 25, 2025
@cocoyoon cocoyoon added the enhancement New feature or request label Jun 25, 2025
@cocoyoon cocoyoon added this to the DKG 1.0 milestone Jun 25, 2025
@cocoyoon cocoyoon changed the title Feat/work raf DKG 1.0 Enhancement Jun 25, 2025
@cocoyoon cocoyoon marked this pull request as ready for review June 25, 2025 01:43
cocoyoon added 30 commits August 4, 2025 23:31
- Add SKDE key generator component with full SKDE integration
- Create operator service for managing node authentication and state
- Add operator task worker for handling operator-specific operations
- Implement modular architecture for different node functionality
- Update primitive types to support new operator architecture
- Refine error handling and event types for modular components
- Adjust model types for improved key generator integration
- Enhance trait definitions for better separation of concerns
- Update cluster RPC handlers to work with operator pattern
- Refine external RPC endpoints for key generator integration
- Improve synchronization handlers for new service architecture
- Enhance RPC routing and message handling efficiency
BREAKING CHANGE: Legacy authentication system and verifier components removed

- Remove auth.rs with DefaultAuthService and blockchain integration
- Delete key_service module in favor of separate key generator components
- Remove verifier task worker as verification is now handled by operators
- Simplify architecture by removing deprecated authentication patterns
- Add comprehensive project documentation for Claude Code integration
- Include build and test commands with single-threading requirements
- Document architecture overview and workspace structure
- Provide node running examples and key implementation details
…PC URLs

- Replace single auth_service_url with blockchain_http_rpc_url and blockchain_ws_rpc_url
- Update CLI arguments to support both HTTP and WebSocket RPC endpoints
- Modify TOML configuration to include separate RPC URL fields
- Update NodeConfig to handle dual RPC URL configuration
…ndling

- Rename BlockchainOperatorService to BAppService for clarity
- Add support for WebSocket event subscriptions alongside HTTP state queries
- Implement TaskCompleted event handling in addition to TaskCreated and TaskResponse
- Improve event stream processing with proper error handling and type safety
- Add separate contract instances for state queries and event subscriptions
…ed components

BREAKING CHANGE: Committee worker implementation has been redesigned

- Replace single CommitteeWorker with specialized components:
  - CommitteeSessionWorker: handles session lifecycle and key coordination
  - KeyManager: manages key operations and solver event processing
- Improve separation of concerns between session management and key handling
- Enhanced async task coordination with dedicated event channels
BREAKING CHANGE: Event system now uses separate channels for different event types

- Add DkgEvent enum to unify SessionEvent and SolverEvent handling
- Replace single event channel with dual channels:
  - session_event_tx/rx: for session lifecycle events
  - solver_event_tx/rx: for solver-specific events
- Update DefaultTaskExecutor to route events to appropriate channels
- Add session_duration field to BasicDkgService configuration
- Enhanced async event coordination between committee and solver components
- Update operator module to work with restructured worker system
- Modify RPC task handling for enhanced event coordination
- Update solver module and worker to support dual-channel event system
- Improve async task processing integration across service components
- Update cluster RPC handlers (sync_dec_key, sync_finalized_enc_keys, sync_start_time)
- Modify external RPC module and handlers (submit_dec_key, submit_enc_key)
- Align RPC layer with restructured worker system and dual-channel events
- Improve consistency in async RPC request handling
- Update traits to support new worker pattern and event system
- Enhance error types for improved error handling across components
- Update model types to align with restructured service architecture
- Improve type safety and consistency in primitive abstractions
…idance

- Add debug build, format, and lint commands for development workflow
- Include individual test execution examples and patterns
- Add detailed node startup commands with proper port configurations
- Document development scripts directory and automation tools
- Expand architecture details with event system and configuration priority
- Add comprehensive testing infrastructure documentation
- Include technical details like Rust toolchain, storage, and private keys
…al architecture

- Add professional header with badges, branding, and navigation links
- Create detailed architecture diagrams using Mermaid for system overview
- Include comprehensive sequential diagrams for complete key lifecycle
- Add state transition diagrams showing key generation phases
- Enhance quick start with one-click demo and step-by-step manual setup
- Document port configuration, testing infrastructure, and project structure
- Add real-world use cases for privacy-preserving and blockchain applications
- Include detailed development workflow and contribution guidelines
- Transform basic README into enterprise-grade project documentation
- Add formatting and dependencies information to development notes
- Document workspace structure with key crates breakdown
- Include process cleanup instructions and external dependencies
- Improve testing infrastructure documentation
- Upgrade alloy to v1.0.24 with improved consensus and network types
- Update various crypto and async dependencies to latest versions
- Remove operator workspace member from Cargo.toml
- Maintain rust toolchain consistency
BREAKING CHANGE: Operator module has been removed

- Remove node/operator/ crate with blockchain integration
- Simplify architecture by removing SSV operator functionality
- Focus on core DKG operations without external blockchain dependencies
- Update consensus module with enhanced commitment and payload types
- Refactor event system with SessionEvent and SolverEvent structures
- Improve error handling and model definitions
- Update node primitives with simplified authentication and key services
- Remove operator task worker module
- Update committee workers with improved session management
- Enhance solver worker with better key handling
- Refactor main task module structure and worker coordination
- Update service dependencies and remove operator references
- Update cluster RPC handlers for improved key synchronization
- Enhance external RPC endpoints with better error handling
- Remove submit_final_reveal handler as part of protocol simplification
- Update payload system and library exports for new architecture
…dling

- Update SKDE key generator implementation for better compatibility
- Improve error handling and type safety in cryptographic operations
- Enhance integration with the new event-driven architecture
…cumentation

- Add validation task module for enhanced security and verification
- Include ISSUES.md for tracking development priorities and known issues
- Integrate validation functionality into task system
- Add FromUtf8Error import
- Add InvalidDecKey variant to KeyGeneratorError enum
- Enable proper error handling for decryption key UTF-8 conversion
- Replace serde_json deserialization with String::from_utf8 for dec_key
- Add detailed logging for verification process
- Add get_enc_key helper method
- Improve error handling with InvalidDecKey error type
- Fix gen_dec_key to use into_bytes() instead of serde_json::to_vec
- Rename ActiveOperatorList to ActiveCommitteeList across codebase
- Update method names: update_active_operator_list -> update_active_committee_list
- Update method names: get_active_operator_list -> get_active_committee_list
- Add get_enc_key method to KeyGenerator trait
- Simplify error handling by removing redundant map_err
- Remove redundant info log (verification logging moved to keygen layer)
- Replace ActiveOperatorList with ActiveCommitteeList across RPC modules
- Remove solver-only check from sync_finalized_enc_keys handler
- Improve get_encryption_key to use KeyGenerator::get_enc_key for String conversion
- Add logging to get_decryption_key handler
- Enable multicast_enc_key_ack in submit_enc_key handler
- Update helper functions to use get_active_committee_list
- Improve logging levels (info -> debug for non-leader cases)
- Replace ActiveOperatorList with ActiveCommitteeList across service modules
- Update method calls: update_active_operator_list -> update_active_committee_list
- Rename get_trusted_setup to init_trusted_setup for clarity
- Update create_pub_sub to create_pub_sub_with_signer
- Add early return for authority role in run_node
- Improve async event subscription handling with proper error handling
- Fix DecKeyPayload constructor parameter order
- Add debug logging for sessions_per_round
- Remove unnecessary blank lines and improve code formatting
- Remove Clone derive from DefaultTaskExecutor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant