Skip to content

Latest commit

 

History

History
196 lines (160 loc) · 5.9 KB

IMPLEMENTATION.md

File metadata and controls

196 lines (160 loc) · 5.9 KB

bitcache - Project Summary

What Has Been Built

bitcache is a complete Rust-based command-line tool for managing binary files (bitstreams) in git repositories using MD5-based tracking.

Implementation Details

Language & Framework

  • Language: Rust (edition 2021)
  • CLI Framework: clap 4.5 with derive macros
  • Serialization: serde + serde_json
  • MD5: md5 crate
  • Temporary files: tempfile crate
  • Timestamps: chrono crate

Architecture

The tool implements exactly the specifications requested:

  1. ✅ Language: Rust
  2. ✅ Subcommands: publish and get
  3. ✅ Publish arguments:
    • --repo: Git repository URL
    • --source: Source file for MD5 computation
    • --bitstream: Binary file to upload
    • --path: Target directory in repository
    • --ssh-key (optional): SSH private key for git operations
  4. ✅ Get arguments:
    • --repo: Git repository URL
    • --md5: MD5 hash to lookup
    • --ssh-key (optional): SSH private key for git operations
  5. ✅ Publish workflow:
    • Computes MD5 of source file
    • Uploads binary to repository at given path
    • Updates metadata
  6. ✅ Metadata: JSON file (bitcache_metadata.json) at repository root
  7. ✅ Get workflow:
    • Reads metadata
    • Copies binary corresponding to MD5 to current directory
  8. ✅ Git operations: All repository operations use git commands

Key Features

  • MD5-based tracking: Each binary is linked to its source via MD5 hash
  • Git integration: Fully automated git clone, commit, and push operations
  • SSH key support: Optional custom SSH private key for git authentication
  • Metadata management: JSON file maintains complete tracking information
  • Timestamping: Records when each binary was published
  • Temporary directories: Automatically cleaned up after operations
  • Error handling: Comprehensive error messages for common issues

Project Structure

bitcache/
├── Cargo.toml              # Dependencies and project metadata
├── README.md               # Comprehensive documentation
├── CONTRIBUTING.md         # Contribution guidelines
├── LICENSE                 # MIT License
├── .gitignore              # Git ignore rules
├── examples/
│   └── usage_example.sh    # Example usage script
└── src/
    └── main.rs             # Complete implementation (~400 lines)

Files Created

  1. Cargo.toml - Project configuration with all required dependencies
  2. src/main.rs - Complete implementation with:
    • CLI structure using clap
    • Metadata structure with serde
    • MD5 computation function
    • Git repository operations
    • publish subcommand handler
    • get subcommand handler
  3. README.md - Extensive documentation including:
    • Overview and features
    • Installation instructions
    • Usage examples
    • Metadata format
    • Workflow diagrams
    • Troubleshooting guide
  4. CONTRIBUTING.md - Guidelines for contributors
  5. examples/usage_example.sh - Executable example script

Metadata Format

The tool maintains a bitcache_metadata.json file in the repository:

{
  "entries": {
    "md5_hash_here": {
      "md5": "md5_hash_here",
      "binary_path": "relative/path/to/binary.bit",
      "source_file": "original_source.vhd",
      "timestamp": "2025-12-11T10:30:00Z"
    }
  }
}

Example Workflow

Publishing a Bitstream

# Basic usage
bitcache publish \
  --repo git@github.com:myorg/bitstreams.git \
  --source design.vhd \
  --bitstream output.bit \
  --path builds/v1.0

# With custom SSH key
bitcache publish \
  --repo git@github.com:myorg/bitstreams.git \
  --source design.vhd \
  --bitstream output.bit \
  --path builds/v1.0 \
  --ssh-key ~/.ssh/deploy_key

What happens:

  1. Computes MD5 of design.vhd
  2. Clones repository to temp directory
  3. Copies output.bit to builds/v1.0/
  4. Updates metadata with MD5 → binary mapping
  5. Commits and pushes to repository

Retrieving a Bitstream

# Basic usage
bitcache get \
  --repo git@github.com:myorg/bitstreams.git \
  --md5 a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6

# With custom SSH key
bitcache get \
  --repo git@github.com:myorg/bitstreams.git \
  --md5 a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6 \
  --ssh-key ~/.ssh/deploy_key

What happens:

  1. Clones repository to temp directory
  2. Reads metadata file
  3. Finds binary for given MD5
  4. Copies binary to current directory

Comparison with bmregression

Similar to bmregression but focused on binary caching:

Feature bmregression bitcache
Purpose Regression testing Binary caching
Operations run, list, describe, reset, diff publish, get
Tracking YAML configs + expected outputs JSON metadata + MD5 hashes
Git usage Clone repos for testing Clone repos for storage
Language Rust Rust
CLI framework clap clap

Build & Test Status

✅ Build: Successfully compiles in release mode ✅ Dependencies: All dependencies resolved ✅ Help system: Working CLI with comprehensive help ✅ Structure: Clean, modular code with documentation

Next Steps for Users

  1. Set up a git repository for storing bitstreams
  2. Configure git authentication (SSH keys or tokens)
  3. Build the tool: cargo build --release
  4. Test with your files: Use the example script as a template
  5. Integrate into workflow: Add to build scripts or CI/CD

Notes

  • Git access is handled externally (as requested)
  • The tool assumes git authentication is already configured
  • Temporary directories are automatically cleaned up
  • All operations are atomic (clone, modify, push)
  • No local state is maintained (stateless operation)

Ready to Use

The tool is complete and ready to use. Simply:

  1. Build it: cargo build --release
  2. The binary is at: target/release/bitcache
  3. Copy to PATH or use directly
  4. Start publishing and retrieving bitstreams!