CONTRIBUTING.md

Contributing to Zerobus SDK for Python

We happily welcome contributions to the Zerobus SDK for Python. We use GitHub Issues to track community reported issues and GitHub Pull Requests for accepting changes.

Contributions are licensed on a license-in/license-out basis.

Communication

Before starting work on a major feature, please open a GitHub issue. We will make sure no one else is already working on it and that it is aligned with the goals of the project.

A "major feature" is defined as any change that is > 100 LOC altered (not including tests), or changes any user-facing behavior.

We will use the GitHub issue to discuss the feature and come to agreement. This is to prevent your time being wasted, as well as ours. The GitHub review process for major features is also important so that organizations with commit access can come to agreement on design.

If it is appropriate to write a design document, the document must be hosted either in the GitHub tracking issue, or linked to from the issue and hosted in a world-readable location.

Small patches and bug fixes don't need prior communication.

Development Setup

Prerequisites

• Python 3.9 or higher
• Git
• Rust toolchain (required for building the native extension)
  • Install from rustup.rs
  • Verify: rustc --version
• maturin (for building Rust-Python bindings)
  • Install: pip install maturin

Setting Up Your Development Environment

Since v0.3.0, the SDK uses a Rust core with Python bindings built via maturin and PyO3.

1. Clone the repository:

   git clone https://github.com/databricks/zerobus-sdk-py.git
   cd zerobus-sdk-py

2. Create and activate a virtual environment:

   make dev

   This will:

   • Create a virtual environment in .venv
   • Install the package in development mode with all dev dependencies
   • Note: This uses pip install -e . which builds the Rust extension automatically

3. Activate the virtual environment:

   source .venv/bin/activate  # On Windows: .venv\Scripts\activate

Building the Rust Extension

The SDK's core is implemented in Rust. There are two ways to build it:

Option 1: Development Mode (Recommended for active development)

make develop-rust

This compiles the Rust code in debug mode and installs it directly into your virtual environment. After modifying Rust code, re-run this command to rebuild.

Option 2: Release Mode (For benchmarking or distribution)

make build-rust

This creates an optimized wheel in target/wheels/. Use this for performance testing or creating release builds.

Coding Style

Code style is enforced by a formatter check in your pull request. We use Black to format our code. Run make fmt to ensure your code is properly formatted prior to raising a pull request.

Running the Formatter

Format your code before committing:

make fmt

This runs:

• Black: Code formatting
• autoflake: Remove unused imports
• isort: Sort imports

Running Linters

Check your code for issues:

make lint

This runs:

• pycodestyle: Style guide enforcement
• autoflake: Check for unused imports

Running Tests

Run the test suite to ensure your changes don't break existing functionality:

make test

This runs:

• pytest: Unit tests with coverage reports
• Generates HTML and XML coverage reports in htmlcov/ and coverage.xml

Open the coverage report in your browser:

open htmlcov/index.html  # macOS
xdg-open htmlcov/index.html  # Linux

Pull Request Process

1. Create a feature branch:

   git checkout -b feature/your-feature-name

2. Make your changes:

   • Write clear, concise commit messages
   • Follow existing code style
   • Update documentation as needed

3. Format and test your code:

   make fmt
   make test

4. Commit your changes:

   git add .
   git commit -m "Add feature: description of your changes"

5. Push to your fork:

   git push origin feature/your-feature-name

6. Create a Pull Request:

   • Provide a clear description of changes
   • Reference any related issues
   • Ensure all CI checks pass

Signed Commits

This repo requires all contributors to sign their commits. To configure this, you can follow Github's documentation to create a GPG key, upload it to your Github account, and configure your git client to sign commits.

Developer Certificate of Origin

To contribute to this repository, you must sign off your commits to certify that you have the right to contribute the code and that it complies with the open source license. The rules are pretty simple, if you can certify the content of DCO, then simply add a "Signed-off-by" line to your commit message to certify your compliance. Please use your real name as pseudonymous/anonymous contributions are not accepted.

Signed-off-by: Joe Smith <joe.smith@email.com>

If you set your user.name and user.email git configs, you can sign your commit automatically with git commit -s:

git commit -s -m "Your commit message"

Code Review Guidelines

When reviewing code:

• Check for adherence to code style
• Look for potential edge cases
• Consider performance implications
• Ensure documentation is updated

Commit Message Guidelines

Follow these conventions for commit messages:

• Use present tense: "Add feature" not "Added feature"
• Use imperative mood: "Fix bug" not "Fixes bug"
• First line should be 50 characters or less
• Reference issues: "Fix #123: Description of fix"

Example:

Add async stream creation example

- Add async_example.py demonstrating non-blocking ingestion
- Update README with async API documentation

Fixes #42

Documentation

Updating Documentation

• Update docstrings for all public APIs
• Use Google-style docstrings
• Include examples in docstrings where helpful
• Update README.md for user-facing changes
• Update examples/ for new features

Example docstring:

def ingest_record(self, record) -> RecordAcknowledgment:
    """
    Submits a single record for ingestion into the stream.

    This method may block if the maximum number of in-flight records
    has been reached.

    Args:
        record: The Protobuf message object to be ingested.

    Returns:
        RecordAcknowledgment: An object to wait on for the server's acknowledgment.

    Raises:
        ZerobusException: If the stream is not in a valid state for ingestion.

    Example:
        >>> record = AirQuality(device_name="sensor-1", temp=25)
        >>> ack = stream.ingest_record(record)
        >>> ack.wait_for_ack()
    """

Continuous Integration

All pull requests must pass CI checks:

• fmt: Runs formatting checks (black, autoflake, isort)
• lint: Runs linting checks (pycodestyle, autoflake)
• tests-ubuntu: Runs tests on Ubuntu with Python 3.9, 3.10, 3.11, 3.12
• tests-windows: Runs tests on Windows with Python 3.9, 3.10, 3.11, 3.12

The formatting check runs make dev fmt and then checks for any git differences. If there are differences, the check will fail.

You can view CI results in the GitHub Actions tab of the pull request.

Makefile Targets

Python Targets

• make dev - Set up development environment (creates venv, installs deps, builds Rust extension)
• make install - Install package in editable mode
• make build - Build wheel package using maturin (use PYTHON=python3.X to specify version)
• make install-wheel - Install the built wheel
• make fmt - Format code with black, autoflake, and isort
• make lint - Run linting with pycodestyle and autoflake
• make test - Run unit tests with pytest and generate coverage reports
• make clean - Remove build artifacts (Python and Rust)
• make help - Show all available targets

Rust Targets (v0.3.0+)

• make build-rust - Build Rust extension in release mode (optimized, slower compile)
• make develop-rust - Build Rust extension in debug mode and install to venv (faster compile)
• make clean-rust - Remove Rust build artifacts
• make test-rust - Run Rust unit tests (runs cargo test in rust/ directory)

Common Workflows

Daily development (Python changes only):

make fmt        # Format your code
make lint       # Check for issues
make test       # Run tests

After modifying Rust code:

make develop-rust   # Rebuild and install Rust extension
make test          # Test Python bindings

Creating a release build:

make build-rust    # Build optimized wheel
# Wheel will be in target/wheels/

Versioning

We follow Semantic Versioning:

• MAJOR: Incompatible API changes
• MINOR: Backwards-compatible functionality additions
• PATCH: Backwards-compatible bug fixes

Getting Help

• Issues: Open an issue on GitHub for bugs or feature requests
• Discussions: Use GitHub Discussions for questions
• Documentation: Check the README and examples/

Package Name

The package is published on PyPI as databricks-zerobus-ingest-sdk.

Code of Conduct

• Be respectful and inclusive
• Welcome newcomers
• Focus on constructive feedback
• Follow the Python Community Code of Conduct