README.md

SpectrumX Data System | SDK

• SpectrumX Data System | SDK
  • SDK Usage Guide
  • System dependencies
  • SDK Development
    • Just recipes
    • Integration Testing
    • More testing options

The SDK is the primary form that clients interact with SDS: either directly by installing the Python package from PyPI, or indirectly by using the SDS Web UI or the Visualization Component.

SDK Usage Guide

This is the development README. See users' readme for a guide on how to install and use the SDK.

System dependencies

• uv - Python environment manager
• just - saves time by automating common tasks
• GNU coreutils, GNU tar.
• Optional:
  • gh - GitHub CLI for running GitHub Actions.
  • gh-act - gh extension for running GitHub Actions locally (incl. pyright action).

Everything else is installed by uv in the virtual environment whenever a uv sync or uv run command is executed.

SDK Development

GitHub Actions automate code quality checks, testing, and package publishing. See the justfile and the GitHub Actions workflows for specific details.

Just recipes

just --list

Available recipes:
    default                                         # show available recipes

    [development]
    dev-setup                                       # installs the development environment with pre-commit hooks and synced deps [alias: install]
    gact *args                                      # runs GitHub Actions locally
    pre-commit                                      # runs pre-commit checks locally, incl. deptry for dependency issues [alias: hooks]
    pyrefly *args                                   # runs the Pyrefly type checker
    pyright *args                                   # runs Pyright checks locally
    update                                          # updates development dependencies and pre-commit hooks

    [documentation]
    docs-build                                      # build SDK documentation
    docs-serve port="1314" *args=""                 # serve SDK documentation locally
    docs-serve-compose *args                        # builds and serves SDK documentation via nginx in docker compose

    [packaging]
    build *args                                     # builds the package and runs smoke tests on the built artifact
    check-acceptance                                # checks the built package acceptance tests
    clean                                           # removes temporary and build files
    publish                                         # publishes the built package to PyPI (prefer the trusted publishers method in GitHub)

    [qa]
    check-acceptance                                # checks the built package acceptance tests
    gact *args                                      # runs GitHub Actions locally
    pre-commit                                      # runs pre-commit checks locally, incl. deptry for dependency issues [alias: hooks]
    pyrefly *args                                   # runs the Pyrefly type checker
    pyright *args                                   # runs Pyright checks locally
    serve-coverage                                  # serves the HTML coverage report at http://localhost:1313 [alias: serve]
    test python=python_version *pytest_args         # runs tests using the default dependency resolution
    test-all *args                                  # tests against all supported python versions and dep lower bounds (no integration tests)
    test-integration python=python_version *pytest_args # runs integration tests using the default dependency resolution
    test-integration-verbose python=python_version *pytest_args # runs integration tests in verbose mode with stdout capture; useful for debugging test failures
    test-lowest python=python_version *pytest_args  # runs tests against the lowest dependency versions
    test-verbose python=python_version *pytest_args # runs local tests in verbose mode with stdout capture; useful for debugging test failures

Examples:

just dev-setup
just test
just test 3.12 --no-cov -k test_authentication
just pre-commit
just build

Tip

If testing a python version other than the default, you can run just dev-setup again to reset the environment to the default version after running the tests

Integration Testing

Integration tests need more setup. You need to deploy the Gateway and Network components, create a test user, and set up the integration test environment:

1. Follow the Network setup instructions in the Network README;
   1. Adjust TLS according to your setup;
   2. Modify your local DNS resolution if you'd like to simulate requests to a custom domain;
2. Follow the Gateway instructions in the Gateway README; In summary:
   1. Deploy the Docker Compose stack;
   2. Create a MinIO user and bucket with same credentials as in minio.env;
3. Create a test user and API key:
   1. Create a Gateway superuser and a regular user (they may be the same);
   2. Enable their is_approved flag in the admin panel;
   3. With the flag enabled, login as that user and create an API key for them;
   4. Copy / save the key.
4. Integration test setup:
   1. Create tests/integration/integration.env from its example counterpart
   2. Set the api key to the one created.
   3. Run the just test-integration target.

Then you can run the integration tests:

just test-integration
# runs integration tests with pytest

just test-integration-verbose
# like just test-integration, but with extra verbosity and log capture

More testing options

# simple tests execution (similar to `just test`)
uv run pytest -v

# running similar tests (substring match)
uv run pytest -v -k test_authentication

# running a specific test
uv run pytest -v tests/test_usage.py::test_authentication_200_succeeds

# verbose with colored log messages (similar to `just test-verbose`)
uv run pytest -vvv --show-capture=stdout -o log_cli=true --log-cli-level=DEBUG --capture=no