diff --git a/README.md b/README.md index b28475d..f11999d 100644 --- a/README.md +++ b/README.md @@ -1,278 +1,21 @@ # Adagio CLI -Command-line runner for Adagio pipeline files +`adagio-cli` is the Python command-line interface for Adagio pipeline execution. -## Requirements +For user-facing documentation and product guides, please reference the docs: -- Python 3.10+ -- `uv` (recommended for development) -- Docker for the default task runtime -- Apptainer or Singularity when using `kind = "apptainer"` with local `.sif` images +- [Adagio Docs](https://docs.adagiodata.com) -## Installation +The adagio frontend is used to build pipelines that can be run with this package on the command line +It can be found here: -Install from PyPI: - -```bash -pip install adagio-cli -``` - -Install a prerelease from TestPyPI while validating a release candidate: - -```bash -pip install \ - --index-url https://test.pypi.org/simple/ \ - --extra-index-url https://pypi.org/simple \ - adagio-cli -``` - -Install from the current checkout: - -```bash -pip install . -``` - -Or with `uv`: - -```bash -uv pip install . -``` - -Verify install: - -```bash -adagio --version -adagio --help -``` - -## Usage - -### Run a pipeline - -Show command help: - -```bash -adagio run --help -``` - -Run with a pipeline file: - -```bash -adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache -``` - -`adagio run` executes each plugin task in its own task environment. -The default task environment is a Docker image in GHCR derived from the plugin -name in the pipeline spec, for example `dada2` -> `ghcr.io/cymis/qiime2-plugin-dada2:2026.1`. -Runtime config can override that per default/plugin/task and switch selected work to -Apptainer/Singularity with a local `.sif` image path. The cache directory is required -and is reused across reruns by default so unchanged successful tasks can be replayed. - -Equivalent positional form: - -```bash -adagio run path/to/pipeline.json --cache-dir /path/to/cache -``` - -Use an arguments file: - -```bash -adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache --arguments path/to/arguments.json -``` - -Use a runtime config file with defaults, plugin-level overrides, and optional task overrides: - -```bash -adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache --config path/to/runtime.toml -``` - -Control which dynamic flags are shown in help: - -```bash -adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache --show-params required -# choices: all | missing | required -``` - -Disable reuse for a run while still writing outputs into the selected cache directory: - -```bash -adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache --no-reuse -``` - -The same boolean pair is available as `--reuse` / `--no-reuse`. `--reuse` is the default. - -Clear an existing cache directory: - -```bash -adagio cache clear --cache-dir /path/to/cache -``` - -### Inspect a pipeline - -Print a dependency-ordered summary of the plugin actions in a pipeline: - -```bash -adagio pipeline show path/to/pipeline.json -``` - -### Arguments file format - -`--arguments` can be downloaded from Adagio directly in the "Run" workflow : - -```json -{ - "version": 1, - "inputs": { - "input_name": "/path/to/input.qza" - }, - "parameters": { - "param_name": "value" - }, - "outputs": "/path/to/output-dir" -} -``` - -`outputs` may also be a map keyed by output name (WIP: not currently generated by Adagio): - -```json -{ - "outputs": { - "output_a": "/path/to/output-a", - "output_b": "/path/to/output-b" - } -} -``` - -If outputs are omitted, defaults are generated under `./adagio-outputs`. - -### Runtime config format - -`--config` accepts TOML. Defaults apply first, then plugin keys, then task keys: - -```toml -version = 1 - -[defaults] -platform = "linux/amd64" - -[plugins] -dada2 = { image = "ghcr.io/cymis/qiime2-plugin-dada2:2026.1" } -demux = { image = "ghcr.io/cymis/qiime2-plugin-demux:2026.1" } - -[tasks] -"dada2.denoise_single" = { image = "registry.internal/custom-dada2:1.0", platform = "linux/amd64" } -``` - -`kind`, `image`, and `platform` are all optional on defaults, plugin entries, and task entries. -`kind` may be `docker` or `apptainer`. `image` remains the environment reference: -for Docker it is the container image, and for Apptainer it must be a local `.sif` path. - -Precedence is `task override > plugin override > defaults > default resolver`. - -Task lookup supports graph node `id`, optional task `name` when present in the -pipeline, and `plugin.action` as a fallback. Plugin lookup uses the pipeline's -plugin name. If `platform` is omitted all the way through, Adagio uses normal -Docker platform resolution with no implicit fallback. Anything not listed in the -config uses the default plugin image resolver. - -Concrete Apptainer example: - -```toml -version = 1 - -[defaults] -kind = "docker" - -[plugins] -bowtie2 = { kind = "apptainer", image = "/shared/qiime-images/q2-bowtie2-test.sif" } -``` - -For `kind = "apptainer"`, Adagio prefers the `apptainer` executable and falls back to -`singularity`. The current implementation supports only local `.sif` paths and runs -tasks serially; no scheduler submission or remote image pull behavior is included. - -### QAPI generation/submission - -Generate and submit plugin metadata from the active QIIME environment: - -```bash -adagio qapi build --action-url http://localhost:81/api/v1 -``` - -Submit to a protected deployment such as `adagio.run` with a scoped submission token: - -```bash -export ACTION_URL=https://adagio.run/api/v1 -export QAPI_SUBMISSION_TOKEN= -uv run adagio qapi build -``` - -You can also pass `--submission-token`, but the environment variable is safer because it does -not end up in shell history. - -Write payload to disk without submitting: - -```bash -adagio qapi build --output qapi.json --dry-run -``` - -Submit selected plugins only: - -```bash -adagio qapi build --plugin dada2 --plugin feature-table -``` +- [Adagio](https://adagio.run) ## Development -### Setup - -Install runtime and dev dependencies: +Set up the project and run the test suite with: ```bash uv sync --group dev -``` - -Run commands inside the project environment: - -```bash -uv run adagio --help -``` - -### Linting - -```bash -uv run ruff check . -uv run ruff format --check . -uv run ruff format . -``` - -### Tests - -```bash uv run pytest ``` - -### Build distributions - -```bash -uv run python -m build -uv run python -m twine check dist/* -``` - -### Running locally during development - -```bash -uv run adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache -``` - -### Runtime entrypoint (container/integration use) - -The `runtime` subcommand is intended for runtime-adapter jobs: - -```bash -uv run adagio runtime --spec spec.json --config runtime.toml --arguments arguments.json --cache-dir /path/to/cache -``` - -### Releasing - -See [RELEASING.md](RELEASING.md) for the one-time PyPI/GitHub setup, version/tag workflow, and TestPyPI/PyPI publishing steps.