From bcc0affb6165ecc725af3ebeb09531de00814747 Mon Sep 17 00:00:00 2001 From: John Chase Date: Wed, 15 Apr 2026 21:12:22 -0700 Subject: [PATCH 1/2] Reduces README size --- CHANGELOG.md | 2 + README.md | 276 +++------------------------------------------------ 2 files changed, 18 insertions(+), 260 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6df60c8..0fdec1a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ using [PEP 440](https://packaging.python.org/en/latest/specifications/version-sp ## [Unreleased] +## [0.1.0a1] - 2026-04-15 + ### Added - GitHub Actions CI for linting, tests, and build verification. diff --git a/README.md b/README.md index b28475d..afb9d77 100644 --- a/README.md +++ b/README.md @@ -1,278 +1,34 @@ # Adagio CLI -Command-line runner for Adagio pipeline files +`adagio-cli` is the Python command-line interface for Adagio pipeline execution. -## Requirements +This README is intentionally brief. For user-facing documentation and product +guides, use: -- Python 3.10+ -- `uv` (recommended for development) -- Docker for the default task runtime -- Apptainer or Singularity when using `kind = "apptainer"` with local `.sif` images +- [Adagio Docs](https://docs.adagiodata.com) +- [Adagio](https://adagio.run) -## Installation +## Scope -Install from PyPI: +This package is mainly for developers working on: -```bash -pip install adagio-cli -``` - -Install a prerelease from TestPyPI while validating a release candidate: - -```bash -pip install \ - --index-url https://test.pypi.org/simple/ \ - --extra-index-url https://pypi.org/simple \ - adagio-cli -``` - -Install from the current checkout: - -```bash -pip install . -``` - -Or with `uv`: - -```bash -uv pip install . -``` - -Verify install: - -```bash -adagio --version -adagio --help -``` - -## Usage - -### Run a pipeline - -Show command help: - -```bash -adagio run --help -``` - -Run with a pipeline file: - -```bash -adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache -``` - -`adagio run` executes each plugin task in its own task environment. -The default task environment is a Docker image in GHCR derived from the plugin -name in the pipeline spec, for example `dada2` -> `ghcr.io/cymis/qiime2-plugin-dada2:2026.1`. -Runtime config can override that per default/plugin/task and switch selected work to -Apptainer/Singularity with a local `.sif` image path. The cache directory is required -and is reused across reruns by default so unchanged successful tasks can be replayed. - -Equivalent positional form: - -```bash -adagio run path/to/pipeline.json --cache-dir /path/to/cache -``` - -Use an arguments file: - -```bash -adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache --arguments path/to/arguments.json -``` - -Use a runtime config file with defaults, plugin-level overrides, and optional task overrides: - -```bash -adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache --config path/to/runtime.toml -``` - -Control which dynamic flags are shown in help: - -```bash -adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache --show-params required -# choices: all | missing | required -``` - -Disable reuse for a run while still writing outputs into the selected cache directory: - -```bash -adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache --no-reuse -``` - -The same boolean pair is available as `--reuse` / `--no-reuse`. `--reuse` is the default. - -Clear an existing cache directory: - -```bash -adagio cache clear --cache-dir /path/to/cache -``` - -### Inspect a pipeline - -Print a dependency-ordered summary of the plugin actions in a pipeline: - -```bash -adagio pipeline show path/to/pipeline.json -``` - -### Arguments file format - -`--arguments` can be downloaded from Adagio directly in the "Run" workflow : - -```json -{ - "version": 1, - "inputs": { - "input_name": "/path/to/input.qza" - }, - "parameters": { - "param_name": "value" - }, - "outputs": "/path/to/output-dir" -} -``` +- local CLI development and testing +- Adagio service integrations that invoke the CLI +- QAPI submission from a QIIME environment -`outputs` may also be a map keyed by output name (WIP: not currently generated by Adagio): - -```json -{ - "outputs": { - "output_a": "/path/to/output-a", - "output_b": "/path/to/output-b" - } -} -``` - -If outputs are omitted, defaults are generated under `./adagio-outputs`. - -### Runtime config format - -`--config` accepts TOML. Defaults apply first, then plugin keys, then task keys: - -```toml -version = 1 - -[defaults] -platform = "linux/amd64" - -[plugins] -dada2 = { image = "ghcr.io/cymis/qiime2-plugin-dada2:2026.1" } -demux = { image = "ghcr.io/cymis/qiime2-plugin-demux:2026.1" } - -[tasks] -"dada2.denoise_single" = { image = "registry.internal/custom-dada2:1.0", platform = "linux/amd64" } -``` - -`kind`, `image`, and `platform` are all optional on defaults, plugin entries, and task entries. -`kind` may be `docker` or `apptainer`. `image` remains the environment reference: -for Docker it is the container image, and for Apptainer it must be a local `.sif` path. - -Precedence is `task override > plugin override > defaults > default resolver`. - -Task lookup supports graph node `id`, optional task `name` when present in the -pipeline, and `plugin.action` as a fallback. Plugin lookup uses the pipeline's -plugin name. If `platform` is omitted all the way through, Adagio uses normal -Docker platform resolution with no implicit fallback. Anything not listed in the -config uses the default plugin image resolver. - -Concrete Apptainer example: - -```toml -version = 1 - -[defaults] -kind = "docker" - -[plugins] -bowtie2 = { kind = "apptainer", image = "/shared/qiime-images/q2-bowtie2-test.sif" } -``` - -For `kind = "apptainer"`, Adagio prefers the `apptainer` executable and falls back to -`singularity`. The current implementation supports only local `.sif` paths and runs -tasks serially; no scheduler submission or remote image pull behavior is included. - -### QAPI generation/submission - -Generate and submit plugin metadata from the active QIIME environment: - -```bash -adagio qapi build --action-url http://localhost:81/api/v1 -``` - -Submit to a protected deployment such as `adagio.run` with a scoped submission token: - -```bash -export ACTION_URL=https://adagio.run/api/v1 -export QAPI_SUBMISSION_TOKEN= -uv run adagio qapi build -``` - -You can also pass `--submission-token`, but the environment variable is safer because it does -not end up in shell history. - -Write payload to disk without submitting: - -```bash -adagio qapi build --output qapi.json --dry-run -``` - -Submit selected plugins only: - -```bash -adagio qapi build --plugin dada2 --plugin feature-table -``` +For command reference, pipeline authoring guides, and product workflows, prefer +the docs site instead of duplicating that material here. ## Development -### Setup - -Install runtime and dev dependencies: +Set up the project and run the test suite with: ```bash uv sync --group dev -``` - -Run commands inside the project environment: - -```bash -uv run adagio --help -``` - -### Linting - -```bash -uv run ruff check . -uv run ruff format --check . -uv run ruff format . -``` - -### Tests - -```bash uv run pytest ``` -### Build distributions - -```bash -uv run python -m build -uv run python -m twine check dist/* -``` - -### Running locally during development - -```bash -uv run adagio run --pipeline path/to/pipeline.json --cache-dir /path/to/cache -``` - -### Runtime entrypoint (container/integration use) - -The `runtime` subcommand is intended for runtime-adapter jobs: - -```bash -uv run adagio runtime --spec spec.json --config runtime.toml --arguments arguments.json --cache-dir /path/to/cache -``` - -### Releasing +## Monorepo -See [RELEASING.md](RELEASING.md) for the one-time PyPI/GitHub setup, version/tag workflow, and TestPyPI/PyPI publishing steps. +`adagio-cli` lives inside the Adagio monorepo. For local stack setup and the +surrounding services, see the repository root README. From f6b6ddf5949018057ee509af9c1530451a81416c Mon Sep 17 00:00:00 2001 From: John Chase Date: Wed, 15 Apr 2026 21:17:38 -0700 Subject: [PATCH 2/2] Fixes README --- README.md | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index afb9d77..f11999d 100644 --- a/README.md +++ b/README.md @@ -2,22 +2,14 @@ `adagio-cli` is the Python command-line interface for Adagio pipeline execution. -This README is intentionally brief. For user-facing documentation and product -guides, use: +For user-facing documentation and product guides, please reference the docs: - [Adagio Docs](https://docs.adagiodata.com) -- [Adagio](https://adagio.run) - -## Scope - -This package is mainly for developers working on: -- local CLI development and testing -- Adagio service integrations that invoke the CLI -- QAPI submission from a QIIME environment +The adagio frontend is used to build pipelines that can be run with this package on the command line +It can be found here: -For command reference, pipeline authoring guides, and product workflows, prefer -the docs site instead of duplicating that material here. +- [Adagio](https://adagio.run) ## Development @@ -27,8 +19,3 @@ Set up the project and run the test suite with: uv sync --group dev uv run pytest ``` - -## Monorepo - -`adagio-cli` lives inside the Adagio monorepo. For local stack setup and the -surrounding services, see the repository root README.