diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 00000000..a4d25ba0 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,57 @@ +--- +name: Documentation + +on: + push: + branches: [main] + paths: + - "docs/**" + - ".github/workflows/docs.yml" + workflow_dispatch: + +permissions: + contents: read + pages: write + id-token: write + +concurrency: + group: pages + cancel-in-progress: true + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: "3.13" + + - uses: astral-sh/setup-uv@v4 + + - name: Install docs dependencies + working-directory: docs + run: | + uv venv .venv + uv pip install sphinx sphinx-autobuild \ + myst_parser shibuya sphinx-design sphinx-copybutton + + - name: Build documentation + working-directory: docs + run: .venv/bin/sphinx-build sources html + + - uses: actions/upload-pages-artifact@v3 + with: + path: docs/html + + deploy: + needs: build + runs-on: ubuntu-latest + environment: + name: github-pages + url: "${{ steps.deployment.outputs.page_url }}" + steps: + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@v4 diff --git a/MANIFEST.in b/MANIFEST.in index bda1f01f..8d567f40 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -10,6 +10,16 @@ recursive-include src *.sh recursive-include src *.txt recursive-include src *.yml +recursive-include docs *.gitignore +recursive-include docs *.ini +recursive-include docs *.md +recursive-include docs *.py +recursive-include docs *.txt +recursive-include docs Makefile + +recursive-exclude docs/.venv * +recursive-exclude docs/.mxmake * +recursive-exclude docs/html * recursive-exclude news * recursive-exclude config * recursive-exclude shared-workflows * diff --git a/README.md b/README.md index 7f758cbd..9eb16d08 100644 --- a/README.md +++ b/README.md @@ -1,1147 +1,52 @@ # plone.meta -`plone.meta` defines and standardizes a common set of configuration files across Plone related Python repositories. -It does not cover the following. +`plone.meta` standardizes configuration files across Plone-related Python repositories. -- Volto or any other JavaScript-based project, which has its own ecosystem. -- Monorepo projects with backend and frontend code bases, such as those created by [Cookieplone](https://github.com/plone/cookieplone). - Repositories must have a single Python package at the top level. -- Projects that support multiple versions of Plone in the same branch. +It generates and manages `.editorconfig`, `.flake8`, `.gitignore`, `pyproject.toml`, `tox.ini`, `.pre-commit-config.yaml`, and CI pipelines (GitHub Actions, GitLab CI) from Jinja2 templates, with per-repository customization via `.meta.toml`. -## Setup - -Create a Python virtual environment and install `plone.meta` via pip. +## Quick Start ```shell python3 -m venv venv venv/bin/pip install plone.meta +venv/bin/config-package /path/to/your/package ``` -## `config-package` usage - -The command `config-package` from `plone.meta` creates or overwrites configuration files for your project. -See a current list of [configuration files](#config-package-configuration-files) that it will create or overwrite. - -This command has several [command line options](#config-package-command-line-options) that you can use to override the default options. - -When you run this command, it automatically goes through the following steps. - -1. It creates a new git branch from the current branch in your project. -1. If the file {file}`.meta.toml` is not present in the project, then it creates this and the other new configuration files from `plone.meta`'s Jinja2 templates. - Otherwise, it reads the file {file}`.meta.toml` for regenerating the configuration files. -1. It writes the configuration files. -1. It creates a change log entry. -1. By default, it commits changes. -1. It optionally adds packages, pushes commits, or runs tox from the configuration files. - -> [!TIP] -> If you prefer to name the new git branch instead of letting the command name it using its default naming scheme, then either create a new branch `my-new-branch`, switch to it, and use the `--branch current` option, or do all that in one step with the `--branch my-new-branch` option. - -> [!TIP] -> If you prefer to review changes before committing them, then use the `--no-commit` option. - -For help for `config-package`, use the following command. - -```shell -venv/bin/config-package --help -``` - -You can request more extension points if `plone.meta` does not fulfill your needs in the [issue tracker](https://github.com/plone/meta/issues/new). - - -### Generate configuration files - -Now you can run the command `config-package` to generate configuration files from Jinja2 template files to manage your project. - -```shell -venv/bin/config-package [OPTIONS] PATH/TO/PACKAGE -``` - - -### Manage configuration files - -For each of the configuration files, you should edit its [corresponding stanza](#config-package-configuration-files) in the file `.meta.toml` to customize them. - -> [!WARNING] -> Do not directly edit the configuration files that `plone.meta` manages. -Anytime someone runs the command `config-package`, any changes made in these files will get clobbered. - -Commit your changes, then run the command `config-package` to regenerate configuration files from your project's `.meta.toml`. - -```shell -venv/bin/config-package [OPTIONS] PATH/TO/PACKAGE -``` - - -### `config-package` command line options - -`config-package` supports the following command line options. +## Documentation -- `--branch BRANCH_NAME`: Define a specific git branch name to create for the changes. - By default, the script creates one, which includes the name of the configuration type. - > [!TIP] - > Use `current` to update the current branch. -- `--commit-msg MSG`: Use `MSG` as the commit message instead of the default one. -- `--no-commit`: Don't automatically commit changes after the configuration run. -- `-h, --help`: Display help. -- `--push`: Push changes at the end of the configuration run. - By default, the changes are _not_ pushed. -- `--tox`: Whether to run `tox` on the repository. - By default, it does not run. -- `--track`: Whether the package being configured should be added to `defaults/packages.txt`. - By default, they are _not_ added. -- `-t, --type`: define the configuration type. - At this time, `default` is the only option. - This option is only needed one time as their values are stored in `.meta.toml`. +Full documentation is available at **https://plone.github.io/meta/** +- [Tutorials](https://plone.github.io/meta/tutorials/) -- Step-by-step lessons to get started +- [How-To Guides](https://plone.github.io/meta/how-to/) -- Solutions to specific problems +- [Reference](https://plone.github.io/meta/reference/) -- Configuration options and CLI details +- [Explanation](https://plone.github.io/meta/explanation/) -- Architecture, design decisions, and philosophy -### `config-package` configuration files -`plone.meta` generates configuration files in your local repository. -Currently, the files that `plone.meta` manages are the following. +## Commands -- `.meta.toml`: `plone.meta`'s configuration file -- `.editorconfig`: configuration meant to be read by code editors -- `.flake8`: [`flake8`](https://pypi.org/project/flake8) configuration -- `.gitignore`: list of file/folders patterns that `git` should ignore -- `.github/workflows/meta.yml`: GitHub workflows to run the testing and code quality and analysis tools on GitHub, provided the repository is hosted at github.com -- `.gitlab-ci.yml`: GitLab CI configuration, provided the repository is hosted at gitlab.com -- `.pre-commit-config.yaml`: [`pre-commit`](https://pypi.org/project/pre-commit) configuration -- `pyproject.toml`: configuration options for a wide variety of Python tooling -- `tox.ini`: [`tox`](https://pypi.org/project/tox) configuration, _the most important file_ +- `config-package` -- Generate configuration files for a single repository +- `multi-call` -- Apply configuration across multiple repositories +- `re-enable-actions` -- Re-enable auto-disabled GitHub Actions -You can find the _template_ files for each of these files in the `config/default` folder of this `plone.meta` repository. -You will notice that they have a `.jinja` extension. -That's because the files are not merely copied over to the target repository, but rather they are enhanced and adapted on the way to their destination. - -The following sections describe how to configure each of the configuration files. - - -#### `.editorconfig` - -Add the `[editorconfig]` TOML table in `.meta.toml`, and set extra configuration for `editorconfig` under the `extra_lines` key. - -```toml -[editorconfig] -extra_lines = """ -_your own configuration lines_ -""" -``` +## GitHub Actions Shared Automations -If you want to set the indentation to four spaces for frontend related files, add the following to your `.meta.toml`. +`plone.meta` also provides reusable workflows and composite actions for GitHub Actions. +See the [full documentation](https://plone.github.io/meta/reference/shared-workflows/) for details. -```toml -[editorconfig] -extra_lines = """ -[*.{json,jsonl,js,jsx,ts,tsx,css,less,scss}] -indent_size = 4 -""" -``` - - -#### `.flake8` - -Add the `[flake8]` TOML table in `.meta.toml`, and set the extra configuration for `flake8` under the `extra_lines` key. - -```toml -[flake8] -extra_lines = """ -_your own configuration lines_ -""" -``` - - -#### `.gitignore` - -Add the `[gitignore]` TOML table in `.meta.toml`, and set the extra configuration for `git` under the `extra_lines` key. - -```toml -[gitignore] -extra_lines = """ -_your own configuration lines_ -""" -``` +## Scope -#### `.github/workflows/test-matrix.yml` - -Run the distribution test on a combination of Plone and Python versions. - -> [!NOTE] -> See the `test_matrix` option in [`tox`](#toxini) configuration file. - -> [!TIP] -> 🍀 `plone.meta` tries to be a bit more environmentally friendly. -> On GitHub, only the first and last Python versions will be added for testing. - - -#### `.github/workflows/meta.yml` - -Customize the GitHub Action jobs run on every change pushed to GitHub. - -Add the `[github]` TOML table in `.meta.toml`, and set the enabled jobs with the `jobs` key. - -```toml -[github] -jobs = [ - "qa", - "coverage", - "dependencies", - "release_ready", - "circular", - ] -``` - -It is possible to configure from which branch or tag of `plone.meta` to get the workflow files by setting the value of the `ref` key. -In the following example, all GitHub workflows would come from the `v3` tag, instead of the default `main` branch. - -```toml -[github] -ref = "v3" -``` - -To set environment variables for all jobs, specify the `env` key as follows. - -```toml -[github] -env = """ - debug: 1 - image-name: 'org/image' - image-tag: 'latest' -""" -``` - -To install specific operating system level dependencies, which must be Ubuntu package names, specify the following key. - -```toml -[github] -os_dependencies = "git libxml2" -``` - -Extend GitHub workflow configuration with additional jobs by setting the values for the `extra_lines` key. - -```toml -[github] -extra_lines = """ - another: - uses: org/repo/.github/workflows/file.yml@main -""" -``` - - -#### `.gitlab-ci.yml` - -Add the `[gitlab]` TOML table in `.meta.toml`, and set the extra configuration for GitLab CI under the `extra_lines` key. - -```toml -[gitlab] -extra_lines = """ -_your own configuration lines_ -""" -``` - -Specify custom Docker images in the `custom_images` key. - -The dictionary keys needs to be Python versions and the value a Docker image for that Python version. - -```toml -[gitlab] -custom_images = {"3.14" = "python:3.14-trixie", "3.13" = "python:3.13-trixie"} -``` - -> [!TIP] -> To tweak the jobs that will be run, you can customize the `test_matrix` option from `[tox]` table. - -To install specific test and coverage dependencies, add the `os_dependencies` key as follows. - -```toml -[gitlab] -os_dependencies = """ - - apt-get install libxslt libxml2 -""" -``` - -You can customize the enabled GitLab jobs with the `jobs` key. - -```toml -[gitlab] -jobs = [ - "lint", - "release-ready", - "dependencies", - "circular-dependencies", - "testing", - "coverage", -] -``` - - -#### `.pre-commit-config.yaml` - -Add the `[pre_commit]` TOML table in `.meta.toml`, and set the extra configuration for `pre-commit` under the `extra_lines` key. - -```toml -[pre_commit] -extra_lines = """ -_your own configuration lines_ -""" -``` - -Extend [`zpretty`](https://pypi.org/project/zpretty) configuration by setting the values for the `zpretty_extra_lines` key. - -```toml -[pre_commit] -zpretty_extra_lines = """ -_your own configuration lines_ -""" -``` +`plone/meta` covers single-package Python repositories. +It does not cover: -Extend [codespell](https://pypi.org/project/codespell) configuration by setting the values for the `codespell_extra_lines` key. - -```toml -[pre_commit] -codespell_extra_lines = """ -_your own configuration lines_ -""" -``` - -Extend [Flake8](https://pypi.org/project/flake8) configuration by setting the values for the `flake8_extra_lines` key. -For example, to add extra Flake8 plugins, you would specify `additional_dependencies` as shown. - -```toml -[pre_commit] -flake8_extra_lines = """ - additional_dependencies: - - flake8-debugger - - flake8-deprecated -""" -``` - -Extend [i18ndude](https://pypi.org/project/i18ndude) configuration by setting the values for the `i18ndude_extra_lines` key. -For example, to add extra i18ndude plugins, you would specify `additional_dependencies` as shown. - -```toml -[pre_commit] -i18ndude_extra_lines = """ - additional_dependencies: - - toml -""" -``` - -If you want to disable the i18ndude check, add the following pre-commit configuration option to your `.meta.toml` file. - -```toml -[pre_commit] -i18ndude_extra_lines = """ - pass_filenames: false -""" -``` - - -#### `pyproject.toml` - -Add the `[pyproject]` TOML table in `.meta.toml`, and set configuration for any extra tool that you use for the `extra_lines` key. - -```toml -[pyproject] -extra_lines = """ -_your own configuration lines_ -""" -``` - -Extend [codespell](https://pypi.org/project/codespell) configuration by setting the values for the `codespell_ignores` and `codespell_skip` keys. - -```toml -[pyproject] -codespell_ignores = "foo,bar" -codespell_skip = "*.po,*.map,package-lock.json" -``` - -Extend [z3c.dependencychecker](https://pypi.org/project/z3c.dependencychecker) configuration by setting the values for the `dependencies_ignores` and `dependencies_mappings` keys. - -```toml -[pyproject] -dependencies_ignores = "['zestreleaser.towncrier']" -dependencies_mappings = [ - "gitpython = ['git']", - "pygithub = ['github']", -] -``` - -Extend [check-manifest](https://pypi.org/project/check-manifest) configuration by setting the values for the `check_manifest_ignores` key. - -```toml -[pyproject] -check_manifest_ignores = """ - "*.map.js", - "*.pyc", -""" -``` - -Extend [Black](https://pypi.org/project/black) configuration by setting the values for the `black_extra_lines` key. - -```toml -[pyproject] -black_extra_lines = """ -_custom configuration_ -""" -``` - -Extend [isort](https://pypi.org/project/isort) configuration by setting the values for the `isort_extra_lines` key. - -```toml -[pyproject] -isort_extra_lines = """ -_custom configuration_ -""" -``` - - -##### `towncrier` configuration - -If your project contains a `news/` folder, `plone.meta` will add the configuration for `towncrier`. - -If your `CHANGES` file has the extension `.md`, a `changelog_template.jinja` template will be generated inside the `news/` folder. - -Configure [`towncrier`](https://pypi.org/project/towncrier) [`issue_format`](https://towncrier.readthedocs.io/en/stable/configuration.html) by setting the new format in the `towncrier_issue_format` key. - -```toml -[pyproject] -towncrier_issue_format = "[#{issue}](https://github.com/plonegovbr/plonegovbr.portal/issues/{issue})" -``` - -Extend [`towncrier`](https://pypi.org/project/towncrier) configuration by setting the values for the `towncrier_extra_lines` key. - -```toml -[pyproject] -towncrier_extra_lines = """ -_custom configuration_ -""" -``` - - -#### `tox.ini` - -Depending on the test runner that you want to use, `plone.meta` will adapt `tox.ini` to it. - -In the `[tox]` TOML table in `.meta.toml`, set the value for the key `test_runner` to `pytest` if you want to use [`pytest`](https://pypi.org/project/pytest). -By default, it falls back to use [zope.testrunner](https://pypi.org/project/zope.testrunner). - -Likewise, the root path where the tests are to be found can be specified under the key `test_path`. -By default, it is set to nothing, that is, the repository's top level is already importable and thus the tests can be found directly. - -If either a `tests` or `src` folder exists, then they are used as safe fallbacks. - -Add the `[tox]` TOML table in `.meta.toml`, and set the extra configuration for `tox` under the `extra_lines` key. - -```toml -[tox] -extra_lines = """ -_your own configuration lines_ -""" -``` - -`plone.meta` generates a list of Python and Plone version combinations to run the distribution tests. - -You can customize that by defining your testing matrix: - -```toml -[tox] -test_matrix = { "6.1" = ["3.13", "3.10"], "6.0" = ["*"] } -``` - -When the list of Python versions is `*`, `plone.meta` replaces it with the default Python version list for this Plone version. - -If a test matrix will not work for your project, you can disable the test matrix entirely: - -```toml -[tox] -use_test_matrix = false -``` - -Extend the list of default `tox` environments in the `envlist_lines` key. -Add extra top level configuration for `tox` in the `config_lines` key. - -```toml -[tox] -envlist_lines = """ - my_other_environment -""" -config_lines = """ -my_extra_top_level_tox_configuration_lines -""" -``` - -Customize the default values for all `tox` environments in the `testenv_options` key. - -```toml -[tox] -testenv_options = """ -basepython = /usr/bin/python3.8 -""" -``` - -Almost all packages have a `test` extra, and we install it. -A few don't, and this gives an error with `tox` 4.36.0+. -So you can skip it: - -```toml -[tox] -skip_test_extra = true -``` - -Extend the list of `extras` that need to be installed for running the test suite and generate the coverage report by setting them on the `test_extras` key. - -```toml -[tox] -test_extras = """ - tests - widgets -""" -``` - -If your package uses [mxdev](https://pypi.org/project/mxdev/) to handle source checkouts for dependencies, you can set the `use_mxdev` key to ensure `tox` will first run mxdev. -You also need to manually set the installation of additional packages that `mxdev` pulls in with the `test_deps_additional` key. - -```toml -[tox] -use_mxdev = true -test_deps_additional = """ - -esources/plonegovbr.portal_base[test] -""" -``` - -When using `plone.meta` outside of plone core packages, there might be extra version pins, or overrides over the official versions. -To specify a custom constraints file, use the `constraints_files` key. - -Generating a custom `constraints.txt` is out of scope for `plone.meta` itself. -There are plenty of tools that can do that though. - -> [!WARNING] -> You need to specify the same Plone versions as in the `test_matrix` option or the default provided by `plone.meta`. - -```toml -[tox] -constraints_files = { "6.1" = "https://my-server.com/constraints-6.1.txt", "6.0" = "https://my-server.com/constraints-6.0.txt" } -``` - -Extend the list of custom environment variables that the test and coverage environments can get in the `test_environment_variables` key. - -```toml -[tox] -test_environment_variables = """ - PIP_EXTRA_INDEX_URL=https://my-pypi.my-server.com/ -""" -``` - -For packages that have `plone.app.robotframework` based tests, it automatically detects it and primes [Playwright](https://playwright.dev/) to install the needed browsers. - - -### Manage multiple repositories with `multi-call` - -The `config-package` command runs only on a single repository. -To update multiple repositories at once, you can use the command `multi-call`. -It runs on all repositories listed in a `packages.txt` file. - -To run `multi-call` on all packages listed in a `packages.txt` file use the following command with positional arguments as shown. - -- The file path the Python script to be called. -- The file path to `packages.txt`, which lists repositories of packages on which the Python script will be called. -- The file path to the directory where the clones of the repositories are stored. -- Arguments to pass into the Python script. - -```shell -bin/multi-call -``` +- Volto or other JavaScript-based projects +- Monorepo projects (backend + frontend) +- Projects supporting multiple Plone versions in the same branch -The script performs the following steps for each line in the given `packages.txt` that does not start with a hash mark (`#`). -1. Check if there is a repository in `` with the name of the repository. - If it does not exist, then clone it. - If it exists, then clean the clone from changes, switch to the `master` branch, and pull from the origin. -2. Call the given script with the package name and arguments for the script. +## License -> [!CAUTION] -> Running this script stashes any uncommitted changes in the repositories. -> Run `git stash pop` to recover them. - - -### Re-enable GitHub Actions with `re-enable-actions` - -After a certain period of time without commits to a repository, GitHub automatically disables Actions. -You can re-enable them manually per repository. -`re-enable-actions` can do this for all repositories. -It does no harm if Actions are already enabled for a repository. - - -#### Setup GitHub CLI - -- Install [GitHub's CLI application](https://github.com/cli/cli). -- Authorize using the application: - - `gh auth login` - - It is probably enough to do it once. - - -#### `re-enable-actions` usage - -Use the following command. - -```shell -venv/bin/re-enable-actions -``` - - -## Explanation - -This section provides explanation of design decisions and capabilities of `plone.meta`. - - -### Project management - -By using `plone.meta`, you can have the same developer experience (DX) -across all Plone related packages. - -The idea is to make it mandatory for repositories under the [GitHub Plone organization](https://github.com/plone), -while encouraging its adoption for the repositories under the [collective Plone organization](https://github.com/collective) -and even your own private packages for your customers. - -With this configuration in place, -any developer has the answer to the following questions at their fingertips: - -- Do the tests of this package pass? -- What's the coverage of the test suite? -- Is the package ready to be released? -- Are all dependencies clearly defined? -- What does the dependency graph look like? -- Are there any circular dependency problems? -- Is the code formatted to some agreed upon standards? -- Do all agreed upon code quality checks pass? - -To find the answers to these questions, you can run the following commands. - -```shell -# run the test suite -tox -e test -# get a coverage report -tox -e coverage -# check if the package is ready to be released -tox -e release-check -# check if the dependencies are all specified -tox -e dependencies -# generate a dependency graph -tox -e dependencies-graph -# check if there are circular dependencies -tox -e circular -# format the code -tox -e format -# run all sorts of QA tools (code smells, typo finder...) -tox -e lint -``` - -As seen above, [`tox`](https://pypi.org/project/tox) provides the answers. - -Tooling is like fashion, it keeps evolving and changing. - -The great power behind `plone.meta` is that when we implement a better solution or tool, -we can swiftly move all packages to the new approach, making it as painless as possible! - - -### Configuration philosophy - -It is one thing to standardize, yet another to be flexible enough to adapt to each repository's particular needs. - -Fortunately `plone.meta` tries its best to accomplish both: - -- it provides sane defaults -- it allows extension of the defaults with custom configuration - -The configuration files have comments all over them -with instructions on how to extend, modify, and influence -what `plone.meta` ends up adding on those files. - -Those options are to be stored, -as it is mentioned on the comments themselves, -in `.meta.toml`. - -This way, when the configuration files get regenerated, -`plone.meta` reads the configuration in `.meta.toml` -and reapplies the specific configuration on the other files. - -See the specific configuration files sections below on how to extend and modify each configuration file. - -The idea behind the configuration system -in `plone.meta` controlled configuration files is to make it as simple as possible. - -Rather than adding plenty of configuration options, -almost all configuration files have an `extra_lines` section -that allows you to paste as much configuration as you want there. - -In this way, it provides a simple, yet powerful, extension mechanism. - -There are a few, and growing, other configuration options in a few files, -where the simple approach described above is not enough. - -## GitHub Actions shared automations - -To [avoid duplicating workflow content](https://docs.github.com/en/actions/sharing-automations/avoiding-duplication) across Plone projects, the [`plone/meta` repository](https://github.com/plone/meta) provides a set of reusable workflows and composite actions. - -These actions and workflows are used by code bases generated with [`Cookieplone`](https://github.com/plone/cookieplone) and [`cookieplone-templates`](https://github.com/plone/cookieplone-templates). - -## Composite actions - -Composite actions combine commonly used steps into reusable units that simplify writing GitHub workflows. - -### `setup_backend_uv` - -The action `setup_backend_uv` sets up and installs a backend code base using [uv](https://github.com/astral-sh/uv). -It also handles package caching using [`actions/cache`](https://github.com/actions/cache). - -This action assumes: - -* The codebase uses uv for dependency management. -* A `Makefile` provides an `install` target. -* The `Makefile` supports overriding `PYTHON_VERSION` and `PLONE_VERSION` via environment variables. - -#### Inputs - -* `python-version`: Python version to use. -* `plone-version`: Plone version to use. -* `working-directory`: Path to the backend code base. - -#### Example usage - -```yaml -- name: Setup backend codebase - uses: plone/meta/.github/actions/setup_backend_uv@2.x - with: - python-version: 3.12 - plone-version: 6.1.1 - working-directory: backend -``` - -### `setup_frontend` - -The action `setup_frontend` sets up and installs a frontend code base using [pnpm](https://pnpm.io/). -It also handles package caching using [`actions/cache`](https://github.com/actions/cache). - -This action assumes: - -* The code base uses pnpm for dependency management and was generated with [Cookieplone](https://github.com/plone/cookieplone). -* A `Makefile` provides an `install` target. - -#### Inputs - -* `node-version`: Node.js version to use. -* `working-directory`: Path to the frontend codebase. - -#### Example usage - -```yaml -- name: Setup frontend codebase - uses: plone/meta/.github/actions/setup_frontend@2.x - with: - node-version: 22.x - working-directory: frontend -``` - -### `setup_uv` - -The action `setup_uv` sets up [uv](https://github.com/astral-sh/uv) and configures package caching. - -This action assumes: - -* The code base uses uv for dependency management. - -#### Inputs - -* `python-version`: Python version to use. -* `working-directory`: Path to the Python code base. - -#### Example usage - -```yaml -- name: Set up uv - uses: plone/meta/.github/actions/setup_uv@2.x - with: - python-version: 3.12 - working-directory: docs -``` - -## Reusable workflows - -Reusable workflows automate testing, building, documentation, and deployment tasks. -They are grouped into: - -- **Backend workflows**: Python-based projects for Plone -- **Frontend workflows**: Node.js-based projects for Plone -- **Documentation workflows**: Building and validating documentation -- **Container workflows**: Building and publishing Docker images - -Each workflow assumes the use of uv for Python-based projects or pnpm for Node.js-based projects for dependency management and standard Makefile targets. - -### Backend workflows - -The following sections describe the workflows for the Plone backend. - - -#### `backend-lint` - -The workflow `backend-lint` lints and performs static analysis on a backend code base, including formatting, XML/ZCML validation, package metadata checking, Python version checking, and optional type checking. - -##### Inputs - -* `python-version`: Python version to use. Required. -* `plone-version`: Plone version to use. Required. -* `working-directory`: Path to the backend code base. Defaults to `.`. -* `check-typing`: Whether to run static typing checks using `mypy`. Defaults to `false`. -* `version-ruff`: Version of [ruff](https://github.com/astral-sh/ruff) to use. Defaults to `latest`. -* `version-zpretty`: Version of [zpretty](https://github.com/collective/zpretty) to use. Defaults to `latest`. -* `version-pyroma`: Version of [pyroma](https://github.com/regebro/pyroma) to use. Defaults to `latest`. -* `version-check-python`: Version of [check-python-versions](https://pypi.org/project/check-python-versions/) to use. Defaults to `latest`. - -##### Example usage - -```yaml -jobs: - lint: - name: "Backend: Lint" - uses: plone/meta/.github/workflows/backend-lint.yml@2.x - with: - python-version: 3.12 - plone-version: 6.1.1 - working-directory: backend -``` - -#### `backend-pytest` - -The workflow `backend-pytest` runs backend tests using [pytest](https://docs.pytest.org/en/stable/). - -##### Inputs - -* `python-version`: Python version to use. Required. -* `plone-version`: Plone version to use. Required. -* `working-directory`: Path to the backend code base. Defaults to `.`. - -##### Example usage - -```yaml -jobs: - test: - name: "Backend: Test" - uses: plone/meta/.github/workflows/backend-pytest.yml@2.x - with: - python-version: 3.12 - plone-version: 6.1.1 - working-directory: backend -``` - -#### `backend-pytest-coverage` - -The workflow `backend-pytest-coverage` runs backend tests and generates a coverage report with [coverage.py](https://coverage.readthedocs.io/en/latest/). - -##### Inputs - -* `python-version`: Python version to use. Required. -* `plone-version`: Plone version to use. Required. -* `working-directory`: Path to the backend code base. Defaults to `.`. - -##### Example usage - -```yaml -jobs: - test: - name: "Backend: Coverage" - uses: plone/meta/.github/workflows/backend-pytest-coverage.yml@2.x - with: - python-version: 3.12 - plone-version: 6.1.1 - working-directory: backend -``` - -### Documentation workflows - -The following sections describe the workflows for documentation. - - -#### `docs-build` - -The workflow `docs-build` builds HTML documentation, checking for MyST syntax errors, and checks for broken links. -It also checks American English spelling, grammar, and syntax, and checks for compliance with the [Microsoft Writing Style Guide](https://learn.microsoft.com/en-us/style-guide/welcome/) using [Vale](https://vale.sh/). - -##### Inputs - -* `python-version`: Python version to use. Defaults to `3.12`. -* `working-directory`: Path to the documentation scaffold. Defaults to `.`. -* `check-links`: Whether to run broken link checks. Defaults to `true`. -* `check-vale`: Whether to run Vale checks. Defaults to `true`. - -##### Example usage - -```yaml -jobs: - docs: - name: "Documentation" - uses: plone/meta/.github/workflows/docs-build.yml@2.x - with: - python-version: 3.12 - working-directory: docs - check-links: true - check-vale: true -``` - - -### Frontend workflows - -The following sections describe the workflows for the Plone frontend. - - -#### `frontend-acceptance` - -The workflow `frontend-acceptance` runs frontend acceptance tests using [Cypress](https://www.cypress.io/), including server startup and artifact uploads with screenshots and videos. - -##### Inputs - -* `node-version`: Node.js version to use. Required. -* `working-directory`: Path to the frontend code base. Defaults to `.`. - -##### Example usage - -```yaml -jobs: - acceptance: - name: "Frontend: Acceptance Tests" - uses: plone/meta/.github/workflows/frontend-acceptance.yml@2.x - with: - node-version: 22.x - working-directory: frontend -``` - -#### `frontend-code` - -The workflow `frontend-code` runs static analysis, or linting, on a frontend code base. - -##### Inputs - -* `node-version`: Node.js version to use. Required. -* `working-directory`: Path to the frontend code base. Defaults to `.`. - -##### Example usage - -```yaml -jobs: - qa: - name: "Frontend: Code Analysis" - uses: plone/meta/.github/workflows/frontend-code.yml@2.x - with: - node-version: 22.x - working-directory: frontend -``` - -#### `frontend-i18n` - -The workflow `frontend-i18n` validates that frontend translation files are up to date. - -##### Inputs - -* `node-version`: Node.js version to use. Required. -* `working-directory`: Path to the frontend code base. Defaults to `.`. - -##### Example usage - -```yaml -jobs: - i18n: - name: "Frontend: i18n Checks" - uses: plone/meta/.github/workflows/frontend-i18n.yml@2.x - with: - node-version: 22.x - working-directory: frontend -``` - -#### `frontend-storybook` - -The workflow `frontend-storybook` builds Storybook documentation and optionally deploys it to GitHub Pages. - -##### Inputs - -* `node-version`: Node.js version to use. Required. -* `working-directory`: Path to the frontend code base. Defaults to `.`. -* `deploy`: Whether to deploy the Storybook build. Defaults to `false`. - -##### Example usage - -```yaml -jobs: - storybook: - name: "Frontend: Storybook" - uses: plone/meta/.github/workflows/frontend-storybook.yml@2.x - with: - node-version: 22.x - working-directory: frontend - deploy: true -``` - -#### `frontend-unit` - -The workflow `frontend-unit` runs unit tests on a frontend code base. - -##### Inputs - -* `node-version`: Node.js version to use. Required. -* `working-directory`: Path to the frontend code base. Defaults to `.`. - -##### Example usage - -```yaml -jobs: - unit: - name: "Frontend: Unit Tests" - uses: plone/meta/.github/workflows/frontend-unit.yml@2.x - with: - node-version: 22.x - working-directory: frontend -``` - -### Container workflows - -The following sections describe the workflows for Plone container images. - - -#### `container-image-build-push` - -The workflow `container-image-build-push` builds and optionally pushes a container image for use in tests or deployments. - -##### Inputs - -* `base-tag`: Base tag for the image, for example, `1.0.0`. Required. -* `working-directory`: Path to the project directory containing the Dockerfile. Defaults to `.`. -* `image-name-prefix`: Image name prefix, for example, the organization or repository name. Required. -* `image-name-suffix`: Image name suffix, for example, the service identifier. Required. -* `platforms`: Platforms for which to build the image. Defaults to `linux/amd64`. -* `dockerfile`: Relative path to the Dockerfile. Defaults to `Dockerfile`. -* `registry`: Container registry URL. Defaults to `ghcr.io`. -* `build-args`: Additional build arguments. Defaults to `""`. -* `push`: Whether to push the built image. Defaults to `true`. - -##### Secrets - -* `username`: Registry username. Required. -* `password`: Registry password. Required. - -##### Example usage - -```yaml -jobs: - build-image: - name: "Container: Build and Push" - uses: plone/meta/.github/workflows/container-image-build-push.yml@2.x - with: - base-tag: 1.0.0 - working-directory: frontend - image-name-prefix: ghcr.io/collective/collective-addon - image-name-suffix: frontend - platforms: linux/amd64 - dockerfile: Dockerfile - registry: ghcr.io - build-args: | - VOLTO_VERSION=18.14.1 - push: true - secrets: - username: ${{ secrets.REGISTRY_USERNAME }} - password: ${{ secrets.REGISTRY_PASSWORD }} -``` - -#### `container-image-build` - -The workflow `container-image-build` builds a container image optimized for caching, typically used for internal tests, for example, acceptance tests. - -##### Inputs - -* `base-tag`: Base tag for the image. Required. -* `working-directory`: Path to the project directory containing the Dockerfile. Defaults to `.`. -* `image-name-prefix`: Image name prefix. Required. -* `image-name-suffix`: Image name suffix. Required. -* `image-cache-suffix`: Suffix for naming cache images. Defaults to `buildcache`. -* `platforms`: Platforms for which to build the image. Defaults to `linux/amd64`. -* `dockerfile`: Relative path to the Dockerfile. Defaults to `Dockerfile`. -* `registry`: Container registry URL. Defaults to `ghcr.io`. -* `build-args`: Additional build arguments. Defaults to `""`. -* `cache-key`: Cache identification key. Defaults to the branch name, for example, `github.ref_name`. - -##### Secrets - -* `username`: Registry username. Required. -* `password`: Registry password. Required. - -##### Example usage - -```yaml -jobs: - build-image: - name: "Container: Build Only" - uses: plone/meta/.github/workflows/container-image-build.yml@2.x - with: - base-tag: branch-x - working-directory: backend - image-name-prefix: ghcr.io/collective/collective-addon - image-name-suffix: backend - image-cache-suffix: buildcache - platforms: linux/amd64 - dockerfile: Dockerfile - registry: ghcr.io - build-args: | - PLONE_VERSION=6.1.1 - cache-key: branch-x - secrets: - username: ${{ secrets.REGISTRY_USERNAME }} - password: ${{ secrets.REGISTRY_PASSWORD }} -``` - -#### `container-image-push` - -The workflow `container-image-push` builds and pushes a container image to a registry, generating multiple tags based on branch, commit, and semantic versioning. - -##### Inputs - -* `base-tag`: Base tag for the image. Required. -* `working-directory`: Path to the project directory containing the Dockerfile. Defaults to `.`. -* `image-name-prefix`: Image name prefix. Required. -* `image-name-suffix`: Image name suffix. Required. -* `image-cache-suffix`: Suffix for naming cache images. Defaults to `buildcache`. -* `platforms`: Platforms for which to build the image. Defaults to `linux/amd64`. -* `dockerfile`: Relative path to the Dockerfile. Defaults to `Dockerfile`. -* `registry`: Container registry URL. Defaults to `ghcr.io`. -* `build-args`: Additional build arguments. Defaults to `""`. -* `cache-key`: Cache identification key. Defaults to the branch name, for example, `github.ref_name`. - -##### Secrets - -* `username`: Registry username. Required. -* `password`: Registry password. Required. - -##### Example usage - -```yaml -jobs: - publish-image: - name: "Container: Build and Push" - uses: plone/meta/.github/workflows/container-image-push.yml@2.x - with: - base-tag: branch-x - working-directory: backend - image-name-prefix: ghcr.io/collective/collective-addon - image-name-suffix: backend - image-cache-suffix: buildcache - platforms: linux/amd64 - dockerfile: Dockerfile - registry: ghcr.io - build-args: | - PLONE_VERSION=6.1.1 - cache-key: branch-x - secrets: - username: ${{ secrets.REGISTRY_USERNAME }} - password: ${{ secrets.REGISTRY_PASSWORD }} -``` +GPLv2. See [LICENSE](LICENSE). diff --git a/docs/.gitignore b/docs/.gitignore new file mode 100644 index 00000000..2a33b725 --- /dev/null +++ b/docs/.gitignore @@ -0,0 +1,3 @@ +/.mxmake +/.venv +/html diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 00000000..a6206e48 --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,310 @@ +############################################################################## +# THIS FILE IS GENERATED BY MXMAKE +# +# DOMAINS: +#: core.base +#: core.mxenv +#: docs.sphinx +# +# SETTINGS (ALL CHANGES MADE BELOW SETTINGS WILL BE LOST) +############################################################################## + +## core.base + +# `deploy` target dependencies. +# No default value. +DEPLOY_TARGETS?= + +# target to be executed when calling `make run` +# No default value. +RUN_TARGET?= + +# Additional files and folders to remove when running clean target +# No default value. +CLEAN_FS?= + +# Optional makefile to include before default targets. This can +# be used to provide custom targets or hook up to existing targets. +# Default: include.mk +INCLUDE_MAKEFILE?=include.mk + +# Optional additional directories to be added to PATH in format +# `/path/to/dir/:/path/to/other/dir`. Gets inserted first, thus gets searched +# first. +# No default value. +EXTRA_PATH?= + +## core.mxenv + +# Primary Python interpreter to use. It is used to create the +# virtual environment if `VENV_ENABLED` and `VENV_CREATE` are set to `true`. +# If global `uv` is used, this value is passed as `--python VALUE` to the venv creation. +# uv then downloads the Python interpreter if it is not available. +# for more on this feature read the [uv python documentation](https://docs.astral.sh/uv/concepts/python-versions/) +# Default: python3 +PRIMARY_PYTHON?=3.13 + +# Minimum required Python version. +# Default: 3.9 +PYTHON_MIN_VERSION?=3.13 + +# Install packages using the given package installer method. +# Supported are `pip` and `uv`. If uv is used, its global availability is +# checked. Otherwise, it is installed, either in the virtual environment or +# using the `PRIMARY_PYTHON`, dependent on the `VENV_ENABLED` setting. If +# `VENV_ENABLED` and uv is selected, uv is used to create the virtual +# environment. +# Default: pip +PYTHON_PACKAGE_INSTALLER?=uv + +# Flag whether to use a global installed 'uv' or install +# it in the virtual environment. +# Default: false +MXENV_UV_GLOBAL?=true + +# Flag whether to use virtual environment. If `false`, the +# interpreter according to `PRIMARY_PYTHON` found in `PATH` is used. +# Default: true +VENV_ENABLED?=true + +# Flag whether to create a virtual environment. If set to `false` +# and `VENV_ENABLED` is `true`, `VENV_FOLDER` is expected to point to an +# existing virtual environment. +# Default: true +VENV_CREATE?=true + +# The folder of the virtual environment. +# If `VENV_ENABLED` is `true` and `VENV_CREATE` is true it is used as the +# target folder for the virtual environment. If `VENV_ENABLED` is `true` and +# `VENV_CREATE` is false it is expected to point to an existing virtual +# environment. If `VENV_ENABLED` is `false` it is ignored. +# Default: .venv +VENV_FOLDER?=.venv + +# mxdev to install in virtual environment. +# Default: mxdev +MXDEV?=mxdev + +# mxmake to install in virtual environment. +# Default: mxmake +MXMAKE?=mxmake + +## docs.sphinx + +# Documentation source folder. +# Default: docs/source +DOCS_SOURCE_FOLDER?=sources + +# Documentation generation target folder. +# Default: docs/html +DOCS_TARGET_FOLDER?=html + +# Documentation linkcheck output folder. +# Default: docs/linkcheck +DOCS_LINKCHECK_FOLDER?=docs/linkcheck + +# Documentation Python requirements to be installed (via pip). +# No default value. +DOCS_REQUIREMENTS?=myst_parser plone-sphinx-theme sphinx-design sphinx-copybutton + +############################################################################## +# END SETTINGS - DO NOT EDIT BELOW THIS LINE +############################################################################## + +INSTALL_TARGETS?= +DIRTY_TARGETS?= +CLEAN_TARGETS?= +PURGE_TARGETS?= + +export PATH:=$(if $(EXTRA_PATH),$(EXTRA_PATH):,)$(PATH) + +# Defensive settings for make: https://tech.davis-hansson.com/p/make/ +SHELL:=bash +.ONESHELL: +# for Makefile debugging purposes add -x to the .SHELLFLAGS +.SHELLFLAGS:=-eu -o pipefail -O inherit_errexit -c +.SILENT: +.DELETE_ON_ERROR: +MAKEFLAGS+=--warn-undefined-variables +MAKEFLAGS+=--no-builtin-rules + +# mxmake folder +MXMAKE_FOLDER?=.mxmake + +# Sentinel files +SENTINEL_FOLDER?=$(MXMAKE_FOLDER)/sentinels +SENTINEL?=$(SENTINEL_FOLDER)/about.txt +$(SENTINEL): $(firstword $(MAKEFILE_LIST)) + @mkdir -p $(SENTINEL_FOLDER) + @echo "Sentinels for the Makefile process." > $(SENTINEL) + +############################################################################## +# mxenv +############################################################################## + +OS?= + +# Determine the executable path +ifeq ("$(VENV_ENABLED)", "true") +export VIRTUAL_ENV=$(abspath $(VENV_FOLDER)) +ifeq ("$(OS)", "Windows_NT") +VENV_EXECUTABLE_FOLDER=$(VIRTUAL_ENV)/Scripts +else +VENV_EXECUTABLE_FOLDER=$(VIRTUAL_ENV)/bin +endif +export PATH:=$(VENV_EXECUTABLE_FOLDER):$(PATH) +MXENV_PYTHON=python +else +MXENV_PYTHON=$(PRIMARY_PYTHON) +endif + +# Determine the package installer +ifeq ("$(PYTHON_PACKAGE_INSTALLER)","uv") +PYTHON_PACKAGE_COMMAND=uv pip +else +PYTHON_PACKAGE_COMMAND=$(MXENV_PYTHON) -m pip +endif + +MXENV_TARGET:=$(SENTINEL_FOLDER)/mxenv.sentinel +$(MXENV_TARGET): $(SENTINEL) +ifneq ("$(PYTHON_PACKAGE_INSTALLER)$(MXENV_UV_GLOBAL)","uvfalse") + @$(PRIMARY_PYTHON) -c "import sys; vi = sys.version_info; sys.exit(1 if (int(vi[0]), int(vi[1])) >= tuple(map(int, '$(PYTHON_MIN_VERSION)'.split('.'))) else 0)" \ + && echo "Need Python >= $(PYTHON_MIN_VERSION)" && exit 1 || : +else + @echo "Use Python $(PYTHON_MIN_VERSION) over uv" +endif + @[[ "$(VENV_ENABLED)" == "true" && "$(VENV_FOLDER)" == "" ]] \ + && echo "VENV_FOLDER must be configured if VENV_ENABLED is true" && exit 1 || : + @[[ "$(VENV_ENABLED)$(PYTHON_PACKAGE_INSTALLER)" == "falseuv" ]] \ + && echo "Package installer uv does not work with a global Python interpreter." && exit 1 || : +ifeq ("$(VENV_ENABLED)", "true") +ifeq ("$(VENV_CREATE)", "true") +ifeq ("$(PYTHON_PACKAGE_INSTALLER)$(MXENV_UV_GLOBAL)","uvtrue") + @echo "Setup Python Virtual Environment using package 'uv' at '$(VENV_FOLDER)'" + @uv venv -p $(PRIMARY_PYTHON) --seed $(VENV_FOLDER) +else + @echo "Setup Python Virtual Environment using module 'venv' at '$(VENV_FOLDER)'" + @$(PRIMARY_PYTHON) -m venv $(VENV_FOLDER) + @$(MXENV_PYTHON) -m ensurepip -U +endif +endif +else + @echo "Using system Python interpreter" +endif +ifeq ("$(PYTHON_PACKAGE_INSTALLER)$(MXENV_UV_GLOBAL)","uvfalse") + @echo "Install uv" + @$(MXENV_PYTHON) -m pip install uv +endif + @$(PYTHON_PACKAGE_COMMAND) install -U pip setuptools wheel + @echo "Install/Update MXStack Python packages" + @$(PYTHON_PACKAGE_COMMAND) install -U $(MXDEV) $(MXMAKE) + @touch $(MXENV_TARGET) + +.PHONY: mxenv +mxenv: $(MXENV_TARGET) + +.PHONY: mxenv-dirty +mxenv-dirty: + @rm -f $(MXENV_TARGET) + +.PHONY: mxenv-clean +mxenv-clean: mxenv-dirty +ifeq ("$(VENV_ENABLED)", "true") +ifeq ("$(VENV_CREATE)", "true") + @rm -rf $(VENV_FOLDER) +endif +else + @$(PYTHON_PACKAGE_COMMAND) uninstall -y $(MXDEV) + @$(PYTHON_PACKAGE_COMMAND) uninstall -y $(MXMAKE) +endif + +INSTALL_TARGETS+=mxenv +DIRTY_TARGETS+=mxenv-dirty +CLEAN_TARGETS+=mxenv-clean + +############################################################################## +# sphinx +############################################################################## + +# additional targets required for building docs. +DOCS_TARGETS+= + +SPHINX_BIN=sphinx-build +SPHINX_AUTOBUILD_BIN=sphinx-autobuild + +DOCS_TARGET:=$(SENTINEL_FOLDER)/sphinx.sentinel +$(DOCS_TARGET): $(MXENV_TARGET) + @echo "Install Sphinx" + @$(PYTHON_PACKAGE_COMMAND) install -U sphinx sphinx-autobuild $(DOCS_REQUIREMENTS) + @touch $(DOCS_TARGET) + +.PHONY: docs +docs: $(DOCS_TARGET) $(DOCS_TARGETS) + @echo "Build sphinx docs" + @$(SPHINX_BIN) $(DOCS_SOURCE_FOLDER) $(DOCS_TARGET_FOLDER) + +.PHONY: docs-live +docs-live: $(DOCS_TARGET) $(DOCS_TARGETS) + @echo "Rebuild Sphinx documentation on changes, with live-reload in the browser" + @$(SPHINX_AUTOBUILD_BIN) $(DOCS_SOURCE_FOLDER) $(DOCS_TARGET_FOLDER) + +.PHONY: docs-linkcheck +docs-linkcheck: $(DOCS_TARGET) $(DOCS_TARGETS) + @echo "Run Sphinx linkcheck" + @$(SPHINX_BIN) -b linkcheck $(DOCS_SOURCE_FOLDER) $(DOCS_LINKCHECK_FOLDER) + +.PHONY: docs-dirty +docs-dirty: + @rm -f $(DOCS_TARGET) + +.PHONY: docs-clean +docs-clean: docs-dirty + @test -e $(MXENV_PYTHON) && $(MXENV_PYTHON) -m pip uninstall -y \ + sphinx sphinx-autobuild $(DOCS_REQUIREMENTS) || : + @rm -rf $(DOCS_TARGET_FOLDER) + +INSTALL_TARGETS+=$(DOCS_TARGET) +DIRTY_TARGETS+=docs-dirty +CLEAN_TARGETS+=docs-clean + +############################################################################## +# Custom includes +############################################################################## + +-include $(INCLUDE_MAKEFILE) + +############################################################################## +# Default targets +############################################################################## + +INSTALL_TARGET:=$(SENTINEL_FOLDER)/install.sentinel +$(INSTALL_TARGET): $(INSTALL_TARGETS) + @touch $(INSTALL_TARGET) + +.PHONY: install +install: $(INSTALL_TARGET) + @touch $(INSTALL_TARGET) + +.PHONY: run +run: $(RUN_TARGET) + +.PHONY: deploy +deploy: $(DEPLOY_TARGETS) + +.PHONY: dirty +dirty: $(DIRTY_TARGETS) + @rm -f $(INSTALL_TARGET) + +.PHONY: clean +clean: dirty $(CLEAN_TARGETS) + @rm -rf $(CLEAN_TARGETS) $(MXMAKE_FOLDER) $(CLEAN_FS) + +.PHONY: purge +purge: clean $(PURGE_TARGETS) + +.PHONY: runtime-clean +runtime-clean: + @echo "Remove runtime artifacts, like byte-code and caches." + @find . -name '*.py[c|o]' -delete + @find . -name '*~' -exec rm -f {} + + @find . -name '__pycache__' -exec rm -fr {} + diff --git a/docs/sources/conf.py b/docs/sources/conf.py new file mode 100644 index 00000000..5531db2f --- /dev/null +++ b/docs/sources/conf.py @@ -0,0 +1,54 @@ +# Configuration file for the Sphinx documentation builder. + +# -- Project information ----------------------------------------------------- + +project = "plone.meta" +copyright = "2023-2026, Plone Foundation" # noqa: A001 +author = "Plone Foundation" +release = "2.5.1" + +# -- General configuration --------------------------------------------------- + +extensions = [ + "myst_parser", + "sphinx_design", + "sphinx_copybutton", +] + +myst_enable_extensions = [ + "deflist", + "colon_fence", + "fieldlist", +] + +exclude_patterns = [] + +# -- Options for HTML output ------------------------------------------------- + +html_theme = "plone_sphinx_theme" + +html_theme_options = { + "icon_links": [ + { + "name": "GitHub", + "url": "https://github.com/plone/meta", + "icon": "fa-brands fa-github", + }, + { + "name": "PyPI", + "url": "https://pypi.org/project/plone.meta/", + "icon": "fa-brands fa-python", + }, + ], + "use_edit_page_button": True, + "navigation_with_keys": True, +} + +html_context = { + "github_user": "plone", + "github_repo": "meta", + "github_version": "2.x", + "doc_path": "docs/sources", +} + +html_extra_path = ["llms.txt"] diff --git a/docs/sources/explanation/architecture.md b/docs/sources/explanation/architecture.md new file mode 100644 index 00000000..179d04ba --- /dev/null +++ b/docs/sources/explanation/architecture.md @@ -0,0 +1,113 @@ +--- +myst: + html_meta: + "description": "Architecture of plone.meta: templates, rendering pipeline, and validation" + "property=og:description": "Architecture of plone.meta: templates, rendering pipeline, and validation" + "property=og:title": "Architecture" + "keywords": "plone.meta, architecture, Jinja2, templates" +--- + +# Architecture + + + +## Overview + +plone.meta is a code generation tool. +It reads per-repository configuration from {file}`.meta.toml`, renders Jinja2 templates, validates the output, and manages git operations. + +## Components + +### Template engine + +The core of plone.meta is a set of Jinja2 templates stored in +{file}`src/plone/meta/default/`. +Each template corresponds to a configuration file in the target repository. + +Templates use `%(variable)s` style placeholders (Python string formatting syntax within the Jinja2 context) for inserting configuration values. +The template engine has `trim_blocks` and `lstrip_blocks` enabled for clean output, and `keep_trailing_newline` preserves proper file endings. + +#### Modular tox templates + +The {file}`tox.ini.j2` template uses a modular architecture with Jinja2 `{% include %}` directives. +Rather than a single monolithic template, the tox configuration is composed from focused sub-templates: + +- {file}`tox-init.j2` -- tox initialization and configuration header +- {file}`tox-base.j2` -- base test environment definition +- {file}`tox-test-runner-specifics.j2` -- test runner specific settings +- {file}`tox-test-coverage.j2` -- coverage environment configuration +- {file}`tox-qa.j2` -- linting and formatting environments +- {file}`tox-plone-depending-qa.j2` -- Plone-specific QA environments + +This modular structure makes the templates easier to maintain and extend. + +### PackageConfiguration class + +The {file}`config_package.py` module contains the `PackageConfiguration` class, which orchestrates the entire process: + +1. **Read** {file}`.meta.toml` from the target repository +2. **Detect** whether the repository is GitHub or GitLab hosted + (based on the git remote URL) +3. **Render** each template with the configuration values +4. **Validate** generated files (TOML via `tomlkit`, YAML via `pyyaml`, + INI via `configparser`, editorconfig via the `editorconfig` library) +5. **Write** files with meta hint comments +6. **Create** a towncrier news entry +7. **Commit** and optionally push + +### File validation + +Every generated file is validated before being written: + +- **TOML files** are parsed with `tomlkit` and additionally validated + with `validate-pyproject` for {file}`pyproject.toml` +- **YAML files** are parsed with `pyyaml` +- **INI files** are parsed with `configparser` +- **editorconfig files** are parsed with the `editorconfig` library + +If validation fails, you are prompted to proceed or abort. + +### Git workflow + +`config-package` manages git operations: + +1. Creates a new branch (or checks out an existing one) +2. Commits all changes with a descriptive message +3. Optionally pushes to the remote + +Branch names follow the pattern `config-with--template-`, where the commit hash refers to the plone.meta repository, making it clear which version of the templates was used. + +## GitHub Actions workflow architecture + +Rather than generating complete CI workflows inline, plone.meta generates a thin {file}`meta.yml` that uses GitHub's `workflow_call` to reference reusable workflows stored in the plone/meta repository itself. +This means: + +- Workflow logic is maintained in one place +- Repositories only need a small dispatch file +- Updates to CI logic do not require re-running `config-package` + (as long as the `ref` points to a branch like `2.x`) + +## Test matrix + +A key architectural feature of plone.meta 2.x is the test matrix. +Rather than testing against a single Python version, plone.meta generates test environments for all combinations of Plone versions and Python versions. +The default matrix covers Plone 6.0, 6.1, and 6.2 across Python 3.10 through 3.14. + +The test matrix is reflected in multiple generated outputs: + +- **tox.ini**: Environments named `py-plone` (e.g., + `py314-plone62`, `py312-plone61`) +- **GitHub Actions**: A dedicated {file}`test-matrix.yml` workflow runs + the upper and lower python version for each Plone version in CI +- **GitLab CI**: Matrix jobs are generated in {file}`.gitlab-ci.yml` + +Each combination uses its own constraints file, allowing different Plone versions to pin different dependency versions. + +## Data flow + +``` +.meta.toml ──> PackageConfiguration ──> Jinja2 templates ──> Rendered files + │ │ + │ v + └── git branch ──> commit ──> (push) Validation +``` diff --git a/docs/sources/explanation/configuration-philosophy.md b/docs/sources/explanation/configuration-philosophy.md new file mode 100644 index 00000000..f5f49647 --- /dev/null +++ b/docs/sources/explanation/configuration-philosophy.md @@ -0,0 +1,54 @@ +--- +myst: + html_meta: + "description": "Configuration philosophy: sane defaults with simple extensibility" + "property=og:description": "Configuration philosophy: sane defaults with simple extensibility" + "property=og:title": "Configuration Philosophy" + "keywords": "plone.meta, configuration, philosophy, defaults" +--- + +# Configuration Philosophy + + + +## Sane defaults with simple extensibility + +plone.meta is designed around two principles: + +1. **Provide sane defaults** that work for the majority of Plone packages. +2. **Allow extension** without complexity. + +## The extra_lines approach + +Rather than adding a configuration option for every possible customization, almost all generated files support an `extra_lines` key in {file}`.meta.toml`. +This key accepts raw configuration text that is appended to the generated file. + +This approach has several advantages: + +- **Simplicity:** No need to learn a custom configuration DSL. You write + the same syntax the target file expects. +- **Flexibility:** Anything the target format supports can be added. +- **Low maintenance:** New configuration options in tools (flake8, tox, + isort, etc.) do not require changes to plone.meta itself. + +The tradeoff is that `extra_lines` content is not validated against the template -- it is simply appended. +But this is intentional: it keeps the tool simple and avoids becoming a configuration management framework. + +## When extra_lines is not enough + +For cases where appending text is insufficient -- where the configuration needs to *change* the template output rather than extend it -- plone.meta provides specific configuration keys. Examples: + +- `test_runner` to switch between pytest and zope.testrunner +- `constraints_file` to override the default pip constraints URL +- `custom_image` to change the GitLab CI Docker image +- `jobs` to select which CI jobs are enabled + +These options are added conservatively. +If you need a customization that plone.meta does not support, you can [request it](https://github.com/plone/meta/issues/new). + +## Don't edit generated files + +This is the cardinal rule of plone.meta: **never directly edit the files that plone.meta manages.** Any changes will be overwritten the next time `config-package` runs. + +All customization goes through {file}`.meta.toml`. +This single file captures the complete delta between the defaults and your repository's needs, making it easy to review, maintain, and update. diff --git a/docs/sources/explanation/index.md b/docs/sources/explanation/index.md new file mode 100644 index 00000000..94050d2a --- /dev/null +++ b/docs/sources/explanation/index.md @@ -0,0 +1,59 @@ +--- +myst: + html_meta: + "description": "Background knowledge, design decisions, and philosophy of plone.meta" + "property=og:description": "Background knowledge, design decisions, and philosophy of plone.meta" + "property=og:title": "Explanation" + "keywords": "plone.meta, explanation, design decisions" +--- + +# Explanation + + + +Background knowledge, design decisions, and project philosophy. +These pages help you understand *why* plone.meta works the way it does. + +::::{grid} 2 +:gutter: 3 + +:::{grid-item-card} Why Standardize? +:link: why-standardize +:link-type: doc + +The motivation behind plone.meta: what problems it solves, what it replaces, and why centralized configuration management matters for the Plone ecosystem. +::: + +:::{grid-item-card} Architecture +:link: architecture +:link-type: doc + +How plone.meta works internally: template rendering, file generation, validation, and the git workflow. +::: + +:::{grid-item-card} Configuration Philosophy +:link: configuration-philosophy +:link-type: doc + +The design principles behind the `extra_lines` approach: sane defaults with simple extensibility. +::: + +:::{grid-item-card} Scope and Limitations +:link: scope +:link-type: doc + +What plone.meta covers and what it intentionally does not cover. +Projects that should not use plone.meta. +::: + +:::: + +```{toctree} +--- +hidden: true +--- +why-standardize +architecture +configuration-philosophy +scope +``` diff --git a/docs/sources/explanation/scope.md b/docs/sources/explanation/scope.md new file mode 100644 index 00000000..b512ff55 --- /dev/null +++ b/docs/sources/explanation/scope.md @@ -0,0 +1,52 @@ +--- +myst: + html_meta: + "description": "Scope and limitations of plone.meta" + "property=og:description": "Scope and limitations of plone.meta" + "property=og:title": "Scope and Limitations" + "keywords": "plone.meta, scope, limitations, Python packages" +--- + +# Scope and Limitations + + + +## What plone.meta covers + +plone.meta manages configuration files for **single-package Python repositories** in the Plone ecosystem. +This includes: + +- Development tool configuration (linting, formatting, editor settings) +- Test environment setup (tox) +- CI/CD pipelines (GitHub Actions, GitLab CI) +- Pre-commit hooks +- Release readiness checks +- Dependency validation + +## What plone.meta does not cover + +### JavaScript and frontend projects + +Volto and other JavaScript-based projects have their own ecosystem with different tooling (ESLint, Prettier, Jest, etc.). +plone.meta is strictly for Python projects. + +### Monorepo projects + +Repositories with both backend and frontend code bases -- such as those created by [Cookieplone](https://github.com/plone/cookieplone) -- are not supported. +plone.meta expects a single Python package at the top level of the repository. + +### Multi-version support + +plone.meta allows testing against multiple Plone versions via the `test_matrix` configuration. +However, it does not support maintaining entirely separate dependency sets or build configurations for different major Plone versions within a single branch. + +### Package scaffolding + +plone.meta does not create new packages. +It configures existing ones. +For creating new Plone packages, use [Cookieplone](https://github.com/plone/cookieplone) or [plonecli](https://pypi.org/project/plonecli/). + +### Custom CI logic + +plone.meta generates standardized CI pipelines. +If your package needs significantly custom CI logic (custom Docker builds, deployment steps, integration tests with external services), you can extend via `extra_lines`, but at some point it may make more sense to maintain CI configuration manually. diff --git a/docs/sources/explanation/why-standardize.md b/docs/sources/explanation/why-standardize.md new file mode 100644 index 00000000..9bb02567 --- /dev/null +++ b/docs/sources/explanation/why-standardize.md @@ -0,0 +1,62 @@ +--- +myst: + html_meta: + "description": "Why standardize configuration across Plone packages" + "property=og:description": "Why standardize configuration across Plone packages" + "property=og:title": "Why Standardize?" + "keywords": "plone.meta, standardization, consistency, developer experience" +--- + +# Why Standardize? + + + +## The problem + +The Plone ecosystem consists of over 100 Python packages, each maintained by different people at different times. +Without coordination, each repository develops its own approach to: + +- Test configuration and runners +- Code formatting and linting rules +- CI pipeline setup +- Pre-commit hooks +- Editor settings +- Release processes + +This leads to fragmentation. +A developer picking up a new package has to figure out from scratch how to run tests, what formatting rules apply, and whether the CI pipeline is current. +Contributions become harder because the answer to "how do I run the tests?" differs for every package. + +## What plone.meta solves + +By using plone.meta, every Plone package answers the same set of questions in the same way: + +- **Do the tests pass?** `tox -e test` +- **What's the test coverage?** `tox -e coverage` +- **Is the package release-ready?** `tox -e release-check` +- **Are dependencies declared correctly?** `tox -e dependencies` +- **Are there circular dependencies?** `tox -e circular` +- **Is the code formatted?** `tox -e format` +- **Do quality checks pass?** `tox -e lint` + +The developer experience is identical everywhere. + +## Centralized updates + +Tooling evolves. +Linting rules change. +New Python versions are released. +CI providers update their APIs. +Without centralization, updating 100+ repositories is impractical -- each one needs manual changes, review, and testing. + +With plone.meta, a tooling update happens in one place (the templates), and is then rolled out to all packages via `config-package` or `multi-call`. +This makes it practical to: + +- Adopt new tools (e.g., switching formatters) +- Update CI pipelines for new Python versions +- Fix a security issue in a CI configuration + +## Adoption + +plone.meta is mandatory for repositories under the [GitHub Plone organization](https://github.com/plone). +It is encouraged for repositories under the [collective Plone organization](https://github.com/collective) and for private packages. diff --git a/docs/sources/how-to/configure-github-actions.md b/docs/sources/how-to/configure-github-actions.md new file mode 100644 index 00000000..38db4bd6 --- /dev/null +++ b/docs/sources/how-to/configure-github-actions.md @@ -0,0 +1,126 @@ +--- +myst: + html_meta: + "description": "Configure GitHub Actions workflows generated by plone.meta" + "property=og:description": "Configure GitHub Actions workflows generated by plone.meta" + "property=og:title": "Configure GitHub Actions" + "keywords": "plone.meta, GitHub Actions, CI/CD, test matrix" +--- + +# Configure GitHub Actions + + + +`plone.meta` generates a {file}`.github/workflows/meta.yml` file that references reusable workflows from the plone/meta repository. + +## Select CI jobs + +Choose which jobs to include in {file}`meta.yml`: + +```toml +[github] +jobs = [ + "qa", + "coverage", + "dependencies", + "release_ready", + "circular", +] +``` + +:::{note} +The `"test"` job is no longer included in the default jobs list. Testing is now handled by the separate {file}`test-matrix.yml` workflow, which is generated automatically when `use_test_matrix` is enabled (the default). +::: + +Available jobs: + +qa +: Runs `tox -e lint` for code quality checks. + +test +: Runs `tox -e test` with a single Python version. Not included by default; + use the test matrix workflow instead (see below). + +coverage +: Runs `tox -e coverage` and outputs a coverage report. + +dependencies +: Validates dependencies with `z3c.dependencychecker`. + +release_ready +: Checks if the package is ready for release. + +circular +: Detects circular dependencies. + +## Pin the workflow version + +By default, workflows reference the `2.x` branch of plone/meta. +Pin to a specific tag: + +```toml +[github] +ref = "v3" +``` + +## Set environment variables + +```toml +[github] +env = """ + debug: 1 + image-name: 'org/image' + image-tag: 'latest' +""" +``` + +## Install OS-level dependencies + +Specify Ubuntu package names: + +```toml +[github] +os_dependencies = "git libxml2 libxslt1-dev" +``` + +## Add lines after OS dependency installation + +Use `extra_lines_after_os_dependencies` to insert additional setup steps that run after the OS-level dependencies are installed but before tests: + +```toml +[github] +extra_lines_after_os_dependencies = """ + run: | + curl -sSL https://example.com/setup.sh | bash +""" +``` + +## Test matrix workflow + +When `use_test_matrix` is enabled (the default), plone.meta generates a separate {file}`test-matrix.yml` workflow that tests all combinations of Plone versions and Python versions. +This replaces the old single-version `test` job. + +The matrix is configured in the `[tox]` section: + +```toml +[tox] +use_test_matrix = true +test_matrix = {"6.2" = ["3.14", "3.13", "3.12", "3.11", "3.10"], "6.1" = ["3.13", "3.12", "3.11", "3.10"]} +``` + +Using `"*"` as shorthand includes all currently supported Python versions for a given Plone version: + +```toml +[tox] +test_matrix = {"6.2" = ["*"]} +``` + +## Add extra workflow jobs + +```toml +[github] +extra_lines = """ + another: + uses: org/repo/.github/workflows/file.yml@main +""" +``` diff --git a/docs/sources/how-to/configure-gitlab-ci.md b/docs/sources/how-to/configure-gitlab-ci.md new file mode 100644 index 00000000..65d97c78 --- /dev/null +++ b/docs/sources/how-to/configure-gitlab-ci.md @@ -0,0 +1,67 @@ +--- +myst: + html_meta: + "description": "Configure GitLab CI pipelines generated by plone.meta" + "property=og:description": "Configure GitLab CI pipelines generated by plone.meta" + "property=og:title": "Configure GitLab CI" + "keywords": "plone.meta, GitLab CI, CI/CD" +--- + +# Configure GitLab CI + + + +For repositories hosted on GitLab, `plone.meta` generates a {file}`.gitlab-ci.yml` file instead of GitHub Actions workflows. + +## Select CI jobs + +```toml +[gitlab] +jobs = [ + "lint", + "release-ready", + "dependencies", + "circular-dependencies", + "testing", + "coverage", +] +``` + +## Use custom Docker images + +Specify Docker images per Python version using the `custom_images` dictionary: + +```toml +[gitlab] +custom_images = {"3.14" = "python:3.14-trixie", "3.13" = "python:3.13-trixie"} +``` + +This allows different Python versions in the CI matrix to use appropriate base images (for example, Debian Trixie for Python 3.14 support). + +## Install OS-level dependencies + +```toml +[gitlab] +os_dependencies = """ + - apt-get install libxslt libxml2 +""" +``` + +## Test matrix support + +GitLab CI also benefits from the test matrix feature. +When `use_test_matrix` is enabled (the default), the generated {file}`.gitlab-ci.yml` will include test jobs for each combination of Plone version and Python version defined in `[tox] test_matrix`. + +See {doc}`/how-to/customize-meta-toml` for details on configuring the test matrix. + +## Add extra configuration + +```toml +[gitlab] +extra_lines = """ +deploy: + stage: deploy + script: + - echo "deploying" +""" +``` diff --git a/docs/sources/how-to/custom-branch.md b/docs/sources/how-to/custom-branch.md new file mode 100644 index 00000000..d0ed7c27 --- /dev/null +++ b/docs/sources/how-to/custom-branch.md @@ -0,0 +1,34 @@ +--- +myst: + html_meta: + "description": "Use a custom branch name with config-package" + "property=og:description": "Use a custom branch name with config-package" + "property=og:title": "Use a Custom Branch Name" + "keywords": "plone.meta, config-package, git branch" +--- + +# Use a Custom Branch Name + + + +By default, `config-package` creates a branch named `config-with-default-template-`. +You can override this. + +## Use a specific branch name + +```shell +venv/bin/config-package --branch my-feature-branch /path/to/package +``` + +If the branch already exists, `config-package` checks it out and updates it. +If it does not exist, it creates a new branch. + +## Update the current branch + +To apply changes without creating a new branch: + +```shell +venv/bin/config-package --branch current /path/to/package +``` + +This is useful when you have already created a branch manually and want to update it with the latest configuration. diff --git a/docs/sources/how-to/customize-meta-toml.md b/docs/sources/how-to/customize-meta-toml.md new file mode 100644 index 00000000..7e5dd12b --- /dev/null +++ b/docs/sources/how-to/customize-meta-toml.md @@ -0,0 +1,286 @@ +--- +myst: + html_meta: + "description": "Customize generated files via .meta.toml" + "property=og:description": "Customize generated files via .meta.toml" + "property=og:title": "Customize via .meta.toml" + "keywords": "plone.meta, .meta.toml, customization, configuration" +--- + +# Customize via .meta.toml + + + +All customizations for generated files go into {file}`.meta.toml` in the target repository. + +:::{tip} +All available configuration options are also documented as comments in each Jinja2 template file under {file}`src/plone/meta/default/`. Browsing the templates is a good way to discover options. +::: + +## General pattern + +Most configuration files support an `extra_lines` key that appends your custom configuration to the generated output: + +```toml +[section_name] +extra_lines = """ +your custom configuration here +""" +``` + +:::{important} +Never directly edit the generated files. Your changes will be overwritten the next time `config-package` runs. +::: + +## .editorconfig + +```toml +[editorconfig] +extra_lines = """ +[*.{json,jsonl,js,jsx,ts,tsx,css,less,scss}] +indent_size = 4 +""" +``` + +## .flake8 + +```toml +[flake8] +extra_lines = """ +per-file-ignores = + setup.py:T20 +""" +``` + +## .gitignore + +```toml +[gitignore] +extra_lines = """ +my-custom-folder/ +*.sqlite +""" +``` + +## .pre-commit-config.yaml + +General extra lines: + +```toml +[pre_commit] +extra_lines = """ +- repo: https://github.com/my/hook + rev: v1.0 + hooks: + - id: my-hook +""" +``` + +Tool-specific extensions: + +```toml +[pre_commit] +zpretty_extra_lines = """ + args: ['--indent', '4'] +""" + +codespell_extra_lines = """ + args: ['--skip', '*.po'] +""" + +flake8_extra_lines = """ + additional_dependencies: + - flake8-debugger + - flake8-deprecated +""" + +i18ndude_extra_lines = """ + additional_dependencies: + - toml +""" +``` + +To disable the i18ndude check: + +```toml +[pre_commit] +i18ndude_extra_lines = """ + pass_filenames: false +""" +``` + +## pyproject.toml + +General extra lines: + +```toml +[pyproject] +extra_lines = """ +[tool.mypy] +ignore_missing_imports = true +""" +``` + +Specific tool configuration: + +```toml +[pyproject] +codespell_ignores = "foo,bar" +codespell_skip = "*.po,*.map,package-lock.json" + +dependencies_ignores = "['zestreleaser.towncrier']" +dependencies_mappings = [ + "gitpython = ['git']", + "pygithub = ['github']", +] + +check_manifest_ignores = """ + "*.map.js", + "*.pyc", +""" + +black_extra_lines = """ +target-version = ["py312"] +""" + +isort_extra_lines = """ +known_third_party = ["plone"] +""" +``` + +### Towncrier configuration + +If your project has a {file}`news/` folder, towncrier configuration is generated automatically. + +Custom issue format: + +```toml +[pyproject] +towncrier_issue_format = "[#{issue}](https://github.com/myorg/myrepo/issues/{issue})" +``` + +Extra towncrier configuration: + +```toml +[pyproject] +towncrier_extra_lines = """ +[[tool.towncrier.type]] +directory = "deprecation" +name = "Deprecations" +showcontent = true +""" +``` + +## tox.ini + +Test runner selection: + +```toml +[tox] +test_runner = "pytest" +# or leave unset for zope.testrunner (default) +``` + +Test path: + +```toml +[tox] +test_path = "src" +``` + +Additional tox environments: + +```toml +[tox] +envlist_lines = """ + my_custom_env +""" +``` + +Extra test dependencies: + +```toml +[tox] +test_extras = """ + tests + widgets +""" +``` + +Custom constraints files (per Plone version): + +``` +[tox] +constraints_files = { + "6.1" = "https://dist.plone.org/release/6.1-latest/constraints.txt", + "6.0" = "https://dist.plone.org/release/6.0-latest/constraints.txt", +} +``` + +Test matrix configuration: + +``` +[tox] +use_test_matrix = true +test_matrix = { + "6.2" = ["3.14", "3.13", "3.12", "3.11", "3.10"], + "6.1" = ["3.13", "3.12", "3.11", "3.10"], + "6.0" = ["3.13", "3.12", "3.11", "3.10"], +} +``` + +Set `use_test_matrix = false` to disable the matrix and use a single test environment. + +Skip the test extra (for packages that do not define a `test` extra): + +```toml +[tox] +skip_test_extra = true +``` + +Test environment variables: + +```toml +[tox] +test_environment_variables = """ + PIP_EXTRA_INDEX_URL=https://my-pypi.example.com/ +""" +``` + +mxdev support for source checkouts: + +```toml +[tox] +use_mxdev = true +test_deps_additional = """ + -esources/my.package[test] +""" +``` + +General extra lines: + +```toml +[tox] +extra_lines = """ +[testenv:custom] +commands = python -c "print('hello')" +""" +``` + +Top-level tox configuration: + +```toml +[tox] +config_lines = """ +my_extra_config = value +""" +``` + +Override default testenv options: + +```toml +[tox] +testenv_options = """ +basepython = /usr/bin/python3.11 +""" +``` diff --git a/docs/sources/how-to/index.md b/docs/sources/how-to/index.md new file mode 100644 index 00000000..b5ee2964 --- /dev/null +++ b/docs/sources/how-to/index.md @@ -0,0 +1,82 @@ +--- +myst: + html_meta: + "description": "How-to guides for common plone.meta tasks" + "property=og:description": "How-to guides for common plone.meta tasks" + "property=og:title": "How-To Guides" + "keywords": "plone.meta, how-to, guides" +--- + +# How-To Guides + + + +Goal-oriented recipes for common tasks. +Each guide answers a specific "how do I..." question. + +::::{grid} 2 +:gutter: 3 + +:::{grid-item-card} Install plone.meta +:link: install +:link-type: doc + +Clone the repository and install plone.meta into a virtual environment. +::: + +:::{grid-item-card} Customize via .meta.toml +:link: customize-meta-toml +:link-type: doc + +Add custom configuration for flake8, tox, pyproject.toml, pre-commit, and more without editing generated files. +::: + +:::{grid-item-card} Configure GitHub Actions +:link: configure-github-actions +:link-type: doc + +Set up CI jobs, Python version matrices, OS dependencies, and environment variables for GitHub-hosted repositories. +::: + +:::{grid-item-card} Configure GitLab CI +:link: configure-gitlab-ci +:link-type: doc + +Set up CI pipelines for GitLab-hosted repositories, including custom Docker images and OS dependencies. +::: + +:::{grid-item-card} Re-enable GitHub Actions +:link: re-enable-actions +:link-type: doc + +Re-enable GitHub Actions workflows that were auto-disabled due to repository inactivity. +::: + +:::{grid-item-card} Use a Custom Branch Name +:link: custom-branch +:link-type: doc + +Control the git branch name created by config-package, or update the current branch. +::: + +:::{grid-item-card} Write Custom Templates +:link: write-custom-templates +:link-type: doc + +Create a custom configuration type with your own Jinja2 templates when `extra_lines` is not enough. +::: + +:::: + +```{toctree} +--- +hidden: true +--- +install +customize-meta-toml +configure-github-actions +configure-gitlab-ci +re-enable-actions +custom-branch +write-custom-templates +``` diff --git a/docs/sources/how-to/install.md b/docs/sources/how-to/install.md new file mode 100644 index 00000000..cda27849 --- /dev/null +++ b/docs/sources/how-to/install.md @@ -0,0 +1,52 @@ +--- +myst: + html_meta: + "description": "Install plone.meta from PyPI or source" + "property=og:description": "Install plone.meta from PyPI or source" + "property=og:title": "Install plone.meta" + "keywords": "plone.meta, installation, uvx, pip" +--- + +# Install plone.meta + + + +## With uvx (recommended) + +The simplest way to use plone.meta is via [uvx](https://docs.astral.sh/uv/), which runs it without a permanent installation: + +```shell +uvx --from=plone.meta config-package /path/to/your-package +``` + +## From PyPI + +Install plone.meta from PyPI into a virtual environment: + +```shell +python3 -m venv .venv +.venv/bin/pip install plone.meta +``` + +The commands `config-package`, `multi-call`, `switch-to-pep420`, and `re-enable-actions` are now available under `.venv/bin/`. + +## From source + +Clone the repository and install in editable mode: + +```shell +git clone https://github.com/plone/meta.git +cd meta +python3 -m venv .venv +.venv/bin/pip install -e . +``` + +This is useful for contributing to plone.meta or testing unreleased changes. + +## Verify the installation + +```shell +config-package --help +``` + +You should see the help output listing all available options. diff --git a/docs/sources/how-to/re-enable-actions.md b/docs/sources/how-to/re-enable-actions.md new file mode 100644 index 00000000..fc85a6e5 --- /dev/null +++ b/docs/sources/how-to/re-enable-actions.md @@ -0,0 +1,34 @@ +--- +myst: + html_meta: + "description": "Re-enable GitHub Actions disabled by inactivity" + "property=og:description": "Re-enable GitHub Actions disabled by inactivity" + "property=og:title": "Re-enable GitHub Actions" + "keywords": "plone.meta, GitHub Actions, re-enable" +--- + +# Re-enable GitHub Actions + + + +GitHub automatically disables Actions on repositories that have not received commits for a certain period. +`plone.meta` provides a command to re-enable them across all tracked packages. + +## Prerequisites + +Install [GitHub's CLI application](https://github.com/cli/cli) and authenticate: + +```shell +gh auth login +``` + +## Re-enable Actions + +From the plone.meta directory: + +```shell +venv/bin/re-enable-actions +``` + +This command iterates over all packages listed in {file}`packages.txt`, checks if a "Meta" workflow exists, and enables it if it was disabled. +It does no harm if Actions are already enabled. diff --git a/docs/sources/how-to/write-custom-templates.md b/docs/sources/how-to/write-custom-templates.md new file mode 100644 index 00000000..13d667cb --- /dev/null +++ b/docs/sources/how-to/write-custom-templates.md @@ -0,0 +1,124 @@ +--- +myst: + html_meta: + "description": "Write custom Jinja2 templates for plone.meta" + "property=og:description": "Write custom Jinja2 templates for plone.meta" + "property=og:title": "Write Custom Templates" + "keywords": "plone.meta, custom templates, Jinja2" +--- + +# Write Custom Templates + + + +If the `default` configuration type does not meet your needs and `extra_lines` is not sufficient, you can create a custom template set. + +:::{note} +Custom templates require modifying the plone.meta source. This is an advanced topic. For most use cases, {doc}`customizing via .meta.toml ` is the recommended approach. +::: + +## How the template system works + +plone.meta uses a Jinja2 `FileSystemLoader` with two search paths, in order: + +1. `src/plone/meta//` -- the active configuration type +2. {file}`src/plone/meta/default/` -- the fallback + +When a template is requested (e.g. {file}`tox.ini.j2`), Jinja2 looks in the config type directory first. +If it finds the file there, it uses that version. +Otherwise, it falls back to `default/`. + +This means you can override *individual* templates while inheriting the rest from `default/`. + +## Create a custom configuration type + +1. In your fork or local clone of plone.meta, create a new directory + alongside `default/`: + + ```shell + mkdir -p src/plone/meta/mytype + ``` + +2. Copy only the templates you want to change: + + ```shell + cp src/plone/meta/default/tox.ini.j2 src/plone/meta/mytype/ + ``` + +3. Edit the copied template. + Templates use a custom Jinja2 syntax with `%(variable)s` for variable substitution instead of the standard `{{ variable }}`: + + ``` + %(extra_lines)s + %(test_runner)s + %(constraints_file)s + ``` + + See {doc}`../reference/generated-files` for the full list of templates and their variables. + +4. Register the new type by adding it to the `choices` list in {file}`config_package.py`: + + ```python + parser.add_argument( + "-t", + "--type", + choices=[ + "default", + "mytype", + ], + ... + ) + ``` + +5. Create a {file}`packages.txt` file in your new directory (can be empty): + + ```shell + touch src/plone/meta/mytype/packages.txt + ``` + +6. Reinstall plone.meta: + + ```shell + venv/bin/pip install -e . + ``` + +## Use the custom type + +Apply the custom configuration to a repository: + +```shell +venv/bin/config-package --type mytype /path/to/package +``` + +The type is stored in {file}`.meta.toml` under `[meta] template`, so subsequent runs do not need the `--type` flag. + +## Template variables + +All templates receive their variables from {file}`.meta.toml` via the `_get_options_for()` method. +The variable names correspond to the keys in each {file}`.meta.toml` section. + +Additionally, some variables are computed: + +`config_type` +: The active configuration type name (e.g. `"default"`, `"mytype"`). + +`news_folder_exists` +: `True` if a {file}`news/` directory exists in the target repository. + +`changes_extension` +: `"md"` or `"rst"` depending on whether {file}`CHANGES.md` exists. + +`prime_robotframework` +: `True` if `plone.app.robotframework` is detected as a dependency. + +`package_name` +: The repository directory name (or overridden via {file}`.meta.toml`). + +## Considerations + +- Keep your custom templates in a fork or branch. Upstream plone.meta + only ships the `default` type. +- When upstream templates change, you will need to merge those changes + into your overridden files. +- If your customization is generally useful, consider + [requesting it upstream](https://github.com/plone/meta/issues/new) as a new `extra_lines` option instead. diff --git a/docs/sources/index.md b/docs/sources/index.md new file mode 100644 index 00000000..01b9fa86 --- /dev/null +++ b/docs/sources/index.md @@ -0,0 +1,91 @@ +--- +myst: + html_meta: + "description": "plone.meta - standardize configuration across Plone Python packages" + "property=og:description": "plone.meta - standardize configuration across Plone Python packages" + "property=og:title": "plone.meta" + "keywords": "plone.meta, Plone, configuration, CI/CD, tox, GitHub Actions" +--- + +# plone.meta + + + +Standardize configuration files across Plone-related Python repositories. + +`plone.meta` generates and manages a consistent set of configuration files -- from CI pipelines and linting to tox environments and editor settings -- so every Plone package provides the same developer experience. + +**Key capabilities:** + +- Generate {file}`.editorconfig`, {file}`.flake8`, {file}`.gitignore`, {file}`pyproject.toml`, {file}`tox.ini`, {file}`.pre-commit-config.yaml` from Jinja2 templates +- GitHub Actions and GitLab CI workflows, centrally managed +- Test matrix: automatically test all combinations of Plone versions (6.0, 6.1, 6.2) and Python versions (3.10--3.14) +- Shared reusable GitHub Actions workflows and composite actions for Cookieplone projects +- Per-repository customization via {file}`.meta.toml` without editing generated files +- Bulk updates across 100+ Plone packages with a single command +- Validation of generated TOML, YAML, INI, and editorconfig files +- Install from PyPI: `pip install plone.meta` + +**Requirements:** Python 3.10+ + +## Documentation + +::::{grid} 2 +:gutter: 3 + +:::{grid-item-card} Tutorials +:link: tutorials/index +:link-type: doc + +**Learning-oriented** -- Step-by-step lessons to get started. + +*Start here if you are new to plone.meta.* +::: + +:::{grid-item-card} How-To Guides +:link: how-to/index +:link-type: doc + +**Goal-oriented** -- Solutions to specific problems. + +*Use these when you need to accomplish something.* +::: + +:::{grid-item-card} Reference +:link: reference/index +:link-type: doc + +**Information-oriented** -- Configuration options and CLI details. + +*Consult when you need detailed information.* +::: + +:::{grid-item-card} Explanation +:link: explanation/index +:link-type: doc + +**Understanding-oriented** -- Architecture, design decisions, and philosophy. + +*Read to deepen your understanding of how and why it works.* +::: + +:::: + +## Quick Start + +1. {doc}`Install plone.meta ` +2. {doc}`Configure your first package ` +3. {doc}`Customize via .meta.toml ` + +```{toctree} +--- +maxdepth: 3 +caption: Documentation +titlesonly: true +hidden: true +--- +tutorials/index +how-to/index +reference/index +explanation/index +``` diff --git a/docs/sources/llms.txt b/docs/sources/llms.txt new file mode 100644 index 00000000..c3f67d2c --- /dev/null +++ b/docs/sources/llms.txt @@ -0,0 +1,128 @@ +# plone.meta + +> Standardize configuration files across Plone-related Python repositories. + +plone.meta generates and manages a consistent set of configuration files for Plone +Python packages: .editorconfig, .flake8, .gitignore, pyproject.toml, tox.ini, +.pre-commit-config.yaml, and CI pipelines (GitHub Actions, GitLab CI). + +## Key Facts + +- Package: plone.meta +- Version: 2.4.1.dev0 +- License: GPLv2 +- Python: 3.9+, with support up to Python 3.14 +- Repository: https://github.com/plone/meta +- PyPI: https://pypi.org/project/plone.meta/ +- Install: pip install plone.meta + +## Commands + +### config-package + +The main command. Generates configuration files for a target Python package +repository from Jinja2 templates. Reads per-repository customizations from +.meta.toml. Creates a git branch, commits changes, and optionally pushes. + +Usage: config-package [OPTIONS] PATH + +Options: + --branch BRANCH_NAME Git branch name (use "current" for current branch) + --commit-msg MSG Custom commit message + --no-commit Don't commit changes + --push Push changes + --tox Run tox after configuration + --track Add package to packages.txt + -t, --type TYPE Configuration type (default: "default") + +### multi-call + +Runs config-package across multiple repositories listed in a packages.txt file. + +Usage: multi-call SCRIPT PACKAGES_FILE CLONES_DIR [ARGS...] + +### switch-to-pep420 + +Converts packages from pkg_resources-style namespace packages to PEP 420 +native namespace packages. + +Usage: switch-to-pep420 [OPTIONS] PATH + +### re-enable-actions + +Re-enables GitHub Actions on repositories where they were auto-disabled. + +Usage: re-enable-actions + +## Generated Files + +- .meta.toml: plone.meta's configuration (the only file you should edit) +- .editorconfig: Editor settings +- .flake8: Flake8 linting rules +- .gitignore: Git ignore patterns +- .github/workflows/meta.yml: GitHub Actions CI (GitHub repos only); default jobs: qa, coverage, dependencies, release_ready, circular (note: "test" no longer included by default) +- .github/workflows/test-matrix.yml: GitHub Actions test matrix workflow (GitHub repos only); generated when use_test_matrix is true (default) +- .gitlab-ci.yml: GitLab CI (GitLab repos only) +- .pre-commit-config.yaml: Pre-commit hooks +- pyproject.toml: Python tooling configuration +- tox.ini: Test environment definitions (uses modular sub-templates: tox-init.j2, tox-base.j2, tox-test-runner-specifics.j2, tox-test-coverage.j2, tox-qa.j2, tox-plone-depending-qa.j2) + +## Test Matrix + +A major feature of plone.meta 2.x. Tests all combinations of Plone versions +and Python versions. Default matrix: +- Plone 6.2: Python 3.14, 3.13, 3.12, 3.11, 3.10 +- Plone 6.1: Python 3.13, 3.12, 3.11, 3.10 +- Plone 6.0: Python 3.13, 3.12, 3.11, 3.10 + +Generated tox environments follow the pattern: py314-plone62, py313-plone61, etc. + +Configurable via [tox] test_matrix and [tox] use_test_matrix in .meta.toml. + +## .meta.toml Configuration + +All customization goes into .meta.toml. Never edit generated files directly. + +Sections: [meta], [editorconfig], [flake8], [gitignore], [github], [gitlab], +[pre_commit], [pyproject], [tox] + +Most sections support extra_lines for appending custom configuration. + +### Key 2.x changes + +- [github] jobs: default no longer includes "test" (replaced by test-matrix.yml) +- [github] extra_lines_after_os_dependencies: replaces py_versions +- [gitlab] custom_images: dict keyed by Python version (replaces custom_image string) +- [tox] constraints_files: dict keyed by Plone version (replaces constraints_file string) +- [tox] test_matrix: dict defining Plone version to Python versions mapping +- [tox] use_test_matrix: boolean (default true) to enable/disable test matrix +- [tox] skip_test_extra: boolean for packages without a test extra + +## Shared Workflows & Actions + +plone/meta provides reusable GitHub Actions workflows and composite actions +for Cookieplone-based projects (not used by config-package): + +Composite actions: setup_backend_uv, setup_frontend, setup_uv +Backend workflows: backend-lint, backend-pytest, backend-pytest-coverage +Documentation workflows: docs-build +Frontend workflows: frontend-acceptance, frontend-code, frontend-i18n, frontend-storybook, frontend-unit +Container workflows: container-image-build-push, container-image-build, container-image-push + +## tox Environments + +- test: Run the test suite (with test matrix: py314-plone62, py313-plone61, etc.) +- coverage: Test coverage reporting +- lint: Code quality checks via pre-commit +- format: Auto-formatting (isort, black, zpretty) +- release-check: Release readiness validation +- dependencies: Dependency declaration check +- circular: Circular dependency detection +- dependencies-graph: Visual dependency graph + +## Documentation Sections + +- Tutorials: Getting started, bulk updates +- How-To Guides: Install, customize .meta.toml, configure CI, re-enable actions +- Reference: CLI options, .meta.toml options, generated files, tox environments, shared workflows, changelog +- Explanation: Why standardize, architecture, configuration philosophy, scope diff --git a/docs/sources/reference/changelog.md b/docs/sources/reference/changelog.md new file mode 100644 index 00000000..94099702 --- /dev/null +++ b/docs/sources/reference/changelog.md @@ -0,0 +1,2 @@ +```{include} ../../../CHANGES.md +``` diff --git a/docs/sources/reference/cli-config-package.md b/docs/sources/reference/cli-config-package.md new file mode 100644 index 00000000..69f13f38 --- /dev/null +++ b/docs/sources/reference/cli-config-package.md @@ -0,0 +1,77 @@ +--- +myst: + html_meta: + "description": "CLI reference for config-package command" + "property=og:description": "CLI reference for config-package command" + "property=og:title": "config-package CLI" + "keywords": "plone.meta, config-package, CLI, reference" +--- + +# config-package CLI + + + +## Synopsis + +``` +config-package [OPTIONS] PATH +``` + +`PATH` is the filesystem path to the target Python package repository. + +## Options + +`--branch BRANCH_NAME` +: Git branch name to create for the changes. + Default: auto-generated as `config-with--template-`. + + :::{important} + Use `--branch current` to update the current branch instead of creating a new one. + This is essential when re-running `config-package` after changing {file}`.meta.toml`. + ::: + +`--commit-msg MSG` +: Custom commit message. + Default: an auto-generated message describing the configuration update. + +`--no-commit` +: Do not automatically commit changes after the configuration run. + Useful for reviewing changes before committing. + +`--push` +: Push changes to the remote after the configuration run. + Default: changes are *not* pushed. + +`--tox` +: Run `tox` on the repository after applying configuration. + Default: tox is *not* run. + +`--track` +: Add the package to {file}`packages.txt` in the plone.meta repository. + Default: packages are *not* tracked. + +`-t, --type TYPE` +: Configuration type. Currently only `default` is available. + Only needed the first time; the value is stored in {file}`.meta.toml`. + +`-h, --help` +: Display help and exit. + +## Behavior + +1. Creates a new git branch from the current branch (unless `--branch current`). +2. Reads {file}`.meta.toml` if present, or creates it with defaults. +3. Renders Jinja2 templates into configuration files. +4. Validates generated files (TOML, YAML, INI, editorconfig). +5. Creates a towncrier news entry. +6. Commits changes (unless `--no-commit`). +7. Optionally pushes and/or runs tox. + +## Exit codes + +`0` +: Success. + +Non-zero +: An error occurred during configuration. The error message is printed to + stderr. You are prompted to proceed or abort. diff --git a/docs/sources/reference/cli-multi-call.md b/docs/sources/reference/cli-multi-call.md new file mode 100644 index 00000000..d48d83a9 --- /dev/null +++ b/docs/sources/reference/cli-multi-call.md @@ -0,0 +1,53 @@ +--- +myst: + html_meta: + "description": "CLI reference for multi-call bulk update command" + "property=og:description": "CLI reference for multi-call bulk update command" + "property=og:title": "multi-call CLI *(experimental)*" + "keywords": "plone.meta, multi-call, CLI, bulk update" +--- + +# multi-call CLI *(experimental)* + + + +:::{warning} +This tool originates from `zope.meta` and has not been extensively tested in the `plone.meta` context. The default branch is `master` instead of `main`. Use with caution. +::: + +## Synopsis + +``` +multi-call SCRIPT PACKAGES_FILE CLONES_DIR [SCRIPT_ARGS...] +``` + +## Positional arguments + +`SCRIPT` +: Path to the Python script to run on each package + (typically `config-package`). + +`PACKAGES_FILE` +: Path to a text file listing repository names, one per line. + Lines starting with `#` are skipped. + +`CLONES_DIR` +: Directory where repositories are cloned. If a repository does not + exist here, it is cloned from GitHub. If it already exists, it is cleaned, switched to `master`, and pulled. + +`SCRIPT_ARGS` +: Additional arguments passed to `SCRIPT` for each package. + +## Behavior + +For each package listed in `PACKAGES_FILE`: + +1. If no clone exists in `CLONES_DIR`, the repository is cloned. +2. If a clone exists, uncommitted changes are stashed, the `master` + branch is checked out, and the latest changes are pulled. +3. The specified script is run with the package path and any extra arguments. + +:::{caution} +Uncommitted changes are stashed automatically. +Use `git stash pop` to recover them. +::: diff --git a/docs/sources/reference/generated-files.md b/docs/sources/reference/generated-files.md new file mode 100644 index 00000000..8b0a2c3b --- /dev/null +++ b/docs/sources/reference/generated-files.md @@ -0,0 +1,128 @@ +--- +myst: + html_meta: + "description": "Reference of all files generated by plone.meta" + "property=og:description": "Reference of all files generated by plone.meta" + "property=og:title": "Generated Files" + "keywords": "plone.meta, generated files, templates, reference" +--- + +# Generated Files + + + +`plone.meta` generates the following files in the target repository. +Each file is rendered from a Jinja2 template in {file}`src/plone/meta/default/`. + +:::{important} +Do not edit these files directly. All customization goes into {file}`.meta.toml`. +::: + +## .meta.toml + +**Template:** Created programmatically +**Purpose:** plone.meta's own configuration file. +Stores the template type, commit ID, and all per-repository customizations. +This is the *only* file you should edit. + +## .editorconfig + +**Template:** {file}`editorconfig.j2` +**Purpose:** Editor configuration for consistent formatting across IDEs. +Sets 4-space indentation for Python, 2-space for XML/YAML/ZCML, and Unix line endings. + +## .flake8 + +**Template:** {file}`flake8.j2` +**Purpose:** Flake8 configuration. +Ignores rules that conflict with Black (E501, W503, E203, E231) and enables doctests. + +## .gitignore + +**Template:** {file}`gitignore.j2` +**Purpose:** Git ignore patterns for Python projects, build artifacts, test output, editor files, Buildout directories, and mxdev artifacts. + +## .github/workflows/meta.yml + +**Template:** {file}`meta.yml.j2` +**Purpose:** GitHub Actions workflow for repositories hosted on GitHub. +Uses `workflow_call` to reference reusable workflows from the plone/meta repository (qa, coverage, dependencies, release_ready, circular). + +:::{note} +On the old `main` branch there used to be a `test` job. Since 2.x, this is handled in {file}`test-matrix.yml` (see below). You can still add `"test"` to the `[github] jobs` list in {file}`.meta.toml` if needed. +::: + +:::{note} +Only generated for GitHub-hosted repositories. +::: + +## .github/workflows/test-matrix.yml + +**Template:** {file}`test-matrix.yml.j2` +**Purpose:** GitHub Actions workflow that runs tests across a matrix of Plone versions and Python versions. +Generated automatically when `use_test_matrix` is enabled (the default). +The matrix is configured via the `[tox] test_matrix` option in +{file}`.meta.toml`. + +The default matrix tests the combinations of Plone versions and Python versions as defined by the Plone community. +See the `[tox] test_matrix` option in {doc}`/reference/meta-toml` for how to customize it. + +:::{note} +Only generated for GitHub-hosted repositories. +::: + +## .github/dependabot.yml + +**Template:** {file}`dependabot.yml` (static) +**Purpose:** Dependabot configuration for automatic GitHub Actions updates on a weekly schedule. + +:::{note} +Only generated for GitHub-hosted repositories. +::: + +## .gitlab-ci.yml + +**Template:** {file}`gitlab-ci.yml.j2` +**Purpose:** GitLab CI pipeline configuration. +Defines jobs for linting, testing, coverage, dependency checking, and release readiness. + +:::{note} +Only generated for GitLab-hosted repositories. +::: + +## .pre-commit-config.yaml + +**Template:** {file}`pre-commit-config.yaml.j2` +**Purpose:** Pre-commit hook configuration. +Includes pyupgrade, isort, black, zpretty, flake8, codespell, check-manifest, pyroma, check-python-versions, and i18ndude. + +## pyproject.toml + +**Template:** {file}`pyproject.toml.j2` +**Purpose:** Python tooling configuration for isort, black, codespell, check-manifest, and z3c.dependencychecker. +Also includes towncrier configuration if a {file}`news/` folder exists. + +:::{note} +plone.meta overwrites {file}`pyproject.toml` completely, like all other generated files. All customization must go through {file}`.meta.toml`. +::: + +## tox.ini + +**Template:** {file}`tox.ini.j2` +**Purpose:** Tox environment definitions for testing, linting, coverage, dependency checking, release readiness, and circular dependency detection. +This is considered the most important generated file. + +The main template {file}`tox.ini.j2` uses a modular architecture with `{% include %}` directives to compose the output from sub-templates: + +- {file}`tox-init.j2` -- tox initialization and configuration header +- {file}`tox-base.j2` -- base test environment definition +- {file}`tox-test-runner-specifics.j2` -- test runner specific settings (pytest or zope.testrunner) +- {file}`tox-test-coverage.j2` -- coverage environment configuration +- {file}`tox-qa.j2` -- linting and formatting environments +- {file}`tox-plone-depending-qa.j2` -- Plone-specific QA environments (dependencies, release-check, circular) + +## news/.changelog_template.jinja + +**Template:** {file}`changelog_template.jinja` (static) +**Purpose:** Towncrier template for Markdown-formatted changelogs. +Only generated if a {file}`news/` folder exists and {file}`CHANGES.md` is used. diff --git a/docs/sources/reference/index.md b/docs/sources/reference/index.md new file mode 100644 index 00000000..15f02771 --- /dev/null +++ b/docs/sources/reference/index.md @@ -0,0 +1,89 @@ +--- +myst: + html_meta: + "description": "Technical reference for plone.meta CLI, configuration, and generated files" + "property=og:description": "Technical reference for plone.meta CLI, configuration, and generated files" + "property=og:title": "Reference" + "keywords": "plone.meta, reference, CLI, configuration" +--- + +# Reference + + + +Technical specifications, CLI details, and configuration option reference for plone.meta. + +::::{grid} 2 +:gutter: 3 + +:::{grid-item-card} config-package CLI +:link: cli-config-package +:link-type: doc + +Complete reference for all command-line options of `config-package`. +::: + +:::{grid-item-card} multi-call CLI *(experimental)* +:link: cli-multi-call +:link-type: doc + +Command-line reference for `multi-call`, the bulk update tool. +::: + +:::{grid-item-card} .meta.toml Options +:link: meta-toml +:link-type: doc + +All configuration options available in {file}`.meta.toml`, organized by section. +::: + +:::{grid-item-card} Generated Files +:link: generated-files +:link-type: doc + +Description of each file that plone.meta generates, its source template, and its purpose. +::: + +:::{grid-item-card} tox Environments +:link: tox-environments +:link-type: doc + +Reference for all tox environments generated by plone.meta and what each one does. +::: + +:::{grid-item-card} Shared Workflows & Actions +:link: shared-workflows +:link-type: doc + +Reusable GitHub Actions workflows and composite actions provided by plone/meta for Cookieplone projects. +::: + +:::{grid-item-card} Upgrade Guide +:link: upgrade-guide +:link-type: doc + +Breaking changes between major versions and how to migrate. +::: + +:::{grid-item-card} Changelog +:link: changelog +:link-type: doc + +Version history with release notes. +::: + +:::: + +```{toctree} +--- +hidden: true +--- +cli-config-package +cli-multi-call +meta-toml +generated-files +tox-environments +shared-workflows +upgrade-guide +changelog +``` diff --git a/docs/sources/reference/meta-toml.md b/docs/sources/reference/meta-toml.md new file mode 100644 index 00000000..77ce26b1 --- /dev/null +++ b/docs/sources/reference/meta-toml.md @@ -0,0 +1,206 @@ +--- +myst: + html_meta: + "description": "Complete reference of .meta.toml configuration options" + "property=og:description": "Complete reference of .meta.toml configuration options" + "property=og:title": ".meta.toml Options" + "keywords": "plone.meta, .meta.toml, options, reference" +--- + +# .meta.toml Options + + + +{file}`.meta.toml` is the per-repository configuration file for plone.meta. +It controls how each generated file is customized. + +:::{tip} +The Jinja2 template files in {file}`src/plone/meta/default/` contain comments documenting each available option. Check the templates for the most up-to-date list. +::: + +## `[meta]` + +Managed by plone.meta itself. Do not edit manually. + +`template` +: Configuration template type. Currently always `"default"`. + +`commit-id` +: 8-character commit hash of the plone.meta version used. + +## `[editorconfig]` + +`extra_lines` +: Additional editorconfig rules appended to the generated file. + +## `[flake8]` + +`extra_lines` +: Additional flake8 configuration appended to the generated file. + +## `[gitignore]` + +`extra_lines` +: Additional gitignore patterns appended to the generated file. + +## `[github]` + +`jobs` +: List of CI jobs to enable in {file}`meta.yml`. Available: `"qa"`, `"test"`, + `"coverage"`, `"dependencies"`, `"release_ready"`, `"circular"`. + Default: `["qa", "coverage", "dependencies", "release_ready", "circular"]`. + + :::{note} + `"test"` is no longer included in the default jobs list. + Testing is now handled by the separate {file}`test-matrix.yml` workflow, which is generated automatically when `use_test_matrix` is enabled (the default). + You can still add `"test"` to the jobs list if you need the legacy single-version test job. + ::: + +`ref` +: Branch or tag of plone/meta to reference for workflow files. + Default: `"2.x"`. + +`env` +: YAML-formatted environment variables for all jobs. + +`os_dependencies` +: Space-separated Ubuntu package names to install before tests. + +`extra_lines_after_os_dependencies` +: Additional YAML lines inserted after the OS dependency installation step + in the workflow. Useful for custom setup steps that need to run before tests (e.g., installing additional tools or configuring the environment). + +`extra_lines` +: Additional YAML appended to the workflow file. + +## `[gitlab]` + +`jobs` +: List of CI jobs. Available: `"lint"`, `"release-ready"`, + `"dependencies"`, `"circular-dependencies"`, `"testing"`, `"coverage"`. + +`custom_images` +: Dictionary of Docker images keyed by Python version. Allows specifying + different images for different Python versions in the CI matrix. + + Example: + + ```toml [gitlab] custom_images = {"3.14" = "python:3.14-trixie", "3.13" = "python:3.13-trixie"} ``` + +`os_dependencies` +: YAML-formatted apt-get install commands. + +`extra_lines` +: Additional YAML appended to the CI configuration. + +## `[pre_commit]` + +`extra_lines` +: Additional pre-commit hook configuration. + +`zpretty_extra_lines` +: Extra configuration for the zpretty hook. + +`codespell_extra_lines` +: Extra configuration for the codespell hook. + +`flake8_extra_lines` +: Extra configuration for the flake8 hook (e.g., additional_dependencies). + +`i18ndude_extra_lines` +: Extra configuration for the i18ndude hook. Set `pass_filenames: false` + to disable the check. + +## `[pyproject]` + +`extra_lines` +: Additional pyproject.toml tool configuration. + +`codespell_ignores` +: Comma-separated words to ignore in codespell. + +`codespell_skip` +: Comma-separated file patterns to skip in codespell. + +`dependencies_ignores` +: Python list string of packages to ignore in dependency checks. + +`dependencies_mappings` +: List of import-to-package mappings for z3c.dependencychecker. + +`check_manifest_ignores` +: Additional patterns for check-manifest to ignore. + +`check_manifest_extra_lines` +: Extra check-manifest configuration. + +`black_extra_lines` +: Additional Black formatter configuration. + +`isort_extra_lines` +: Additional isort configuration. + +`towncrier_issue_format` +: Custom issue URL format for towncrier. + +`towncrier_extra_lines` +: Extra towncrier configuration. + +## `[tox]` + +`test_runner` +: Test runner to use. `"pytest"` or `"zope.testrunner"` (default). + +`test_path` +: Root path for test discovery. Auto-detected from `tests/` or `src/` + if not set. + +`test_extras` +: Additional extras to install for the test and coverage environments. + +`test_deps_additional` +: Additional test dependencies (typically with mxdev source checkouts). + +`test_environment_variables` +: Environment variables for test and coverage environments. + +`constraints_files` +: Dictionary of pip constraints file URLs keyed by Plone version. + This allows specifying different constraints for each Plone version in the test matrix. + + Example: + + ``` + [tox] constraints_files = { "6.1" = "https://dist.plone.org/release/6.1-latest/constraints.txt", "6.0" = "https://dist.plone.org/release/6.0-latest/constraints.txt", } ``` + +`use_test_matrix` +: Boolean. When `true` (the default), generates test environments for + all combinations of Plone versions and Python versions defined in `test_matrix`. Set to `false` to disable the test matrix and use a single test environment instead. + +`test_matrix` +: Dictionary defining which Python versions to test against each Plone + version. Only used when `use_test_matrix` is `true`. + + Default: + + ``` + [tox] test_matrix = { "6.2" = ["3.14", "3.13", "3.12", "3.11", "3.10"], "6.1" = ["3.13", "3.12", "3.11", "3.10"], "6.0" = ["3.13", "3.12", "3.11", "3.10"], } ``` + +`skip_test_extra` +: Boolean. Set to `true` for packages that do not define a `test` extra + in their packaging metadata. When enabled, the test environments will not attempt to install `[test]` extras. + +`use_mxdev` +: Set to `true` to enable mxdev source checkout support. + +`envlist_lines` +: Additional tox environment names to include. + +`config_lines` +: Extra top-level tox configuration. + +`testenv_options` +: Override default testenv options (e.g., basepython). + +`extra_lines` +: Additional tox configuration sections. diff --git a/docs/sources/reference/shared-workflows.md b/docs/sources/reference/shared-workflows.md new file mode 100644 index 00000000..1c2aa998 --- /dev/null +++ b/docs/sources/reference/shared-workflows.md @@ -0,0 +1,260 @@ +--- +myst: + html_meta: + "description": "Reusable GitHub Actions workflows and composite actions from plone.meta" + "property=og:description": "Reusable GitHub Actions workflows and composite actions from plone.meta" + "property=og:title": "Shared Workflows & Actions" + "keywords": "plone.meta, GitHub Actions, workflows, composite actions" +--- + +# Shared Workflows & Actions + + + +plone.meta provides reusable GitHub Actions workflows and composite actions for use by Cookieplone-based projects. +These are **not** used by the `config-package` tool; they are designed to be called directly from downstream project workflows. + +All workflows and actions are located in the plone.meta repository and referenced via `uses:` in your project's workflow files. + +## Composite Actions + +### setup_backend_uv + +Sets up a Python backend environment using [uv](https://docs.astral.sh/uv/) as the package installer. + +**Inputs:** + +| Input | Description | Required | Default | +|-------|-------------|----------|---------| +| `python-version` | Python version to install | No | `"3.12"` | + +**Example usage:** + +```yaml +steps: + - uses: actions/checkout@v4 + - uses: plone/meta/.github/actions/setup_backend_uv@2.x + with: + python-version: "3.13" +``` + +### setup_frontend + +Sets up a Node.js frontend environment with dependency installation. + +**Inputs:** + +| Input | Description | Required | Default | +|-------|-------------|----------|---------| +| `node-version` | Node.js version to install | No | `"22.x"` | + +**Example usage:** + +```yaml +steps: + - uses: actions/checkout@v4 + - uses: plone/meta/.github/actions/setup_frontend@2.x + with: + node-version: "22.x" +``` + +### setup_uv + +Sets up the [uv](https://docs.astral.sh/uv/) package installer. + +**Inputs:** + +| Input | Description | Required | Default | +|-------|-------------|----------|---------| +| `uv-version` | Version of uv to install | No | `"latest"` | + +**Example usage:** + +```yaml +steps: + - uses: actions/checkout@v4 + - uses: plone/meta/.github/actions/setup_uv@2.x +``` + +## Backend Workflows + +### backend-lint + +Runs backend linting checks. + +**Example usage:** + +```yaml +jobs: + backend-lint: + uses: plone/meta/.github/workflows/backend-lint.yml@2.x +``` + +### backend-pytest + +Runs backend tests with pytest. + +**Example usage:** + +```yaml +jobs: + backend-pytest: + uses: plone/meta/.github/workflows/backend-pytest.yml@2.x +``` + +### backend-pytest-coverage + +Runs backend tests with coverage reporting. + +**Example usage:** + +```yaml +jobs: + backend-pytest-coverage: + uses: plone/meta/.github/workflows/backend-pytest-coverage.yml@2.x +``` + +## Documentation Workflows + +### docs-build + +Builds project documentation. + +**Example usage:** + +```yaml +jobs: + docs-build: + uses: plone/meta/.github/workflows/docs-build.yml@2.x +``` + +## Frontend Workflows + +### frontend-acceptance + +Runs frontend acceptance (end-to-end) tests. + +**Example usage:** + +```yaml +jobs: + frontend-acceptance: + uses: plone/meta/.github/workflows/frontend-acceptance.yml@2.x +``` + +### frontend-code + +Runs frontend code quality checks (linting, formatting). + +**Example usage:** + +```yaml +jobs: + frontend-code: + uses: plone/meta/.github/workflows/frontend-code.yml@2.x +``` + +### frontend-i18n + +Validates frontend internationalization (i18n) setup. + +**Example usage:** + +```yaml +jobs: + frontend-i18n: + uses: plone/meta/.github/workflows/frontend-i18n.yml@2.x +``` + +### frontend-storybook + +Builds and validates Storybook stories. + +**Example usage:** + +```yaml +jobs: + frontend-storybook: + uses: plone/meta/.github/workflows/frontend-storybook.yml@2.x +``` + +### frontend-unit + +Runs frontend unit tests. + +**Example usage:** + +```yaml +jobs: + frontend-unit: + uses: plone/meta/.github/workflows/frontend-unit.yml@2.x +``` + +## Container Image Workflows + +### container-image-build-push + +Builds and pushes a container image in a single workflow. +Combines the build and push steps for convenience. + +**Secrets:** + +| Secret | Description | Required | +|--------|-------------|----------| +| `registry-username` | Container registry username | Yes | +| `registry-password` | Container registry password or token | Yes | + +**Example usage:** + +```yaml +jobs: + container-image: + uses: plone/meta/.github/workflows/container-image-build-push.yml@2.x + secrets: + registry-username: ${{ secrets.REGISTRY_USERNAME }} + registry-password: ${{ secrets.REGISTRY_PASSWORD }} +``` + +### container-image-build + +Builds a container image without pushing. +Useful for validation in pull requests. + +**Example usage:** + +```yaml +jobs: + container-image-build: + uses: plone/meta/.github/workflows/container-image-build.yml@2.x +``` + +### container-image-push + +Pushes a previously built container image to a registry. + +**Secrets:** + +| Secret | Description | Required | +|--------|-------------|----------| +| `registry-username` | Container registry username | Yes | +| `registry-password` | Container registry password or token | Yes | + +**Example usage:** + +```yaml +jobs: + container-image-push: + uses: plone/meta/.github/workflows/container-image-push.yml@2.x + secrets: + registry-username: ${{ secrets.REGISTRY_USERNAME }} + registry-password: ${{ secrets.REGISTRY_PASSWORD }} +``` + +## Version pinning + +All examples above use `@2.x` to track the latest 2.x release. +You can pin to a specific tag (e.g., `@v2.4.0`) for reproducible builds: + +```yaml +uses: plone/meta/.github/workflows/backend-lint.yml@v2.4.0 +``` diff --git a/docs/sources/reference/tox-environments.md b/docs/sources/reference/tox-environments.md new file mode 100644 index 00000000..a618cb72 --- /dev/null +++ b/docs/sources/reference/tox-environments.md @@ -0,0 +1,70 @@ +--- +myst: + html_meta: + "description": "Reference for tox environments generated by plone.meta" + "property=og:description": "Reference for tox environments generated by plone.meta" + "property=og:title": "tox Environments" + "keywords": "plone.meta, tox, environments, testing" +--- + +# tox Environments + + + +The generated {file}`tox.ini` provides the following environments. + +## `test` + +Runs the test suite using the configured test runner. +plone.meta chooses the runner based on the `test_runner` option in +{file}`.meta.toml`: either `pytest` or `zope.testrunner` (the default). +The exact commands are managed by tox and may change between releases. + +Installs the package with the `test` extra (unless `skip_test_extra` is set) and any additional extras specified in `test_extras`. + +Automatically detects and primes Playwright if `plone.app.robotframework` is a dependency. + +### Test matrix + +When `use_test_matrix` is enabled (the default), the `test` environment is expanded into a matrix of environments combining Plone versions with Python versions. +Environment names follow the pattern `py-plone`, for example: + +- `py314-plone62` +- `py312-plone61` + +Each generated environment uses the appropriate constraints file for its Plone version (configured via `constraints_files` in the `[tox]` section). + +The matrix is configured via the `[tox] test_matrix` option in +{file}`.meta.toml`. +See {doc}`/reference/meta-toml` for details. + +## `coverage` + +Runs the test suite with coverage measurement and generates reports in Markdown, XML, and HTML formats. + +## `release-check` + +Validates the package is ready for release by checking changelog entries, building the distribution, and verifying the result. + +## `lint` + +Runs code quality and formatting checks via `pre-commit run --all-files`. + +## `format` + +Applies auto-formatting by running each formatter individually via `pre-commit run `. +Each formatter is invoked separately so that its changes can be committed and inspected independently. +This is particularly useful when porting old code, where subtle autoformatter changes might break the test suite. + +## `dependencies` + +Validates that all dependencies are properly declared using `z3c.dependencychecker`. + +## `dependencies-graph` + +Generates a visual dependency graph using `pipdeptree` and `graphviz`. + +## `circular` + +Checks for circular dependencies using `pipdeptree` and `pipforester`. +Requires `libgraphviz-dev` for graph visualization. diff --git a/docs/sources/reference/upgrade-guide.md b/docs/sources/reference/upgrade-guide.md new file mode 100644 index 00000000..6f301b1c --- /dev/null +++ b/docs/sources/reference/upgrade-guide.md @@ -0,0 +1,75 @@ +--- +myst: + html_meta: + "description": "Upgrade guide for breaking changes between plone.meta versions" + "property=og:description": "Upgrade guide for breaking changes between plone.meta versions" + "property=og:title": "Upgrade Guide" + "keywords": "plone.meta, upgrade, migration, breaking changes" +--- + +# Upgrade Guide + + + +This guide describes the breaking changes between major versions. + +## From `main` to `2.x` + +### Test matrix + +You can now define the combinations of Plone versions and Python versions to be tested. + +`plone.meta` provides a default combination (the same as used by Plone itself), but it can be overridden in {file}`.meta.toml`: + +```toml +[tox] +test_matrix = {"6.2" = ["3.14", "3.11"], "6.1" = ["3.13", "3.12", "3.11", "3.10"]} +``` + +This generates the necessary `tox` environments and the GitHub Actions that run on each pull request. + +:::{tip} +`plone.meta` tries to be a bit more environmentally friendly. +On GitHub, only the first and last Python versions are added for testing. +::: + +### Constraints + +The `constraints_file` option in {file}`.meta.toml`'s `[tox]` table was renamed to `constraints_files`, and the type of its value was changed from a string to a dictionary. + +This option continues to be optional. + +The dictionary keys must be Plone versions, and each key's value must be the constraints file for that Plone version. + +```toml +[tox] +# OLD +constraints_file = "https://example.org/my-custom-constraints.txt" +# NEW +constraints_files = {"6.2" = "https://example.org/constraints.6.2.txt", "6.1" = "https://example.org/constraints.6.1.txt"} +``` + +### GitHub Actions + +The `py_versions` option in {file}`.meta.toml`'s `[github]` table is deprecated. +Use the new `test_matrix` option from the `[tox]` table instead, as plone.meta can now run multiple Python versions from within `tox` itself. + +### GitHub variables + +The GitHub variables `TEST_OS_VERSIONS` and `TEST_PYTHON_VERSIONS` are deprecated and no longer used. + +### GitLab images + +The `custom_image` option in {file}`.meta.toml`'s `[gitlab]` table was renamed to `custom_images`, and the type of its value was changed from a string to a dictionary. + +This option continues to be optional. + +The dictionary keys must be Python versions, and the values a Docker image for that Python version. + +```toml +[gitlab] +# OLD +custom_image = "python:3.11-bullseye" +# NEW +custom_images = {"3.14" = "python:3.14-trixie", "3.13" = "python:3.13-trixie"} +``` diff --git a/docs/sources/tutorials/first-package.md b/docs/sources/tutorials/first-package.md new file mode 100644 index 00000000..cf508002 --- /dev/null +++ b/docs/sources/tutorials/first-package.md @@ -0,0 +1,136 @@ +--- +myst: + html_meta: + "description": "Tutorial: configure your first package with plone.meta" + "property=og:description": "Tutorial: configure your first package with plone.meta" + "property=og:title": "Configure Your First Package" + "keywords": "plone.meta, tutorial, first package, config-package" +--- + +# Configure Your First Package + + + +Run `config-package` on a Plone repository, inspect the generated files, and customize via {file}`.meta.toml`. +This workflow is especially useful for [collective](https://github.com/collective) repositories adopting standard Plone tooling. + +## Prerequisites + +- Python 3.10 or later (with [uv](https://docs.astral.sh/uv/) recommended) +- Git +- A Plone Python package repository to configure (or you can use any + single-package Python repository) + +## Run config-package + +Pick a target repository. +For this example, use `plone.api`: + +```shell +git clone https://github.com/plone/plone.api.git +``` + +Run `config-package` using `uvx`: + +```shell +uvx --from=plone.meta config-package plone.api +``` + +:::{note} +If you prefer a local installation instead of `uvx`, see +{doc}`/how-to/install` for the virtual environment approach, then run +`.venv/bin/config-package plone.api`. +::: + +The output shows: +1. A new git branch (named `config-with-default-template-`) +2. Configuration files written +3. A commit created + +## Inspect the results + +Look at the generated files: + +```shell +cd plone.api +git log --oneline -1 +git diff HEAD~1 --stat +``` + +`config-package` created or updated these files: + +- {file}`.meta.toml` -- plone.meta's own configuration +- {file}`.editorconfig` -- editor settings +- {file}`.flake8` -- flake8 linting rules +- {file}`.gitignore` -- git ignore patterns +- {file}`.github/workflows/meta.yml` -- GitHub Actions CI (for GitHub-hosted repos) +- {file}`.pre-commit-config.yaml` -- pre-commit hooks +- {file}`pyproject.toml` -- Python tooling configuration +- {file}`tox.ini` -- tox test environments + +## Understand .meta.toml + +Open {file}`.meta.toml`, the central configuration file: + +```toml +[meta] +template = "default" +commit-id = "a1b2c3d4" + +[tox] +test_runner = "zope.testrunner" +``` + +All your customizations go into {file}`.meta.toml`. +Re-running `config-package` reads this file and regenerates everything else. + +:::{important} +Do not edit generated files directly. +They are overwritten each time `config-package` runs. +All customization goes into {file}`.meta.toml`. +::: + +## Try a customization + +Add extra lines to the {file}`.flake8` configuration by editing {file}`.meta.toml`: + +```toml +[flake8] +extra_lines = """ +per-file-ignores = + setup.py:T20 +""" +``` + +Re-run `config-package` on the current branch: + +```shell +uvx --from=plone.meta config-package --branch current . +``` + +Check {file}`.flake8` -- your custom lines appear below the standard configuration. + +## Run the generated tox environments + +The generated {file}`tox.ini` provides several environments: + +```shell +# Run the test suite +tox -e test + +# Check code formatting +tox -e lint + +# Check if the package is release-ready +tox -e release-check +``` + +## Summary + +You now know how to: + +- Run `config-package` on a repository +- Inspect the generated files +- Customize behaviour through {file}`.meta.toml` +- Re-run `config-package` after configuration changes +- Use the generated tox environments diff --git a/docs/sources/tutorials/index.md b/docs/sources/tutorials/index.md new file mode 100644 index 00000000..46aa5ccf --- /dev/null +++ b/docs/sources/tutorials/index.md @@ -0,0 +1,33 @@ +--- +myst: + html_meta: + "description": "Step-by-step tutorials for plone.meta" + "property=og:description": "Step-by-step tutorials for plone.meta" + "property=og:title": "Tutorials" + "keywords": "plone.meta, tutorials, getting started" +--- + +# Tutorials + + + +Step-by-step lessons that guide you through using `plone.meta`. + +::::{grid} 2 +:gutter: 3 + +:::{grid-item-card} Configure Your First Package +:link: first-package +:link-type: doc + +Run `config-package` on a repository, and understand the files it generates. Learn how {file}`.meta.toml` controls the output. +::: + +:::: + +```{toctree} +--- +hidden: true +--- +first-package +```