Skip to content

Consistent phpunit runners for local and CI#493

Open
kasparsd wants to merge 71 commits into
fairpm:release_1.4.1from
kasparsd:add-phpstan-fresh
Open

Consistent phpunit runners for local and CI#493
kasparsd wants to merge 71 commits into
fairpm:release_1.4.1from
kasparsd:add-phpstan-fresh

Conversation

@kasparsd

@kasparsd kasparsd commented May 15, 2026

Copy link
Copy Markdown
Contributor

Fixes #479, fixes #383.

A follow-up to #382.

Development workflow consistency:

  • Standardizes local development and CI around wp-env so both paths use the same environment bootstrapping and runtime logic.
  • Adds a small npm command layer for common tasks like CLI access, linting, formatting, PHPUnit, and coverage, so developers and CI invoke the same commands.
  • Updates the coding standards workflow to run through the wp-env-based toolchain instead of relying on a separate runner-level PHP setup.

Local setup and onboarding:

  • Commits a Node version via .nvmrc and upgrades @wordpress/env, reducing local setup drift and keeping machines closer to CI.
  • Expands the contributing docs to make wp-env the default local workflow, covering start/log/stop commands, PHPUnit usage, coverage, and version overrides.
  • Supports both .wp-env.override.json and WP_ENV_PHP_VERSION / WP_ENV_CORE env vars, making local environment customization straightforward without changing repo defaults.

Local PHPUnit execution:

  • Moves PHPUnit execution into the wp-env tests container, removing the old split between custom local setup and CI-only test installation.
  • Adds a dedicated tests:cli path and install flow for test dependencies, so local test runs are easier to reason about and reuse.
  • Aligns the single-site and multisite PHPUnit config files with this workflow, including cleaner cache and coverage paths.

WordPress and PHP version matrix support:

  • Makes it possible to run PHPUnit locally against different WordPress core and PHP combinations by driving wp-env from override files or environment variables.
  • Broadens Composer dev tooling to support multiple PHPUnit versions and include the WP-CLI bundle needed for this workflow.
  • Replaces the hardcoded GitHub Actions exclusion list with a generated PHP/WordPress compatibility matrix, centralizing supported combinations in one script.

CI parity with local testing:

  • Uses the generated matrix in GitHub Actions so each job boots the requested WordPress and PHP combination the same way it can be done locally.
  • Refreshes Composer dependencies per matrix job so the right PHPUnit version is installed for that exact WordPress/PHP pairing.
  • Extends the same wp-env-based approach to coverage runs, improving parity between local verification and CI.

CI timing

Previously the phpunit setup in CI didn't use wp-env leading to two different ways of running the same tests. Switching to wp-env ensures that we can run the tests with any combination of PHP and WP core both locally and during CI.

Running the phpunit test matrix in CI before the change took 1min 42s:

matrix-before

while now it takes 3min 16s (or 2min 30s in recent runs):

matrix-after

which is a minor increase but offers consistency and predictability.

@github-actions

Copy link
Copy Markdown
Contributor

@kasparsd kasparsd force-pushed the add-phpstan-fresh branch from 783b8d0 to 15e03a5 Compare May 15, 2026 13:43
@cdils cdils changed the base branch from main to release_1.5.0 May 15, 2026 13:52
@cdils cdils added this to the 1.5.0 milestone May 15, 2026
kasparsd added 26 commits May 15, 2026 16:55
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
During CI we require in versions as needed

Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
kasparsd added 6 commits May 15, 2026 16:55
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
@kasparsd kasparsd force-pushed the add-phpstan-fresh branch from 051ece5 to 7ec0085 Compare May 15, 2026 13:55
Comment thread .github/workflows/coding-standards.yml
Comment thread .github/workflows/phpunit-tests.yml
Comment thread .github/workflows/phpunit-tests.yml
Comment thread inc/icons/svg.php
Comment thread composer.json
Comment thread composer.json
Comment thread phpcs.xml.dist
Comment thread phpcs.xml.dist
Comment thread phpstan.dist.neon
Comment thread phpunit-multisite.xml.dist
kasparsd added 4 commits May 15, 2026 17:05
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
@kasparsd kasparsd changed the base branch from release_1.5.0 to release_1.4.1 May 22, 2026 13:45
@chuckadams

Copy link
Copy Markdown
Contributor

I personally have less than zero interest in using wp-env as the standard environment: a dependency on node in order to run bash scripts in order to run PHP. If this is the easiest way to switch WP versions quickly in CI, especially if it can cache the different versions, then I'm not going to stand in the way, but I would very much prefer to avoid gratuitous Node dependencies as far as the officially sanctioned dev workflow goes, and would rather see something like DDEV be it instead.

@kasparsd

Copy link
Copy Markdown
Contributor Author

if this is the easiest way to switch WP versions quickly in CI, especially if it can cache the different versions

Yeah, this setup (and the updated documentation) allows us to consistently test the plugin against any version of PHP and WP core. Previously the setup used locally (wp-env) was different from what the CI would use, so there was no clear way to replicate failing builds.

I absolutely don't love wp-env but it gets the job done and is a shared convention that others know how to debug and use. The previous mix of bash scripts, system dependencies installed only in CI to run the same tests and linters was arguably more convoluted and harder to maintain.

@cdils

cdils commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

@kasparsd Beyond resolving conflicts, are you waiting for further review?

@kasparsd

Copy link
Copy Markdown
Contributor Author

@cdils if we agree on the concept (same tooling locally and in CI for consistency) then I believe this is ready for merge.

I'll resolve the merge conflicts now.

kasparsd added 2 commits June 15, 2026 21:59
…fresh

Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
@kasparsd

Copy link
Copy Markdown
Contributor Author

@cdils the conflicts have been resolved ✅

kasparsd added 4 commits June 15, 2026 22:20
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Signed-off-by: Kaspars Dambis <hi@kaspars.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🚀 Feature Request: Review and update Unit tests 🚀 Feature Request: Use wp-env tooling for PHPUnit CI runs

4 participants