Skip to content

Move pytest config into a pytest.ini#12621

Open
aiolibsbot wants to merge 8 commits into
aio-libs:masterfrom
aiolibsbot:koan/implement-12620
Open

Move pytest config into a pytest.ini#12621
aiolibsbot wants to merge 8 commits into
aio-libs:masterfrom
aiolibsbot:koan/implement-12620

Conversation

@aiolibsbot
Copy link
Copy Markdown
Contributor

@aiolibsbot aiolibsbot commented May 18, 2026

What do these changes do?

Port and adapt the shared aio-libs pytest.ini layout (from yarl,
frozenlist, propcache, cheroot, pylibssh, awx-plugins, and
sphinx-contrib/towncrier) and move the existing pytest configuration
out of setup.cfg into a dedicated top-level pytest.ini.

All aiohttp-specific settings are preserved: asyncio_mode = auto,
the -m "not dev_mode and not autobahn and not internal" default
marker filter, -p no:aiohttp to keep the entry-point plugin out of
the coverage data, the full filterwarnings list, the custom
markers (autobahn, dev_mode, internal, skip_blockbuster),
and junit_suite_name = aiohttp_test_suite.

The port also picks up the aio-libs defaults that aiohttp's
setup.cfg lacked: --numprocesses=auto (already used in CI and
documented in AGENTS.md), --strict-markers,
empty_parameter_set_mark = xfail, faulthandler_timeout = 30,
doctest_optionflags, the richer junit_* reporting flags, and an
expanded norecursedirs list.

Coverage flags (--cov, --cov-config) and --doctest-modules
were deliberately not added, so existing local and CI pytest
invocations behave the same — CI continues to drive coverage via
--cov=aiohttp/ --cov=tests/ on the command line and the
.coveragerc.toml it already points at.

Are there changes in behavior for the user?

No runtime behavior changes. Contributor-facing only: pytest now picks
up its configuration from pytest.ini instead of the
[tool:pytest] section of setup.cfg.

Is it a substantial burden for the maintainers to support this?

No — the new file mirrors the layout already used across other
aio-libs projects, so future syncs are a one-file diff.

Related issue number

Closes #12620.

Checklist

  • I think the code is well written
  • Unit tests for the changes exist — N/A (config-only change; the existing test suite continues to exercise it).
  • Documentation reflects the changes — N/A (no user-facing API surface).
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt — N/A (bot author).
  • Add a new news fragment into the CHANGES/ folder (CHANGES/12620.contrib.rst).

Drafted with Claude Opus 4.7 via Kōan; reviewed by .


Quality Report

Changes: 3 files changed, 115 insertions(+), 62 deletions(-)

Code scan: clean

Tests: failed (FAILED)

Branch hygiene: clean

Generated by Kōan post-mission quality pipeline

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label May 18, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.95%. Comparing base (37cd4a8) to head (e9fd583).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #12621   +/-   ##
=======================================
  Coverage   98.95%   98.95%           
=======================================
  Files         131      131           
  Lines       46620    46620           
  Branches     2418     2418           
=======================================
  Hits        46132    46132           
  Misses        366      366           
  Partials      122      122           
Flag Coverage Δ
Autobahn 22.40% <ø> (ø)
CI-GHA 98.92% <ø> (ø)
OS-Linux 98.67% <ø> (ø)
OS-Windows 97.03% <ø> (-0.01%) ⬇️
OS-macOS 97.93% <ø> (+<0.01%) ⬆️
Py-3.10 98.15% <ø> (ø)
Py-3.11 98.41% <ø> (+<0.01%) ⬆️
Py-3.12 98.50% <ø> (-0.01%) ⬇️
Py-3.13 98.47% <ø> (+<0.01%) ⬆️
Py-3.14 98.49% <ø> (-0.01%) ⬇️
Py-3.14t 97.55% <ø> (+<0.01%) ⬆️
Py-pypy-3.11 97.42% <ø> (+<0.01%) ⬆️
VM-macos 97.93% <ø> (+<0.01%) ⬆️
VM-ubuntu 98.67% <ø> (ø)
VM-windows 97.03% <ø> (-0.01%) ⬇️
cython-coverage 37.94% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 18, 2026

Merging this PR will not alter performance

✅ 68 untouched benchmarks
⏩ 72 skipped benchmarks1


Comparing aiolibsbot:koan/implement-12620 (e9fd583) with master (37cd4a8)

Open in CodSpeed

Footnotes

  1. 72 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@bdraco
Copy link
Copy Markdown
Member

bdraco commented May 18, 2026

note for an hour from now
will need to rebase to pickup #12624 after it merges so it gets lifted as well

@bdraco
Copy link
Copy Markdown
Member

bdraco commented May 18, 2026

@aiolibsbot rr

Comment thread CHANGES/12620.contrib.rst Outdated
@@ -0,0 +1,3 @@
Moved the pytest configuration out of ``setup.cfg`` into a dedicated
``pytest.ini`` file, aligning the layout with other ``aio-libs``
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use :file: role here and reduce verbosity

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, it reduced verbosity of the change note, but I originally meant to drop -v from the config. Another time, then.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — since this PR is already rewriting the entire pytest config block, it's a natural place to drop -v rather than deferring. I'll do that in the next push: removing the -v line (and its leading # A bit of verbosity doesn't hurt: comment) from addopts. Contributors who want verbose output can still pass -v/-vv on the command line, and CI logs will shrink noticeably across the matrix. If you'd rather keep the scope of this PR strictly to the file move and handle -v separately, happy to defer — just let me know.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in the latest push — the -v line and its leading comment have been dropped from addopts in pytest.ini. Contributors who want verbose output can pass -v/-vv on the command line, and CI logs should shrink across the matrix. Sorry for the misread on the original request.

@aiolibsbot
Copy link
Copy Markdown
Contributor Author

aiolibsbot commented May 18, 2026

PR Review — Move pytest config into a pytest.ini

Mechanical move of pytest config is mostly clean and the file mirrors the shared aio-libs layout as intended. However, three things block readiness: (1) the post-mission quality report says tests are FAILED — most likely the new --strict-markers flag, which must be reconciled before merge; (2) empty_parameter_set_mark = xfail and the (mis)claimed --numprocesses=auto contradict the 'no runtime behavior change' framing in the PR description; (3) webknjaz's pytest-cov request is still unresolved. Once those are addressed and the suite is green, this is a straightforward port.


🟡 Important

1. `--numprocesses=auto` is missing from addopts despite PR description (`pytest.ini`, L2-26)

The PR description states the port picks up the aio-libs default --numprocesses=auto, but the new addopts block does not contain it. Either add --numprocesses=auto to addopts (which would actually change behavior since CI/AGENTS.md already pass it explicitly, but local single-file invocations would now parallelize), or remove the claim from the PR description so reviewers aren't misled. Given the PR otherwise advertises 'no runtime behavior changes', dropping the claim from the description is the safer move.

addopts =
  --durations=10
  -ra
  --showlocals
  --strict-markers
  -m "not dev_mode and not autobahn and not internal"
  -p no:aiohttp
2. `empty_parameter_set_mark = xfail` is a behavior change, not a no-op port (`pytest.ini`, L32-33)

pytest's default for empty_parameter_set_mark is skip. Switching to xfail means any @pytest.mark.parametrize("x", []) site that currently silently skips will now be reported as xfailed — and combined with xfail_strict = true already in the config, an empty parameter set could become a hard failure if it ever 'unexpectedly passes' (which is conceptually weird but possible if pytest treats the no-collection case that way). The PR description says 'no runtime behavior changes', but this is one. Worth a sanity grep for empty parametrize lists in the suite, and a sentence in the PR description acknowledging the semantics shift — or revert this single line to default to keep the port truly behavior-preserving.

# Marks tests with an empty parameterset as xfail(run=False)
empty_parameter_set_mark = xfail
3. Newly added `--strict-markers` may explain the failing tests (`pytest.ini`, L22)

Tests are reported as failing in the quality report. --strict-markers is new to this config and will hard-fail any test that uses a marker not listed in the markers = block (lines 76–81). The block lists autobahn, dev_mode, internal, skip_blockbuster — but pytest plugins in use (pytest-asyncio adds asyncio, pytest-xdist adds nothing problematic, pytest-trio if used, etc.) and the suite itself may apply markers not enumerated here. Please run PYTHONPATH='.' pytest --collect-only and confirm no PytestUnknownMarkWarning previously hidden gets promoted to a collection error. If undocumented markers exist in the suite, either register them in this list or drop --strict-markers for now.

  --strict-markers
4. Resolve coverage/pytest-cov disagreement with webknjaz before merge

webknjaz explicitly asked to 'Recover the pytest-cov related settings' in their review. The PR description argues these were deliberately omitted because CI passes --cov flags on the command line. That's a defensible position, but it's an unresolved request-changes signal from a maintainer — please either (a) push back in-thread with the rationale (CI invocation already covers it; adding --cov to addopts here would double-instrument or conflict with the CI-side .coveragerc.toml), or (b) restore the coverage flags as requested. Don't let this PR go to ready-for-review with the maintainer disagreement unaddressed.


Checklist

  • No hardcoded secrets
  • Config matches PR description claims — warning #1
  • Behavior-preserving as advertised — warning #2
  • Existing test suite passes locally — warning #4
  • Maintainer review feedback addressed — warning #6
  • Changelog fragment present and formatted correctly

Summary

Mechanical move of pytest config is mostly clean and the file mirrors the shared aio-libs layout as intended. However, three things block readiness: (1) the post-mission quality report says tests are FAILED — most likely the new --strict-markers flag, which must be reconciled before merge; (2) empty_parameter_set_mark = xfail and the (mis)claimed --numprocesses=auto contradict the 'no runtime behavior change' framing in the PR description; (3) webknjaz's pytest-cov request is still unresolved. Once those are addressed and the suite is green, this is a straightforward port.


Automated review by Kōand501c6c
ed63d09
4776798
336a37a
531e23d
fdb6cd6
4ae930a

@aiolibsbot
Copy link
Copy Markdown
Contributor Author

Rebase with requested adjustments

Branch koan/implement-12620 was rebased onto master and review feedback was applied.

Changes applied

  • Applied @bdraco's feedback:
  • Switched setup.cfg and pytest.ini literals to :file: role per reviewer request.
  • Tightened wording from three lines to two, dropping the "aligning with other aio-libs projects" clause to reduce verbosity.
  • CI fix: Applied pre-push CI fix
  • CI fix: Pre-push CI fix applied

Stats

3 files changed, 115 insertions(+), 65 deletions(-)
Actions performed
  • Already-solved check: negative (confidence=high, reasoning=No recent master commit moves pytest config from setup.cfg to a dedicated pytest.ini.)
  • Resolved merge conflicts (1 round(s))
  • Rebased koan/implement-12620 onto upstream/master
  • Applied review feedback
  • Pre-push CI check: previous run #26039514171 failed
  • Applied pre-push CI fix
  • Pre-push CI fix applied
  • Force-pushed koan/implement-12620 to origin
  • CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.


Automated by Kōan

aiolibsbot added a commit to aiolibsbot/aiohttp that referenced this pull request May 18, 2026
aiolibsbot added a commit to aiolibsbot/aiohttp that referenced this pull request May 18, 2026
@aiolibsbot aiolibsbot force-pushed the koan/implement-12620 branch from ff0490d to 80f140b Compare May 18, 2026 15:17
@bdraco bdraco added the backport-3.14 Trigger automatic backporting to the 3.14 release branch by Patchback robot label May 18, 2026
@webknjaz webknjaz added the backport-3.13 Trigger automatic backporting to the 3.13 release branch by Patchback robot label May 18, 2026
@webknjaz webknjaz marked this pull request as ready for review May 18, 2026 15:42
@webknjaz webknjaz requested a review from asvetlov as a code owner May 18, 2026 15:42
@webknjaz webknjaz requested a review from bdraco May 18, 2026 15:42
@webknjaz webknjaz enabled auto-merge (squash) May 18, 2026 18:25
@webknjaz
Copy link
Copy Markdown
Member

@aiolibsbot rr

@aiolibsbot
Copy link
Copy Markdown
Contributor Author

@bdraco:

Acknowledged — will rebase once the recent master commits (including #12624's per-test timeout work) are in, so the timeout = 120 line moves over cleanly without a conflict. Will also verify locally that no tooling still reads [tool:pytest] from setup.cfg after the rebase, since the local quality pipeline currently reports a test failure that codecov doesn't see.

@aiolibsbot
Copy link
Copy Markdown
Contributor Author

Rebase with requested adjustments

Branch koan/implement-12620 was rebased onto master and review feedback was applied.

Changes applied

  • Summary:
  • Dropped -v flag and its # A bit of verbosity doesn't hurt: comment from pytest.ini addopts, per @webknjaz's request (clarified after @bdraco's :file: note) and @aiolibsbot's agreement to handle it in-scope. Contributors can still pass -v/-vv on the command line.
  • :file: role in CHANGES/12620.contrib.rst already addressed in the pushed diff (no further change needed).

Stats

3 files changed, 112 insertions(+), 65 deletions(-)
Actions performed
  • Already-solved check: negative (confidence=high, reasoning=No commit on master moves pytest config from setup.cfg to a dedicated pytest.ini.)
  • Rebased koan/implement-12620 onto upstream/master
  • Applied review feedback
  • Pre-push CI check: previous run #26042614602 failed
  • Pre-push CI fix: no changes needed or Claude found nothing to fix
  • Force-pushed koan/implement-12620 to origin
  • CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.


Automated by Kōan

aiolibsbot added a commit to aiolibsbot/aiohttp that referenced this pull request May 18, 2026
aiolibsbot added a commit to aiolibsbot/aiohttp that referenced this pull request May 18, 2026
aiolibsbot added a commit to aiolibsbot/aiohttp that referenced this pull request May 18, 2026
auto-merge was automatically disabled May 18, 2026 19:35

Head branch was pushed to by a user without write access

@aiolibsbot aiolibsbot force-pushed the koan/implement-12620 branch from 80f140b to 7fef07b Compare May 18, 2026 19:36
@webknjaz webknjaz enabled auto-merge (squash) May 18, 2026 20:04
webknjaz pushed a commit to aiolibsbot/aiohttp that referenced this pull request May 18, 2026
webknjaz pushed a commit to aiolibsbot/aiohttp that referenced this pull request May 18, 2026
webknjaz pushed a commit to aiolibsbot/aiohttp that referenced this pull request May 18, 2026
@webknjaz webknjaz force-pushed the koan/implement-12620 branch from 7fef07b to 531e23d Compare May 18, 2026 23:21
@Dreamsorcerer Dreamsorcerer removed the backport-3.13 Trigger automatic backporting to the 3.13 release branch by Patchback robot label May 19, 2026
@webknjaz webknjaz changed the title Move pytest config into a dedicated pytest.ini Move pytest config into a pytest.ini May 19, 2026
@webknjaz webknjaz added the backport-3.13 Trigger automatic backporting to the 3.13 release branch by Patchback robot label May 19, 2026
Comment thread pytest.ini Outdated
vendor
venv
virtualenv
aiohttp.egg-info
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
aiohttp.egg-info
*.egg-info

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fold this in on the next push. I'll replace the explicit aiohttp.egg-info entry with the generic *.egg-info glob so the pattern covers any future packaged subpackages too, and keep *.egg alongside it since they match different layouts.

Copy link
Copy Markdown
Member

@webknjaz webknjaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recover the pytest-cov related settings. Also, pytest.ini is not deprecated. The point is having a dedicated file that can be easily synchronized.

@aiolibsbot rr

@bdraco
Copy link
Copy Markdown
Member

bdraco commented May 19, 2026

@aiolibsbot rr

@bdraco
Copy link
Copy Markdown
Member

bdraco commented May 19, 2026

it feel asleep. I woke it up

@aiolibsbot
Copy link
Copy Markdown
Contributor Author

@bdraco:

Acknowledged — will rebase onto master once #12624 lands so the lift is picked up here as well. If #12624 ends up touching setup.cfg or pytest.ini directly, I'll resolve the conflict in favor of the aio-libs-style layout.

@aiolibsbot
Copy link
Copy Markdown
Contributor Author

Rebase with requested adjustments

Branch koan/implement-12620 was rebased onto master and review feedback was applied.

Changes applied

  • Summary:
  • Replaced aiohttp.egg-info with *.egg-info in pytest.ini norecursedirs per @webknjaz suggestion at pytest.ini:96, so the pattern covers any packaged subpackages, not only the top-level aiohttp.egg-info directory.
  • Other review threads already addressed in prior commits on this branch (:file: role and -v removal); @webknjaz's pytest-cov request was reverted by maintainer Sam Bull (commit 2e00f68) so left as-is.

Stats

3 files changed, 112 insertions(+), 65 deletions(-)
Actions performed
  • Already-solved check: negative (confidence=high, reasoning=No recent master commit moves pytest config from setup.cfg to pytest.ini.)
  • Rebased koan/implement-12620 onto upstream/master
  • Applied review feedback
  • Pre-push CI check: previous run #26098771055 failed
  • Pre-push CI fix: no changes needed or Claude found nothing to fix
  • Force-pushed koan/implement-12620 to origin
  • CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.


Automated by Kōan

auto-merge was automatically disabled May 19, 2026 17:49

Head branch was pushed to by a user without write access

@aiolibsbot aiolibsbot force-pushed the koan/implement-12620 branch from 4ae930a to e9fd583 Compare May 19, 2026 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-3.13 Trigger automatic backporting to the 3.13 release branch by Patchback robot backport-3.14 Trigger automatic backporting to the 3.14 release branch by Patchback robot bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pytest config sync

4 participants