Skip to content

Add opt-in TCPConnector(use_truststore=True) for OS-native trust store integration (#11705)#12702

Open
Krishnachaitanyakc wants to merge 4 commits into
aio-libs:masterfrom
Krishnachaitanyakc:truststore-opt-in-11705
Open

Add opt-in TCPConnector(use_truststore=True) for OS-native trust store integration (#11705)#12702
Krishnachaitanyakc wants to merge 4 commits into
aio-libs:masterfrom
Krishnachaitanyakc:truststore-opt-in-11705

Conversation

@Krishnachaitanyakc
Copy link
Copy Markdown
Contributor

Closes

Closes #11705 (in part — this is the additive, opt-in step that the issue's action list anticipates as "follow pip's example and make it optional first").

What this changes

Adds a new keyword-only use_truststore: bool = False parameter to TCPConnector. When set to True, certificate verification is delegated to the OS-native trust store via the truststore library, fixing CERTIFICATE_VERIFY_FAILED errors for users behind enterprise TLS-intercepting proxies whose root CA is installed in the macOS Keychain or Windows certificate stores but not in OpenSSL's default paths.

  • New dedicated optional extra: pip install aiohttp[truststore] (kept separate from speedups because trust-store integration is a correctness/functionality concern, not a perf accelerator).
  • use_truststore=True without the dependency installed raises a friendly RuntimeError at TCPConnector construction, not deep in an async handshake stack.
  • use_truststore=True together with ssl=False raises ValueError — there is no concept of an "unverified truststore" context.
  • An explicit ssl=<SSLContext> always wins over use_truststore=True.
  • Default behaviour is unchanged. Existing users who pass ssl=, ssl_context=, or rely on defaults see zero behaviour change.
  • truststore is an OPTIONAL dependency. Without it installed, aiohttp works exactly as today.
  • Per-connector context instance, not a module-level singleton — preserves zero-import-cost for users who do not opt in.

Why this is "step 1"

The issue body explicitly leaves the rollout strategy open:

truststore should probably be a mandatory runtime dependency in packaging core metadata; although, maybe we need to follow pip's example and make it optional first (via extras or manual install) and then add it unconditionally later.

This PR is the "optional first" step. A follow-up PR can flip the default and add truststore as a hard dependency once the opt-in path has bake-in time, and that follow-up will also expand THREAT_MODEL.md to cover the new default trust source. Keeping the steps separate makes the design conversation per PR small and reviewable.

Documentation

  • docs/client_advanced.rst: fixed the misleading "By default, Python uses the system CA certificates." sentence (which is wrong on macOS, partially wrong on Windows) and added a new "Example: Use truststore for system trust stores" section explaining the corporate-proxy use case.
  • docs/client_reference.rst: added use_truststore to the TCPConnector signature block and a parameter description with .. versionadded:: 3.14.

Tests

12 new tests in tests/test_truststore.py covering all branches: default off, off-no-import, on-with-truststore, on-without-truststore, ssl=False guard, explicit-context precedence, per-instance isolation, dispatch return value, fingerprint composition, unverified-path ignores flag, ALPN compatibility, and the import helper itself. Truststore-dependent tests use pytest.importorskip so the suite still runs without the optional dependency.

Local test results: 12 passed, 0 failed. Full tests/test_connector.py regression run: 179 passed, 7 skipped, 1 xfailed — no regression.

Manual reproduction

End-to-end smoke test (trustme-issued self-signed cert)

TCPConnector(use_truststore=True) constructs a truststore.SSLContext instance. A separate HTTPS round trip against a self-signed trustme-issued cert confirms the wider SSL pipeline still works under truststore monkey-patching. Output:

OK: TCPConnector(use_truststore=True) built a truststore.SSLContext
OK: HTTPS round trip succeeded: status=200 body='ok-truststore'

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • Added a CHANGES file (CHANGES/11705.feature.rst)
  • Added myself to CONTRIBUTORS.txt
  • Followed AGENTS.md branching and draft-PR rules

Credit

Thank you @sethmlarson for the truststore library, and @webknjaz for the structured tracking issue and the open call for new volunteers.

aio-libs#11705)

Add a keyword-only ``use_truststore: bool = False`` parameter to
``TCPConnector``. When set to ``True``, certificate verification is
delegated to the OS-native trust store via the ``truststore`` library,
fixing ``CERTIFICATE_VERIFY_FAILED`` errors for users behind enterprise
TLS-intercepting proxies whose root CA is installed in the macOS Keychain
or Windows certificate stores but not in OpenSSL's default paths.

- New dedicated optional extra: ``pip install aiohttp[truststore]``
- ``use_truststore=True`` without the dependency installed raises a
  friendly ``RuntimeError`` at construction time
- ``use_truststore=True`` together with ``ssl=False`` raises ``ValueError``
- Explicit ``ssl=<SSLContext>`` always wins over ``use_truststore=True``
- Default behaviour is unchanged
- Per-connector context instance, not a module-level singleton

Also fixes a misleading sentence in ``docs/client_advanced.rst`` that
claimed Python uses the system CA certificates by default (untrue on
macOS, partially wrong on Windows).

This is the "optional first" step from @webknjaz's structured plan in
the issue; a follow-up PR can flip the default and add truststore as a
hard dependency once the opt-in path has bake-in time.

Tests: 12 new tests in ``tests/test_truststore.py`` covering all
branches. Full ``tests/test_connector.py`` regression run: 179 passed,
7 skipped, 1 xfailed (no regression).
@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label May 26, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 26, 2026

Codecov Report

❌ Patch coverage is 99.26471% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 98.95%. Comparing base (7f0532c) to head (f2940f3).
⚠️ Report is 2 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
tests/test_truststore.py 99.13% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           master   #12702    +/-   ##
========================================
  Coverage   98.95%   98.95%            
========================================
  Files         131      132     +1     
  Lines       46688    46822   +134     
  Branches     2421     2427     +6     
========================================
+ Hits        46200    46333   +133     
  Misses        366      366            
- Partials      122      123     +1     
Flag Coverage Δ
Autobahn 22.42% <25.00%> (+<0.01%) ⬆️
CI-GHA 98.92% <98.52%> (-0.01%) ⬇️
OS-Linux 98.67% <98.52%> (-0.01%) ⬇️
OS-Windows 97.03% <96.32%> (-0.01%) ⬇️
OS-macOS 97.92% <96.32%> (-0.01%) ⬇️
Py-3.10 98.14% <96.32%> (-0.02%) ⬇️
Py-3.11 98.41% <96.32%> (-0.01%) ⬇️
Py-3.12 98.49% <96.32%> (-0.01%) ⬇️
Py-3.13 98.46% <96.32%> (-0.01%) ⬇️
Py-3.14 98.48% <96.32%> (-0.02%) ⬇️
Py-3.14t 97.55% <96.32%> (-0.02%) ⬇️
Py-pypy-3.11 97.30% <55.14%> (-0.15%) ⬇️
VM-macos 97.92% <96.32%> (-0.01%) ⬇️
VM-ubuntu 98.67% <98.52%> (-0.01%) ⬇️
VM-windows 97.03% <96.32%> (-0.01%) ⬇️
cython-coverage 37.84% <3.67%> (-0.11%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 26, 2026

Merging this PR will not alter performance

✅ 72 untouched benchmarks
⏩ 72 skipped benchmarks1


Comparing Krishnachaitanyakc:truststore-opt-in-11705 (f2940f3) with master (7f0532c)2

Open in CodSpeed

Footnotes

  1. 72 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on master (b866c68) during the generation of this report, so 7f0532c was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

Krishnachaitanyakc and others added 3 commits May 26, 2026 08:50
…1705)

Three classes of CI failure from prior commit:

1. `mypy` (Linter job) — `Cannot find implementation or library stub for
   module named "truststore"`. CI does not install the optional dep, so
   `[mypy-truststore] ignore_missing_imports` alone is insufficient
   (only takes effect when mypy CAN find the module). Add per-import
   `# type: ignore[import-not-found,unused-ignore]` so the type-check
   passes whether or not the dep is present. Keep the .mypy.ini section
   for the case when the dep IS installed locally and somebody runs
   mypy with full env.

2. `flake8-requirements` (I900) — `'truststore' not listed as a
   requirement`. Same root cause: the optional dep is not in the
   default test requirements. Add `# noqa: I900` on each truststore
   import. Also add `truststore` (gated on Python >=3.10 cpython) to
   `requirements/test-common.in` so the test environment installs it
   and the skipif-gated tests actually run -- this also addresses the
   codecov patch-coverage drop from the prior commit (every truststore
   test was being skipped because the dep was missing).

3. `flake8-docstrings` D205/D209 — multi-line docstrings without a
   blank line between summary and description and with closing quotes
   on the body line. Collapse the affected docstrings to single-line
   summaries; preserve multi-line form only where there is genuine
   prose after the summary (and add the blank line / standalone closing
   quotes where preserved).

Verified locally with truststore installed and uninstalled: 0 errors
on `aiohttp/connector.py` and `tests/test_truststore.py` under
`mypy aiohttp/connector.py tests/test_truststore.py` and `flake8 -j 1
aiohttp/connector.py tests/test_truststore.py`.

Note: requirements/*.txt lock files are not regenerated in this commit
because pip-compile output depends on the resolving Python version and
produces off-target diffs; the maintainer can regenerate via the
project's standard workflow.

Refs aio-libs#11705
@Krishnachaitanyakc Krishnachaitanyakc marked this pull request as ready for review May 26, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[TODO] Use truststore in place of ssl by default

1 participant