Add opt-in TCPConnector(use_truststore=True) for OS-native trust store integration (#11705)#12702
Conversation
aio-libs#11705) Add a keyword-only ``use_truststore: bool = False`` parameter to ``TCPConnector``. When set to ``True``, certificate verification is delegated to the OS-native trust store via the ``truststore`` library, fixing ``CERTIFICATE_VERIFY_FAILED`` errors for users behind enterprise TLS-intercepting proxies whose root CA is installed in the macOS Keychain or Windows certificate stores but not in OpenSSL's default paths. - New dedicated optional extra: ``pip install aiohttp[truststore]`` - ``use_truststore=True`` without the dependency installed raises a friendly ``RuntimeError`` at construction time - ``use_truststore=True`` together with ``ssl=False`` raises ``ValueError`` - Explicit ``ssl=<SSLContext>`` always wins over ``use_truststore=True`` - Default behaviour is unchanged - Per-connector context instance, not a module-level singleton Also fixes a misleading sentence in ``docs/client_advanced.rst`` that claimed Python uses the system CA certificates by default (untrue on macOS, partially wrong on Windows). This is the "optional first" step from @webknjaz's structured plan in the issue; a follow-up PR can flip the default and add truststore as a hard dependency once the opt-in path has bake-in time. Tests: 12 new tests in ``tests/test_truststore.py`` covering all branches. Full ``tests/test_connector.py`` regression run: 179 passed, 7 skipped, 1 xfailed (no regression).
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #12702 +/- ##
========================================
Coverage 98.95% 98.95%
========================================
Files 131 132 +1
Lines 46688 46822 +134
Branches 2421 2427 +6
========================================
+ Hits 46200 46333 +133
Misses 366 366
- Partials 122 123 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Merging this PR will not alter performance
Comparing Footnotes
|
…1705) Three classes of CI failure from prior commit: 1. `mypy` (Linter job) — `Cannot find implementation or library stub for module named "truststore"`. CI does not install the optional dep, so `[mypy-truststore] ignore_missing_imports` alone is insufficient (only takes effect when mypy CAN find the module). Add per-import `# type: ignore[import-not-found,unused-ignore]` so the type-check passes whether or not the dep is present. Keep the .mypy.ini section for the case when the dep IS installed locally and somebody runs mypy with full env. 2. `flake8-requirements` (I900) — `'truststore' not listed as a requirement`. Same root cause: the optional dep is not in the default test requirements. Add `# noqa: I900` on each truststore import. Also add `truststore` (gated on Python >=3.10 cpython) to `requirements/test-common.in` so the test environment installs it and the skipif-gated tests actually run -- this also addresses the codecov patch-coverage drop from the prior commit (every truststore test was being skipped because the dep was missing). 3. `flake8-docstrings` D205/D209 — multi-line docstrings without a blank line between summary and description and with closing quotes on the body line. Collapse the affected docstrings to single-line summaries; preserve multi-line form only where there is genuine prose after the summary (and add the blank line / standalone closing quotes where preserved). Verified locally with truststore installed and uninstalled: 0 errors on `aiohttp/connector.py` and `tests/test_truststore.py` under `mypy aiohttp/connector.py tests/test_truststore.py` and `flake8 -j 1 aiohttp/connector.py tests/test_truststore.py`. Note: requirements/*.txt lock files are not regenerated in this commit because pip-compile output depends on the resolving Python version and produces off-target diffs; the maintainer can regenerate via the project's standard workflow. Refs aio-libs#11705
for more information, see https://pre-commit.ci
Closes
Closes #11705 (in part — this is the additive, opt-in step that the issue's action list anticipates as "follow pip's example and make it optional first").
What this changes
Adds a new keyword-only
use_truststore: bool = Falseparameter toTCPConnector. When set toTrue, certificate verification is delegated to the OS-native trust store via thetruststorelibrary, fixingCERTIFICATE_VERIFY_FAILEDerrors for users behind enterprise TLS-intercepting proxies whose root CA is installed in the macOS Keychain or Windows certificate stores but not in OpenSSL's default paths.pip install aiohttp[truststore](kept separate fromspeedupsbecause trust-store integration is a correctness/functionality concern, not a perf accelerator).use_truststore=Truewithout the dependency installed raises a friendlyRuntimeErroratTCPConnectorconstruction, not deep in an async handshake stack.use_truststore=Truetogether withssl=FalseraisesValueError— there is no concept of an "unverified truststore" context.ssl=<SSLContext>always wins overuse_truststore=True.ssl=,ssl_context=, or rely on defaults see zero behaviour change.truststoreis an OPTIONAL dependency. Without it installed, aiohttp works exactly as today.Why this is "step 1"
The issue body explicitly leaves the rollout strategy open:
This PR is the "optional first" step. A follow-up PR can flip the default and add truststore as a hard dependency once the opt-in path has bake-in time, and that follow-up will also expand
THREAT_MODEL.mdto cover the new default trust source. Keeping the steps separate makes the design conversation per PR small and reviewable.Documentation
docs/client_advanced.rst: fixed the misleading "By default, Python uses the system CA certificates." sentence (which is wrong on macOS, partially wrong on Windows) and added a new "Example: Use truststore for system trust stores" section explaining the corporate-proxy use case.docs/client_reference.rst: addeduse_truststoreto theTCPConnectorsignature block and a parameter description with.. versionadded:: 3.14.Tests
12 new tests in
tests/test_truststore.pycovering all branches: default off, off-no-import, on-with-truststore, on-without-truststore, ssl=False guard, explicit-context precedence, per-instance isolation, dispatch return value, fingerprint composition, unverified-path ignores flag, ALPN compatibility, and the import helper itself. Truststore-dependent tests usepytest.importorskipso the suite still runs without the optional dependency.Local test results:
12 passed, 0 failed. Fulltests/test_connector.pyregression run:179 passed, 7 skipped, 1 xfailed— no regression.Manual reproduction
End-to-end smoke test (trustme-issued self-signed cert)
TCPConnector(use_truststore=True)constructs atruststore.SSLContextinstance. A separate HTTPS round trip against a self-signedtrustme-issued cert confirms the wider SSL pipeline still works under truststore monkey-patching. Output:Checklist
CHANGES/11705.feature.rst)CONTRIBUTORS.txtCredit
Thank you @sethmlarson for the
truststorelibrary, and @webknjaz for the structured tracking issue and the open call for new volunteers.