Mirror fallback: ClientPayloadError when json-mirror drops connection mid-download (no graceful fallback to json-nvd)

[Chetan-code-lrca]

Environment

• OS: Windows 11
• Python: 3.14.3
• cve-bin-tool: 3.4
• aiohttp: 3.13.3

Steps to reproduce

pip install cve-bin-tool # installs under Python 3.14
cve-bin-tool --input-file requirements.txt

Expected

CVE database downloads successfully

Actual

Crashes at ~67% with:
ClientPayloadError: Response payload is not completed:
<ContentLengthError: 400, message='Not enough data to satisfy content length header.'>

Full traceback

C:\Users\cheta\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\aiohttp\client_proto.py:144 in connection_lost
C:\Users\cheta\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\cve_bin_tool\cli.py:891 in main
C:\Users\cheta\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\cve_bin_tool\cvedb.py:316 in get_cvelist_if_stale
C:\Users\cheta\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\cve_bin_tool\cvedb.py:299 in refresh_cache_and_update_db
C:\Users\cheta\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\cve_bin_tool\async_utils.py:90 in run_coroutine
C:\Users\cheta\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\cve_bin_tool\cvedb.py:293 in refresh
C:\Users\cheta\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\cve_bin_tool\data_sources\nvd_source.py:100 in get_cve_data
C:\Users\cheta\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\cve_bin_tool\data_sources\nvd_source.py:418 in fetch_cves
C:\Users\cheta\AppData\Local\Python\pythoncore-3.14-64\Lib\site-packages\cve_bin_tool\data_sources\nvd_source.py:541 in cache_update

ContentLengthError: 400, message='Not enough data to satisfy content length header.'
ClientPayloadError: Response payload is not completed: <ContentLengthError: 400, message='Not enough data to satisfy content length header.'>

Root cause hypothesis

aiohttp's async HTTP parser behavior changed in Python 3.14.
The ContentLengthError originates in aiohttp/client_proto.py:144
during connection_lost, suggesting the asyncio event loop
integration has a breaking change in 3.14.
Same aiohttp version (3.13.3) works correctly under Python 3.11.

Workaround

Install and run under Python 3.11.

[ffontaine]

This error is not related to python 3.14, it happens randomly with the json mirror (v4.mirror.cveb.in). Using -n json-nvd will fix the issue. I'm really wondering with the mirror returns those kind of failures. @anthonyharrison do you know who is in charge of it?

[Chetan-code-lrca]

Thanks @ffontaine — tested with -n json-nvd on Python 3.14 and it works successfully. You're correct that the issue is with the json mirror (v4.mirror.cveb.in) dropping connections mid-download, not Python 3.14 itself. I'll update the title and root cause.
One follow-up question: should the tool automatically fall back to json-nvd when the mirror returns a ContentLengthError instead of crashing with an unhandled ClientPayloadError? A graceful fallback or a clearer error message pointing users to -n json-nvd would improve the Windows/first-run experience significantly.

[ffontaine]

I don’t think switching from json-mirror to json-nvd on failure is the right solution, as it would just “hide” the underlying issue. The real problem seems to be with those random failures on json-mirror, and it would be better to investigate and fix it there.

If the issue can’t be resolved, I would prefer reverting the default back to json-nvd. @b31ngd3v changed the default from json-nvd to json-mirror in commit 0abbddd about two years ago.

Do you have any idea what might be going wrong? Pinging @warthog9 and @alex-ter as well for any insights.

[anthonyharrison]

@ffontaine Try @terriko or @warthog9 for info on the mirror

[warthog9]

Mirror system is going to be pretty straight forward, you are going into v4.mirror.cveb.in and then it's giving you a redirect out to the mm.fcix.net cluster, which is just more non-dynamic web servers (and in fact they move a lot of other data and we aren't seeing any issues or reports so I don't think there's a gotcha there).

I suspect we'd need further back in the output to tell what's going on. If it's somewhat randomly (and by randomly I mean, try 1 works, try 2 breaks weirdly), you are likely getting pointed at a system that's got something wonky on it, but there's not enough data in the above to tell which one that is.

3.13 behaving consistently vs. 3.14 does point to something wonky changing with aiohttp there

[ffontaine]

Thanks for your feedback @warthog9, on CI/CD tests, connection timeouts (400, 404, etc.) to the mirror are randomly raised with python 3.13:

• https://github.com/ossf/cve-bin-tool/actions/runs/22921280561/job/66520013737:

/opt/hostedtoolcache/Python/3.13.12/x64/lib/python3.13/site-packages/aiohttp/helpers.py:713: TimeoutError

• https://github.com/ossf/cve-bin-tool/actions/runs/22942233419/job/66586163170:

FAILED test/test_cvedb.py::TestCVEDB::test_refresh_nvd_json - aiohttp.client_exceptions.ClientResponseError: 404, message='Not Found', url='https://v4.mirror.cveb.in/nvd/json/cve/2.0/nvdcve-2.0-2013.json.gz'

• https://github.com/ossf/cve-bin-tool/actions/runs/22847604952/job/66267828105:

FAILED test/test_cli.py::TestCLI::test_update - aiohttp.client_exceptions.ClientResponseError: 404, message='Not Found', url='https://v4.mirror.cveb.in/nvd/json/cve/2.0/nvdcve-2.0-2025.json.gz'

I was unable to find any errors on CI/CD with Python versions 3.9, 3.10, 3.11, or 3.12. However, I was able to reproduce the issue on my local machine with Python 3.12.

I conducted numerous tests to try to resolve this problem (such as forcing disconnections, limiting concurrency, implementing retry mechanisms, etc.) but without success.

Eventually, I managed to take a screenshot of https://v4.mirror.cveb.in/nvd/json/cve/2.0:

In this screenshot, it’s clear that nvdcve-2.0-2013.json.gz is missing. Additionally, there are strange files such as nvdcve-2.0-2013.json.2c.delayed and nvdcve-2.0-2013.json.2f.delayed.

Until the mirror is fixed, I will likely revert the default to json-nvd, as this issue significantly impacts cve-bin-tool for many users (a second related issue has been opened today).

[warthog9]

Ok did some poking at the syncing script (takes a while to run because NIST is the slow part, yay), should be happier now that I've worked around the work arounds for the last time upstream updated something (sure wish they'd just provide an rsync path I could yank from), and I've added an explicit bit to check if all the files we have in the mirror are still present up on upstream as well (if they aren't it yanks them, so that should clear the earlier delayed files that seem to be left around).

We should probably flip to using https://cveb.in/get/nvd/json/cve/2.0 for things instead of the mirror url, mostly so that it's actually using the mirrors instead of continuing to rely on the root VM as it currently is. Don't know that that would exactly help the timeout issues, mostly because that's just the way the internet works. I'd recommend a retry 2 or 3 times if it encounters oddball problems like the timeouts you were seeing.

Quick glance I think that clears up at least the 2013 file problem you've noted

[warthog9]

I'll also note: the reason we moved to the mirror was using the nvd server itself was differently problematically slow and cumbersome, it's probably not a good idea to revert the commit universally.

[ffontaine]

Thanks for your help, I'll update the URL as suggested.

By the way, yesterday, the NVD server itself was returning a ClientPayloadError with truncated content, so reverting doesn't seem like a good option after all. I believe the best solution would be, to replace the RateLimiter class with a mechanism that also manages retries, for example using tenacity. I was working on the code, but by the time it was ready, the NVD server was back to normal. I'll make additional tests in the next days