Skip to content

fix(consume): add ethereum/execution-specs to SUPPORTED_REPOS#2849

Merged
marioevz merged 1 commit into
ethereum:forks/amsterdamfrom
edg-l:fix/consume-cache-execution-specs-repo
May 13, 2026
Merged

fix(consume): add ethereum/execution-specs to SUPPORTED_REPOS#2849
marioevz merged 1 commit into
ethereum:forks/amsterdamfrom
edg-l:fix/consume-cache-execution-specs-repo

Conversation

@edg-l
Copy link
Copy Markdown
Contributor

@edg-l edg-l commented May 13, 2026

Summary

FixtureDownloader.get_cache_path versions the on-disk cache directory by release tag only when the release URL's owner/repo appears in SUPPORTED_REPOS. Otherwise it falls through to the unversioned cache_folder / "other" / <archive_name> path.

When the BAL fixture release was moved from ethereum/execution-spec-tests to ethereum/execution-specs for tests-bal@v7.1.0, the new URL stopped being recognized as a release URL. A v7.0.0 download from a prior session, cached at other/fixtures_bal/, kept satisfying every subsequent consume request, including ones explicitly pointing at v7.1.0. Consumers got stale fixtures with no warning.

This PR adds ethereum/execution-specs to SUPPORTED_REPOS so its release URLs hit the versioned cache path (execution-specs/tests-bal@v7.1.0/fixtures_bal/).

How this was found

While validating ethrex's bal-devnet-7 branch against tests-bal@v7.1.0:

  • The downloaded tarball at https://github.com/ethereum/execution-specs/releases/download/tests-bal%40v7.1.0/fixtures_bal.tar.gz has the correct v7.1.0 BAL (sender post-balance reflects no SD-refund per feat(spec-specs, tests): remove SD state gas refunds from EIP-8037 #2845).
  • The on-disk cache used by consume had the v7.0.0 BAL (still applied the SD refund).
  • 76 SD-related Amsterdam tests appeared to fail with a +1,836,000 wei = +1× NEW_ACCOUNT × cpsb × gas_price mismatch; the exact amount of the removed v7.0.0 SD refund.
  • The fixture file extracted from a fresh v7.1.0 tarball download matched ethrex's execution exactly. The wire-BAL the consume harness sent matched a v7.0.0 tarball's _info.url.
  • Clearing ~/.cache/ethereum-execution-spec-tests/cached_downloads/ made all 2,145 amsterdam tests pass on the same ethrex commit.

So this was a stale-cache issue, not a spec or client bug. The fix prevents it from recurring once execution-specs-hosted releases become standard.

Changes

  • releases.py: add ethereum/execution-specs to SUPPORTED_REPOS.
  • tests/test_releases.py: add parametrized regression test for is_release_url covering every entry in SUPPORTED_REPOS, plus an explicit assertion that ethereum/execution-specs is present.

Follow-up worth considering (not in this PR)

The cache_folder / "other" / <archive_name> fallback in get_cache_path is a latent footgun: any future release hosted on a repo not in SUPPORTED_REPOS will silently collide with prior downloads sharing an archive filename. A defensive change, e.g. incorporating a hash of the URL into the "other" path, would close the door entirely. Happy to follow up in a separate PR if maintainers agree it's worth doing.

Test plan

  • uv run pytest packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/tests/test_releases.py: 12/12 pass.
  • uv run ruff check && uv run ruff format --check: clean.
  • Manual: clear cache, re-run consume against tests-bal@v7.1.0, observe 2,145/2,145 pass on ethrex bal-devnet-7.

`FixtureDownloader.get_cache_path` versions the on-disk cache directory
by release tag only when the release URL's owner/repo appears in
`SUPPORTED_REPOS`. Otherwise it falls through to the unversioned
`cache_folder / "other" / <archive_name>` path.

When the BAL fixture release was moved from
`ethereum/execution-spec-tests` to `ethereum/execution-specs` for
`tests-bal@v7.1.0`, the new URL stopped being recognized as a release
URL. A v7.0.0 download from a prior session, cached at
`other/fixtures_bal/`, kept satisfying every subsequent request,
including ones for v7.1.0. Consumers got stale fixtures with no warning.

Add `ethereum/execution-specs` to `SUPPORTED_REPOS` so its release URLs
hit the versioned cache path. Add regression tests covering
`is_release_url` against every supported repo and asserting the new
entry is present.

Discovered while validating ethrex against tests-bal@v7.1.0: 76
SD-related ef-tests appeared to fail due to a spec mismatch, but the
client was actually executing v7.1.0 semantics against v7.0.0 fixtures
served by the stale cache. After clearing
`~/.cache/ethereum-execution-spec-tests/cached_downloads/` all 2145
amsterdam tests pass.
@edg-l edg-l force-pushed the fix/consume-cache-execution-specs-repo branch from e8bb28c to 7875f05 Compare May 13, 2026 14:49
@edg-l edg-l marked this pull request as ready for review May 13, 2026 14:49
@codecov
Copy link
Copy Markdown

codecov Bot commented May 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.01%. Comparing base (5c6cec7) to head (7875f05).

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #2849   +/-   ##
================================================
  Coverage            90.01%   90.01%           
================================================
  Files                  539      539           
  Lines                32618    32618           
  Branches              3030     3030           
================================================
  Hits                 29361    29361           
  Misses                2699     2699           
  Partials               558      558           
Flag Coverage Δ
unittests 90.01% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@marioevz marioevz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for this!

@marioevz marioevz merged commit b087114 into ethereum:forks/amsterdam May 13, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants