Skip to content

fix(ci): redirect Go caches to a per-job temp dir#2774

Open
danceratopz wants to merge 1 commit into
ethereum:forks/amsterdamfrom
danceratopz:silence-go-cache-errors
Open

fix(ci): redirect Go caches to a per-job temp dir#2774
danceratopz wants to merge 1 commit into
ethereum:forks/amsterdamfrom
danceratopz:silence-go-cache-errors

Conversation

@danceratopz
Copy link
Copy Markdown
Member

@danceratopz danceratopz commented Apr 28, 2026

🗒️ Description

The fill (benchmark) job in the release_fixture_feature workflow (and any other CI step that runs actions/setup-go on a self-hosted-ghr runner) emits hundreds of error lines like:

##[error]/usr/bin/tar: ../../../../../data/gh-home/go/pkg/mod/go.opentelemetry.io/otel/trace@v1.40.0/tracestate_benchkmark_test.go: Cannot open: File exists
...
/usr/bin/tar: Exiting with failure status due to previous errors
##[warning]Failed to restore: "/usr/bin/tar" failed with error: The process '/usr/bin/tar' failed with exit code 2

Example: https://github.com/ethereum/execution-specs/actions/runs/25045144626/job/73358194540#step:5:2143

The self-hosted runner pool persists $GOMODCACHE (/data/gh-home/go/pkg/mod) across jobs, so when actions/setup-go extracts the GHA module-cache tarball it lands on top of files that already exist. tar aborts each one with File exists, the cache restore is skipped, and the job continues using the on-disk modules. The build still succeeds, but the log noise is alarming and the ~500 MB cache download is wasted.

The same job ran cleanly in tests-benchmark@v0.0.7 (Feb 2026) on the same runner labels, confirming the persistent $GOMODCACHE only began colliding recently. tests-benchmark@v0.0.8 happened to run build-benchmark-genesis in the hosted combine job under the split-mode release path, so it didn't hit this either.

This change exports GOMODCACHE and GOCACHE to \${{ runner.temp }}/go/... immediately before every setup-go invocation in the affected workflows / composite actions:

  • .github/actions/build-benchmark-genesis/action.yaml
  • .github/actions/build-evm-client/geth/action.yaml
  • .github/workflows/hive-execute.yaml
  • .github/workflows/hive-consume.yaml

runner.temp is wiped at the end of each job, so the cache always lands in an empty, job-scoped directory. Behaviour on GitHub-hosted runners is unchanged; self-hosted runners stop colliding without losing the GHA cache.

🔗 Related Issues or PRs

N/A.

✅ Checklist

  • All: Ran fast static checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    ```console
    just static
    ```
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start `type(scope):`.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).
image

Self-hosted runners persist `$GOMODCACHE` (`/data/gh-home/go/pkg/mod`)
across jobs, so the `actions/setup-go` cache restore extracts a tarball
on top of files that already exist and emits hundreds of `tar: Cannot
open: File exists` lines before bailing out with `Failed to restore`.
The build still succeeds because the on-disk modules are reused, but
the log noise is alarming and the ~500 MB cache download is wasted.

Set `GOMODCACHE` and `GOCACHE` to `${{ runner.temp }}/go/...` before
each `setup-go` invocation so the cache always lands in an empty,
job-scoped directory. Behaviour on GitHub-hosted runners is unchanged;
self-hosted runners stop colliding without losing the GHA cache.
@danceratopz danceratopz added the A-ci Area: Continuous Integration label Apr 28, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 28, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.17%. Comparing base (c48ecea) to head (483b42e).
⚠️ Report is 2 commits behind head on forks/amsterdam.

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #2774   +/-   ##
================================================
  Coverage            88.17%   88.17%           
================================================
  Files                  577      577           
  Lines                35659    35659           
  Branches              3490     3490           
================================================
  Hits                 31442    31442           
  Misses                3654     3654           
  Partials               563      563           
Flag Coverage Δ
unittests 88.17% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ci Area: Continuous Integration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants