ci: multi Bitcoin Core RPC testing

[federico-stacks]

Description

This PR introduces a new CI workflow designed to run our Bitcoin RPC test suite against multiple versions of Bitcoin Core, ensuring continued compatibility across supported major releases.

What's new:

• Added testcontainers as a development dependency to run a containerized Bitcoin Core service, encapsulated in the BitcoinCoreContainer struct.
• Updated the RPC test suite to use BitcoinCoreContainer, replacing the previous BitcoinCoreController.
• Implemented a new workflow, bitcoin-rpc-tests.yml (triggered from ci.yml), which:
  • Dynamically discovers the latest available Docker image versions for interesting major Bitcoin Core releases (starting from a configurable major version).
  • Executes the RPC test suite using a version matrix based on the discovered images.
  • The idea is to keeps the test suite automatically aligned with the latest published minor versions for each major release.

Note: To improve maintainability and testability, the Docker image discovery script has been extracted to .github/workflow/scripts/docker-bitcoin-majors.

Job execution example:

• https://github.com/stacks-network/stacks-core/actions/runs/22309577031/
• Look for "Bitcoin RPC Tests" jobs

Applicable issues

• fixes testing: multi-version Bitcoin Core Rpc automation #6363

Additional info (benefits, drawbacks, caveats)

> Alternative execution strategy (nightly runs)
As a potential improvement, we could consider moving the multi-version compatibility workflow to a scheduled nightly run instead of executing it on every PR. This would allow us to continuously validate Bitcoin RPC compatibility across all supported major versions, even in the absence of active pull requests.

In this model, the CI pipeline for PRs could be simplified to run the test suite only against the default Bitcoin Core version (as is currently done for other integration tests), while the full dynamically discovered version matrix would execute nightly.

> Other approaches evaluated

• Using rstest with compile-time configuration:
  • Pros: Explicit list of tested image tags
  • Cons: Requires manual updates whenever new versions are released.
• Installing Bitcoin Core directly on the CI VM.
  • Pros: No changes required to the existing test structure
  • Cons: Results in a more complex and less portable workflow; makes local test execution more cumbersome.

> Generic CI improvement
In the bitcoin-rpc-tests.yml workflow, stacks-network/actions/stacks-core/testenv@main is reused to set up the test environment. This action also restores a cached Bitcoin binary, which is not used in this job since tests rely exclusively on the containerized Bitcoin Core instance.
Although the cache restore only adds a few seconds to the runtime (so no big issue there), we could streamline the workflow by skipping the Bitcoin cache if not explicitly requested.

Checklist

• Test coverage for new or modified code paths
• For new Clarity features or consensus changes, add property tests (see docs/property-testing.md)
• Changelog is updated
• Required documentation changes (e.g., rpc/openapi.yaml for RPC endpoints, event-dispatcher.md for new events)
• New clarity functions have corresponding PR in clarity-benchmarking repo

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 55.80%. Comparing base (2847ab5) to head (0c1362d).
⚠️ Report is 161 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #6927      +/-   ##
===========================================
+ Coverage    54.68%   55.80%   +1.12%     
===========================================
  Files          412      412              
  Lines       218662   219958    +1296     
  Branches       338      338              
===========================================
+ Hits        119571   122756    +3185     
+ Misses       99091    97202    -1889     

see 257 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2847ab5...0c1362d. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[wileyj]

From the CI side, the changes appear fine - but the question i have is more around rate-limits from dockerhub.
If this job is triggered several times by several PR's in an hour, is it possible a rate-limit my be hit and the job(s) fail?

https://docs.docker.com/docker-hub/usage/

100 per IPv4 address or IPv6 /64 subnet

I think the question here is - will gha pull these images from a new ip for each runner, or would they use a shared egress? or is there some agreement so gha isn't affected at all? 🤔

if the above is possible, we do have some options though (ex: building the docker image ourselves per PR, or in a bitcoin fork where the binaries come from - we could push to ghcr)

[federico-stacks]

From the CI side, the changes appear fine - but the question i have is more around rate-limits from dockerhub. If this job is triggered several times by several PR's in an hour, is it possible a rate-limit my be hit and the job(s) fail?

https://docs.docker.com/docker-hub/usage/

100 per IPv4 address or IPv6 /64 subnet

I think the question here is - will gha pull these images from a new ip for each runner, or would they use a shared egress? or is there some agreement so gha isn't affected at all? 🤔

if the above is possible, we do have some options though (ex: building the docker image ourselves per PR, or in a bitcoin fork where the binaries come from - we could push to ghcr)

Fair point.
For what I knew, these Docker Hub rate limits do not apply when:

• Using GitHub-hosted runners, and
• Pulling public images

Here the official reference: https://docs.github.com/en/actions/reference/limits#docker-hubs-rate-limit-for-github-actions

So we should be safe from hitting those limits in our current setup.

[wileyj]

lgtm

[federico-stacks]

@wileyj just in case you miss the Additional Info section. I would like to know your thoughts about the option to run this workflow nightly? Or do you prefer start like this and improve later or in a follow-up PR?

[wileyj]

@wileyj just in case you miss the Additional Info section. I would like to know your thoughts about the option to run this workflow nightly? Or do you prefer start like this and improve later or in a follow-up PR?

i don't have a strong opinion for either, but when i looked at the job - it takes roughly ~10 minutes in total. based on that, i think it's fine to run for each PR vs nightly (i would reserver nightly jobs for longer tasks or non-blocking workflows).

this workflow specifically, i feel has a good signal - if the rpc breaks for a major version of bitcoin, it's something we should know about (and today it relies on manuallly testing it).

i say keep it the way you wrote it. if it causes pain later, it may be adjusted.

edit: there is one thing to consider actually - if kept as-is, and for example and older version of btc causes a test failure due to a deprecated rpc or something, then the entire ci workflow will appear as "failed". one option to prevent a (visual only) failed ci is to trigger this workflow at the same time as the main ci does, i.e.

on:
  merge_group:
    types:
      - checks_requested
  push:
    branches:
      - master
      - develop
      - next
    paths-ignore:
      - "**.md"
      - "**.yml"
  workflow_dispatch:
  pull_request:
    types:
      - opened
      - reopened
      - synchronize

[federico-stacks]

i don't have a strong opinion for either, but when i looked at the job - it takes roughly ~10 minutes in total. based on that, i think it's fine to run for each PR vs nightly (i would reserver nightly jobs for longer tasks or non-blocking workflows).

this workflow specifically, i feel has a good signal - if the rpc breaks for a major version of bitcoin, it's something we should know about (and today it relies on manuallly testing it).

i say keep it the way you wrote it. if it causes pain later, it may be adjusted.

OK. My concern wasn’t about performance, but rather about continuously validating Bitcoin RPC compatibility across all supported major versions, even when there are no active pull requests.

That said, I’m fine starting with the current setup (running on each PR). If we later feel that continuous validation is important enough, we can always switch it to a nightly schedule.

edit: there is one thing to consider actually - if kept as-is, and for example and older version of btc causes a test failure due to a deprecated rpc or something, then the entire ci workflow will appear as "failed". one option to prevent a (visual only) failed ci is to trigger this workflow at the same time as the main ci does, i.e.

That might actually be desirable. If something breaks on the RPC side, it’s probably something we should address promptly.

Also, if we run this workflow in parallel with the main CI, wouldn’t we need to rebuild the binaries separately increasing load on the github runners for each PR? (Or eventually we should try to serialize the workflows)

If you don’t have a strong opinion here, I’d lean toward keeping the workflow as-is for now.

[francesco-stacks]

lgtm!

[francesco-stacks]

lgtm

[coveralls]

Pull Request Test Coverage Report for Build 23488680821

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 209 unchanged lines in 32 files lost coverage.
• Overall coverage decreased (-0.07%) to 85.641%

---

Files with Coverage Reduction	New Missed Lines	%
stackslib/src/burnchains/bitcoin/network.rs	1	79.31%
stackslib/src/chainstate/stacks/miner.rs	1	83.48%
stackslib/src/clarity_vm/clarity.rs	1	93.7%
stackslib/src/net/download/nakamoto/download_state_machine.rs	1	90.57%
stackslib/src/net/mod.rs	1	77.81%
stackslib/src/net/relay.rs	1	74.54%
stacks-signer/src/client/stacks_client.rs	1	86.19%
stacks-signer/src/v0/signer.rs	1	87.47%
stacks-common/src/deps_common/bitcoin/util/hash.rs	2	81.45%
stacks-common/src/util/pipe.rs	2	89.52%

Totals
Change from base Build 23400582460:	-0.07%
Covered Lines:	186426
Relevant Lines:	217682

---

💛 - Coveralls

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.