feat(pytest): add minimal test set infrastructure for json_infra#1919
Open
DeborahOlaboye wants to merge 5 commits intoethereum:forks/amsterdamfrom
Open
feat(pytest): add minimal test set infrastructure for json_infra#1919DeborahOlaboye wants to merge 5 commits intoethereum:forks/amsterdamfrom
DeborahOlaboye wants to merge 5 commits intoethereum:forks/amsterdamfrom
Conversation
Fixes ethereum#1020 Implements minimize-tests integration to dramatically reduce json_infra test execution time from hours to minutes while maintaining full coverage. Changes: - Added minimize-tests dependency to pyproject.toml test group - Created generate_minimal_tests.py script to compute minimal test set - Added tox environment 'json_infra_min' to run minimal tests with coverage enforcement - Added tox environment 'json_infra_generate' to regenerate minimal test list - Created minimal_tests.txt placeholder for generated test IDs How it works: 1. Run all json_infra tests with --cov-context=test to track coverage per test 2. Use minimize-tests tool to solve set cover problem (find minimum tests covering all lines) 3. Run only the minimal test set with --cov-fail-under to enforce coverage threshold 4. Regenerate when coverage drops or code changes significantly Usage: tox -e json_infra_min # Run minimal tests (fast) tox -e json_infra_generate # Regenerate minimal test list tox -e json_infra # Run full suite (slow) Expected impact: 20-50x speedup for json_infra tests while maintaining same code coverage as the full test suite.
DeborahOlaboye
changed the title
fselmo
force-pushed
the
forks/amsterdam
branch
from
3d2eede to
9e161b6
Compare
Contributor
|
We haven't quite decided if we're going to take this approach. Running minimize tests takes an age, so it isn't exactly ideal. |
|
This PR has had no recent activity and has been marked as stale. |
github-actions
Bot
added
the
stale
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🗒️ Description
Implements the
minimize-testsapproach to dramatically reducejson_infratest execution time from hours to minutes while maintaining full code coverage.This PR adds infrastructure to generate and run a minimal subset of
json_infratests using set cover optimization. Thejson_infratests verify that EELS can execute filled JSON fixtures, so we don't need to run all fixtures.Key additions:
tox -e json_infra_generate: Generates minimal test list using minimize-tests tooltox -e json_infra_min: Runs minimal test set with coverage enforcement (fast)tox -e json_infra: Unchanged (full suite)🔗 Related Issues or PRs
Closes #1020
✅ Checklist
toxchecks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:uvx tox -e statictype(scope):.mkdocs servelocally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.@ported_frommarker.Cute Animal Picture