Skip to content

perf(test): remove redundant integration test executions #972

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Stevengre wants to merge 11 commits into
base: master
Choose a base branch
from
+30 −3
Open

perf(test): remove redundant integration test executions #972

Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
eb489e6
perf(ci): skip redundant integration tests via pytest -k filter
Stevengre Mar 4, 2026
992569c
fix(ci): avoid parentheses in pytest -k filter expression
Stevengre Mar 4, 2026
871776b
fix(ci): quote pytest -k expression in TEST_ARGS
Stevengre Mar 4, 2026
dc5eda8
.github/workflows/test.yml: split test-suite rather than removing
ehildenb Mar 18, 2026
fc600b9
.github/workflows/test: increase test parallelism
ehildenb Mar 18, 2026
c6707c4
.github/workflows/test.yml: more stratification of the tests
ehildenb Mar 18, 2026
171ec97
.github/test.yml: more stratification
ehildenb Mar 18, 2026
ea89e5f
.github/test: fix quotations
ehildenb Mar 18, 2026
d64d30e
.github/test: refine test.yml more
ehildenb Mar 18, 2026
dd534bf
.github/test: compact tests, up timeouts for testing
ehildenb Mar 18, 2026
e4d0cd8
.github/test: tighten timeouts
ehildenb Mar 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,37 @@ jobs:
with:
version: ${{ steps.uv_release.outputs.uv_version }}
- name: 'Run unit tests'
run: make test-unit
run: make test-unit PARALLEL=12

integration-tests:
needs: code-quality-checks
name: 'Integration Tests'
name: 'Integration Tests ${{ matrix.name }}'
runs-on: [self-hosted, linux, normal]
timeout-minutes: ${{ matrix.timeout }}
strategy:
fail-fast: true
matrix:
include:
- name: 'LLVM Concrete Tests'
test-args: '-k "llvm or test_run_smir_random"'
parallel: 12
timeout: 20
- name: 'Haskell Exec SMIR'
test-args: '-k "test_exec_smir and haskell"'
parallel: 6
timeout: 20
Comment on lines +65 to +68
Copy link
Member

@ehildenb ehildenb Mar 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the tests originally proposed to be removed I believe. In other semantics, we do not test the concrete tests on both backends, they are just to test that the semantics is correct (assuming the backends agree on them). But this is also not dominating execution time, the Haskell Proofs phase does.

- name: 'Haskell Termination'
test-args: '-k test_prove_termination'
parallel: 6
timeout: 20
- name: 'Haskell Proofs'
test-args: '-k "test_prove and not test_prove_termination"'
parallel: 6
timeout: 90
- name: 'Remainder'
test-args: '-k "not llvm and not test_run_smir_random and not test_exec_smir and not test_prove_termination and not test_prove"'
parallel: 6
timeout: 15
steps:
- name: 'Check out code'
uses: actions/checkout@v4
Expand All @@ -66,7 +91,9 @@ jobs:
- name: 'Build stable-mir-json and kmir'
run: docker exec --user github-user mir-semantics-ci-${GITHUB_SHA} make build
- name: 'Run integration tests'
run: docker exec --user github-user mir-semantics-ci-${GITHUB_SHA} make test-integration
run: |
docker exec --user github-user mir-semantics-ci-${GITHUB_SHA} make test-integration \
TEST_ARGS='${{ matrix.test-args }}' PARALLEL=${{ matrix.parallel }}
- name: 'Tear down Docker'
if: always()
run: docker stop --time 0 mir-semantics-ci-${GITHUB_SHA}
Expand Down
Loading