Skip to content

Attempted fix / investigate Action caching issues #264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bouwew merged 29 commits into from
Jun 19, 2025
Merged

Attempted fix / investigate Action caching issues #264

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
250c840
Fix biome url
CoMPaTech Jun 18, 2025
51ee340
Add inheritance of prepare to maintain cache
CoMPaTech Jun 18, 2025
4cbc0e3
Add pytest-cov
CoMPaTech Jun 18, 2025
93f1d83
Hash correctly
CoMPaTech Jun 18, 2025
65948b9
Less copy/pasting
CoMPaTech Jun 18, 2025
6f2c609
Wrong dir
CoMPaTech Jun 18, 2025
d4ca86e
Rework cache init
CoMPaTech Jun 18, 2025
c4bc9ac
pytest uses other versioning
CoMPaTech Jun 18, 2025
96d4d15
pytest uses other versioning
CoMPaTech Jun 18, 2025
779123a
Ensure saving
CoMPaTech Jun 18, 2025
fe7f78d
Include requirements for matrix building + single-build for default
CoMPaTech Jun 18, 2025
b030ce5
Include requirements for matrix building + single-build for default
CoMPaTech Jun 18, 2025
ef0bbe2
Include requirements for matrix building + single-build for default
CoMPaTech Jun 18, 2025
73a9fca
Include requirements for matrix building + single-build for default
CoMPaTech Jun 18, 2025
b1cb78f
Include requirements for matrix building + single-build for default
CoMPaTech Jun 18, 2025
aed710d
Improve cache usage
CoMPaTech Jun 18, 2025
2275d7b
Tidy up / fix broken publish
CoMPaTech Jun 18, 2025
776147b
Tidy up / fix broken publish
CoMPaTech Jun 18, 2025
83ff43c
Tidy up / fix broken publish
CoMPaTech Jun 18, 2025
1c960d7
Dependency
CoMPaTech Jun 18, 2025
1941dd8
Rework env variables
CoMPaTech Jun 18, 2025
1bbdd41
Rework env variables
CoMPaTech Jun 18, 2025
f549de6
Rework env variables - missing line
CoMPaTech Jun 18, 2025
741647d
Rework env variables - typo
CoMPaTech Jun 18, 2025
a958174
Rework env variables, ingest generated key
CoMPaTech Jun 18, 2025
48fa433
Try caching pre-commit install and bump cache
CoMPaTech Jun 18, 2025
6573379
Try caching pre-commit install and bump cache - change dir
CoMPaTech Jun 18, 2025
87be92e
Bump changelog, mostly to check cache re-usage
CoMPaTech Jun 18, 2025
954cd87
Cleanup
CoMPaTech Jun 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
112 changes: 31 additions & 81 deletions .github/workflows/verify.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,21 +28,13 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ env.DEFAULT_PYTHON }}
- name: Restore base Python ${{ env.DEFAULT_PYTHON }} virtual environment
- name: Ensure/restore Python ${{ env.DEFAULT_PYTHON }} virtual environment
id: cache-venv
uses: actions/cache@v4
with:
path: venv
key: >-
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-base-venv-${{
steps.python.outputs.python-version }}-${{
hashFiles('requirements_test.txt') }}-${{
hashFiles('setup.py') }}
restore-keys: |
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-base-venv-${{ steps.python.outputs.python-version }}-${{ hashFiles('requirements_test.txt') }}-${{ hashFiles('setup.py') }}-
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-base-venv-${{ steps.python.outputs.python-version }}-${{ hashFiles('requirements_test.txt') }}
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-base-venv-${{ steps.python.outputs.python-version }}-
- name: Create Python virtual environment
key: ${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-cache-${{ env.CACHE_VERSION }}-${{ steps.python.outputs.python-version }}-${{ hashFiles('pyproject.toml', 'requirements_test.txt') }}
- name: Create Python virtual environment (from cache)
Comment thread
CoMPaTech marked this conversation as resolved.
Outdated
Comment thread
CoMPaTech marked this conversation as resolved.
Outdated
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
pip install virtualenv --upgrade
Expand All @@ -51,15 +43,14 @@ jobs:
pip install uv
uv pip install -U pip setuptools wheel
uv pip install -r requirements_test.txt -r requirements_commit.txt
- name: Restore pre-commit environment from cache
- name: Ensure/restore python and pre-commit environment (from cache)
id: cache-precommit
uses: actions/cache@v4
with:
path: ${{ env.PRE_COMMIT_HOME }}
key: |
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
restore-keys: |
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-pre-commit-
path: |
venv
${{ env.PRE_COMMIT_HOME }}
key: ${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-cache-${{ env.CACHE_VERSION }}-${{ steps.python.outputs.python-version }}-${{ hashFiles('.pre-commit-config.yaml') }}
Comment thread
CoMPaTech marked this conversation as resolved.
Outdated
- name: Install pre-commit dependencies
if: steps.cache-precommit.outputs.cache-hit != 'true'
run: |
Expand All @@ -80,20 +71,19 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ env.DEFAULT_PYTHON }}
- name: Restore base Python ${{ env.DEFAULT_PYTHON }} virtual environment
- name: Ensure/restore pre-commit and Python ${{ env.DEFAULT_PYTHON }} virtual environment
id: cache-venv
uses: actions/cache@v4
with:
path: venv
path: |
venv
${{ env.PRE_COMMIT_HOME }}
key: >-
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-base-venv-${{
steps.python.outputs.python-version }}-${{
hashFiles('requirements_test.txt') }}-${{
hashFiles('setup.py') }}
${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-cache-${{ env.CACHE_VERSION }}-${{ steps.python.outputs.python-version }}-${{ hashFiles('.pre-commit-config.yaml') }}
Comment thread
CoMPaTech marked this conversation as resolved.
Outdated
- name: Fail job if Python cache restore failed
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
echo "Failed to restore Python ${{ env.DEFAULT_PYTHON }} virtual environment from cache"
echo "Failed to restore pre-commit and Python ${{ env.DEFAULT_PYTHON }} virtual environment from cache"
exit 1
- name: Ruff (check)
run: |
Expand Down Expand Up @@ -126,32 +116,18 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ env.DEFAULT_PYTHON }}
- name: Restore base Python ${{ env.DEFAULT_PYTHON }} virtual environment
- name: Restore pre-commit and Python ${{ env.DEFAULT_PYTHON }} virtual environment
id: cache-venv
uses: actions/cache@v4
with:
path: venv
key: >-
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-base-venv-${{
steps.python.outputs.python-version }}-${{
hashFiles('requirements_test.txt') }}-${{
hashFiles('setup.py') }}
path: |
venv
${{ env.PRE_COMMIT_HOME }}
key: ${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-cache-${{ env.CACHE_VERSION }}-${{ steps.python.outputs.python-version }}-${{ hashFiles('.pre-commit-config.yaml') }}
Comment thread
CoMPaTech marked this conversation as resolved.
Outdated
- name: Fail job if Python cache restore failed
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
echo "Failed to restore Python ${{ env.DEFAULT_PYTHON }} virtual environment from cache"
exit 1
- name: Restore pre-commit environment from cache
id: cache-precommit
uses: actions/cache@v4
with:
path: ${{ env.PRE_COMMIT_HOME }}
key: |
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
- name: Fail job if cache restore failed
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
echo "Failed to restore pre-commit environment from cache"
echo "Failed to restore pre-commit and Python ${{ env.DEFAULT_PYTHON }} virtual environment from cache"
exit 1
- name: Verify commit
run: |
Expand Down Expand Up @@ -182,19 +158,12 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Restore full Python ${{ matrix.python-version }} virtual environment
- name: Restore Python ${{ matrix.python-version }} virtual environment
id: cache-venv
uses: actions/cache@v4
with:
path: venv
key: >-
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-${{
matrix.python-version }}-${{ hashFiles('requirements_test.txt')
}}-${{ hashFiles('setup.py') }}
restore-keys: |
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-${{ matrix.python-version }}-${{ hashFiles('requirements_test.txt') }}-${{ hashFiles('setup.py') }}
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-${{ matrix.python-version }}-${{ hashFiles('requirements_test.txt') }}
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-${{ matrix.python-version }}-
key: ${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-cache-${{ env.CACHE_VERSION }}-${{ steps.python.outputs.python-version }}-${{ hashFiles('pyproject.toml', 'requirements_test.txt') }}
- name: Create full Python ${{ matrix.python-version }} virtual environment
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
Expand Down Expand Up @@ -228,10 +197,7 @@ jobs:
uses: actions/cache@v4
with:
path: venv
key: >-
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-${{
matrix.python-version }}-${{ hashFiles('requirements_test.txt')
}}-${{ hashFiles('setup.py') }}
key: ${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-cache-${{ env.CACHE_VERSION }}-${{ steps.python.outputs.python-version }}-${{ hashFiles('pyproject.toml', 'requirements_test.txt') }}
- name: Fail job if Python cache restore failed
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
Expand Down Expand Up @@ -263,16 +229,12 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ env.DEFAULT_PYTHON }}
- name: Restore base Python ${{ env.DEFAULT_PYTHON }} virtual environment
- name: Restore Python ${{ env.DEFAULT_PYTHON }} virtual environment
id: cache-venv
uses: actions/cache@v4
with:
path: venv
key: >-
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-base-venv-${{
steps.python.outputs.python-version }}-${{
hashFiles('requirements_test.txt') }}-${{
hashFiles('setup.py') }}
key: ${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-cache-${{ env.CACHE_VERSION }}-${{ steps.python.outputs.python-version }}-${{ hashFiles('pyproject.toml', 'requirements_test.txt') }}
- name: Fail job if Python cache restore failed
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
Expand Down Expand Up @@ -321,16 +283,12 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ env.DEFAULT_PYTHON }}
- name: Restore base Python ${{ env.DEFAULT_PYTHON }} virtual environment
- name: Restore Python ${{ env.DEFAULT_PYTHON }} virtual environment
id: cache-venv
uses: actions/cache@v4
with:
path: venv
key: >-
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-base-venv-${{
steps.python.outputs.python-version }}-${{
hashFiles('requirements_test.txt') }}-${{
hashFiles('setup.py') }}
key: ${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-cache-${{ env.CACHE_VERSION }}-${{ steps.python.outputs.python-version }}-${{ hashFiles('pyproject.toml', 'requirements_test.txt') }}
- name: Fail job if Python cache restore failed
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
Expand Down Expand Up @@ -366,16 +324,12 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ env.DEFAULT_PYTHON }}
- name: Restore base Python ${{ env.DEFAULT_PYTHON }} virtual environment
- name: Restore Python ${{ env.DEFAULT_PYTHON }} virtual environment
id: cache-venv
uses: actions/cache@v4
with:
path: venv
key: >-
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-base-venv-${{
steps.python.outputs.python-version }}-${{
hashFiles('requirements_test.txt') }}-${{
hashFiles('setup.py') }}
key: ${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-cache-${{ env.CACHE_VERSION }}-${{ steps.python.outputs.python-version }}-${{ hashFiles('pyproject.toml', 'requirements_test.txt') }}
- name: Fail job if Python cache restore failed
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
Expand Down Expand Up @@ -409,16 +363,12 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ env.DEFAULT_PYTHON }}
- name: Restore base Python ${{ env.DEFAULT_PYTHON }} virtual environment
- name: Restore Python ${{ env.DEFAULT_PYTHON }} virtual environment
id: cache-venv
uses: actions/cache@v4
with:
path: venv
key: >-
usb-${{ env.CACHE_VERSION}}-${{ runner.os }}-base-venv-${{
steps.python.outputs.python-version }}-${{
hashFiles('requirements_test.txt') }}-${{
hashFiles('setup.py') }}
key: ${{ env.CACHE_VERSION}}-${{ runner.os }}-venv-cache-${{ env.CACHE_VERSION }}-${{ steps.python.outputs.python-version }}-${{ hashFiles('pyproject.toml', 'requirements_test.txt') }}
- name: Fail job if Python cache restore failed
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
Expand Down
6 changes: 3 additions & 3 deletions scripts/setup_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ uv pip install --upgrade -e . -r requirements_test.txt -c https://raw.githubuser
echo "Fetching/updating biome cli"
arch=$(uname -m)
case "$arch" in
aarch64|arm64) url="biome-darwin-arm64" ;;
x86_64) url="biome-linux-x64" ;;
aarch64|arm64) use_arch="darwin-arm64" ;;
x86_64) use_arch="linux-x64" ;;
Comment thread
CoMPaTech marked this conversation as resolved.
*) echo "Unsupported arch for biome cli version: $arch"; exit 2 ;;
esac
curl -sL "https://github.com/biomejs/biome/releases/latest/download/biome-$url" -o "${my_path}/tmp/biome"
curl -sL "https://github.com/biomejs/biome/releases/latest/download/biome-${use_arch}" -o "${my_path}/tmp/biome"

# Make biome executable (if necessary)
chmod +x "${my_path}/tmp/biome"
Expand Down
Loading