Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
4d52e1d
Python 3.13
johnewart Nov 21, 2025
80e1635
Replace str Enum mixin pattern with enum.StrEnum, minor other fixes f…
johnewart Dec 9, 2025
f93cbb2
Only build 3.13.x
johnewart Dec 9, 2025
03f9235
Minor updates
johnewart Dec 9, 2025
db127ef
Remove pandas dependency
johnewart Dec 9, 2025
1da1248
Linting
johnewart Dec 9, 2025
4259bcd
Merge branch 'main' into johnewart/python3.13
johnewart Dec 9, 2025
67e8141
Replace pandas CSV reader with built-in csv library in a few tests
johnewart Dec 9, 2025
4e45016
Cache the nox virtual environments
johnewart Dec 9, 2025
9dd5594
Cache container builds to GH actions cache if possible
johnewart Dec 10, 2025
0114c14
Fix pydantic function call
johnewart Dec 10, 2025
6a23cc2
Cache redis and postgres rather than pull them from dockerhub if poss…
johnewart Dec 10, 2025
cb32a67
Move uses to after checkout
johnewart Dec 10, 2025
44d4b32
Workflow tweak
johnewart Dec 10, 2025
75a7c26
Workflow tweak
johnewart Dec 10, 2025
ee7926a
Workflow tweak
johnewart Dec 10, 2025
5369aaa
Workflow tweak
johnewart Dec 10, 2025
6a55b7b
Workflow tweak
johnewart Dec 10, 2025
f741b4d
Some test failure fixes
johnewart Dec 10, 2025
272d405
parallel run fix
johnewart Dec 10, 2025
ecd1a0a
Fix pydantic create_model call
johnewart Dec 10, 2025
11f6e2b
Bump typer to fix a click issue
johnewart Dec 10, 2025
5e31d2e
Bump python version in other workflows
johnewart Dec 10, 2025
80dcfc5
reformat tests with black
johnewart Dec 10, 2025
259315f
Add nox caching to static checks
johnewart Dec 10, 2025
fd486d0
A few straggling enum string mixins
johnewart Dec 10, 2025
5afc0b8
Format noxfiles
johnewart Dec 10, 2025
c17b763
isort...
johnewart Dec 10, 2025
d4c2a4c
Fix a pydantic version in a test string
johnewart Dec 10, 2025
895c16d
Replace straggling str/Enum mixin and sets with lists that are used f…
johnewart Dec 10, 2025
b7cbfc6
Fix a few tests with AsyncClient taking app
johnewart Dec 10, 2025
3be84a5
Don't fail builds if Coverage doesn't upload
johnewart Dec 10, 2025
a1ffcb5
Minor test fixes
johnewart Dec 10, 2025
3f779d3
Fix some tests
johnewart Dec 10, 2025
e348c13
Don't use xdist for integration tests
johnewart Dec 10, 2025
b5f8e86
Fix import
johnewart Dec 10, 2025
c20852c
Fix noxfile
johnewart Dec 10, 2025
76455ea
Fix some Pydantic dynamic model creation in SaaS connection secrets
johnewart Dec 11, 2025
11f6b39
Pydantic fix
johnewart Dec 11, 2025
d322a86
Handle single value in dynamic model
johnewart Dec 11, 2025
e2c0165
Merge branch 'main' into johnewart/python3.13
johnewart Dec 11, 2025
e570f50
Merge branch 'main' into johnewart/python3.13
galvana Dec 11, 2025
6d8db09
Fixing fideslog dependency
galvana Dec 11, 2025
f3971ad
Fix pytest_nox method
johnewart Dec 11, 2025
2fab9ff
xfail only dsr2.0 tests
johnewart Dec 11, 2025
a6835f4
Updating docs Dockerfile
galvana Dec 11, 2025
07de19d
Fixing pytest_nox
galvana Dec 11, 2025
208fc5d
Fixing CLI tests
galvana Dec 11, 2025
e827e15
Fixing Pydantic schemas
galvana Dec 11, 2025
d8b25c1
Fixing DSR 2.0
galvana Dec 12, 2025
aa7c99e
Removing select DSR 2.0 tests
galvana Dec 12, 2025
588e206
Fixing flaky test
galvana Dec 12, 2025
e091e47
Updating test to use string keys instead of CollectionAddress objects
galvana Dec 12, 2025
1bb3a07
Re-up parallel workers
johnewart Dec 12, 2025
a45f867
Bump pylint, add pytest-loguru
johnewart Dec 12, 2025
a29499e
Fix test to not be timing sensitive
johnewart Dec 12, 2025
3c7355a
Remove unused variable
johnewart Dec 12, 2025
804db20
Suppress stdout capture and warnings from pytest itself to address th…
johnewart Dec 12, 2025
aea7007
Make static checks non-terminal for now
johnewart Dec 13, 2025
da699c1
XFail the docs test because our schema transmogrification code is bre…
johnewart Dec 13, 2025
5e9f4b6
Update CLI test to use database name based on worker id when using xdist
johnewart Dec 13, 2025
c962121
Update loguru
johnewart Dec 13, 2025
42d6954
Don't use xdist for ctl-not-external tests
johnewart Dec 13, 2025
5f5a4de
Xfail test_scan_dataset_db_input_connection_string for now
johnewart Dec 13, 2025
7807b0e
Fix flaky test and celery session worker to be resilient
johnewart Dec 13, 2025
b69519c
Downgrade click
johnewart Dec 13, 2025
a89735d
Update string enum mixin to StrEnum for PrivacyNoticeRegion
johnewart Dec 16, 2025
7a666e1
Merge branch 'main' into johnewart/python3.13
galvana Dec 17, 2025
79b6bba
Update fideslang version
johnewart Dec 18, 2025
84fffd3
Remove git dependency for fideslog
johnewart Dec 18, 2025
93254f4
Bump fideslang
johnewart Dec 18, 2025
39d0b5e
Merge branch 'main' into johnewart/python3.13
johnewart Jan 5, 2026
a0c4e65
Consolidate multiple StrEnum imports
johnewart Jan 6, 2026
3884d97
Update Dockerfile for docs to be 3.13 instead of 3.12
johnewart Jan 6, 2026
55ad433
Removed unneeded test that failed in 3.13
johnewart Jan 6, 2026
2afba09
Removed nox python versions other than 3.13
johnewart Jan 6, 2026
dca8861
Merge branch 'main' into johnewart/python3.13
johnewart Jan 6, 2026
0fa21bc
Remove outdated ManualTaskLogStatus class
johnewart Jan 6, 2026
baccd22
Merge branch 'main' into johnewart/python3.13
johnewart Jan 6, 2026
034ec4f
Merge branch 'main' into johnewart/python3.13
johnewart Jan 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 31 additions & 37 deletions .github/workflows/backend_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ permissions:

env:
IMAGE: ethyca/fides:local
DEFAULT_PYTHON_VERSION: "3.10.16"
DEFAULT_PYTHON_VERSION: "3.13.11"
# Docker auth with read-only permissions.
DOCKER_USER: ${{ secrets.DOCKER_USER }}
DOCKER_RO_TOKEN: ${{ secrets.DOCKER_RO_TOKEN }}
Expand Down Expand Up @@ -59,7 +59,6 @@ jobs:
Collect-Tests:
needs: Check-Backend-Changes
if: needs.Check-Backend-Changes.outputs.has_backend_changes == 'true'

runs-on: ubuntu-latest
steps:
- name: Checkout
Expand Down Expand Up @@ -88,10 +87,6 @@ jobs:
Build:
needs: [Check-Backend-Changes, Collect-Tests]
if: needs.Check-Backend-Changes.outputs.has_backend_changes == 'true'
strategy:
matrix:
# NOTE: These are the currently supported/tested Python Versions
python_version: ["3.9.21", "3.10.16"]
runs-on: ubuntu-latest
steps:
- name: Checkout
Expand All @@ -106,20 +101,19 @@ jobs:
with:
builder: ${{ steps.buildx.outputs.name }}
context: .
build-args: PYTHON_VERSION=${{ matrix.python_version }}
build-args: PYTHON_VERSION=${{ env.DEFAULT_PYTHON_VERSION }}
target: prod
outputs: type=docker,dest=/tmp/python-${{ matrix.python_version }}.tar
outputs: type=docker,dest=/tmp/python-${{ env.DEFAULT_PYTHON_VERSION }}.tar
push: false
tags: ${{ env.IMAGE }}
cache-from: type=gha,scope=${{ matrix.python_version }}
cache-to: type=gha,mode=max,scope=${{ matrix.python_version }}

cache-from: type=gha
cache-to: type=gha,mode=max

- name: Upload container
uses: actions/upload-artifact@v4
with:
name: python-${{ matrix.python_version }}
path: /tmp/python-${{ matrix.python_version }}.tar
name: python-${{ env.DEFAULT_PYTHON_VERSION }}
path: /tmp/python-${{ env.DEFAULT_PYTHON_VERSION }}.tar
retention-days: 1

##################
Expand Down Expand Up @@ -325,7 +319,7 @@ jobs:
path: .nox/
key: ${{ runner.os }}-nox-${{ github.job }}-${{ hashFiles('noxfile.py') }}-${{ hashFiles('noxfiles/**.py') }}-${{ hashFiles('pyproject.toml') }}
restore-keys: |
${{ runner.os }}-nox-${{ github.job }}-
${{ runner.os }}-nox-${{ github.job }}-qq

- name: Login to Docker Hub
uses: docker/login-action@v3
Expand All @@ -351,7 +345,6 @@ jobs:
strategy:
fail-fast: false
matrix:
python_version: ["3.9.21", "3.10.16"]
test_selection:
- "ctl-not-external"
- "ops-unit-api"
Expand All @@ -365,14 +358,21 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Pull Docker images in background
run: |
docker pull postgres:16 > /dev/null 2>&1 &
docker pull redis:8.0-alpine > /dev/null 2>&1 &
echo "Docker pull initiated in background."
shell: bash

- name: Download container
uses: actions/download-artifact@v4
with:
name: python-${{ matrix.python_version }}
name: python-${{ env.DEFAULT_PYTHON_VERSION }}
path: /tmp/

- name: Load image
run: docker load --input /tmp/python-${{ matrix.python_version }}.tar
run: docker load --input /tmp/python-${{ env.DEFAULT_PYTHON_VERSION }}.tar

- name: Checkout
uses: actions/checkout@v4
Expand All @@ -390,9 +390,9 @@ jobs:
uses: actions/cache@v4
with:
path: .nox/
key: ${{ runner.os }}-nox-${{ github.job }}-${{ matrix.python_version }}-${{ matrix.test_selection }}-${{ hashFiles('noxfile.py') }}-${{ hashFiles('noxfiles/**.py') }}-${{ hashFiles('pyproject.toml') }}
key: ${{ runner.os }}-nox-${{ github.job }}-${{ matrix.test_selection }}-${{ hashFiles('noxfile.py') }}-${{ hashFiles('noxfiles/**.py') }}-${{ hashFiles('pyproject.toml') }}
restore-keys: |
${{ runner.os }}-nox-${{ github.job }}-${{ matrix.python_version }}-${{ matrix.test_selection }}
${{ runner.os }}-nox-${{ github.job }}-${{ matrix.test_selection }}

- name: Login to Docker Hub
uses: docker/login-action@v3
Expand Down Expand Up @@ -425,19 +425,17 @@ jobs:
if: needs.Check-Backend-Changes.outputs.has_backend_changes == 'true' && (contains(github.event.pull_request.labels.*.name, 'run unsafe ci checks') || github.event_name == 'push' || github.event_name == 'merge_group')
strategy:
max-parallel: 1 # This prevents collisions in shared external resources
matrix:
python_version: ["3.9.21", "3.10.16"]
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Download container
uses: actions/download-artifact@v4
with:
name: python-${{ matrix.python_version }}
name: python-${{ env.DEFAULT_PYTHON_VERSION }}
path: /tmp/

- name: Load image
run: docker load --input /tmp/python-${{ matrix.python_version }}.tar
run: docker load --input /tmp/python-${{ env.DEFAULT_PYTHON_VERSION }}.tar

- name: Checkout
uses: actions/checkout@v4
Expand All @@ -455,9 +453,9 @@ jobs:
uses: actions/cache@v4
with:
path: .nox/
key: ${{ runner.os }}-nox-${{ github.job }}-${{ matrix.python_version }}-${{ hashFiles('noxfile.py') }}-${{ hashFiles('noxfiles/**.py') }}-${{ hashFiles('pyproject.toml') }}
key: ${{ runner.os }}-nox-${{ github.job }}-${{ hashFiles('noxfile.py') }}-${{ hashFiles('noxfiles/**.py') }}-${{ hashFiles('pyproject.toml') }}
restore-keys: |
${{ runner.os }}-nox-${{ github.job }}-${{ matrix.python_version }}
${{ runner.os }}-nox-${{ github.job }}-

- name: Login to Docker Hub
uses: docker/login-action@v3
Expand Down Expand Up @@ -495,19 +493,17 @@ jobs:
if: needs.Check-Backend-Changes.outputs.has_backend_changes == 'true' && (contains(github.event.pull_request.labels.*.name, 'run unsafe ci checks') || github.event_name == 'push' || github.event_name == 'merge_group')
strategy:
max-parallel: 1 # This prevents collisions in shared external resources
matrix:
python_version: ["3.9.21", "3.10.16"]
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- name: Download container
uses: actions/download-artifact@v4
with:
name: python-${{ matrix.python_version }}
name: python-${{ env.DEFAULT_PYTHON_VERSION }}
path: /tmp/

- name: Load image
run: docker load --input /tmp/python-${{ matrix.python_version }}.tar
run: docker load --input /tmp/python-${{ env.DEFAULT_PYTHON_VERSION }}.tar

- name: Checkout
uses: actions/checkout@v4
Expand All @@ -525,9 +521,9 @@ jobs:
uses: actions/cache@v4
with:
path: .nox/
key: ${{ runner.os }}-nox-${{ github.job }}-${{ matrix.python_version }}-${{ hashFiles('noxfile.py') }}-${{ hashFiles('noxfiles/**.py') }}-${{ hashFiles('pyproject.toml') }}
key: ${{ runner.os }}-nox-${{ github.job }}-${{ hashFiles('noxfile.py') }}-${{ hashFiles('noxfiles/**.py') }}-${{ hashFiles('pyproject.toml') }}
restore-keys: |
${{ runner.os }}-nox-${{ github.job }}-${{ matrix.python_version }}-
${{ runner.os }}-nox-${{ github.job }}-

- name: Login to Docker Hub
uses: docker/login-action@v3
Expand Down Expand Up @@ -613,17 +609,15 @@ jobs:
id-token: write
strategy:
max-parallel: 1 # This prevents collisions in shared external resources
matrix:
python_version: ["3.9.21", "3.10.16"]
steps:
- name: Download container
uses: actions/download-artifact@v4
with:
name: python-${{ matrix.python_version }}
name: python-${{ env.DEFAULT_PYTHON_VERSION }}
path: /tmp/

- name: Load image
run: docker load --input /tmp/python-${{ matrix.python_version }}.tar
run: docker load --input /tmp/python-${{ env.DEFAULT_PYTHON_VERSION }}.tar

- name: Checkout
uses: actions/checkout@v4
Expand All @@ -641,9 +635,9 @@ jobs:
uses: actions/cache@v4
with:
path: .nox/
key: ${{ runner.os }}-nox-${{ github.job }}-${{ matrix.python_version }}-${{ hashFiles('noxfile.py') }}-${{ hashFiles('noxfiles/**.py') }}-${{ hashFiles('pyproject.toml') }}
key: ${{ runner.os }}-nox-${{ github.job }}-${{ hashFiles('noxfile.py') }}-${{ hashFiles('noxfiles/**.py') }}-${{ hashFiles('pyproject.toml') }}
restore-keys: |
${{ runner.os }}-nox-${{ github.job }}-${{ matrix.python_version }}-
${{ runner.os }}-nox-${{ github.job }}-

- name: Login to Docker Hub
uses: docker/login-action@v3
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/cli_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ on:
- "release-**"

env:
DEFAULT_PYTHON_VERSION: "3.10.16"
DEFAULT_PYTHON_VERSION: "3.13.11"

jobs:
Check-CLI-Changes:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/cypress_e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ env:
# Docker auth with read-only permissions.
DOCKER_USER: ${{ secrets.DOCKER_USER }}
DOCKER_RO_TOKEN: ${{ secrets.DOCKER_RO_TOKEN }}
DEFAULT_PYTHON_VERSION: "3.10.16"
DEFAULT_PYTHON_VERSION: "3.13.11"

jobs:
Check-E2E-Changes:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish_docker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ env:
# Docker auth with read-write (publish) permissions. Set as env in workflow root as auth is required in multiple jobs.
DOCKER_USER: ${{ secrets.DOCKER_USER }}
DOCKER_TOKEN: ${{ secrets.DOCKER_TOKEN }}
DEFAULT_PYTHON_VERSION: "3.10.16"
DEFAULT_PYTHON_VERSION: "3.13.11"

jobs:
ParseTags:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish_docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:
env:
TAG: ${{ github.event.release.tag_name }}
PROD_PUBLISH: true
DEFAULT_PYTHON_VERSION: "3.10.16"
DEFAULT_PYTHON_VERSION: "3.13.11"

jobs:
publish_docs:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish_package.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ jobs:
with:
fetch-depth: 0 # This is required to properly tag packages

- name: Setup Python 3.9
- name: Setup Python 3.13.11
uses: actions/setup-python@v5
with:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

syntax: Python version 3.11.11 doesn't exist (should be 3.13.11 to match the Dockerfile and other workflow)

Suggested change
with:
python-version: 3.13.11

python-version: 3.9
python-version: 3.11.11

- name: Use Node.js 20
uses: actions/setup-node@v4
Expand Down
8 changes: 6 additions & 2 deletions .github/workflows/static_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@ name: Backend Static Code Checks
on:
pull_request:
merge_group:
types: [checks_requested]
types: [ checks_requested ]
push:
branches:
- "main"
- "release-**"

env:
IMAGE: ethyca/fides:local
DEFAULT_PYTHON_VERSION: "3.10.16"
DEFAULT_PYTHON_VERSION: "3.13.11"
# Docker auth with read-only permissions.
DOCKER_USER: ${{ secrets.DOCKER_USER }}
DOCKER_RO_TOKEN: ${{ secrets.DOCKER_RO_TOKEN }}
Expand Down Expand Up @@ -88,9 +88,13 @@ jobs:
- name: Install Dev Requirements
run: pip install -r dev-requirements.txt

# The workflow will proceed even if this fails because it should be non-blocking
- name: Run Static Check
run: nox -s ${{ matrix.session_name }}
continue-on-error: true



# Summary job for branch protection
Static-Checks-Summary:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# If you update this, also update `DEFAULT_PYTHON_VERSION` in the GitHub workflow files
ARG PYTHON_VERSION="3.10.16"
ARG PYTHON_VERSION="3.13.11"
#########################
## Compile Python Deps ##
#########################
Expand Down
11 changes: 6 additions & 5 deletions dev-requirements.txt
Original file line number Diff line number Diff line change
@@ -1,21 +1,22 @@
black==24.3.0
debugpy==1.6.3
debugpy~=1.8.0
Faker==14.1.0
freezegun==1.0.0
freezegun==1.5.5
GitPython==3.1.41
isort==5.12.0
moto[s3]==5.1.0
mypy==1.10.0
nox==2022.8.7
pre-commit==2.20.0
pylint==3.2.5
pylint~=3.3.2
pytest-asyncio==0.19.0
pytest-celery==1.2.1
pytest-cov==4.0.0
pytest-env==0.7.0
pytest-loguru==0.4.0
pytest-mock==3.14.0
pytest-rerunfailures==14.0
pytest-xdist==3.6.1
pytest==7.2.2
pytest==8.4.2
pyyaml==6.0.1
requests-mock==1.10.0
setuptools>=64.0.2
Expand Down
14 changes: 10 additions & 4 deletions docs/fides/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
FROM python:3.10.16-slim-bookworm AS build
FROM python:3.13-slim-bookworm AS build

RUN apt-get update && \
apt-get install -y --no-install-recommends \
g++ \
gnupg \
gcc \
git \
python3-wheel \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
Expand All @@ -30,7 +31,12 @@ COPY . .
RUN pip install -U pip && pip install . && pip install -r docs/fides/requirements.txt


FROM python:3.10.16-slim-bookworm AS docs
FROM python:3.12-slim-bookworm AS docs

# Add the fidesuser user
RUN addgroup --system --gid 1001 fidesgroup
RUN adduser --system --uid 1001 --home /home/fidesuser fidesuser

RUN apt-get update && \
apt-get install -y --no-install-recommends \
git \
Expand All @@ -39,8 +45,8 @@ RUN apt-get update && \

WORKDIR /docs

COPY --from=build /opt/venv /opt/venv
COPY --from=build /docs/fides .
COPY --from=build --chown=fidesuser:fidesgroup /opt/venv /opt/venv
COPY --from=build --chown=fidesuser:fidesgroup /docs/fides .

ENV PATH="/opt/venv/bin:$PATH"

Expand Down
2 changes: 1 addition & 1 deletion noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
# pylint: enable=unused-wildcard-import, wildcard-import, wrong-import-position

REQUIRED_DOCKER_VERSION = "20.10.17"
REQUIRED_PYTHON_VERSIONS = ["3.9", "3.10"]
REQUIRED_PYTHON_VERSIONS = ["3.13"]

nox.options.sessions = ["open_docs"]

Expand Down
Loading
Loading