Skip to content

Matrix coverage: stop hand-maintaining the per-type snapshot list in CI #249

@tobyhede

Description

@tobyhede

Problem

Adding a scalar encrypted-domain type currently requires a manual edit to CI that is easy to forget. The matrix-coverage job in .github/workflows/test-eql.yml hardcodes the snapshot files it diffs:

# .github/workflows/test-eql.yml  (matrix-coverage job)
- name: Regenerate and verify the matrix test-name inventory
  run: |
    mise run test:matrix:inventory
    git diff --exit-code -- tests/sqlx/snapshots/int4_matrix_tests.txt \
                            tests/sqlx/snapshots/int2_matrix_tests.txt \
      || { echo "Coverage inventory stale — run 'mise run test:matrix:inventory' and commit."; exit 1; }

The same per-type hardcoding exists in the test:matrix:inventory task in mise.toml, which has one grep '^scalars::<T>' ... > snapshots/<T>_matrix_tests.txt block per type.

So every new type (int8, bool, date, …) means editing two places:

    git diff --exit-code -- tests/sqlx/snapshots/int4_matrix_tests.txt \
                            tests/sqlx/snapshots/int2_matrix_tests.txt \
                            tests/sqlx/snapshots/int8_matrix_tests.txt \   # <-- easy to forget

If either edit is forgotten, the new type's snapshot is silently not coverage-gated — which is exactly the failure mode this job exists to prevent.

Asks

  1. Abstract the per-type list so the extra step isn't required. Enumerate the scalar types dynamically from tasks/codegen/types/*.toml (the single source of truth) instead of hand-listing snapshot files — in both test:matrix:inventory (mise.toml) and the CI diff step. This is the same enumeration mise run codegen:domain:all and mise run fixture:generate:all already use, so adding a type is picked up automatically with no CI/task edit.

  2. Add a check that catches types with no snapshot. Fail CI if a type manifest (with a [fixture]/matrix surface) has no committed tests/sqlx/snapshots/<T>_matrix_tests.txt, and ideally the converse (a snapshot with no manifest — i.e. a removed type left a stale snapshot). Today a missing snapshot is simply uncovered rather than a hard error.

Prior art in this repo

fixture:generate:all (tasks/fixtures.toml) and codegen:domain:all (mise.toml) already enumerate tasks/codegen/types/*.toml and act on every type. The matrix-coverage inventory should follow the same pattern so the type list lives in exactly one place.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions