Skip to content

Commit 6ad2b84

Browse files
constantine2ndconstantine2nd
committed
ci: split test matrix from 3 to 4 shards; fix catch-all to cover shard 4
Shard 1 now runs v4_0_0 alone (~258s). Former shard-1 packages minus v4 move to shard 2 (adds v6_0_0, ~267s). Former shard-2 becomes shard 3 (~252s). Former shard-3 with catch-all becomes shard 4 (~232s). Catch-all block updated in both workflow files: shard check changed from "3" to "4", ASSIGNED now references SHARD1+SHARD2+SHARD3, echo message updated. CLAUDE.md shard table updated to match.
1 parent 03efaad commit 6ad2b84

3 files changed

Lines changed: 60 additions & 51 deletions

File tree

‎.github/workflows/build_container.yml‎

Lines changed: 28 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -10,12 +10,12 @@ env:
1010

1111
# ---------------------------------------------------------------------------
1212
# compile — compiles everything once, packages the JAR, uploads classes
13-
# test — 3-way matrix downloads compiled output and runs a shard of tests
13+
# test — 4-way matrix downloads compiled output and runs a shard of tests
1414
# docker — downloads compiled output, builds and pushes the container image
1515
#
1616
# Wall-clock target:
1717
# compile ~10 min (parallel with setup of test shards)
18-
# tests ~8 min (3 shards in parallel after compile finishes)
18+
# tests ~8 min (4 shards in parallel after compile finishes)
1919
# docker ~3 min (after all shards pass)
2020
# total ~21 min (vs ~30 min single-job)
2121
# ---------------------------------------------------------------------------
@@ -68,12 +68,13 @@ jobs:
6868
path: push/
6969

7070
# --------------------------------------------------------------------------
71-
# Job 2: test (3-way matrix)
71+
# Job 2: test (4-way matrix)
7272
#
73-
# Shard assignment (based on actual build #48 timings):
74-
# Shard 1 ~440s v4_0_0(292) v5_0_0(47) v3_0_0(42) v2_1_0(37) v2_2_0(11) …
75-
# Shard 2 ~460s v1_2_1(175) v6_0_0(162) ResourceDocs(82) util(15) berlin(41) …
76-
# Shard 3 ~420s v5_1_0(156) v3_1_0(124) http4sbridge(53) v7_0_0(27) code.api(26) …
73+
# Shard assignment (based on actual clean-run timings):
74+
# Shard 1 ~258s v4_0_0(258)
75+
# Shard 2 ~267s v6_0_0(122) v5_0_0(42) v3_0_0(39) v2_1_0(35) v2_2_0(12) …
76+
# Shard 3 ~252s v1_2_1(137) ResourceDocs(67) berlin(34) util(12) …
77+
# Shard 4 ~232s v5_1_0(79) v3_1_0(65) http4sbridge(52) v7_0_0(45) … + catch-all
7778
# --------------------------------------------------------------------------
7879
test:
7980
needs: compile
@@ -83,11 +84,15 @@ jobs:
8384
matrix:
8485
include:
8586
- shard: 1
86-
name: "v4 + v5_0 + v3_0 + v2 + small"
87-
# ~440s of test work
88-
# Space-separated package prefixes for scalatest wildcardSuites (-w)
87+
name: "v4 only"
88+
# ~258s of test work
8989
test_filter: >-
9090
code.api.v4_0_0
91+
- shard: 2
92+
name: "v6 + v5_0 + v3_0 + v2 + small"
93+
# ~267s of test work
94+
test_filter: >-
95+
code.api.v6_0_0
9196
code.api.v5_0_0
9297
code.api.v3_0_0
9398
code.api.v2_1_0
@@ -105,12 +110,11 @@ jobs:
105110
code.bankaccountcreation
106111
code.bankconnectors
107112
code.container
108-
- shard: 2
109-
name: "v1_2_1 + v6 + ResourceDocs + util + berlin + small"
110-
# ~460s of test work
113+
- shard: 3
114+
name: "v1_2_1 + ResourceDocs + berlin + util + small"
115+
# ~252s of test work
111116
test_filter: >-
112117
code.api.v1_2_1
113-
code.api.v6_0_0
114118
code.api.ResourceDocs1_4_0
115119
code.api.util
116120
code.api.berlin
@@ -121,9 +125,9 @@ jobs:
121125
code.usercustomerlinks
122126
code.customer
123127
code.errormessages
124-
- shard: 3
128+
- shard: 4
125129
name: "v5_1 + v3_1 + http4sbridge + v7 + code.api + util + connector"
126-
# ~420s of test work
130+
# ~232s of test work + catch-all for any new packages
127131
# Root-level code.api tests use class-name prefix matching (lowercase classes)
128132
test_filter: >-
129133
code.api.v5_1_0
@@ -238,19 +242,20 @@ jobs:
238242
# The YAML >- scalar collapses newlines to spaces, so we convert here.
239243
FILTER=$(echo "${{ matrix.test_filter }}" | tr ' ' ',')
240244
241-
# Shard 3 is the catch-all: append any test package not explicitly
242-
# assigned to shard 1 or shard 2, so new packages are never silently skipped.
243-
if [ "${{ matrix.shard }}" = "3" ]; then
244-
SHARD1="code.api.v4_0_0 code.api.v5_0_0 code.api.v3_0_0 code.api.v2_1_0 \
245+
# Shard 4 is the catch-all: append any test package not explicitly
246+
# assigned to shards 1–3, so new packages are never silently skipped.
247+
if [ "${{ matrix.shard }}" = "4" ]; then
248+
SHARD1="code.api.v4_0_0"
249+
SHARD2="code.api.v6_0_0 code.api.v5_0_0 code.api.v3_0_0 code.api.v2_1_0 \
245250
code.api.v2_2_0 code.api.v2_0_0 code.api.v1_4_0 code.api.v1_3_0 \
246251
code.api.UKOpenBanking code.atms code.branches code.products code.crm \
247252
code.accountHolder code.entitlement code.bankaccountcreation \
248253
code.bankconnectors code.container"
249-
SHARD2="code.api.v1_2_1 code.api.v6_0_0 code.api.ResourceDocs1_4_0 \
254+
SHARD3="code.api.v1_2_1 code.api.ResourceDocs1_4_0 \
250255
code.api.util code.api.berlin code.management code.metrics \
251256
code.model code.views code.usercustomerlinks code.customer \
252257
code.errormessages"
253-
ASSIGNED="$SHARD1 $SHARD2 ${{ matrix.test_filter }}"
258+
ASSIGNED="$SHARD1 $SHARD2 $SHARD3 ${{ matrix.test_filter }}"
254259
255260
# Discover all packages that contain at least one .scala test file
256261
ALL_PKGS=$(find obp-api/src/test/scala obp-commons/src/test/scala \
@@ -269,7 +274,7 @@ jobs:
269274
[ "$covered" = "false" ] && EXTRAS="$EXTRAS,$pkg"
270275
done
271276
272-
[ -n "$EXTRAS" ] && echo "Catch-all extras added to shard 3:$EXTRAS"
277+
[ -n "$EXTRAS" ] && echo "Catch-all extras added to shard 4:$EXTRAS"
273278
FILTER="${FILTER}${EXTRAS}"
274279
fi
275280

‎.github/workflows/build_pull_request.yml‎

Lines changed: 26 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -68,12 +68,13 @@ jobs:
6868
path: pull/
6969

7070
# --------------------------------------------------------------------------
71-
# Job 2: test (3-way matrix)
71+
# Job 2: test (4-way matrix)
7272
#
73-
# Shard assignment (based on actual build #48 timings):
74-
# Shard 1 ~440s v4_0_0(292) v5_0_0(47) v3_0_0(42) v2_1_0(37) v2_2_0(11) …
75-
# Shard 2 ~460s v1_2_1(175) v6_0_0(162) ResourceDocs(82) util(15) berlin(41) …
76-
# Shard 3 ~420s v5_1_0(156) v3_1_0(124) http4sbridge(53) v7_0_0(27) code.api(26) …
73+
# Shard assignment (based on actual clean-run timings):
74+
# Shard 1 ~258s v4_0_0(258)
75+
# Shard 2 ~267s v6_0_0(122) v5_0_0(42) v3_0_0(39) v2_1_0(35) v2_2_0(12) …
76+
# Shard 3 ~252s v1_2_1(137) ResourceDocs(67) berlin(34) util(12) …
77+
# Shard 4 ~232s v5_1_0(79) v3_1_0(65) http4sbridge(52) v7_0_0(45) … + catch-all
7778
# --------------------------------------------------------------------------
7879
test:
7980
needs: compile
@@ -83,11 +84,15 @@ jobs:
8384
matrix:
8485
include:
8586
- shard: 1
86-
name: "v4 + v5_0 + v3_0 + v2 + small"
87-
# ~440s of test work
88-
# Space-separated package prefixes for scalatest wildcardSuites (-w)
87+
name: "v4 only"
88+
# ~258s of test work
8989
test_filter: >-
9090
code.api.v4_0_0
91+
- shard: 2
92+
name: "v6 + v5_0 + v3_0 + v2 + small"
93+
# ~267s of test work
94+
test_filter: >-
95+
code.api.v6_0_0
9196
code.api.v5_0_0
9297
code.api.v3_0_0
9398
code.api.v2_1_0
@@ -105,12 +110,11 @@ jobs:
105110
code.bankaccountcreation
106111
code.bankconnectors
107112
code.container
108-
- shard: 2
109-
name: "v1_2_1 + v6 + ResourceDocs + util + berlin + small"
110-
# ~460s of test work
113+
- shard: 3
114+
name: "v1_2_1 + ResourceDocs + berlin + util + small"
115+
# ~252s of test work
111116
test_filter: >-
112117
code.api.v1_2_1
113-
code.api.v6_0_0
114118
code.api.ResourceDocs1_4_0
115119
code.api.util
116120
code.api.berlin
@@ -121,9 +125,9 @@ jobs:
121125
code.usercustomerlinks
122126
code.customer
123127
code.errormessages
124-
- shard: 3
128+
- shard: 4
125129
name: "v5_1 + v3_1 + http4sbridge + v7 + code.api + util + connector"
126-
# ~420s of test work
130+
# ~232s of test work + catch-all for any new packages
127131
# Root-level code.api tests use class-name prefix matching (lowercase classes)
128132
test_filter: >-
129133
code.api.v5_1_0
@@ -238,19 +242,20 @@ jobs:
238242
# The YAML >- scalar collapses newlines to spaces, so we convert here.
239243
FILTER=$(echo "${{ matrix.test_filter }}" | tr ' ' ',')
240244
241-
# Shard 3 is the catch-all: append any test package not explicitly
242-
# assigned to shard 1 or shard 2, so new packages are never silently skipped.
243-
if [ "${{ matrix.shard }}" = "3" ]; then
244-
SHARD1="code.api.v4_0_0 code.api.v5_0_0 code.api.v3_0_0 code.api.v2_1_0 \
245+
# Shard 4 is the catch-all: append any test package not explicitly
246+
# assigned to shards 1–3, so new packages are never silently skipped.
247+
if [ "${{ matrix.shard }}" = "4" ]; then
248+
SHARD1="code.api.v4_0_0"
249+
SHARD2="code.api.v6_0_0 code.api.v5_0_0 code.api.v3_0_0 code.api.v2_1_0 \
245250
code.api.v2_2_0 code.api.v2_0_0 code.api.v1_4_0 code.api.v1_3_0 \
246251
code.api.UKOpenBanking code.atms code.branches code.products code.crm \
247252
code.accountHolder code.entitlement code.bankaccountcreation \
248253
code.bankconnectors code.container"
249-
SHARD2="code.api.v1_2_1 code.api.v6_0_0 code.api.ResourceDocs1_4_0 \
254+
SHARD3="code.api.v1_2_1 code.api.ResourceDocs1_4_0 \
250255
code.api.util code.api.berlin code.management code.metrics \
251256
code.model code.views code.usercustomerlinks code.customer \
252257
code.errormessages"
253-
ASSIGNED="$SHARD1 $SHARD2 ${{ matrix.test_filter }}"
258+
ASSIGNED="$SHARD1 $SHARD2 $SHARD3 ${{ matrix.test_filter }}"
254259
255260
# Discover all packages that contain at least one .scala test file
256261
ALL_PKGS=$(find obp-api/src/test/scala obp-commons/src/test/scala \
@@ -269,7 +274,7 @@ jobs:
269274
[ "$covered" = "false" ] && EXTRAS="$EXTRAS,$pkg"
270275
done
271276
272-
[ -n "$EXTRAS" ] && echo "Catch-all extras added to shard 3:$EXTRAS"
277+
[ -n "$EXTRAS" ] && echo "Catch-all extras added to shard 4:$EXTRAS"
273278
FILTER="${FILTER}${EXTRAS}"
274279
fi
275280

‎CLAUDE.md‎

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -172,18 +172,17 @@ The 12 pure-unit suites (172 tests, 1.3s total):
172172

173173
**`ResourceDocsTest` / `SwaggerDocsTest`** — 34s + 24s = 58s, averaging 0.85s/test — the slowest per-test cost in the suite. Each test serializes the entire API surface (633+ endpoints) into JSON/Swagger. Cost scales linearly with endpoint count. Will worsen as the http4s migration adds endpoints unless ResourceDoc serialization is cached or the heavy tests are isolated.
174174

175-
**Shard imbalance**: shard2 runs ~100s longer than shard3 because it holds `API1_2_1Test`. Reassigning that one suite to its own shard or splitting it would balance all three shards to ~9m15s.
176-
177175
### Shard assignment
178176

179-
Shards are defined by explicit package-prefix allowlists in `.github/workflows/build_pull_request.yml` (lines 89–139). Shard 3 also runs a **catch-all**: any `.scala` test file whose package is not covered by shards 1 or 2 is appended automatically at runtime — new packages are never silently skipped. Extras are printed in the step log under `"Catch-all extras added to shard 3:"`.
177+
Shards are defined by explicit package-prefix allowlists in `.github/workflows/build_pull_request.yml` (lines 89–143). Shard 4 also runs a **catch-all**: any `.scala` test file whose package is not covered by shards 1–3 is appended automatically at runtime — new packages are never silently skipped. Extras are printed in the step log under `"Catch-all extras added to shard 4:"`.
180178

181179
| Package prefix | Shard |
182180
|---|---|
183-
| `code.api.v4_0_0`, `code.api.v5_0_0`, `code.api.v3_0_0`, `code.api.v2_*`, `code.api.v1_[34]_0`, `code.api.UKOpenBanking`, `code.atms`, `code.branches`, `code.products`, `code.crm`, `code.accountHolder`, `code.entitlement`, `code.bankaccountcreation`, `code.bankconnectors`, `code.container` | 1 |
184-
| `code.api.v1_2_1`, `code.api.v6_0_0`, `code.api.ResourceDocs1_4_0`, `code.api.util`, `code.api.berlin`, `code.management`, `code.metrics`, `code.model`, `code.views`, `code.usercustomerlinks`, `code.customer`, `code.errormessages` | 2 |
185-
| `code.api.v5_1_0`, `code.api.v3_1_0`, `code.api.http4sbridge`, `code.api.v7_0_0`, `code.api.Authentication*`, `code.api.DirectLoginTest`, `code.api.dauthTest`, `code.api.gateWayloginTest`, `code.api.OBPRestHelperTest`, `code.util`, `code.connector` | 3 |
186-
| anything else | **3** (catch-all) |
181+
| `code.api.v4_0_0` | 1 |
182+
| `code.api.v6_0_0`, `code.api.v5_0_0`, `code.api.v3_0_0`, `code.api.v2_*`, `code.api.v1_[34]_0`, `code.api.UKOpenBanking`, `code.atms`, `code.branches`, `code.products`, `code.crm`, `code.accountHolder`, `code.entitlement`, `code.bankaccountcreation`, `code.bankconnectors`, `code.container` | 2 |
183+
| `code.api.v1_2_1`, `code.api.ResourceDocs1_4_0`, `code.api.util`, `code.api.berlin`, `code.management`, `code.metrics`, `code.model`, `code.views`, `code.usercustomerlinks`, `code.customer`, `code.errormessages` | 3 |
184+
| `code.api.v5_1_0`, `code.api.v3_1_0`, `code.api.http4sbridge`, `code.api.v7_0_0`, `code.api.Authentication*`, `code.api.DirectLoginTest`, `code.api.dauthTest`, `code.api.gateWayloginTest`, `code.api.OBPRestHelperTest`, `code.util`, `code.connector` | 4 |
185+
| anything else | **4** (catch-all) |
187186

188187
To explicitly move a package to a different shard, add it to that shard's `test_filter` block — it will be excluded from the catch-all automatically.
189188

0 commit comments

Comments
 (0)