Skip to content

Commit 39cdbcb

Browse files
authored
Merge: Use Custom instead of OHE encoded categorical parameter for Non-SHAP tests (emdgroup#732)
There seem to be machine-dependent non-deterministic failures for the LIME method which is part of our SHAP insight (as non-SHAP extension handled mostly via shap). The failures do not indicate any particular problem on our side but some internal process causing NaN's when calculating model regression metrics. The test only seems to fail when there is the `Categorical_1` parameter involved, which is a 3 category, OHE encoded parameter. This was replaced with the custom parameter as it avoids degenerate values which hopefully contribute to avoiding the NaN generations in LIME. I've [tried this fix 3 times in the `Regular` action](https://github.com/emdgroup/baybe/actions/runs/21355269316) (had to ignore one pip-audit vulnerability to do so in a temporary commit which has been removed again). All runs had no issue, which I take as indication the fix worked. If the issue appears again I would probably add another parameter to the test searchspaces to avoid generation of degenerate points.
2 parents 43a921e + b40ce27 commit 39cdbcb

2 files changed

Lines changed: 3 additions & 4 deletions

File tree

.github/workflows/regular.yml

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -140,8 +140,7 @@ jobs:
140140
py-version: [ {semantic: '3.10', tox: 'py310'},
141141
{semantic: '3.11', tox: 'py311'},
142142
{semantic: '3.12', tox: 'py312'},
143-
{semantic: '3.13', tox: 'py313'},
144-
{semantic: '3.14', tox: 'py314'} ]
143+
{semantic: '3.13', tox: 'py313'} ]
145144
name: Full Tests ${{ matrix.py-version.semantic }}
146145
runs-on: ubuntu-latest
147146
steps:

tests/insights/test_shap.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,8 +55,8 @@ def batch_size():
5555
["Conti_finite1", "Conti_finite2"],
5656
["Num_disc_1", "Fraction_1"],
5757
["Num_disc_1", "Conti_finite1"],
58-
["Num_disc_1", "Categorical_1"],
59-
["Conti_finite1", "Categorical_1"],
58+
["Num_disc_1", "Custom_1"],
59+
["Conti_finite1", "Custom_1"],
6060
],
6161
ids=[
6262
"params_conti",

0 commit comments

Comments
 (0)