Tests fail in GitHub Action, but not locally

[JR-1991]

While updating and preparing #28 for merge, the tests in the GitHub Action are failing but working locally. I've been using 6.6 throughout, but I am a bit puzzled as to why this happens.

@pdurbin, do you have any ideas on why this is not working? This is the last workflow run and in the following my local output.

[pdurbin]

the last workflow run

Huh, no nothing jumps to mind. I'm not sure why other ids would change. 🤔

=========================== short test summary info ============================
FAILED tests/integration/test_dataset_update.py::TestDatasetUpdate::test_update_edit - AssertionError: The updated dataset does not have the expected number of other ids.
assert 4 == 2
 +  where 4 = len([{'otherIdAgency': {'multiple': False, 'typeClass': 'primitive', 'typeName': 'otherIdAgency', 'value': 'Software Heritage1'}, 'otherIdValue': {'multiple': False, 'typeClass': 'primitive', 'typeName': 'otherIdValue', 'value': 'softwareid1'}}, {'otherIdAgency': {'multiple': False, 'typeClass': 'primitive', 'typeName': 'otherIdAgency', 'value': 'Software Heritage2'}, 'otherIdValue': {'multiple': False, 'typeClass': 'primitive', 'typeName': 'otherIdValue', 'value': 'softwareid2'}}, {'otherIdAgency': {'multiple': False, 'typeClass': 'primitive', 'typeName': 'otherIdAgency', 'value': 'Software Heritage1 updated'}, 'otherIdValue': {'multiple': False, 'typeClass': 'primitive', 'typeName': 'otherIdValue', 'value': 'softwareid1'}}, {'otherIdAgency': {'multiple': False, 'typeClass': 'primitive', 'typeName': 'otherIdAgency', 'value': 'Software Heritage2'}, 'otherIdValue': {'multiple': False, 'typeClass': 'primitive', 'typeName': 'otherIdValue', 'value': 'softwareid2'}}])
=========== 1 failed, 8 passed, 48 deselected, 283 warnings in 7.56s ===========

[JR-1991]

The test validates the update method by using the editMetadata endpoint. Depending on whether replace=false is specified, this endpoint either completely replaces the current metadata with new data or selectively updates it. For further details, check out the implementation and the corresponding test.

I’m curious, as to why it works locally but not within the action. Could there be a specific Dataverse configuration setting that governs this edit behavior that the Dataverse Action does not have?

[pdurbin]

Good news, I guess? I can reproduce the problem locally. This is what I'm seeing:

% pytest -vv tests/integration/test_dataset_update.py
====================================== test session starts ======================================
platform darwin -- Python 3.12.4, pytest-8.3.5, pluggy-1.5.0 -- /Users/PDurbin/github/gdcc/easyDataverse/venv/bin/python3.12
cachedir: .pytest_cache
rootdir: /Users/PDurbin/github/gdcc/easyDataverse
configfile: pytest.ini
plugins: anyio-4.9.0, cov-6.1.1
collected 3 items                                                                               

tests/integration/test_dataset_update.py::TestDatasetUpdate::test_dataset_update PASSED   [ 33%]
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_dataset_update_wo_replace PASSED [ 66%]
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_update_edit FAILED      [100%]

=========================================== FAILURES ============================================
______________________________ TestDatasetUpdate.test_update_edit _______________________________

self = <tests.integration.test_dataset_update.TestDatasetUpdate object at 0x104d64620>
credentials = ('http://localhost:8080', '7e1ac48d-7f93-466d-9f38-734aea3b6275')
minimal_upload = {'datasetVersion': {'metadataBlocks': {'citation': {'fields': [{'multiple': True, 'typeClass': 'compound', 'typeName':...bject', 'value': [...]}, {'multiple': False, 'typeClass': 'primitive', 'typeName': 'title', 'value': 'My dataset'}]}}}}

    @pytest.mark.integration
    def test_update_edit(
        self,
        credentials,
        minimal_upload,
    ):
        # Arrange
        base_url, api_token = credentials
        url = f"{base_url}/api/dataverses/root/datasets"
        response = requests.post(
            url=url,
            json=minimal_upload,
            headers={
                "X-Dataverse-key": api_token,
                "Content-Type": "application/json",
            },
        )
    
        response.raise_for_status()
        pid = response.json()["data"]["persistentId"]
    
        # Act
        dataverse = Dataverse(
            server_url=base_url,
            api_token=api_token,
        )
    
        # Fetch the dataset and update the title
        dataset = dataverse.load_dataset(pid)
    
        # Check if multiple compound changes are tracked too
        dataset.citation.add_other_id(agency="Software Heritage1", value="softwareid1")
        dataset.citation.add_other_id(agency="Software Heritage2", value="softwareid2")
    
        dataset.update(replace=False)
    
        # Fetch another time and edit the first entry
        dataset = dataverse.load_dataset(pid)
        dataset.citation.other_id[0].agency = "Software Heritage1 updated"
    
        dataset.update(replace=False)
    
        # Re-fetch the dataset
        url = (
            f"{base_url}/api/datasets/:persistentId/versions/:draft?persistentId={pid}"
        )
    
        response = requests.get(
            url,
            headers={"X-Dataverse-key": api_token},
        )
    
        response.raise_for_status()
        updated_dataset = response.json()
        other_id_fields = next(
            filter(
                lambda x: x["typeName"] == "otherId",
                updated_dataset["data"]["metadataBlocks"]["citation"]["fields"],
            )
        )["value"]
    
        # Assert
>       assert (
            len(other_id_fields) == 2
        ), "The updated dataset does not have the expected number of other ids."
E       AssertionError: The updated dataset does not have the expected number of other ids.
E       assert 4 == 2
E        +  where 4 = len([{'otherIdAgency': {'typeName': 'otherIdAgency', 'multiple': False, 'typeClass': 'primitive', 'value': 'Software Heritage1'}, 'otherIdValue': {'typeName': 'otherIdValue', 'multiple': False, 'typeClass': 'primitive', 'value': 'softwareid1'}}, {'otherIdAgency': {'typeName': 'otherIdAgency', 'multiple': False, 'typeClass': 'primitive', 'value': 'Software Heritage2'}, 'otherIdValue': {'typeName': 'otherIdValue', 'multiple': False, 'typeClass': 'primitive', 'value': 'softwareid2'}}, {'otherIdAgency': {'typeName': 'otherIdAgency', 'multiple': False, 'typeClass': 'primitive', 'value': 'Software Heritage1 updated'}, 'otherIdValue': {'typeName': 'otherIdValue', 'multiple': False, 'typeClass': 'primitive', 'value': 'softwareid1'}}, {'otherIdAgency': {'typeName': 'otherIdAgency', 'multiple': False, 'typeClass': 'primitive', 'value': 'Software Heritage2'}, 'otherIdValue': {'typeName': 'otherIdValue', 'multiple': False, 'typeClass': 'primitive', 'value': 'softwareid2'}}])

tests/integration/test_dataset_update.py:228: AssertionError
------------------------------------- Captured stdout call --------------------------------------


🎉 Connected to 'http://localhost:8080/'

Fetching dataset 'doi:10.5072/FK2/FPZXL6' from 'http://localhost:8080/'

╭─ Dataset Information ─╮
│ Title: My dataset     │
│ Version: latest       │
│ Files: 0              │
╰───────────────────────╯
Fetching dataset 'doi:10.5072/FK2/FPZXL6' from 'http://localhost:8080/'

╭─ Dataset Information ─╮
│ Title: My dataset     │
│ Version: latest       │
│ Files: 0              │
╰───────────────────────╯
======================================= warnings summary ========================================
tests/integration/test_dataset_update.py: 56 warnings
  /Users/PDurbin/github/gdcc/easyDataverse/easyDataverse/base.py:32: PydanticDeprecatedSince211: Accessing the 'model_fields' attribute on the instance is deprecated. Instead, you should access this attribute from the model class. Deprecated in Pydantic V2.11 to be removed in V3.0.
    if name in self.model_fields:

tests/integration/test_dataset_update.py: 54 warnings
  /Users/PDurbin/github/gdcc/easyDataverse/easyDataverse/base.py:262: PydanticDeprecatedSince211: Accessing the 'model_fields' attribute on the instance is deprecated. Instead, you should access this attribute from the model class. Deprecated in Pydantic V2.11 to be removed in V3.0.
    for attr, field in self.model_fields.items():

tests/integration/test_dataset_update.py: 28 warnings
  /Users/PDurbin/github/gdcc/easyDataverse/easyDataverse/base.py:210: PydanticDeprecatedSince211: Accessing the 'model_fields' attribute on the instance is deprecated. Instead, you should access this attribute from the model class. Deprecated in Pydantic V2.11 to be removed in V3.0.
    for name, field in self.model_fields.items():

tests/integration/test_dataset_update.py::TestDatasetUpdate::test_dataset_update
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_dataset_update
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_dataset_update_wo_replace
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_update_edit
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_update_edit
  /Users/PDurbin/github/gdcc/easyDataverse/easyDataverse/base.py:193: PydanticDeprecatedSince211: Accessing the 'model_fields' attribute on the instance is deprecated. Instead, you should access this attribute from the model class. Deprecated in Pydantic V2.11 to be removed in V3.0.
    field = self.model_fields[name]

tests/integration/test_dataset_update.py::TestDatasetUpdate::test_dataset_update
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_dataset_update
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_dataset_update_wo_replace
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_dataset_update_wo_replace
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_update_edit
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_update_edit
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_update_edit
tests/integration/test_dataset_update.py::TestDatasetUpdate::test_update_edit
  /Users/PDurbin/github/gdcc/easyDataverse/easyDataverse/base.py:128: PydanticDeprecatedSince211: Accessing the 'model_fields' attribute on the instance is deprecated. Instead, you should access this attribute from the model class. Deprecated in Pydantic V2.11 to be removed in V3.0.
    for attr, field in self.model_fields.items():

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================== short test summary info ====================================
FAILED tests/integration/test_dataset_update.py::TestDatasetUpdate::test_update_edit - AssertionError: The updated dataset does not have the expected number of other ids.
assert 4 == 2
 +  where 4 = len([{'otherIdAgency': {'typeName': 'otherIdAgency', 'multiple': False, 'typeClass': 'primitive', 'value': 'Software Heritage1'}, 'otherIdValue': {'typeName': 'otherIdValue', 'multiple': False, 'typeClass': 'primitive', 'value': 'softwareid1'}}, {'otherIdAgency': {'typeName': 'otherIdAgency', 'multiple': False, 'typeClass': 'primitive', 'value': 'Software Heritage2'}, 'otherIdValue': {'typeName': 'otherIdValue', 'multiple': False, 'typeClass': 'primitive', 'value': 'softwareid2'}}, {'otherIdAgency': {'typeName': 'otherIdAgency', 'multiple': False, 'typeClass': 'primitive', 'value': 'Software Heritage1 updated'}, 'otherIdValue': {'typeName': 'otherIdValue', 'multiple': False, 'typeClass': 'primitive', 'value': 'softwareid1'}}, {'otherIdAgency': {'typeName': 'otherIdAgency', 'multiple': False, 'typeClass': 'primitive', 'value': 'Software Heritage2'}, 'otherIdValue': {'typeName': 'otherIdValue', 'multiple': False, 'typeClass': 'primitive', 'value': 'softwareid2'}}])
=========================== 1 failed, 2 passed, 151 warnings in 3.69s ===========================

This is how the dataset looks:

[JR-1991]

@pdurbin That's good news, thanks for testing! There may be something I have missed. I will look into the PR again and pin down the issue 🙌