Skip to content

Maint/to pytest test dataset sparse dataset #1418

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Taniya-Das wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Maint/to pytest test dataset sparse dataset #1418

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
18c64d1
resolve merge conflict
Taniya-Das Jun 18, 2025
261849b
test get_sparse_data
Taniya-Das Jun 18, 2025
e5267d6
convert test_get_sparse_daatset_rowid_and_ignore_and_target
Taniya-Das Jun 18, 2025
d73df95
convert test_sparse_categorical_395
Taniya-Das Jun 19, 2025
879fe7a
remove repetition using fixture
Taniya-Das Jun 19, 2025
a6aae77
remove old fixture
Taniya-Das Jun 19, 2025
69e8072
minor correction
Taniya-Das Jun 19, 2025
cbaf4bf
merge conflict resolve
Taniya-Das Jun 19, 2025
8a75cb4
Merge branch 'develop' into maint/to_pytest_test_dataset_OpenMLdatase…
Taniya-Das Jun 19, 2025
28fff8a
resolve comments
Taniya-Das Jun 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion openml/_api_calls.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,6 @@ def _download_text_file(
start = time.time()
response = __read_url(source, request_method="get", md5_checksum=md5_checksum)
downloaded_file = response.text

if output_path is None:
logging.info(
"%.7fs taken for [%s] request for the URL %s",
Expand Down
43 changes: 43 additions & 0 deletions tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,3 +307,46 @@ def workdir(tmp_path):
os.chdir(tmp_path)
yield tmp_path
os.chdir(original_cwd)

@pytest.fixture
def mock_data_base_path(test_files_directory):
"""Fixture to provide the base path for mock data files."""
return test_files_directory / "mock_responses"


@pytest.fixture
def mock_sparse_categorical_395(requests_mock, mock_data_base_path):

description_file = mock_data_base_path / "datasets" / "sparse_categorical_395" / "description.xml"
requests_mock.get(
"https://www.openml.org/api/v1/xml/data/395",
Copy link
Contributor

@LennartPurucker LennartPurucker Jun 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe make the API base path a fixture as well

Copy link
Member Author

@Taniya-Das Taniya-Das Jun 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, but it doesn't make a lot of difference. the generic base is just test_files_directory / "mock_responses".

text=description_file.read_text()
)

data_file = mock_data_base_path / "datasets" / "sparse_categorical_395" / "dataset.arff"
requests_mock.get(
"https://api.openml.org/data/v1/download/52507/re1.wc.sparse_arff",
text=data_file.read_text()
)

feature_file = mock_data_base_path / "datasets" / "sparse_categorical_395" / "features.xml"
requests_mock.get(
"https://www.openml.org/api/v1/xml/data/features/395",
text=feature_file.read_text()
)
yield

@pytest.fixture
def mock_sparse_dataset(requests_mock, mock_data_base_path):
content_file = (
mock_data_base_path / "datasets" / "sparse_dataset" /"data_description.xml"
)
requests_mock.get("https://www.openml.org/api/v1/xml/data/4136", text=content_file.read_text())

sparse_arff_file = (
mock_data_base_path / "datasets" / "sparse_dataset" /"sparse_arff.arff"
)
requests_mock.get("https://api.openml.org/data/v1/download/1681111/Dexter.sparse_arff", text = sparse_arff_file.read_text())


yield
Loading
Loading