Skip to content

tests: Update pygrass doctests for NumPy 2 compatibility (#4100)#7194

Open
Khansa435 wants to merge 5 commits intoOSGeo:mainfrom
Khansa435:fix-numpy2-doctests
Open

tests: Update pygrass doctests for NumPy 2 compatibility (#4100)#7194
Khansa435 wants to merge 5 commits intoOSGeo:mainfrom
Khansa435:fix-numpy2-doctests

Conversation

@Khansa435
Copy link
Copy Markdown

Description

Fixes #4100
(and also addresses the underlying cause behind #4116 )

This PR updates the gunittest doctest suite to be fully compatible with NumPy 2.0+.

NumPy 2.0 introduced changes to the string representation of arrays and scalar types (e.g., scalars now print as np.int32(1) instead of 1, and arrays with non-default dtypes are formatted differently depending on the default platform size). This caused several doctests to fail, leading to test_pygrass_raster_doctests.py being excluded from the test suite.

As discussed with @echoix in the issue thread, since main is moving towards the 8.6 release, the strategy here is to explicitly target the NumPy 2 string representations natively rather than retaining backward compatibility with NumPy 1.x string outputs.

Changes Made

  • Bumped CI requirements: Updated numpy to numpy>=2.0.0 in .github/workflows/python_requirements.txt to guarantee the test environments use NumPy 2.
  • Un-excluded raster doctests: Re-enabled python/grass/pygrass/raster/testsuite/test_pygrass_raster_doctests.py in .gunittest.cfg so the test suite will run these tests again.
  • Updated scalar representations: Fixed the expected output in python/grass/pygrass/utils.py (get_raster_for_points) to expect the explicit np.int32(1) NumPy 2 scalar wrapper.
  • Updated array representations: Removed the dtype= parameter from expected Buffer outputs in python/grass/pygrass/raster/init.py. In NumPy 2 on standard configurations, these standard buffer arrays omit the dtype parameter string during formatting.

@github-actions github-actions bot added CI Continuous integration Python Related code is in Python libraries tests Related to Test Suite labels Mar 18, 2026
@Khansa435 Khansa435 changed the title test: Update pygrass doctests for NumPy 2 compatibility (#4100) tests: Update pygrass doctests for NumPy 2 compatibility (#4100) Mar 18, 2026
@Khansa435
Copy link
Copy Markdown
Author

Khansa435 commented Mar 19, 2026

Hi @echoix @wenzeslaus ! I've updated this PR with a fix for NumPy 2.x compatibility using # doctest: +ELLIPSIS wildcards.

This approach supports both NumPy 2 and the older NumPy 1.x used in the Ubuntu CI. Could you please approve the workflows to run as they require approval from a maintainer? :)

@Khansa435 Khansa435 force-pushed the fix-numpy2-doctests branch from b706b30 to 829c621 Compare March 19, 2026 14:43
@Khansa435
Copy link
Copy Markdown
Author

Khansa435 commented Mar 19, 2026

@echoix @wenzeslaus @ninsbl
Can you please take a look and review the changes I made :)

@Khansa435
Copy link
Copy Markdown
Author

Hi @echoix @wenzeslaus @ninsbl
Just a polite followup on requesting for the review of my code changes please :)

@Khansa435
Copy link
Copy Markdown
Author

Hi @echoix, Can you please review the changes :)
Thanks.

@echoix
Copy link
Copy Markdown
Member

echoix commented Mar 26, 2026

Can you please take a look at the CI logs and understand why the some tests (including doctests) fail there?

@Khansa435
Copy link
Copy Markdown
Author

Khansa435 commented Mar 28, 2026

@echoix
CI logs error details are showing that there's just 1 failing test file test_pygrass_raster_doctests.py but there's no detail to it.

Running ./python/grass/pygrass/raster/testsuite/test_pygrass_raster_doctests.py...

..========================================================================
FAILED ./python/grass/pygrass/raster/testsuite/test_pygrass_raster_doctests.py

Initially, there was a failing check for numpy compatibilty issue but that gave details like "Expected: " and "Got: " which I fixed but now this one is not clear as to why these are failing.

@ninsbl
Copy link
Copy Markdown
Member

ninsbl commented Mar 28, 2026

You could try to run failing tests locally...

@echoix
Copy link
Copy Markdown
Member

echoix commented Mar 28, 2026

Download the artifacts in the summary page of the failing tests, unzip it, and it's a website you can open locally and browse detailed info. That was made for before, when CI and detailed logs weren't a thing, so it's closer to a dashboard
image

@Khansa435
Copy link
Copy Markdown
Author

Khansa435 commented Mar 28, 2026

@echoix @ninsbl
I ran this command "pytest.main(['test_pygrass_raster_doctests.py', '-v', '-s'])" locally in GRASS python console first. It worked perfectly - no crashes, just the expected behavior:

collected 0 items # Normal for doctest-only files
1 warning in 1.50s # Just a pytest config warning about 'timeout'
exit code 5 # pytest "tests found but skipped" = clean pass

But the CI Summary and artifacts downloads result showed return code -11 (segfault) on dataset maybe.

So it looks like my numpy2 doctest fixes are working fine locally. The CI crash seems to be environment/dataset specific rather than a code problem...

What do you think? Or am I missing something here? :/

@echoix
Copy link
Copy Markdown
Member

echoix commented Mar 28, 2026

In your pytest run, how many were marked "passed"? To make sure they work, change the file with the doctest your trying, and deliberately change a number to make it fail. If it doesn't fail, either the file you changed isn't the one actually running, or your method of invoking it doesn't work.

But, I'm pretty sure that the doctests aren't collected and running through pytest. They have a different setup I think, that Pytest isn't configured to set it. They are supposed (for now) to be run with gunittest, with the appropriate dataset for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Continuous integration libraries Python Related code is in Python tests Related to Test Suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Python doctests must be updated for numpy 2+ or some later versions of numpy 1

3 participants