proposal: colocate tests with files being tested #83
Description
tl;dr -- move unit tests to be located next to the files they test to improve coverage discoverability.
I will do this if we have a consensus.
My proposal is each python file has a *_test.py file associated with it that lives in the same folder. This file contains unit tests for objects implemented in the python file. We would still have some integration, example, or similar tests that only make sense inside a top-level "tests" folder.
Arguments for
- test location and coverage discoverability is higher (imo)
- Reduces the cognitive load of navigating between source files and a separate test suite
- tests can be run by a user after installing the package to ensure the install went well (discussion)
- easy to run sub-tests (pytest goodpractices discussion)
- easy maintenance & refactoring
- if you move files around you don't have to repeat the process in a mirrored directory
Arguments against
- tests will ship with production code, which inflates the package size. I think this is not a real issue for research code. And I think it's currently happening anyway.
- there is some work to reorganize the current repo structure
Examples in the wild
Numpy does something similar, although they group tests into lowest level folders. For example, here's the polynomial folder.
This is not an industry standard in Python repos. Many big repos use top-level folders, like Pytorch, matplotlib, and Pandas.
What I'm proposing is standard practice in lower-level languages like Rust.