Refactor the tests into a more standard structure

[joanise]

In g2p and everyvoice, we have tests under <REPO>/<projectroot>/tests, but in Studio we have them directly under <REPO>/test.
The way we've installed them, the standard python -m unittest invocation cannot be used to run a class or a single function, instead we have to one or more files at a time in our unusual custom way.

Tasks:

• Move tests under readalongs/tests
• change the import of basic_test_case and sound_swallower_stub to proper relative import, i.e., from .basic_test_case import ...
• address other dependencies we had to the current file structure
• exclude readalongs/tests from wheel and dist, the way everyvoice does, so they don't bloat our published wheels
• replace test/run.py by run_tests.py at the root like in g2p
• make sure code coverage analysis excludes readalongs/tests/
• Update the documentation on how to run the tests

[dhdaines]

Putting tests inside the package directory isn't really a standard structure, and in my experience not very many packages do it. The Python Packaging Authority recommends against it, for instance:

https://packaging.python.org/en/latest/tutorials/packaging-projects/#creating-the-package-files

(also we should call our test directory tests not test).

There are some arguments for or against, you can read more about it in the pytest documentation and some other places:

https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html

I think actual problem with running individual modules/files with python -m unittest is that they use implicit relative imports:

(readalongs) dhd@minipc:~/work/ReadAlongs/Studio$ python -m unittest test.test_audio
E
======================================================================
ERROR: test_audio (unittest.loader._FailedTest.test_audio)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_audio
Traceback (most recent call last):
  File "/usr/lib/python3.11/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dhd/work/ReadAlongs/Studio/test/test_audio.py", line 10, in <module>
    from basic_test_case import BasicTestCase
ModuleNotFoundError: No module named 'basic_test_case'

[dhdaines]

TL;DR instead of doing this we should use pytest

[joanise]

Interesting. I haven't used pytest before, but it looks like if I make the imports explicitly relative we can run the suite with pytest with little change.

[joanise]

TL;DR instead of doing this we should use pytest

@dhdaines sadly pytest doens't solve the problem of not finding basic_test_case. As far as I can tell, the only to have importable shared code in the tests is to have that file installed somewhere. But, you have more experience that I do here: would you know how to make pytest work with our tests?

One of the sales pitches for pytest is you can just start using it with your existing tests, and then later you can start using its features. But with all our projects, I'm hitting a wall on "just start using it"... Our existing tests are apparently really not compatible with pytest as they are.

[dhdaines]

You could name basic_test_case explicitly in [tool.pytest] I think... let me see!

[dhdaines]

Oh right, basic_test_case is an imported module, sorry, I forgot about that!

[dhdaines]

Ok, this all just basically works once we do the things in your list 😄 I'll make a PR now.