Windows functional test

[Sam071100]

Addresses Issue #27

[darosior]

Discussed this with @Sam071100 outside of Github, and we need to find a workaround for the OpenSSL-on-Windows issue: petertodd/python-bitcoinlib#262 .

The current explored option is to drop the python-bitcoinlib dependency, which we virtually only use for the RPC connection to bitcoind. This could be trivially replaced by HTTP requests to the endpoint.

[darosior]

@Sam071100 are you still planning to work on this?

[darosior]

Rebased and updated this PR.

[danielabrozzoni]

Some issues we have:

• Sometimes the JSON RPC path is too long:

b"[1639825344][revaultd][INFO] Using Noise static public key: '3698d872edc4446733657398e3b4e9d11dab14cec0ac2f83558450246458fc4d'"
b"[1639825344][revaultd][DEBUG] Coordinator static public key: 'd91563973102454a7830137e92d0548bc83b4ea2799f1df04622ca1307381402'"
b'[1639825344][revaultd::daemon][INFO] Setting up database'
b'[1639825344][revaultd::daemon::database::actions][INFO] No database at "\\\\\\\\?\\\\C:\\\\Users\\\\Daniela\\\\AppData\\\\Local\\\\Temp\\\\revaultd-_6mjdeyh\\\\test_reorged_deposit_1\\\\revaultd\\\\regtest\\\\revaultd.sqlite3", creating a new one.'
b'[1639825344][revaultd::daemon][INFO] Setting up bitcoind connection'
b'[1639825344][revaultd::daemon][INFO] Starting JSONRPC server'
b'[1639825344][revaultd::daemon][ERROR] panic occurred at line 47 of file src\\daemon\\mod.rs: Some("Setting up JSONRPC server: Custom { kind: InvalidInput, error: \\"path must be shorter than SUN_LEN\\" }")'

setting the environment variable TEST_DIR helps.

• We have some troubles with the JSON RPC (maybe windows: parse error of JSONRPC request #339 is related?). Calling getinfo works, as it doesn't have arguments, but calling listvaults with arguments throws:

(venv) C:\Users\Daniela\Downloads\revaultd-Windows_Functional_Test\revaultd-Windows_Functional_Test\tests>pytest -k test_listvaults
=================================================================================================================================================== test session starts ====================================================================================================================================================
platform win32 -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-0.13.1
rootdir: C:\Users\Daniela\Downloads\revaultd-Windows_Functional_Test\revaultd-Windows_Functional_Test, configfile: pyproject.toml
plugins: forked-1.4.0, timeout-1.3.4, xdist-1.31.0
collected 41 items / 40 deselected / 1 selected

test_rpc.py::test_listvaults
------------------------------------------------------------------------------------------------------------------------------------------------------ live log setup ------------------------------------------------------------------------------------------------------------------------------------------------------
INFO     root:bitcoind.py:82 BitcoinD started
FAILED                                                                                                                                                                                                                                                                                                                [100%]

========================================================================================================================================================= FAILURES =========================================================================================================================================================
_____________________________________________________________________________________________________________________________________________________ test_listvaults ______________________________________________________________________________________________________________________________________________________

revaultd_manager = <test_framework.revaultd.ManagerRevaultd object at 0x000001E6D6C57BB0>, bitcoind = <test_framework.bitcoind.BitcoinD object at 0x000001E6D6A1F280>

    def test_listvaults(revaultd_manager, bitcoind):
        res = revaultd_manager.rpc.call("listvaults")
        assert res["vaults"] == []

        # Send to a deposit address, we detect one unconfirmed vault
        amount_sent = 0.75
        addr = revaultd_manager.rpc.call("getdepositaddress")["address"]
        txid = bitcoind.rpc.sendtoaddress(addr, amount_sent)
        revaultd_manager.wait_for_log("Got a new unconfirmed deposit")
        vault_list = revaultd_manager.rpc.call("listvaults")["vaults"]
        assert len(vault_list) == 1
        assert vault_list[0]["status"] == "unconfirmed"
        assert vault_list[0]["txid"] == txid
        assert vault_list[0]["amount"] == amount_sent * 10 ** 8
        assert vault_list[0]["address"] == addr
        assert vault_list[0]["derivation_index"] == 0
        assert vault_list[0]["blockheight"] == 0
        assert vault_list[0]["funded_at"] is None
        assert vault_list[0]["secured_at"] is None
        assert vault_list[0]["delegated_at"] is None
        assert vault_list[0]["moved_at"] is None
        assert revaultd_manager.rpc.call("getinfo")["vaults"] == 1

        # Generate 5 blocks, it is still unconfirmed
        bitcoind.generate_block(5)
        assert (
            revaultd_manager.rpc.call("listvaults")["vaults"][0]["status"] == "unconfirmed"
        )

        # 1 more block will get it confirmed
        bitcoind.generate_block(1)
        revaultd_manager.wait_for_log(f"Vault at .*{txid}.* is now confirmed")
        vault = revaultd_manager.rpc.call("listvaults")["vaults"][0]
        assert vault["status"] == "funded"
        assert vault["funded_at"] is not None
        assert vault["secured_at"] is None
        assert vault["delegated_at"] is None
        assert vault["moved_at"] is None
        assert vault["blockheight"] == bitcoind.rpc.getblockcount() - 5

        # Of course, it persists across restarts.
        revaultd_manager.rpc.call("stop")
        revaultd_manager.proc.wait(TIMEOUT)
        revaultd_manager.start()
        vault_list = revaultd_manager.rpc.call("listvaults")["vaults"]
        assert len(vault_list) == 1
        assert vault_list[0]["status"] == "funded"
        assert vault_list[0]["txid"] == txid
        assert vault_list[0]["amount"] == amount_sent * 10 ** 8
        assert vault_list[0]["address"] == addr
        assert vault["funded_at"] is not None
        assert vault["secured_at"] is None
        assert vault["delegated_at"] is None
        assert vault["moved_at"] is None
        assert vault_list[0]["derivation_index"] == 0

        # And we can filter the result by status
>       vault_list = revaultd_manager.rpc.call("listvaults", [["unconfirmed"]])["vaults"]

test_rpc.py:98:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_framework\utils.py:385: in call
    resp = subprocess.check_output(
..\..\..\..\AppData\Local\Programs\Python\Python310\lib\subprocess.py:420: in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
..\..\..\..\AppData\Local\Programs\Python\Python310\lib\subprocess.py:501: in run
    with Popen(*popenargs, **kwargs) as process:
..\..\..\..\AppData\Local\Programs\Python\Python310\lib\subprocess.py:966: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
..\..\..\..\AppData\Local\Programs\Python\Python310\lib\subprocess.py:1375: in _execute_child
    args = list2cmdline(args)
..\..\..\..\AppData\Local\Programs\Python\Python310\lib\subprocess.py:561: in list2cmdline
    for arg in map(os.fsdecode, seq):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

filename = ['unconfirmed']

    def fsdecode(filename):
        """Decode filename (an os.PathLike, bytes, or str) from the filesystem
        encoding with 'surrogateescape' error handler, return str unchanged. On
        Windows, use 'strict' error handler if the file system encoding is
        'mbcs' (which is the default encoding).
        """
>       filename = fspath(filename)  # Does type-checking of `filename`.
E       TypeError: expected str, bytes or os.PathLike object, not list

..\..\..\..\AppData\Local\Programs\Python\Python310\lib\os.py:822: TypeError
-------------------------------------------------------------------------------------------------------------------------------------------------- Captured stdout setup ---------------------------------------------------------------------------------------------------------------------------------------------------
Running tests in C:\Users\Daniela\RevaultdTests\revaultd-jo8mi550
-------------------------------------------------------------------------------------------------------------------------------------------------- Captured stderr setup ---------------------------------------------------------------------------------------------------------------------------------------------------
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
---------------------------------------------------------------------------------------------------------------------------------------------------- Captured log setup ----------------------------------------------------------------------------------------------------------------------------------------------------
INFO     root:bitcoind.py:82 BitcoinD started
------------------------------------------------------------------------------------------------------------------------------------------------- Captured stdout teardown -------------------------------------------------------------------------------------------------------------------------------------------------
Test failed, leaving directory 'C:\Users\Daniela\RevaultdTests\revaultd-jo8mi550\test_listvaults_1' intact
Leaving base dir 'C:\Users\Daniela\RevaultdTests\revaultd-jo8mi550' as it still contains ['test_listvaults_1']
===================================================================================================================================================== warnings summary =====================================================================================================================================================
tests/test_rpc.py: 4340 warnings
  C:\Users\Daniela\Downloads\revaultd-Windows_Functional_Test\revaultd-Windows_Functional_Test\tests\test_framework\utils.py:516: DeprecationWarning: notifyAll() is deprecated, use notify_all() instead
    self.logs_cond.notifyAll()

-- Docs: https://docs.pytest.org/en/stable/warnings.html
================================================================================================================================================= short test summary info ==================================================================================================================================================
FAILED test_rpc.py::test_listvaults - TypeError: expected str, bytes or os.PathLike object, not list
===================================================================================================================================== 1 failed, 40 deselected, 4340 warnings in 27.04s =====================================================================================================================================