Synchronous Outlet for zero-copying socket writing

[cboulay]

This implements a code path that allows for synchronous (blocked) calls to push_* by setting an outlet constructor argument do_async to false or 0. In this path, liblsl performs 0 (zero) copies on the data, compared to the 2 copies performed wen doing asynchronous writing. This can have dramatic speed / CPU usage improvements when using high bandwidth data (18%->5% in one of my tests).

Note that this contains some commits from the double_buflen branch. Currently the only relevant commit to examine is c64aeb9.

Some concerns:

• The time spent in push_* depends on how many consumers are added. This can lead to unpredictable user experience unless the user application calls push_* from a non-interactive thread.
• tcp_server -- I tried to skip launching the transfer_samples_thread but this did not work out. So now we have a thread doing nothing. I will try again.
• I only enabled the do_async flag on the very new lsl_create_outlet_ex C method, and preserved the API of the existing lsl_create_outlet method.
• stream_outlet_impl.h now has #include <asio/buffer.hpp> for private member std::vector<asio::const_buffer> sync_buffs_. A future revision should change this to std::vector<const_buffer_p> defined elsewhere.
• I would like to implement an outlet method to push_buffers_directly(std::vector<buffer_type> buffers, double timestamp, std::vector<std::size_t> frame_indices) where frame_indices gives the indices in buffers that start a new frame (frame := multi-channel sample). The goal is to enable the user to push discontiguous frames (e.g., disabled channels still allocated in device buffer) without first copying to a contiguous frame. I'm not sure I actually need this.

[cboulay]

Disabling the transfer_samples_thread leads to errors on the inlet at chunk boundaries on line https://github.com/sccn/liblsl/blob/cboulay/outlet_sync/src/data_receiver.cpp#L317-L318 . If I leave the do-nothing thread in place then it seems to work fine. 🤷

[cboulay]

Note to self: disable synchronous-write-mode when data format is string or when byte order is not default.

[cboulay]

Thanks @tstenner for pointing out that the transfer_samples_thread becomes the sole owner of the client_session. Without the thread we would need the client_session to exist elsewhere. Maybe in a container with the socket? I don't know yet.

[cboulay]

Rebased on #146 which was recently rebased on #155. Still a little work to do here.

[tstenner]

The new push_numeric_bufs should be part of a separate PR; as it is it's easy to misuse the API / provoke undefined behavior and the casts should be handled more safely by the C++ API.

[cboulay]

I don't disagree, but unfortunately I need this functionality from pure C. So it was either this, or add 14 functions to the C API. We can do that eventually, after all the inner workings are settled on.

[tstenner]

Suggestion: remove enqueue_sync and make this a template that takes a pair of asio::const_buffer iterators/pointers

[tstenner]

This waits for each preceding outlet to acknowledge that the complete data has been received before transmitting data to the next outlet. Suggestion: keep this code when there's only one inlet, start async writes to all inlets and wait for all writes to succeed before returning.

[cboulay]

I don't know of an easy way to do this without semaphores (C++20). Do you?

[tstenner]

For sync outlets you don't have a data transfer thread, so the calling thread is responsible for performing the IO, i.e. call send() or send_async()+io_context::run(). In pseudocode:

if consumers.size == 0:
    return
if consumers.size() == 1:
    res = consumers[0].send(data)
    if res != ok:
        handle_error()
else:
    io_ctx.restart()
    for consumer in consumers:
        consumer.send_async(data, [](err_t err) {
            if res != ok:
                handle_error()
            })
    steady_timer timer;
    timer.wait(3s, [](err_t err) { if err != ok: … });
    io_ctx.run();

[cboulay]

Assuming here that consumers is a tcp::socket, unfortunately socket.send(...) often fails to send the entire set of buffers. I was getting garbled data until I switched to asio::write(socket, bufs).

I don't know about socket.send_async vs asio::async_write. I hope the latter can successfully send everything (eventually) without having to monitor bytes_sent and re-entering the send condition.
I don't know if that changes the interaction with io_ctx. I'll give it a try when I get back to this. Thanks!

[tstenner]

You're right about socket.send_async(), so that would need to be asio::async_write() and also asio::write(). It doesn't change anything for the io_ctx, as asio::(async_)write just keeps track of the data already sent and starts a new send operation if it isn't finished yet.

[cboulay]

I made some changes based on your suggestions. However, the multiple-client code is probably still broken. I had to deprioritize that due to time constraints. I hope to get back to this soon.

[cboulay]

I discovered that outlet.have_consumers doesn't work properly in sync mode.

[cboulay]

Closing in favour of #170