Optimize uv_write() + uv_fs_sendfile()

[bnoordhuis]

A standard trick on Linux to pack response headers and body into a single TCP packet is to sendmsg() the headers with flags=MSG_MORE and sendfile() or splice() the body afterwards. That kind of optimized mix-and-match I/O is not something you can do with libuv.

In theory, libuv could apply the sendmsg+sendfile trick implicitly by correlating write and sendfile requests for the same file descriptor. That wouldn't need any new APIs but it would require deep cooperation between normal socket I/O and the thread pool. (The time gap between the system calls should be short in order to be effective. Wait too long and the kernel will flush the headers automatically. Then you not only have two TCP packets but also extra latency.)

Alternatively, one could conceive an "I/O plan" API that lets the user compile a sequence of I/O actions. I'm not sure what it would look like but it's potentially more general, extensible and reliable than applying heuristics.

[aktau]

That would be really cool, though I have not idea what it would look like either way. Afaik splice()'ing to TCP can only be from a pipe so it would depend on the input source.

Maybe there wouldn't be a need for magical correlating if the uv_write_t could be use to pass something like MSG_MORE.

If it's a userspace buffer and libuv has full control over the buffer, vmsplice() first seemed like an option. But as is mentioned in the docs, this usually requires protocol cooperation, which libuv can't mandate. So while that sounds nice, it doesn't seem feasible at libuv's level.