Implement non-blocking IO operations

[dometto]

This should be done on the adapter level. @brodock suggests implementing EventMachine support for the ordinary git adapter:

The idea of introducing eventmachine to the IO part of the code, is to be able to do more non blocking operations, which will improve the amount of connections grack can handle.

Thin uses eventmachine already, which should be the most plug an play implementation for this specific case, but it doesn't handle blocking operations automatically, and any blocking IO you make during a request will block the main thread, see: http://stackoverflow.com/questions/19738489/does-thin-block-main-thread-when-doing-io

When you use thin and eventmachine for IO operations together, you can get the best amount of concurrency, that's even better than puma with real threads.

As a simple example, imagine you have 100 clients trying to get files, but 20 of them are requesting from a high latency connection (3G mobile). And the other 80 are on high speed fiber optic connection.

The ones on high latency will block the connection code for a lot longer than the others with fiber, and because of that, everybody will be slowed down.

In an evented environment, the IO part of the code will be resumed in a high frequency by the fibers one, and in a low frequency by the 3G mobile clients, but speed, from the perspective of server load/work, will be the same for both, or in other words, will be evenly distributed.

While I focused more on the clients accessing a resource (this is mostly handled by thin itself), the other important piece in the puzzle is the resource you are accessing that needs to go to the disk and do IO (git).

While disk can be faster, slow, depending on a lot of factors, when you use evented communications, it's the same principle I've exemplified above. While your code is waiting for the chunck of data, it will be doing other tasks that are being required.

For Java, some other implementation is likely necessary.

[javanthropus]

In the upcoming version 0.1.0 release, I believe the only place we need to involve EM is in the pipe handling between Grack and Git. Regular files such as .idx files are handled by the app container (Puma, Thin, etc.) directly. Grack itself does not attempt to stream them, so it's up to the container to behave efficiently.

It looks like http://www.rubydoc.info/github/eventmachine/eventmachine/EventMachine.popen may get us pretty close. Unfortunately, the method is documented as being unsupported on Windows. We could fall back to the current logic there, but that would be a bit surprising for consumers of the library.

Another tack would be to wrap the chunk handling in the write and read segments of the pipe processing such that we basically yield to EM after handling each chunk. This would get pretty complicated from what I can tell and would still be vulnerable to blocking in the communication with Git. I think the only benefit would be to possibly prevent EM starvation when copying Git's response to the socket.

I'm not sure how much evented IO on the pipe between Grack and Git would get us in the end. Git in this context is working with local data on the filesystem and may not realistically block. The socket into which we copy the Git response is much more likely to block, and that's where we'll encounter EM when run under a container such as Thin. IOW, we should test that this feature is even necessary.

[dometto]

IOW, we should test that this feature is even necessary.

Agreed. I have little experience with eventing and such, but if there's anything I can do to help out, please let me know!

It looks like http://www.rubydoc.info/github/eventmachine/eventmachine/EventMachine.popen may get us pretty close. Unfortunately, the method is documented as being unsupported on Windows. We could fall back to the current logic there, but that would be a bit surprising for consumers of the library.

If we mark this clearly in the README, I think EM support on all platforms other than Windows is worth the trouble -- provided that it is necessary at all.