Linux async sockets - SOCKETS_INLINE_COMPLETIONS and scheduling SocketIOEvent on threadpool

[petrroll]

Let's say I have a client library which manages its own sockets (e.g. StackExchangeRedis) and I want to make sure managing these sockets (ping, pongs) happens even if the instance is overloaded and general threadpool stops being able to make fast enough progress due to too many tasks scheduled on it (e.g.: StackExchange/StackExchange.Redis#3060).

From what I understand, there's little a library author can do in that case, regrettably. As the async sockets work roughly like this in linux:

1. Caller schedules a task
2. Dedicated epoll threads (can't controll them) read from OS sockets and:

• If SOCKETS_INLINE_COMPLETIONS==true, run SocketIOEvent completion inline
• else (which is default): schedule the SocketIOEvent on general threadpool using global queue

3. SocketIOEvent runs and:

• if it's on general threadpool && no execution context, schedules the user code continuation on threadpool preferLocal: true
• else schedules via execution context on threadpool

Which means that, assuming SOCKETS_INLINE_COMPLETIONS == false (which is default), one cannot make sure all of the tasks from OS socket getting data to user's continuation of the read operation run outside of the shared ThreadPool.

One could control the execution context of user code continuation and force scheduling that continuation on not shared, dedicated, possibly higher prio threads threadpool.

But there'd still be the de-bounce of SocketIOEvent via common threadpool (epoll threads schedule it, and this contination then schedules our user code continuation). And even worse the debounce happens via the global queue. Which means that if the global queue is not being drained enough, we'd never get to the user code.

If SOCKETS_INLINE_COMPLETIONS was true, it'd be possible. We could have the SocketIOEvent run on dedicated IO thread (not bogged down by run-away common threadpool) and the user continuation could run by our-controlled threadpool which could have dedicated high prio threads.

---

1. Is above roughly correct understanding?

2. If it is, what's the status of SOCKETS_INLINE_COMPLETIONS? are there any plans to enable it globally? If not, what are the concerns?

3. What about the worry that socket reads can get stalled if the threadpool is bogged down? Wouldn't it make sense to schedule these on dedicated threadpool or (if on the common) at least on the high priority global queue?

• I understand that with the high prio queue there's a risk of starving user continuations if the user continuations are then scheduled with preferLocal: true which can skip the queue a bit; but isn't that worth it?

Any other thoughts?