RFC: Middleware framework

[pguillory]

There are several kinds of standard functionality we need to apply on every Thrift endpoint.

• Metrics (QPS, latency, request/response payload size)
• Authorization (is the client allowed to call this endpoint?)
• Rate/concurrency limiting per client type
• Timeouts
• Logging

These generally need to run before and sometimes after every request, and in some cases make a decision about whether the request is allowed to happen. Web servers have similar needs that are served with a middleware framework like Plug, or Rack in the Ruby world. I propose we have a middleware framework for elixir-thrift.

When creating the server, we'd pass an additional option like:

tcp_opts: ...,
ssl_opts: ...
middleware: [
  Thrift.Middleware.Logging,
  Thrift.Middleware.RateLimiting,
  MyProprietaryService.Middleware.Authorization,
]

When handling a request, the server would call the first middleware with a callback function to invoke the subsequent chain of middleware. The last middleware's callback function will invoke the normal, existing request handling. A stub middleware implementation would look like this:

defmodule StubMiddleware do
  def handle_request(request, func) do
    # <- Before subsequent middlewares have been called.
    response = func(request)
    # <- After subsequent middlewares have been called.
    response
  end
end

We'd have a handful of standard middleware types available for common things like logging, but this would also provide an extension point for users to implement custom ones. For instance we want to authorize every request using a proprietary system with which it wouldn't make sense to integrate elixir-thrift directly.

The request object would be a struct. Spitballing:

%Thrift.Request{
  args: %MyService.FooArgs{x: 1, y: 2},
  handler_module: MyService.Handler,
  method: "foo",
  serialized_args: <<...>>,
  socket: #Port<0.17595>,
  ssl: false,
}

[yawaramin]

I like and use middleware a lot, but defining a fixed list of middleware for every call could be a bit inflexible. We actually don't need any specific support to add middleware to every call, we can just do it manually and it's even more flexible. E.g. from the readme,

defmodule Calculator.ServiceHandler do
  @behaviour Calculator.Generated.Service.Handler

  @impl true
  def add(left, right) do
    middleware([left, right]) do
      left + right
    end
  end
end

This middleware can do anything with the given arguments--log them, use them for access control purposes, etc. The middlewares can be nested or composed together to create middleware stacks that do everything you need.

But you may be thinking, this would require manually injecting the middleware into every handler. In fact, with Elixir macros it should be fairly easy to have e.g. a defthrift macro that defines a handler and applies a middleware stack to it at the same time. Then just replace def with defthrift.