Skip to content

Incorrect handling of ESI calls initiated after POST request #479

Open
Open
Incorrect handling of ESI calls initiated after POST request#479
@jwadolowski

Description

@jwadolowski
jwadolowski
opened on Jan 15, 2019

Bug description

knot.x version: 1.4.0

Email signup is one of the features on my website. User initiates POST request which bypasses knot.x completely. The response contains ESI markup, so CDN triggers sub-requests, which go via knot.x instance. knot.x returns 500 for each subrequest call.

This situation doesn't happen when the entire flow starts with GET request.

Here's a diagram with end-to-end HTTP flow:

kx_req_flow

Steps to reproduce

  1. Send a POST request with Content-Type: application/x-www-form-urlencoded header
  2. Make sure response contains ESI markup
  3. ESI sub-requests should go via knot.x (they inherit request headers from original request - only those that came from the user, Content-Type is one of them)
  4. Each ESI call ends with 500 response

Response headers:

HTTP/1.1 500 java.nio.file.AccessDeniedException: /file-uploads
Content-Type: text/html
Connection: close
Content-Length: 9372

Response body:

<html>
  <head>
    <title>java.nio.file.AccessDeniedException: /file-uploads</title>
    <style>
        body {
          margin: 0;
          padding: 80px 100px;
          font: 13px "Helvetica Neue", "Lucida Grande", "Arial";
          background: #ECE9E9 -webkit-gradient(linear, 0% 0%, 0% 100%, from(#fff), to(#ECE9E9));
          background: #ECE9E9 -moz-linear-gradient(top, #fff, #ECE9E9);
          background-repeat: no-repeat;
          color: #555;
          -webkit-font-smoothing: antialiased;
        }
        h1, h2, h3 {
          margin: 0;
          font-size: 22px;
          color: #343434;
        }
        h1 em, h2 em {
          padding: 0 5px;
          font-weight: normal;
        }
        h1 {
          font-size: 60px;
        }
        h2 {
            margin-top: 10px;
        }
        h3 {
          margin: 5px 0 10px 0;
          padding-bottom: 5px;
          border-bottom: 1px solid #eee;
          font-size: 18px;
        }
        ul {
          margin: 0;
          padding: 0;
        }
        ul li {
          margin: 5px 0;
          padding: 3px 8px;
          list-style: none;
        }
        ul li:hover {
          cursor: pointer;
          color: #2e2e2e;
        }
        p {
          line-height: 1.5;
        }
        a {
          color: #555;
          text-decoration: none;
        }
        a:hover {
          color: #303030;
        }
        #stacktrace {
            margin-top: 15px;
        }
        .directory h1 {
          margin-bottom: 15px;
          font-size: 18px;
        }
    </style>
  </head>
  <body>
    <div id="wrapper">
      <h1>Matron!</h1>
      <h2><em>500</em> java.nio.file.AccessDeniedException: /file-uploads</h2>
      <ul id="stacktrace"><li>io.vertx.core.file.impl.FileSystemImpl$11.perform(FileSystemImpl.java:678)</li><li>io.vertx.core.file.impl.FileSystemImpl$11.perform(FileSystemImpl.java:660)</li><li>io.vertx.core.file.impl.FileSystemImpl.mkdirsBlocking(FileSystemImpl.java:248)</li><li>io.vertx.ext.web.handler.impl.BodyHandlerImpl$BHandler.makeUploadDir(BodyHandlerImpl.java:171)</li><li>io.vertx.ext.web.handler.impl.BodyHandlerImpl$BHandler.<init>(BodyHandlerImpl.java:138)</li><li>io.vertx.ext.web.handler.impl.BodyHandlerImpl.handle(BodyHandlerImpl.java:72)</li><li>io.vertx.ext.web.handler.impl.BodyHandlerImpl.handle(BodyHandlerImpl.java:42)</li><li>io.vertx.reactivex.ext.web.handler.BodyHandler.handle(BodyHandler.java:74)</li><li>io.vertx.reactivex.ext.web.handler.BodyHandler.handle(BodyHandler.java:37)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:155)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:153)</li><li>io.vertx.ext.web.impl.RouteImpl.handleContext(RouteImpl.java:219)</li><li>io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:120)</li><li>io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:133)</li><li>io.vertx.ext.web.handler.impl.CookieHandlerImpl.handle(CookieHandlerImpl.java:66)</li><li>io.vertx.ext.web.handler.impl.CookieHandlerImpl.handle(CookieHandlerImpl.java:42)</li><li>io.vertx.reactivex.ext.web.handler.CookieHandler.handle(CookieHandler.java:73)</li><li>io.vertx.reactivex.ext.web.handler.CookieHandler.handle(CookieHandler.java:36)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:155)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:153)</li><li>io.vertx.ext.web.impl.RouteImpl.handleContext(RouteImpl.java:219)</li><li>io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:120)</li><li>io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:133)</li><li>io.vertx.reactivex.ext.web.RoutingContext.next(RoutingContext.java:128)</li><li>io.knotx.server.SupportedMethodsAndPathsHandler.handle(SupportedMethodsAndPathsHandler.java:52)</li><li>io.knotx.server.SupportedMethodsAndPathsHandler.handle(SupportedMethodsAndPathsHandler.java:27)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:155)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:153)</li><li>io.vertx.ext.web.impl.RouteImpl.handleContext(RouteImpl.java:219)</li><li>io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:120)</li><li>io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:133)</li><li>io.vertx.reactivex.ext.web.RoutingContext.next(RoutingContext.java:128)</li><li>io.knotx.server.KnotxHeaderHandler.handle(KnotxHeaderHandler.java:41)</li><li>io.knotx.server.KnotxHeaderHandler.handle(KnotxHeaderHandler.java:23)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:155)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:153)</li><li>io.vertx.ext.web.impl.RouteImpl.handleContext(RouteImpl.java:219)</li><li>io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:120)</li><li>io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:133)</li><li>io.vertx.ext.web.handler.impl.LoggerHandlerImpl.handle(LoggerHandlerImpl.java:178)</li><li>io.vertx.ext.web.handler.impl.LoggerHandlerImpl.handle(LoggerHandlerImpl.java:47)</li><li>io.vertx.reactivex.ext.web.handler.LoggerHandler.handle(LoggerHandler.java:73)</li><li>io.vertx.reactivex.ext.web.handler.LoggerHandler.handle(LoggerHandler.java:36)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:155)</li><li>io.vertx.reactivex.ext.web.Route$1.handle(Route.java:153)</li><li>io.vertx.ext.web.impl.RouteImpl.handleContext(RouteImpl.java:219)</li><li>io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:120)</li><li>io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:133)</li><li>io.vertx.ext.web.impl.RouterImpl.accept(RouterImpl.java:79)</li><li>io.vertx.reactivex.ext.web.Router.accept(Router.java:94)</li><li>io.knotx.server.KnotxServerVerticle.routeSafe(KnotxServerVerticle.java:181)</li><li>io.knotx.server.KnotxServerVerticle.lambda$start$8(KnotxServerVerticle.java:164)</li><li>io.vertx.reactivex.core.http.HttpServer$1.handle(HttpServer.java:111)</li><li>io.vertx.reactivex.core.http.HttpServer$1.handle(HttpServer.java:109)</li><li>io.vertx.core.http.impl.Http1xServerConnection.processMessage(Http1xServerConnection.java:453)</li><li>io.vertx.core.http.impl.Http1xServerConnection.handleMessage(Http1xServerConnection.java:144)</li><li>io.vertx.core.http.impl.HttpServerImpl$ServerHandlerWithWebSockets.handleMessage(HttpServerImpl.java:666)</li><li>io.vertx.core.http.impl.HttpServerImpl$ServerHandlerWithWebSockets.handleMessage(HttpServerImpl.java:619)</li><li>io.vertx.core.net.impl.VertxHandler.lambda$channelRead$1(VertxHandler.java:146)</li><li>io.vertx.core.impl.ContextImpl.lambda$wrapTask$2(ContextImpl.java:337)</li><li>io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:195)</li><li>io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:144)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)</li><li>io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)</li><li>io.vertx.core.http.impl.HttpServerImpl$Http2UpgradeHandler.channelRead(HttpServerImpl.java:968)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)</li><li>io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)</li><li>io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)</li><li>io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)</li><li>io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)</li><li>io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)</li><li>io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)</li><li>io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)</li><li>io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141)</li><li>io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)</li><li>io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)</li><li>io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)</li><li>io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)</li><li>io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)</li><li>io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)</li><li>java.lang.Thread.run(Thread.java:748)</li></ul>
    </div>
  </body>
</html>

Expected behavior

ESI calls are regular GET requests. The only difference in comparison to other situations is the fact that sub-requests inherit parent request headers, so CDN sends GET with Content-Type: application/x-www-form-urlencoded.

knot.x should either reject such request with 400 status code or ignore Content-Type header sent with GET request, as it doesn't make any sense.

Screenshots

N/A

Additional context

The following curl command was enough to reproduce the problem:

$ curl localhost:8092/path/to/esi/snippet.html -H "Content-Type: application/x-www-form-urlencoded"

Those 500s are visible in knotx-access.log file, but knotx.log stays empty despite of the fact I increased log level to DEBUG.

10.251.35.221 - - [Mon, 14 Jan 2019 22:59:25 GMT] "GET /path/to/esi/snippet.html HTTP/1.1" 500 8931 "http://example.org/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"

It could be related to #321

Possible workaround - "sanitize" HTTP request before it gets processed by knot.x and remove Content-Type from GET requests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions