-
Notifications
You must be signed in to change notification settings - Fork 38.9k
Description
Description
I see Netty leak reports during multipart uploads when the client aborts the request mid‑upload. Leak detector logs point to the inbound HTTP pipeline / multipart parsing path. This reproduces quickly when the client timeout is small and the
uploaded files are large.
Environment
- OS: Linux
- JDK: 21
- Spring Boot: 3.5.10
- Spring Framework: 6.x (from Boot 3.5.10 BOM)
- Reactor Netty: 1.2.14
- Netty: 4.1.130.Final
- Leak detection:
-Dio.netty.leakDetection.level=advanced (also tried paranoid)
-Dio.netty.leakDetection.targetRecords=32
-Dio.netty.leakDetection.samplingInterval=64
Dependencies (relevant)
org.springframework.boot:spring-boot-starter-reactor-netty:3.5.10
└─ io.projectreactor.netty:reactor-netty-http:1.2.14
├─ io.netty:netty-codec-http:4.1.130.Final
├─ io.netty:netty-buffer:4.1.130.Final
├─ io.netty:netty-transport:4.1.130.Final
├─ io.netty:netty-handler:4.1.130.Final
└─ io.projectreactor.netty:reactor-netty-core:1.2.14
Reproduction
Minimal repro project:
https://github.com/dmittriy13/webflux-multipart-leak-repro
Summary of steps:
- Start the app.
- Run k6 load test with a short client timeout so requests are aborted mid‑upload.
- Observe leak logs.
Expected behavior
No ByteBuf leak reports on client abort. Inbound buffers should be released even if the request is cancelled.
Actual behavior
Leak logs appear immediately after client abort. Example:
Full leak stack trace
2026-02-05T09:56:34.932Z ERROR 1 --- [r-http-epoll-13] io.netty.util.ResourceLeakDetector : LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more
information.
app-1 | Recent access records:
app-1 | #1:
app-1 | io.netty.handler.codec.http.DefaultHttpContent.release(DefaultHttpContent.java:92)
app-1 | io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:90)
app-1 | reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:296)
app-1 | reactor.netty.channel.FluxReceive.lambda$request$1(FluxReceive.java:136)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | #2:
app-1 | io.netty.buffer.AdvancedLeakAwareByteBuf.forEachByte(AdvancedLeakAwareByteBuf.java:677)
app-1 | org.springframework.core.io.buffer.NettyDataBuffer.forEachByte(NettyDataBuffer.java:134)
app-1 | org.springframework.core.io.buffer.DataBufferUtils$AbstractNestedMatcher.match(DataBufferUtils.java:882)
app-1 | org.springframework.http.codec.multipart.MultipartParser$BodyState.onNext(MultipartParser.java:522)
app-1 | org.springframework.http.codec.multipart.MultipartParser.hookOnNext(MultipartParser.java:123)
app-1 | org.springframework.http.codec.multipart.MultipartParser.hookOnNext(MultipartParser.java:52)
app-1 | reactor.core.publisher.BaseSubscriber.onNext(BaseSubscriber.java:160)
app-1 | reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
app-1 | reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200)
app-1 | reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
app-1 | reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:292)
app-1 | reactor.netty.channel.FluxReceive.lambda$request$1(FluxReceive.java:136)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | #3:
app-1 | org.springframework.core.io.buffer.NettyDataBufferFactory.wrap(NettyDataBufferFactory.java:94)
app-1 | reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:106)
app-1 | reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200)
app-1 | reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
app-1 | reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:292)
app-1 | reactor.netty.channel.FluxReceive.lambda$request$1(FluxReceive.java:136)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | #4:
app-1 | reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:185)
app-1 | reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
app-1 | reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:292)
app-1 | reactor.netty.channel.FluxReceive.lambda$request$1(FluxReceive.java:136)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
app-1 | io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | #5:
app-1 | Hint: 'reactor.right.reactiveBridge' will handle the message from this point.
app-1 | io.netty.handler.codec.http.DefaultHttpContent.touch(DefaultHttpContent.java:86)
app-1 | io.netty.handler.codec.http.DefaultHttpContent.touch(DefaultHttpContent.java:25)
app-1 | io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:115)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:417)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | reactor.netty.http.server.HttpTrafficHandler.channelRead(HttpTrafficHandler.java:326)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:361)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:325)
app-1 | io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
app-1 | io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:805)
app-1 | io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | #6:
app-1 | Hint: 'reactor.left.httpTrafficHandler' will handle the message from this point.
app-1 | io.netty.handler.codec.http.DefaultHttpContent.touch(DefaultHttpContent.java:86)
app-1 | io.netty.handler.codec.http.DefaultHttpContent.touch(DefaultHttpContent.java:25)
app-1 | io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:115)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:417)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:361)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:325)
app-1 | io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
app-1 | io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:805)
app-1 | io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
app-1 | Created at:
app-1 | io.netty.buffer.SimpleLeakAwareByteBuf.unwrappedDerived(SimpleLeakAwareByteBuf.java:144)
app-1 | io.netty.buffer.SimpleLeakAwareByteBuf.readRetainedSlice(SimpleLeakAwareByteBuf.java:67)
app-1 | io.netty.buffer.AdvancedLeakAwareByteBuf.readRetainedSlice(AdvancedLeakAwareByteBuf.java:108)
app-1 | io.netty.handler.codec.http.HttpObjectDecoder.decode(HttpObjectDecoder.java:459)
app-1 | io.netty.handler.codec.http.HttpServerCodec$HttpServerRequestDecoder.decode(HttpServerCodec.java:167)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:545)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:484)
app-1 | io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
app-1 | io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
app-1 | io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
app-1 | io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
app-1 | io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868)
app-1 | io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:805)
app-1 | io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
app-1 | io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
app-1 | io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
app-1 | io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
app-1 | io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
app-1 | java.base/java.lang.Thread.run(Unknown Source)
Additional notes
- The issue reproduces only for multipart. A raw (non‑multipart) upload endpoint does not leak under the same abort conditions.
- The leak appears even if application logic is effectively a no‑op (no downstream calls, no further processing).
Questions
- Is this a known issue in multipart parsing under client abort?
- Is there a recommended way to ensure inbound buffers are released in this scenario?