Skip to content

fix(transport): resolve SSE stream timeout in JdkHttpTransport#1322

Open
jujn wants to merge 1 commit into
agentscope-ai:mainfrom
jujn:fix_1302
Open

fix(transport): resolve SSE stream timeout in JdkHttpTransport#1322
jujn wants to merge 1 commit into
agentscope-ai:mainfrom
jujn:fix_1302

Conversation

@jujn
Copy link
Copy Markdown
Contributor

@jujn jujn commented Apr 30, 2026

Description

Close #1302
This PR introduces a structural fix for JdkHttpTransport when handling long-running Server-Sent Events (SSE) streams, particularly for LLMs with long Time-To-First-Token (TTFT) requirements.
Previously, the combination of JDK HttpClient's absolute timeout semantics and synchronous stream reading within a reactive pipeline led to premature request terminations, leaked connections, and severe NIO thread pool starvation.

Key Changes & Architectural Improvements:

  • Removed Absolute Timeout for Streams: Bypassed the java.net.http.HttpRequest.timeout() for streaming requests. The JDK's absolute timeout forcefully cuts off connections regardless of ongoing data transfer, which is fundamentally incompatible with LLM streaming.

  • Dual-Stage Reactive Timeout Strategy:
    Moved timeout management to the Reactor layer using a custom timeout() operator:

    • responseTimeout (TTFT): Tolerates long-thinking models waiting for the first byte/token (Default: 5 mins).
    • streamIdleTimeout (Inter-token gap): Resets upon receiving every data chunk (Default: 30 seconds).
  • Prevent NIO Thread Starvation:
    Added .subscribeOn(Schedulers.boundedElastic()) after the HTTP response is received. This guarantees that the blocking I/O operations (BufferedReader.readLine()) used to parse the NDJSON/SSE stream do not hijack and block the JDK HttpClient's highly limited internal NIO selector threads.

  • Fixed "Ghost Connection" Leaks:
    Wrapped the asynchronous JDK request in Mono.fromFuture(() -> client.sendAsync(...)). The Supplier ensures lazy execution, and using fromFuture (instead of fromCompletionStage) guarantees that Reactor's cancellation signals correctly trigger future.cancel(true), closing the underlying socket if the downstream aborts or times out before headers arrive.

Checklist

Please check the following items before code is ready to be reviewed.

  • Code has been formatted with mvn spotless:apply
  • All tests are passing (mvn test)
  • Javadoc comments are complete and follow project conventions
  • Related documentation has been updated (e.g. links, examples, etc.)
  • Code is ready for review

@jujn jujn requested a review from a team April 30, 2026 10:39
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 30, 2026

Codecov Report

❌ Patch coverage is 84.44444% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...ntscope/core/model/transport/JdkHttpTransport.java 78.78% 3 Missing and 4 partials ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]:SSE流式请求首次/闲置后超时 + HttpTransportException#getStatusCode() NPE 异常

1 participant