Commit 0ecde36
committed
fix(providers): add 60s timeout to all OpenAI provider fetch requests
Currently, fetch() calls to OpenAI endpoints in the Responses API provider
have no timeout, which means HTTP requests can hang indefinitely on network
issues, server unresponsiveness, or slow model responses.
This causes:
- Simulations stuck waiting forever for LLM responses
- Resource exhaustion when multiple sims run concurrently
- Poor user experience (no error feedback, hanging UI)
- Wasted compute resources on hung HTTP requests
This adds a 60-second timeout using AbortSignal.timeout() while preserving
any existing abort signals via AbortSignal.any().
**PeakInfer Issue:** Missing timeout on LLM API HTTP requests
**Impact:** Prevents indefinite hangs and improves reliability
**Category:** Reliability + Latency
Changes:
- Added 60s timeout to postResponses() fetch (line 265-268)
- Added 60s timeout to streaming fetch (line 293-296)
- Added 60s timeout to final streaming fetch after tools (line 718-721)
- Preserves existing abortSignal functionality via AbortSignal.any()
- Applies to all OpenAI-compatible providers (OpenAI, Azure, etc.)
This follows PeakInfer best practices for production LLM systems:
- Prevents resource exhaustion from hung requests
- Enables faster error detection and recovery
- Improves system resilience under network issues
- 60s timeout balances patience for long responses vs system health
🤖 Generated with PeakInfer LLM inference optimization1 parent 1de514b commit 0ecde36
1 file changed
+21
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
261 | 261 | | |
262 | 262 | | |
263 | 263 | | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
264 | 270 | | |
265 | 271 | | |
266 | 272 | | |
267 | 273 | | |
268 | | - | |
| 274 | + | |
269 | 275 | | |
270 | 276 | | |
271 | 277 | | |
| |||
283 | 289 | | |
284 | 290 | | |
285 | 291 | | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
286 | 298 | | |
287 | 299 | | |
288 | 300 | | |
289 | 301 | | |
290 | | - | |
| 302 | + | |
291 | 303 | | |
292 | 304 | | |
293 | 305 | | |
| |||
702 | 714 | | |
703 | 715 | | |
704 | 716 | | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
705 | 723 | | |
706 | 724 | | |
707 | 725 | | |
708 | 726 | | |
709 | | - | |
| 727 | + | |
710 | 728 | | |
711 | 729 | | |
712 | 730 | | |
| |||
0 commit comments