Handle Ollama offline gracefully with retry and 503 response #41
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Problem: When the Ollama provider was offline or unreachable, the raw TypeError
from the failed fetch bubbled up to the Express error middleware, producing an
unhelpful 500 error. The retry client also did not recognize ECONNREFUSED as a
retryable error, so connection-refused failures failed immediately without any
backoff attempt.
Changes implemented:
1. Add ECONNREFUSED to retryable errors (src/clients/retry.js)
- Added ECONNREFUSED to the retryableErrors list alongside ECONNRESET, etc.
- Added check for nested .cause.code on errors, since Node's undici wraps
connection errors as TypeError with the actual code on .cause
- Connection-refused errors now get retried with exponential backoff
2. Wrap invokeModel in try/catch (src/orchestrator/index.js)
- Catches connection errors (ECONNREFUSED, fetch failed) from invokeModel
- Returns a structured 503 response with provider_unreachable error type
- Prevents raw TypeError from bubbling to Express error middleware
- Non-connection errors are re-thrown unchanged
Testing:
- Stopped Ollama, sent a request: returns clean 503 with provider_unreachable
- Previously: raw TypeError crash in Express middleware
- Started Ollama back up: requests resume normally
- Retry behavior verified: ECONNREFUSED triggers backoff retries before 503
- npm run test:unit passes with no regressions
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Problem
When the Ollama provider was offline, the raw TypeError from the failed fetch crashed through to Express error middleware, producing an unhelpful 500 error. The retry client also didn't recognize ECONNREFUSED, so it failed immediately without retrying.
Changes
ECONNREFUSEDto retryable errors list; added check for nested.cause.codesince Node's undici wraps connection errors as TypeErrorinvokeModelcall in try/catch that detects connection errors and returns a structured 503 withprovider_unreachableerror typeTesting
provider_unreachablenpm run test:unitpasses with no regressions