-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
fix(core): Preserve OpenAI APIPromise methods like withResponse() #18402
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
The OpenAI SDK returns APIPromise objects with additional methods like .withResponse(). The previous async/await instrumentation converted these to regular Promises, losing those methods. Use Proxy + handleCallbackErrors pattern (matching anthropic-ai) to preserve the original return type.
packages/core/test/tracing/openai-integration-functions.test.ts
Outdated
Show resolved
Hide resolved
- Use startInactiveSpan instead of startSpan/startSpanManual because they internally use handleCallbackErrors which calls .then() on Promises, creating a new instance and losing APIPromise methods - Add try-catch for synchronous exceptions - Add tests for error handling (sync throw + async reject) - Update tests to match real OpenAI SDK behavior"
- Use startInactiveSpan instead of startSpan/startSpanManual because they internally use handleCallbackErrors which calls .then() on Promises, creating a new instance and losing APIPromise methods - Add try-catch for synchronous exceptions - Use .finally() to ensure span always ends even if attribute processing throws - Add tests for error handling (sync throw + async reject) - Update tests to match real OpenAI SDK behavior
| const span = startInactiveSpan({ | ||
| name: `${operationName} ${model}`, | ||
| op: getSpanOperation(methodPath), | ||
| attributes: requestAttributes as Record<string, SpanAttributeValue>, | ||
| }); | ||
|
|
||
| if (options.recordInputs && params) { | ||
| addRequestAttributes(span, params); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Unhandled exceptions from addRequestAttributes at line 209 can prevent spans from ending, leading to incomplete tracing.
Severity: HIGH | Confidence: High
🔍 Detailed Analysis
The addRequestAttributes function, called at line 209, is not wrapped in a try-catch block. This function internally calls getTruncatedJsonString(), which uses JSON.stringify(). JSON.stringify() can throw exceptions, for example, when encountering circular references or BigInt values without a replacer. If such an exception occurs, it propagates out of the Proxy's apply handler, preventing the span created at line 202 from being properly ended. This violates the span's lifecycle and leads to incomplete tracing data, contrasting with the protected addResponseAttributes call at line 238.
💡 Suggested Fix
Wrap the call to addRequestAttributes(span, params) at line 209 in a try-catch block to gracefully handle potential errors, mirroring the error handling for addResponseAttributes.
🤖 Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.
Location: packages/core/src/tracing/openai/index.ts#L202-L210
Potential issue: The `addRequestAttributes` function, called at line 209, is not wrapped
in a try-catch block. This function internally calls `getTruncatedJsonString()`, which
uses `JSON.stringify()`. `JSON.stringify()` can throw exceptions, for example, when
encountering circular references or `BigInt` values without a replacer. If such an
exception occurs, it propagates out of the Proxy's `apply` handler, preventing the span
created at line 202 from being properly ended. This violates the span's lifecycle and
leads to incomplete tracing data, contrasting with the protected `addResponseAttributes`
call at line 238.
Did we get this right? 👍 / 👎 to inform future reviews.
Reference ID: 5640211
|
|
||
| if (options.recordInputs && params) { | ||
| addRequestAttributes(span, params); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Unprotected addRequestAttributes call can leave span open
The addRequestAttributes call on line 209 is outside the try/catch block that surrounds target.apply(). If addRequestAttributes throws an exception (e.g., from JSON.stringify failing on circular references in user-provided params), the span created on line 202 will never be ended - it's started but the span.end() calls only happen inside the try/catch error handler or the promise's finally() block. In the previous implementation, this call was inside the try/catch. This could cause orphaned spans and potential memory leaks.
Summary
The OpenAI SDK returns
APIPromiseobjects with additional methods like.withResponse().The previous
async/awaitinstrumentation converted these to regular Promises, losing those methods.This PR uses
Proxy+handleCallbackErrorspattern (matchinganthropic-ai) to preserve the original return type.Changes
Proxyinstead ofasync functionwrapperhandleCallbackErrorsfor non-streaming responses (preserves Promise subclass)handleStreamingErrorhelper for consistencyAPIPromise.withResponse()preservationTest Plan
yarn build:devpassesyarn testpassesyarn lintpasses