feat: add @eggjs/agent-tracing package for AI agent tracing#412
feat: add @eggjs/agent-tracing package for AI agent tracing#412jerryliang64 merged 3 commits intomasterfrom
Conversation
Add tracing support for AI agents built with LangGraph and Claude Agent SDK. - LangGraphTracer: extends LangChain BaseTracer, hooks into graph lifecycle - ClaudeAgentTracer: converts Claude SDK messages to LangChain Run format - TraceSession: streaming support for real-time message processing - TracingService: shared log formatting, OSS upload, and log service sync - AbstractOssClient/AbstractLogServiceClient: IoC injection interfaces - Comprehensive mocha tests (52 passing) Adapted from eggjs/egg#5822 for tegg monorepo conventions (CJS, mocha, lerna). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a dedicated tracing package for AI agents, enhancing observability by capturing detailed execution flows from LangGraph and Claude Agent SDK interactions. It centralizes logging, data storage, and external service integration through a flexible architecture, allowing for custom implementations of OSS and log services. The primary goal is to provide developers with better insights into AI agent behavior and performance. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new package @eggjs/agent-tracing for AI agent tracing, which is a significant and well-architected feature. It supports both LangGraph and Claude Agent SDKs, with a clear separation of concerns between tracers and a shared TracingService. The code is well-documented and comes with a comprehensive test suite, which is excellent. My review focuses on improving type safety in a few areas to enhance maintainability.
…spacing) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughWalkthroughAdds a new Changes
Sequence Diagram(s)sequenceDiagram
participant Client as Client
participant Session as TraceSession
participant Tracer as ClaudeAgentTracer
participant Service as TracingService
participant OSS as OSS
participant LogSvc as LogService
Client->>Session: processMessage(systemInit)
Session->>Tracer: convertSDKMessage / start root run
Tracer->>Service: logTrace(rootRun, START)
Client->>Session: processMessage(assistant / tool-use)
Session->>Tracer: convertSDKMessage / create llm/tool runs (START)
Tracer->>Service: logTrace(childRuns, START)
Client->>Session: processMessage(user with tool_result)
Session->>Tracer: complete pending tool runs (END)
Tracer->>Service: logTrace(toolRuns, END)
Client->>Session: processMessage(result)
Session->>Tracer: finalize root run (END/ERROR)
Tracer->>Service: logTrace(rootRun, END/ERROR)
Service->>OSS: uploadToOss(key, largeField) (async)
OSS-->>Service: success/skip
Service->>LogSvc: syncLocalToLogService(log) [if local env]
LogSvc-->>Service: success/skip
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…ailable) @langchain/core requires ReadableStream which is only globally available in Node.js >= 18. Follow the same pattern as core/vitest. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (8)
core/agent-tracing/tsconfig.json (1)
1-12: Consider consolidating duplicate TypeScript configs.Both
tsconfig.jsonandtsconfig.pub.jsonhave identical content. If these configs serve distinct purposes (e.g., dev vs. publish), consider documenting the distinction. Otherwise, one could extend the other to reduce duplication.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/agent-tracing/tsconfig.json` around lines 1 - 12, The two identical TypeScript configs (tsconfig.json and tsconfig.pub.json) should be consolidated: pick one as the canonical config (e.g., tsconfig.json) and have the other extend it (using "extends") or delete the duplicate, and add a brief comment or README entry describing their distinct purpose if they must remain; update the "compilerOptions" and "exclude" usage accordingly so only the canonical file contains the full settings and the secondary file merely references it to avoid duplication.core/agent-tracing/test/TestUtils.ts (2)
4-4: ImportTracingServiceis only used for type casting.The import on line 4 is only used in
as unknown as TracingServicetype assertion. Consider using a type-only import to make the intent clearer and potentially improve tree-shaking.Suggested change
-import { TracingService } from '../src/TracingService'; +import type { TracingService } from '../src/TracingService';🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/agent-tracing/test/TestUtils.ts` at line 4, The import of TracingService in TestUtils.ts is only used for a type assertion (as unknown as TracingService); change it to a type-only import to clarify intent and aid bundlers/tree-shaking by replacing the current import with "import type { TracingService } from '../src/TracingService';" and keep the existing type assertion usage unchanged (referencing TracingService) so runtime behavior is unaffected.
35-47: Mock logger may be missingdebugmethod.The mock logger implements
info,warn, anderror, but many Logger interfaces also expect adebugmethod. If tests exercise code paths that calllogger.debug(), this could cause runtime errors.Consider adding debug method
export function createMockLogger(logs?: string[]): Logger { return { + debug: (msg: string) => { + logs?.push(msg); + }, info: (msg: string) => { logs?.push(msg); },🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/agent-tracing/test/TestUtils.ts` around lines 35 - 47, The mock logger returned by createMockLogger lacks a debug method which can cause runtime errors when code calls logger.debug; add a debug: (msg: string) => { logs?.push(msg); } implementation to the returned object (alongside info/warn/error) so the mock fully implements the expected Logger shape, then keep the existing "as unknown as Logger" cast or update typing if desired.core/agent-tracing/test/TracingService.test.ts (1)
258-262: Log parsing regex may be fragile to format changes.The regex
,run=({.*})$assumes the run JSON is always the last element in the log line. If the log format changes, these tests will silently fail to extract the JSON rather than failing explicitly.Consider extracting this pattern to a helper function with clearer error handling.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/agent-tracing/test/TracingService.test.ts` around lines 258 - 262, The test's inline regex extraction using /,run=({.*})$/ is fragile; replace it with a dedicated helper (e.g., extractRunJsonFromLog) that searches for the `,run=` key more robustly (non-greedy, supports trailing text) and returns parsed JSON or throws a clear assertion error; update the test to call this helper instead of using the ad-hoc `runJson` regex so failures explicitly surface (refer to variables/functions: runJson, parsed, and the JSON parsing/assertion logic in TracingService.test.ts).core/agent-tracing/src/types.ts (1)
126-127:FIELDS_TO_OSSdefinition and export are separated.The constant is defined on line 126 but exported on line 147. While this works, co-locating the export with the definition would improve readability.
Suggested consolidation
-const FIELDS_TO_OSS = [ 'inputs', 'outputs', 'attachments', 'serialized', 'events' ] as const; +export const FIELDS_TO_OSS = [ 'inputs', 'outputs', 'attachments', 'serialized', 'events' ] as const; ... -export { FIELDS_TO_OSS };Also applies to: 147-147
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/agent-tracing/src/types.ts` around lines 126 - 127, The FIELDS_TO_OSS constant is defined separately from its export; inline the export by changing the standalone definition to an exported declaration (e.g., export const FIELDS_TO_OSS = ...) and remove the later separate export statement to co-locate definition and export for readability; ensure no other code relies on the original separate export name and update imports/uses if necessary (symbol: FIELDS_TO_OSS).core/agent-tracing/src/TracingService.ts (1)
56-56: Complex ternary could be clearer.The expression
process.env.FAAS_ENV || env === 'local' ? '' : \env=${env},`relies on operator precedence that may confuse readers. The||binds tighter than?:, so this evaluates as(process.env.FAAS_ENV || env === 'local') ? '' : ...`.Suggested clarification
- const envSegment = process.env.FAAS_ENV || env === 'local' ? '' : `env=${env},`; + const shouldOmitEnv = process.env.FAAS_ENV || env === 'local'; + const envSegment = shouldOmitEnv ? '' : `env=${env},`;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/agent-tracing/src/TracingService.ts` at line 56, The ternary computing envSegment in TracingService is hard to read due to mixed || and ?: precedence; make the intent explicit by rewriting the condition (e.g., wrap the boolean check in parentheses or use an if/else) so it's clear you're checking whether FAAS_ENV is set or env === 'local' before choosing '' or `env=${env},`; update the expression that assigns envSegment to use (process.env.FAAS_ENV || env === 'local') ? '' : `env=${env},` or an equivalent if/else to improve readability.core/agent-tracing/test/LangGraphTracer.test.ts (1)
55-56: Hardcodedsleep(500)introduces potential test flakiness.Multiple tests use
await sleep(500)to wait for async tracing operations. This can lead to flaky tests on slow CI runners or unnecessary delays on fast machines. Consider using a polling/retry approach or exposing a flush mechanism on the tracer.Example polling approach
async function waitForCapturedRuns( capturedRuns: CapturedEntry[], minCount: number, timeoutMs = 2000 ): Promise<void> { const start = Date.now(); while (capturedRuns.length < minCount && Date.now() - start < timeoutMs) { await sleep(50); } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/agent-tracing/test/LangGraphTracer.test.ts` around lines 55 - 56, The test currently uses a hardcoded await sleep(500) in LangGraphTracer.test.ts which causes flakiness; replace this with a deterministic wait by polling the tracer output or using a flush API: implement and call a helper like waitForCapturedRuns(capturedRuns, expectedCount, timeoutMs) that loops with short sleeps until capturedRuns.length >= expectedCount (or call tracer.flush()/await tracer.waitForFlush() if available), and remove the fixed sleep(500) so the test waits only until the tracer work is actually done.core/agent-tracing/src/LangGraphTracer.ts (1)
77-82: MissingonAgentErrorhook for symmetry with other hook groups.All other hook groups have START/END/ERROR variants (onChainError, onToolError, onLLMError, onRetrieverError), but the agent hooks only have
onAgentAction(START) andonAgentEnd(END). Consider addingonAgentErrorif the base tracer supports it, to ensure error states during agent execution are also captured.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/agent-tracing/src/LangGraphTracer.ts` around lines 77 - 82, Add a symmetric error hook for agents by implementing an onAgentError(run: Run): void | Promise<void> method that mirrors onAgentAction and onAgentEnd; call this.logTrace(run, RunStatus.ERROR) inside it (use the same Run type and RunStatus enum as in onAgentAction/onAgentEnd). Ensure the base tracer or interface supports onAgentError before adding the method so it correctly overrides the hook.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@core/agent-tracing/src/ClaudeAgentTracer.ts`:
- Around line 377-386: The current branch that sets outputs when isToolCall only
writes outputs.tool_calls and drops any companion assistant text parsed into
textBlocks; update the branch in ClaudeAgentTracer.ts (the block that builds
outputs, using variables isToolCall, toolBlocks, and textBlocks) so that when
isToolCall is true you still populate outputs.content with textBlocks.map(c =>
c.text).join('') if textBlocks is non-empty (i.e., include both
outputs.tool_calls and outputs.content), ensuring the span preserves assistant
text alongside tool call entries.
- Around line 154-159: The root result span is dropping structured_output from
SDKResultMessage; update the assignments that set outputs so they preserve
structured_output when present — specifically extend this.rootRun.outputs (in
ClaudeAgentTracer where startTime/end_time are set) to include structured_output
when message.structured_output exists, and likewise update the
normalizeSdkMessage function to copy structured_output into the normalized
output object for success variants; reference the this.rootRun.outputs
assignment and normalizeSdkMessage so both places include structured_output
alongside result, is_error, and num_turns.
- Around line 75-120: convertSDKMessage currently drops
SDKAssistantMessage.error and handleAssistant only creates runs for 'tool_use'
or 'text', so assistant messages that carry only an error are lost; update
convertSDKMessage to extract and retain the optional error property from
SDKAssistantMessage into the converted message, and update handleAssistant to
detect when a message has no tool_use/text but does have an error and emit an
error span/run (use the tracer APIs analogous to
createLLMRunInternal/createToolRunStartInternal—e.g., create an LLM error run or
a dedicated error run via your tracer, attach the error details to the run, push
it into this.rootRun.child_runs, and call this.tracer.logTrace with an error
status like RunStatus.ERROR) so assistant-side SDK errors surface in traces.
- Around line 123-136: handleUser currently only iterates
message.message.content for tool_result blocks and ignores a top-level
SDKUserMessage.tool_use_result; update either the SDKUserMessage->ClaudeMessage
conversion to include the top-level tool_use_result into ClaudeMessage (so it
appears in message.message.content) or extend handleUser to explicitly check
message.tool_use_result and treat it the same as a tool_result block: look up
the tool run via pendingToolUses.get(tool_use_id), call
tracer.completeToolRunInternal(toolRun, toolUseResult, Date.now()), call
tracer.logTrace(toolRun, appropriate RunStatus), and delete the pendingToolUses
entry; reference functions/types: handleUser, ClaudeMessage, SDKUserMessage,
tool_use_result, pendingToolUses, tracer.completeToolRunInternal,
tracer.logTrace.
---
Nitpick comments:
In `@core/agent-tracing/src/LangGraphTracer.ts`:
- Around line 77-82: Add a symmetric error hook for agents by implementing an
onAgentError(run: Run): void | Promise<void> method that mirrors onAgentAction
and onAgentEnd; call this.logTrace(run, RunStatus.ERROR) inside it (use the same
Run type and RunStatus enum as in onAgentAction/onAgentEnd). Ensure the base
tracer or interface supports onAgentError before adding the method so it
correctly overrides the hook.
In `@core/agent-tracing/src/TracingService.ts`:
- Line 56: The ternary computing envSegment in TracingService is hard to read
due to mixed || and ?: precedence; make the intent explicit by rewriting the
condition (e.g., wrap the boolean check in parentheses or use an if/else) so
it's clear you're checking whether FAAS_ENV is set or env === 'local' before
choosing '' or `env=${env},`; update the expression that assigns envSegment to
use (process.env.FAAS_ENV || env === 'local') ? '' : `env=${env},` or an
equivalent if/else to improve readability.
In `@core/agent-tracing/src/types.ts`:
- Around line 126-127: The FIELDS_TO_OSS constant is defined separately from its
export; inline the export by changing the standalone definition to an exported
declaration (e.g., export const FIELDS_TO_OSS = ...) and remove the later
separate export statement to co-locate definition and export for readability;
ensure no other code relies on the original separate export name and update
imports/uses if necessary (symbol: FIELDS_TO_OSS).
In `@core/agent-tracing/test/LangGraphTracer.test.ts`:
- Around line 55-56: The test currently uses a hardcoded await sleep(500) in
LangGraphTracer.test.ts which causes flakiness; replace this with a
deterministic wait by polling the tracer output or using a flush API: implement
and call a helper like waitForCapturedRuns(capturedRuns, expectedCount,
timeoutMs) that loops with short sleeps until capturedRuns.length >=
expectedCount (or call tracer.flush()/await tracer.waitForFlush() if available),
and remove the fixed sleep(500) so the test waits only until the tracer work is
actually done.
In `@core/agent-tracing/test/TestUtils.ts`:
- Line 4: The import of TracingService in TestUtils.ts is only used for a type
assertion (as unknown as TracingService); change it to a type-only import to
clarify intent and aid bundlers/tree-shaking by replacing the current import
with "import type { TracingService } from '../src/TracingService';" and keep the
existing type assertion usage unchanged (referencing TracingService) so runtime
behavior is unaffected.
- Around line 35-47: The mock logger returned by createMockLogger lacks a debug
method which can cause runtime errors when code calls logger.debug; add a debug:
(msg: string) => { logs?.push(msg); } implementation to the returned object
(alongside info/warn/error) so the mock fully implements the expected Logger
shape, then keep the existing "as unknown as Logger" cast or update typing if
desired.
In `@core/agent-tracing/test/TracingService.test.ts`:
- Around line 258-262: The test's inline regex extraction using /,run=({.*})$/
is fragile; replace it with a dedicated helper (e.g., extractRunJsonFromLog)
that searches for the `,run=` key more robustly (non-greedy, supports trailing
text) and returns parsed JSON or throws a clear assertion error; update the test
to call this helper instead of using the ad-hoc `runJson` regex so failures
explicitly surface (refer to variables/functions: runJson, parsed, and the JSON
parsing/assertion logic in TracingService.test.ts).
In `@core/agent-tracing/tsconfig.json`:
- Around line 1-12: The two identical TypeScript configs (tsconfig.json and
tsconfig.pub.json) should be consolidated: pick one as the canonical config
(e.g., tsconfig.json) and have the other extend it (using "extends") or delete
the duplicate, and add a brief comment or README entry describing their distinct
purpose if they must remain; update the "compilerOptions" and "exclude" usage
accordingly so only the canonical file contains the full settings and the
secondary file merely references it to avoid duplication.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: f911f5c1-585b-44fa-a929-d50c908fdd34
📒 Files selected for processing (17)
core/agent-tracing/claude.tscore/agent-tracing/index.tscore/agent-tracing/langgraph.tscore/agent-tracing/package.jsoncore/agent-tracing/src/AbstractLogServiceClient.tscore/agent-tracing/src/AbstractOssClient.tscore/agent-tracing/src/ClaudeAgentTracer.tscore/agent-tracing/src/LangGraphTracer.tscore/agent-tracing/src/TracingService.tscore/agent-tracing/src/types.tscore/agent-tracing/test/ClaudeAgentTracer.test.tscore/agent-tracing/test/Configure.test.tscore/agent-tracing/test/LangGraphTracer.test.tscore/agent-tracing/test/TestUtils.tscore/agent-tracing/test/TracingService.test.tscore/agent-tracing/tsconfig.jsoncore/agent-tracing/tsconfig.pub.json
1 participant
Summary
@eggjs/agent-tracingpackage for AI agent LLM call tracingLangGraphTracer: extends LangChainBaseTracer, hooks into graph lifecycle eventsClaudeAgentTracer: converts Claude SDK messages to LangChain Run format, supports both batch and streaming modesTracingService: shared log formatting, OSS upload, and log service syncAbstractOssClient/AbstractLogServiceClient: IoC injection interfaces for pluggable backendsAdapted from eggjs/egg#5822 for tegg monorepo conventions (CJS module system, mocha tests, lerna versioning).
Test plan
npm test --workspace=core/agent-tracing)tsc --noEmit)core/*conventions🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Tests
Chores