test: add LangGraph node_metrics isolation regression test#176
Open
jsonbailey wants to merge 1 commit intomainfrom
Open
test: add LangGraph node_metrics isolation regression test#176jsonbailey wants to merge 1 commit intomainfrom
jsonbailey wants to merge 1 commit intomainfrom
Conversation
Mirrors the OpenAI regression test added in PR #155. Verifies that LangGraphAgentGraphRunner.run() produces a fresh node_metrics dict on each invocation so callback-handler state cannot leak across runs on the same runner instance.
bd61c31 to
637ee98
Compare
keelerm84
approved these changes
May 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the missing regression test that should have shipped alongside the LangGraph runner migration in #156. Mirrors the OpenAI version added in #155.
Bug class being guarded
The graph-runner pattern uses a callback-style accumulator (
LDMetricsCallbackHandlerfor LangGraph,_node_*state dicts for OpenAI) to collect per-node metrics during arun(). If that accumulator is held on the runner instance acrossrun()calls, the second invocation'snode_metricsends up containing stale entries from the first run.#155 fixed this for the OpenAI runner and shipped a regression test:
packages/ai-providers/server-ai-openai/tests/test_openai_agent_graph_runner.py::test_openai_agent_graph_runner_run_resets_node_metrics_between_runs#156 migrated the LangGraph runner to the same
AgentGraphRunnerResultshape but did not include the equivalent regression test. This PR adds it.What this PR does
Adds
test_langgraph_runner_run_resets_node_metrics_between_runsinpackages/ai-providers/server-ai-langchain/tests/test_langgraph_agent_graph_runner.py. It callsrunner.run()twice on the sameLangGraphAgentGraphRunnerinstance, fires callback events on the handler each time, and asserts that the second result'snode_metrics:LDAIMetricsinstance, not the same object returned on the first runNo fix to the runner itself was needed:
LangGraphAgentGraphRunner.run()already constructs a freshLDMetricsCallbackHandlerper call, so state is isolated by construction. The test locks that behaviour in.Test plan
make test-langchainpasses (86 tests)make testpasses across all packages (server-ai 200, langchain 86, openai 44)make lintpasses (mypy, isort, pycodestyle)Note
Low Risk
Low risk: test-only change that tightens coverage around metrics collection across repeated
run()calls, with no production logic modified.Overview
Adds a new async regression test in
test_langgraph_agent_graph_runner.pythat callsLangGraphAgentGraphRunner.run()twice on the same instance and asserts the second result’smetrics.node_metricsandmetrics.pathare fresh per invocation (no stale/cumulative state from the first run).The test bypasses graph compilation by injecting a mocked compiled graph whose
ainvoketriggers LangChain callback events, specifically guarding against accidental reuse of the metrics callback handler/state across runs.Reviewed by Cursor Bugbot for commit 637ee98. Bugbot is set up for automated code reviews on this repo. Configure here.