refactor(telemetry): align metrics/pricing init with tracing pattern#1252
Open
ajbozarth wants to merge 1 commit into
Open
refactor(telemetry): align metrics/pricing init with tracing pattern#1252ajbozarth wants to merge 1 commit into
ajbozarth wants to merge 1 commit into
Conversation
Wraps init logic in metrics.py and pricing.py in callable `_setup_*()` functions so tests can re-trigger init via a helper instead of `importlib.reload`. Drops the seven module-level env-read constants from metrics.py; env vars are now read at call time inside `_setup_meter_provider()` and hot-path record_* gates check `_meter is None`. Adds test/telemetry/conftest.py with shared `reset_*_state()` helpers that replace ~40 `importlib.reload(mellea.telemetry.*)` calls across all four telemetry test suites. Consolidates three drifted copies of `_reset_tracing_state()` left over from PR generative-computing#1181. Assisted-by: Claude Code Signed-off-by: Alex Bozarth <ajbozart@us.ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Issue
Fixes #
Description
Brings
mellea.telemetry.metricsandmellea.telemetry.pricingonto the same module-init pattern thatmellea.telemetry.tracingadopted in #1181. Eliminatesimportlib.reload(mellea.telemetry.*)from every telemetry test (~40 sites) and consolidates the three drifted copies of_reset_tracing_state()that PR #1181 left behind.No public API change. No production behavior change. Test side absorbed nearly all of the diff.
Why: after #1181,
tracing.pywas the only telemetry pillar with its init logic in a recallable_setup_tracing()function.metrics.pyandpricing.pystill captured env vars into module-level constants at the top of the module, which forced every test that toggled an env var toimportlib.reload(mellea.telemetry.metrics)to re-execute the module body. Each reload re-runs imports and side effects — slow, noisy, and fragile.Source-side changes:
metrics.py— Module-level env-read constants removed. The seven private_METRICS_*/_OTLP_METRICS_ENDPOINT/_SERVICE_NAME/_EXPORT_INTERVAL_MILLISconstants no longer exist; env-var reads live inside_setup_meter_provider(), matchingtracing.py. The hot-path gate in everyrecord_*function checks_meter is Nonedirectly (semantically equivalent to the old_METRICS_ENABLEDcheck). Also fixes a parallel of the tracing-side regression caught in fix: reload module for telemetry testing so all tests can run #805:_meteris now bound off the freshly built provider rather than via OTel's globalmetrics.get_meter(), sinceset_meter_provider()is one-shot per process and the global meter would otherwise fall back to the original (now shutdown) provider after a reset.pricing.py— Init logic wrapped in_setup_pricing(). Same shape as the others; no production behavior change.Minor cleanup: the redundant
if is_tracing_enabled():guard at the module-load_setup_tracing()call site is dropped (the function early-returns on its own), andis_metrics_enabled()/is_tracing_enabled()now cache their result likeis_pricing_enabled()already does.Test-side changes:
test/telemetry/conftest.pywith sharedreset_metrics_state(),reset_tracing_state(),reset_logging_state(),reset_pricing_state()helpers replacing allimportlib.reloadcalls. New regression testtest_meter_remains_functional_after_repeated_resetsexercises_setup_metrics()across multiple reset cycles and asserts recordings still land in the active reader._reset_tracing_state()consolidated.test_tracing.pyandtest_tracing_backend.pypreviously omitted the_tracer_provider.shutdown()call thattest_tracing_plugins.pyhad — the consolidated helper does the shutdown for all three. Strictly safer cleanup; no perceptible slowdown (idleBatchSpanProcessorworkers join in microseconds)._LITELLM_AVAILABLEand_OTEL_AVAILABLE: tests no longer manipulatesys.modulesto fake import failure. They now usemonkeypatch.setattr(...)directly on the module attribute. Same code paths exercised; module-load-timetry: import ...blocks are no longer reverified per-test.test_metrics.pyshifted from cached env captures (_METRICS_CONSOLE is True,_OTLP_METRICS_ENDPOINT == "...", etc.) to behavior on the constructedMeterProvider(resource attributes, attached metric readers, mock-patched exporter call args). Strictly stronger — proves env values were both read and applied.test/telemetry/__init__.pyadded sofrom test.telemetry.conftest import ...resolves under mypy. Same patterntest/plugins/__init__.pyalready uses.What didn't change:
is_metrics_enabled(),is_tracing_enabled(),is_pricing_enabled(),get_otlp_log_handler(), allrecord_*andcreate_*functions — same signatures, same semantics._plugins_registeredsemantics preserved (process-global, intentionally not reset by tests).Testing
2 e2e tests in
test_metrics_backend.py::test_huggingface_token_metrics_integrationfail on local — pre-existinghuggingface.py:1119torch tensor-indexing issue, unrelated to this PR. Not seen in-m "not e2e".Opened #1253
Attribution
Adding a new component, requirement, sampling strategy, or tool?
If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.
NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.