Add new HookMetricEmit that can emit arbitrary metrics#1285
Open
brandur wants to merge 1 commit into
Open
Conversation
brandur
changed the title
HookMetric that can emit arbitrary metricsHookMetricEmit that can emit arbitrary metrics
brandur
added a commit
to riverqueue/rivercontrib
that referenced
this pull request
Jun 12, 2026
Here, build on the proposal in [1] to add a metric hook to River, and start emitting metrics for the time to lock jobs and the number of jobs locked. Especially the first metric is generally very useful for looking for queue degradation due to dead tuples. [1] riverqueue/river#1285
brandur
force-pushed
the
brandur-metrics-hook
branch
from
c9d83e3 to
c3c56ed
Compare
I was thinking about observability in River the other day, and how if one were trying to track River performance degradation due to dead tuples, how you might do so. Surprisingly, I don't think it's really possible under our current framework -- the best proxy for it is time to lock jobs, a metric which we don't provide any way of extracting out of River. Here, add a new hook called `HookMetricEmit`. The idea behind it is that it may receive an arbitrary metric that River starts to emit, and handle it by passing it onto a metrics system of choice, like OTEL, DataDog, or whatever. Within `HookMetricEmit`, add two specific metrics to get us started: * `JobGetAvailableDurationMetric`: Time to lock available jobs. This will start to show significant degradation in case of dead tuples. * `JobGetAvailableCountMetric`: Number of jobs locked while getting available. This is somewhat useful in seeing how well batch locking is working out in a program (i.e. are we locking one job at a time or 100?).
brandur
force-pushed
the
brandur-metrics-hook
branch
from
c3c56ed to
e54c7ad
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I was thinking about observability in River the other day, and how if
one were trying to track River performance degradation due to dead
tuples, how you might do so. Surprisingly, I don't think it's really
possible under our current framework -- the best proxy for it is time to
lock jobs, a metric which we don't provide any way of extracting out of
River.
Here, add a new hook called
HookMetricEmit. The idea behind it is that itmay receive an arbitrary metric that River starts to emit, and handle it
by passing it onto a metrics system of choice, like OTEL, DataDog, or
whatever.
Within
HookMetricEmit, add two specific metrics to get us started:JobGetAvailableDurationMetric: Time to lock available jobs. Thiswill start to show significant degradation in case of dead tuples.
JobGetAvailableCountMetric: Number of jobs locked while gettingavailable. This is somewhat useful in seeing how well batch locking is
working out in a program (i.e. are we locking one job at a time or
100?).