Skip to content

Use CLOCK_MONOTONIC_RAW on macOS for nanosecond precision#5994

Open
eregon wants to merge 1 commit into
masterfrom
macos-CLOCK_MONOTONIC_RAW
Open

Use CLOCK_MONOTONIC_RAW on macOS for nanosecond precision#5994
eregon wants to merge 1 commit into
masterfrom
macos-CLOCK_MONOTONIC_RAW

Conversation

@eregon

@eregon eregon commented Jul 3, 2026

Copy link
Copy Markdown
Member

CLOCK_MONOTONIC on macOS only has microsecond precision, while CLOCK_MONOTONIC_RAW has nanosecond precision.

What does this PR do?

Motivation:

Finer-grained times on macOS, same granularity as on Linux, which may help avoid flaky tests.

Change log entry

None

Additional Notes:

How to test the change?

CLOCK_MONOTONIC on macOS only has microsecond precision, while
CLOCK_MONOTONIC_RAW has nanosecond precision.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@dd-octo-sts dd-octo-sts Bot added core Involves Datadog core libraries profiling Involves Datadog profiling labels Jul 3, 2026
@dd-octo-sts

dd-octo-sts Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Thank you for updating Change log entry section 👏

Visited at: 2026-07-06 08:15:32 UTC

@datadog-official

datadog-official Bot commented Jul 3, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 90.04% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: f80041c | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jul 3, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-07-03 15:52:59

Comparing candidate commit f80041c in PR branch macos-CLOCK_MONOTONIC_RAW with baseline commit 66536f2 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

@ivoanjo ivoanjo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable; no concerns here.

This is a very "rabbit hole" suggestion, and I'd do it as a separate PR, but I wonder if we should instead use CLOCK_MONOTONIC_RAW everywhere.

@eregon

eregon commented Jul 6, 2026

Copy link
Copy Markdown
Member Author

The reason to not use RAW on Linux is mostly this (from Claude but I agree, I think it's important that 1 second is 1 second, not +/- something):

⏺ CLOCK_MONOTONIC is the better default for a profiler. The main tradeoffs:

  • CLOCK_MONOTONIC_RAW avoids NTP frequency slewing, giving "true" hardware time — but NTP slewing is capped at 500 ppm (0.05%), which is negligible for profiling intervals.
  • CLOCK_MONOTONIC is historically faster to read because it was available via vDSO on Linux long before _RAW was (though modern kernels 5.x+ put _RAW in vDSO too). On older kernels, _RAW requires a real syscall — significant overhead when you're calling it on every sample.
  • Portability: CLOCK_MONOTONIC is POSIX. CLOCK_MONOTONIC_RAW is Linux-specific (macOS has it too, but BSDs generally don't).

For a profiler that samples frequently, the cost of reading the clock matters more than sub-ppm accuracy from NTP correction. CLOCK_MONOTONIC is the pragmatic choice unless you're on a modern kernel and need to correlate with a clock source that's completely independent of NTP (e.g., cross-machine hardware timestamp comparison).


So CLOCK_MONOTONIC is the best for speed on Linux and (more arguable) the portable correct clock to use in the first place.
But on macOS CLOCK_MONOTONIC only has microsecond precision and macOS's "real monotonic clock" is mach_absolute_time() and that's the same as CLOCK_MONOTONIC_RAW.

@eregon eregon marked this pull request as ready for review July 6, 2026 08:15
@eregon eregon requested review from a team as code owners July 6, 2026 08:15
@eregon eregon enabled auto-merge July 6, 2026 08:17

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f80041cec6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".


// CLOCK_MONOTONIC on macOS only has microsecond precision, CLOCK_MONOTONIC_RAW has nanosecond precision
#ifdef __APPLE__
static inline long monotonic_wall_time_now_ns(raise_on_failure_setting raise_on_failure) { return retrieve_clock_as_ns(CLOCK_MONOTONIC_RAW, raise_on_failure); }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep coarse and regular monotonic clocks consistent

On macOS with allocation profiling and dynamic sampling enabled, this changes monotonic_wall_time_now_ns to CLOCK_MONOTONIC_RAW, but the skip fast path still calls monotonic_coarse_wall_time_now_ns(), which falls back to CLOCK_MONOTONIC on macOS before should_readjust subtracts it from last_readjust_time_ns (collectors_discrete_dynamic_sampler.c:122-124). Mixing these clock domains means the sampler can readjust too early or never as the adjusted and raw clocks diverge; the macOS fallback should use the same Darwin clock as the regular helper, or the skip path should avoid the coarse helper there.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this might be a problem -- on Linux we use monotonic_coarse_wall_time_now_ns() and monotonic_wall_time_now_ns() but there's an underlying clear assumption -- those are directly comparable, just one is cheaper than the other.

But CLOCK_MONOTONIC I don't think will be comparable to CLOCK_MONOTONIC_RAW, so monotonic_coarse_wall_time_now_ns should probably use CLOCK_MONOTONIC_RAW as well for macOS (and we probably need a comment there saying "whatever gets used here needs to be comparable to monotonic_wall_time_now_ns")

#
# @param unit [Symbol] unit for the resulting value, same as ::Process#clock_gettime, defaults to :float_second
# @return [Float|Integer] timestamp in the requested unit, since some unspecified starting point
MONOTONIC_CLOCK_ID = RUBY_PLATFORM.include?("darwin") ? Process::CLOCK_MONOTONIC_RAW : Process::CLOCK_MONOTONIC

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep configuration clock provider in sync

On macOS this makes Core::Utils::Time.get_time use CLOCK_MONOTONIC_RAW, but the get_time_provider option's default/resetter still return a proc using ::Process::CLOCK_MONOTONIC (settings.rb:834-844). After a custom provider is reset, or when code uses Datadog.configuration.get_time_provider directly, Datadog goes back to the old microsecond-precision clock and can compare Ruby timestamps from CLOCK_MONOTONIC with native timestamps from CLOCK_MONOTONIC_RAW; update the configuration default/resetter to use the same clock id.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point... And this is another very awkward legacy mechanism that's very awkwardly implemented -- we monkey patch ourselves and whatnot.

Also, this gets used in production so it might impact performance if CLOCK_MONOTONIC_RAW is slower than CLOCK_MONOTONIC.

So... in hindsight this part I think it probably needs some validation, as this is not only a "change it for the tests should be fine" change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Involves Datadog core libraries profiling Involves Datadog profiling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants