Skip to content

Chrometrace profiling support#49

Open
lee1258561 wants to merge 13 commits intopinterest/main-2.52.1from
chrometrace-profiling-support
Open

Chrometrace profiling support#49
lee1258561 wants to merge 13 commits intopinterest/main-2.52.1from
chrometrace-profiling-support

Conversation

@lee1258561
Copy link

@lee1258561 lee1258561 commented Mar 13, 2026

Summary

This PR adds Chrome Trace format support for CPU profiling in the Ray Dashboard, enabling temporal/timeline-based profiling
alongside the existing flamegraph format. Users can now choose between three visualization methods when profiling actors, tasks,
jobs, or nodes.

Key Changes

  • Chrome Trace format support: Added chrometrace format option to py-spy profiler, generating timeline-based profiles that
    show when functions execute over time

  • Three visualization options:

    • Speedscope (Flamegraph) - Aggregated view, best for identifying hotspots
    • Perfetto UI - External Google trace viewer (requires user confirmation for data upload)
    • Chrome Trace Viewer - Embedded local viewer, no external data transfer
  • Embedded Catapult Trace Viewer: Bundled the Chromium Catapult trace-viewer for fully local visualization without sending data
    to external services

Visualization Method Comparison

Feature Speedscope (Flamegraph) Perfetto UI Chrome Trace Viewer
View Type Aggregated stack samples Timeline Timeline
Multi-thread Unified View ❌ No ❌ No Yes
Data Privacy Local External (ui.perfetto.dev) Local
Best For Finding hotspots Detailed trace analysis Multi-threaded temporal analysis

Key Insight: Chrome Trace Viewer is the only option that provides a unified stack view across multiple threads, making it
essential for debugging multi-threaded Python applications where understanding cross-thread timing and interactions is critical.

Files Changed

  • python/ray/dashboard/client/src/common/ProfilingLink.tsx - New profiling UI with format selection
  • python/ray/dashboard/modules/reporter/profile_manager.py - Added chrometrace format support
  • python/ray/dashboard/modules/reporter/reporter_head.py - New endpoint for trace viewer
  • python/ray/dashboard/client/public/trace-viewer/ - Embedded Catapult trace viewer
  • src/ray/protobuf/reporter.proto - Added format field to protobuf

Test Plan

Screen.Recording.2026-03-12.at.7.19.20.PM.mov
  • Start Ray cluster with sample actors
  • Navigate to Actors → select actor → CPU Profiling
  • Test "Speedscope (Flamegraph)" - opens flamegraph view
  • Test "Perfetto UI" - shows confirmation dialog, opens external viewer
  • Test "Chrome Trace (Timeline)" - opens embedded trace viewer with timeline
  • Verify loading overlay disappears after trace loads
  • Test with multi-threaded workload to verify unified thread view in Chrome Trace Viewer

lee1258561 and others added 12 commits March 12, 2026 17:33
This change adds support for py-spy's native chrometrace format output,
enabling timeline-based CPU profiling visualization in Perfetto UI.

Backend changes:
- Add 'chrometrace' to supported formats in profile_manager.py
- Add 'rate' parameter for configurable sampling rate (default 100 Hz)
- Update reporter_agent.py to pass rate parameter
- Update reporter_head.py to set Content-Type: application/json and
  CORS headers for chrometrace format to allow Perfetto UI fetching

Frontend changes:
- Add CpuProfilingButton component with dialog for configuring:
  - Format: Flamegraph (SVG), Chrome Trace (Timeline), Speedscope
  - Duration: 1-60 seconds
  - Sampling Rate: 1-1000 Hz
  - Native: Include C/C++ stack frames (Linux only)
- Add "Open in Perfetto" button for chrometrace format
- Replace CpuProfilingLink with CpuProfilingButton in all pages
- Replace TaskCpuProfilingLink with TaskCpuProfilingButton

Protobuf changes:
- Add 'rate' field to CpuProfilingRequest message

Tests:
- Add test_profiler_chrometrace_format test

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The "Open in Viewer" button was failing because Speedscope's URL
constructor requires an absolute URL with protocol and host, but we
were passing a relative path. Now we use window.location.origin to
build the full absolute URL dynamically, which works regardless of
how the dashboard is accessed (localhost, IP, or proxy).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds "Open in Perfetto" button for chrometrace format that opens
ui.perfetto.dev with trace data using postMessage protocol. Includes
a confirmation dialog warning users about sending data to third-party site.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Integrates the open-source chrome://tracing viewer (Catapult) directly into
the Ray dashboard for fully local chrometrace visualization without sending
data to external services.

Changes:
- Add trace-viewer/ static assets (built from Catapult project)
- Add URL loading support via hash parameter (#traceURL=...)
- Add "Open in Trace Viewer" button for chrometrace format
- Rename existing button to "Open in Speedscope" for clarity

Benefits:
- Security: No profile data sent to third-party sites
- Offline: Works in air-gapped environments
- UX: Same experience as chrome://tracing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use original about_tracing.html which contains all required <template>
  elements (like #profiling-view-template) that the JS code needs
- Rename about_tracing.js to tracing.js to match original HTML reference
- Add URL loading script with hash-based traceURL parameter support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The setActiveTrace function is asynchronous - it returns immediately but
loads the model via a promise internally. Instead of hiding the overlay
right after calling setActiveTrace, poll for when the timelineView.model
is populated with processes before removing the loading overlay.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use inline style.display='none' instead of classList.add('hidden')
to override the inline display:flex that has higher CSS specificity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove speedscope from format dropdown, keep flamegraph and chrometrace
- Add new py-spy options exposed in UI: GIL Only, Include Idle, Non-blocking
- Always enable --threads flag for thread ID visibility
- Update protobuf, backend API, and profile manager to support new options

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add getBasePath() helper to extract base path from current URL
- Update openInSpeedscope, openInTraceViewer, and openInPerfetto to
  use base path for correct URL generation behind proxy
- Change CpuProfilerButton and ProfilerButton wrappers from <div> to
  React.Fragment to fix inline layout issues

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link

@lishunyao97 lishunyao97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool work! thanks for adding this

Tests cover button rendering, dialog interactions, py-spy options
(native, GIL, idle, nonblocking), chrometrace viewer buttons
(Speedscope, Trace Viewer, Perfetto), and default format selection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants