Skip to content

Technical review: Document VideoFrame.metadata()#43506

Open
chrisdavidmills wants to merge 4 commits intomdn:mainfrom
chrisdavidmills:video-frame-metadata
Open

Technical review: Document VideoFrame.metadata()#43506
chrisdavidmills wants to merge 4 commits intomdn:mainfrom
chrisdavidmills:video-frame-metadata

Conversation

@chrisdavidmills
Copy link
Copy Markdown
Contributor

Description

Chrome 145 adds support for the VideoFrame.metadata() method; see https://chromestatus.com/feature/5186046555586560.

This PR documents that method, and also adds information about the metadata option to the VideoFrame() constructor page.

Motivation

Additional details

Related issues and pull requests

@chrisdavidmills chrisdavidmills requested a review from a team as a code owner March 20, 2026 10:07
@chrisdavidmills chrisdavidmills requested review from hamishwillee and removed request for a team March 20, 2026 10:07
@github-actions github-actions bot added Content:WebAPI Web API docs size/m [PR only] 51-500 LoC changed labels Mar 20, 2026
@chrisdavidmills chrisdavidmills changed the title Document VideoFrame.metadata() Technical review: Document VideoFrame.metadata() Mar 20, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 20, 2026

Preview URLs (3 pages)

External URLs (2)

URL: /en-US/docs/Web/API/VideoFrame/metadata
Title: VideoFrame: metadata() method


URL: /en-US/docs/Web/API/VideoFrame/VideoFrame
Title: VideoFrame: VideoFrame() constructor

(comment last updated: 2026-03-23 11:07:18)

An object containing metadata describing the video frame, specified by the [WebCodecs VideoFrame Metadata Registry](https://w3c.github.io/webcodecs/video_frame_metadata_registry.html), which can contain the following properties:

- `rtpTimestamp` {{optional_inline}}
- : The RTP timestamp of the corresponding encoded frame. Only video frames originating from [WebRTC](/en-US/docs/Web/API/WebRTC_API) sources will have `rtpTimestamp` metadata. This allows applications using a {{domxref("MediaStreamTrackProcessor")}} (for example, to render decoded WebRTC frames to a `<canvas>`) or [WebCodecs](/en-US/docs/Web/API/WebCodecs_API) (for example, for custom decoding pipelines) to correlate each exposed frame with its original RTP transport timestamp. This is useful for example when aligning video with audio segments or debugging latency issues.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the only frames which have this must come from WebRTC, frames decoded by WebCodecs won't have the value. So you'll instead want to say something about "custom encoding pipelines"

You may want to drop the only for "generally" since user generated frames may add this value at will.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I've updated this description to:

The RTP timestamp of the corresponding encoded frame. Video frames originating from WebRTC sources will have rtpTimestamp metadata. This allows applications using a {{domxref("MediaStreamTrackProcessor")}} (for example, to render decoded WebRTC frames to a <canvas>) or a custom decoding pipeline to correlate each exposed frame with its original RTP transport timestamp. This is useful for example when aligning video with audio segments or debugging latency issues.

So I've removed "only", and the mention of WebCodecs, and kept the bit about custom pipelines. Does that address your concern here?

## Syntax

```js-nolint
// Image constructor
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defer to you but image in this case is a CanvasImageSource so there may be a more helpful comment to have here and below.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I can see the porential for confusion here. I've sidestepped this by just removing the comments (we don't tend to have comments in the syntax code block anyway) and leaving a line break between the first constructor type and the second, to provide the indication that there are two different types. I think this is enough.

- `metadata` {{optional_inline}}
- : An object containing metadata describing the video frame, specified by the [WebCodecs VideoFrame Metadata Registry](https://w3c.github.io/webcodecs/video_frame_metadata_registry.html), which can contain the following properties:
- `rtpTimestamp` {{optional_inline}}
- : The RTP timestamp of the corresponding encoded frame. Only video frames originating from [WebRTC](/en-US/docs/Web/API/WebRTC_API) sources will have `rtpTimestamp` metadata.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not helpful to say this here since this dictionary is user controlled and can have whatever the caller wants in it. I.e., even if the frame is not from WebRTC they could add this value.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, that's fair enough. I've removed the second sentence from the description; in each case, it now reads "The RTP timestamp of the corresponding encoded frame." Does that answer your concern?

- `metadata` {{optional_inline}}
- : An object containing metadata describing the video frame, specified by the [WebCodecs VideoFrame Metadata Registry](https://w3c.github.io/webcodecs/video_frame_metadata_registry.html), which can contain the following properties:
- `rtpTimestamp` {{optional_inline}}
- : The RTP timestamp of the corresponding encoded frame. Only video frames originating from [WebRTC](/en-US/docs/Web/API/WebRTC_API) sources will have `rtpTimestamp` metadata.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto on limitation text

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated as per above comment.

chrisdavidmills and others added 3 commits March 23, 2026 10:37
Co-authored-by: Hamish Willee <hamishwillee@gmail.com>
Co-authored-by: Hamish Willee <hamishwillee@gmail.com>
Copy link
Copy Markdown
Contributor

@dalecurtis dalecurtis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, but can't approve!

An object containing metadata describing the video frame, as specified by the [WebCodecs VideoFrame Metadata Registry](https://w3c.github.io/webcodecs/video_frame_metadata_registry.html).
This can contain the following properties:

- `rtpTimestamp` {{optional_inline}}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd just delete the "... or a custom ..." part since it doesn't really make logical sense. I.e., RTC must have done the decoding or the site attached the metadata to its already decoded frames afterwards. In which case, they probably wouldn't bother with attaching the metadata since it's already known to the site.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Content:WebAPI Web API docs size/m [PR only] 51-500 LoC changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants