Technical review: Document VideoFrame.metadata()#43506
Technical review: Document VideoFrame.metadata()#43506chrisdavidmills wants to merge 4 commits intomdn:mainfrom
Conversation
|
Preview URLs (3 pages)
External URLs (2)URL:
URL:
(comment last updated: 2026-03-23 11:07:18) |
| An object containing metadata describing the video frame, specified by the [WebCodecs VideoFrame Metadata Registry](https://w3c.github.io/webcodecs/video_frame_metadata_registry.html), which can contain the following properties: | ||
|
|
||
| - `rtpTimestamp` {{optional_inline}} | ||
| - : The RTP timestamp of the corresponding encoded frame. Only video frames originating from [WebRTC](/en-US/docs/Web/API/WebRTC_API) sources will have `rtpTimestamp` metadata. This allows applications using a {{domxref("MediaStreamTrackProcessor")}} (for example, to render decoded WebRTC frames to a `<canvas>`) or [WebCodecs](/en-US/docs/Web/API/WebCodecs_API) (for example, for custom decoding pipelines) to correlate each exposed frame with its original RTP transport timestamp. This is useful for example when aligning video with audio segments or debugging latency issues. |
There was a problem hiding this comment.
Because the only frames which have this must come from WebRTC, frames decoded by WebCodecs won't have the value. So you'll instead want to say something about "custom encoding pipelines"
You may want to drop the only for "generally" since user generated frames may add this value at will.
There was a problem hiding this comment.
OK, I've updated this description to:
The RTP timestamp of the corresponding encoded frame. Video frames originating from WebRTC sources will have
rtpTimestampmetadata. This allows applications using a {{domxref("MediaStreamTrackProcessor")}} (for example, to render decoded WebRTC frames to a<canvas>) or a custom decoding pipeline to correlate each exposed frame with its original RTP transport timestamp. This is useful for example when aligning video with audio segments or debugging latency issues.
So I've removed "only", and the mention of WebCodecs, and kept the bit about custom pipelines. Does that address your concern here?
| ## Syntax | ||
|
|
||
| ```js-nolint | ||
| // Image constructor |
There was a problem hiding this comment.
Defer to you but image in this case is a CanvasImageSource so there may be a more helpful comment to have here and below.
There was a problem hiding this comment.
Ah, I can see the porential for confusion here. I've sidestepped this by just removing the comments (we don't tend to have comments in the syntax code block anyway) and leaving a line break between the first constructor type and the second, to provide the indication that there are two different types. I think this is enough.
| - `metadata` {{optional_inline}} | ||
| - : An object containing metadata describing the video frame, specified by the [WebCodecs VideoFrame Metadata Registry](https://w3c.github.io/webcodecs/video_frame_metadata_registry.html), which can contain the following properties: | ||
| - `rtpTimestamp` {{optional_inline}} | ||
| - : The RTP timestamp of the corresponding encoded frame. Only video frames originating from [WebRTC](/en-US/docs/Web/API/WebRTC_API) sources will have `rtpTimestamp` metadata. |
There was a problem hiding this comment.
Not helpful to say this here since this dictionary is user controlled and can have whatever the caller wants in it. I.e., even if the frame is not from WebRTC they could add this value.
There was a problem hiding this comment.
OK, that's fair enough. I've removed the second sentence from the description; in each case, it now reads "The RTP timestamp of the corresponding encoded frame." Does that answer your concern?
| - `metadata` {{optional_inline}} | ||
| - : An object containing metadata describing the video frame, specified by the [WebCodecs VideoFrame Metadata Registry](https://w3c.github.io/webcodecs/video_frame_metadata_registry.html), which can contain the following properties: | ||
| - `rtpTimestamp` {{optional_inline}} | ||
| - : The RTP timestamp of the corresponding encoded frame. Only video frames originating from [WebRTC](/en-US/docs/Web/API/WebRTC_API) sources will have `rtpTimestamp` metadata. |
There was a problem hiding this comment.
Ditto on limitation text
There was a problem hiding this comment.
updated as per above comment.
Co-authored-by: Hamish Willee <hamishwillee@gmail.com>
Co-authored-by: Hamish Willee <hamishwillee@gmail.com>
dalecurtis
left a comment
There was a problem hiding this comment.
lgtm, but can't approve!
| An object containing metadata describing the video frame, as specified by the [WebCodecs VideoFrame Metadata Registry](https://w3c.github.io/webcodecs/video_frame_metadata_registry.html). | ||
| This can contain the following properties: | ||
|
|
||
| - `rtpTimestamp` {{optional_inline}} |
There was a problem hiding this comment.
I'd just delete the "... or a custom ..." part since it doesn't really make logical sense. I.e., RTC must have done the decoding or the site attached the metadata to its already decoded frames afterwards. In which case, they probably wouldn't bother with attaching the metadata since it's already known to the site.
Description
Chrome 145 adds support for the
VideoFrame.metadata()method; see https://chromestatus.com/feature/5186046555586560.This PR documents that method, and also adds information about the
metadataoption to theVideoFrame()constructor page.Motivation
Additional details
Related issues and pull requests