Is there any appetite to try to standardize a capture format?

[greggman]

It came up at the Sept 2025 WebGPU Face To Face meeting to see if there was any desire to try to standardize a WebGPU capture format so that tools like WebGPUReconstruct, and webgpu inspector

This would theoretically allow capturing from one tool and replaying that capture in a different tool which might be useful. On the other hand it requires coordination.

I wasn't sure where to have this conversation so I thought I'd start a thread here. Feel free to move this conversation somewhere more appropriate.

@Chainsawkitten
@brendan-duncan
@stefnotch
@Kangz

wgpu also has a capture and playback facility and dawn has been exploring one as well.

The first proposal is to semi auto-generate a protocol buffer from the webgpu.idl with a little extra sprinkled in where needed.

[brendan-duncan]

That would be interesting, I'd be up for a conversation. I use my recording tool all the time, and have been experimenting with some new designs based on needs I have. I just used it recently to submit some bug report repros to the Mozilla team.

I don't have a strong opinion on the particular format, protocol buffer or otherwise, except that it has to be human readable and editable, and translatable to pure self-contained JS for simple bug repros and experimentation.

[stefnotch]

So we should need to pick a text format, and should store binary assets in an extra file. We could base64 encode them, but that makes the massive text files be unusable in many text editors.

Protobuf has an implemented, but not officially specced text format https://protobuf.dev/reference/protobuf/textformat-spec/ . Which makes it a less practical option.

JSON would be human readable, but the few people that I have talked to have strong opinions against using it.

Which likely means that we have to design our own simple text format and use that.

[greggman]

It was mentioned in the meeting that v8 (or JS?) has 250meg string size limit in many browsers so once that limit is hit you have to start having arrays of JSON or something.

[brendan-duncan]

Alternatively, whatever format can be used for the standard recording format, binary or whatever, as making tooling on top of it to translate it to other formats would be possible.

I use embedded base64 in JS recordings and most text editors are fine, but then again I purposefully try to minimize the scene I'm recording for the bug repro or experimenting with the commands. The benefit of that is that it's a self contained html and doesn't require a local server to run. I could always write a tool to convert a protobuf to my needs there.

Another thing I'm working on for the webgpu_inspector recorder is single frame recording. It tracks all commands and data during the run, then for the frame I want to isolate, it goes back and gathers the commands and data that were recorded in previous frames, to reproduce that single frame, discarding everything else that wasn't needed for that frame.

I suppose a tool could be written on top of the protobuf standard recording to do that, if there was enough data for each command.

[stefnotch]

I took a cursory glance at what other tools do

Renderdoc

• has a custom binary format.
• can convert that to a text format (xml, or zipped xml).

Nvidia Nsight

• I couldn't find documentation about a binary capture format
• can export to a C++ file
  https://docs.nvidia.com/nsight-graphics/2018.4/content/nsight_graphics/save_captures.htm

Microsoft Pix

• has an optimized binary format https://github.com/microsoft/PixEvents?tab=readme-ov-file#why-use-pix-events
• seems to have a csv format for events https://devblogs.microsoft.com/pix/pixtool/

[stefnotch]

Talking to @Chainsawkitten revealed another requirement: We should think about backwards compatibility and versioning, because the format can slightly change over time. Opening old captures should always work.

[brendan-duncan]

Versioning will be important for sure. All of the best formats inevitably change :-)

I just hit a case 10 minutes ago where having a JS recording was necessary. An obscure bug in Firefox I've been trying to help isolate. After seeing this post, it gave me the idea to try WebGPUReconstruct to record the problem and play it back in pure Wgpu. I had been having trouble creating a repro of the problem with wgpu myself, even though the problem definitely occurs in the browser. The same thing that renders incorrectly in the browser, rendered correctly calling directly into Wgpu. So, a recording with native only playback like WebGPUReconstruct wouldn't have helped with reproducing the problem, a JS recording was needed to repro the problem in the browser, but having the native playback work correctly helps identify where the problem might be coming from.

Being human readable/editable, I was also able to strip the recording down to a tiny subset of the commands/data that were in the full recording, isolating the single draw calls.

Again, I think I could write tools on top of whatever format to do what I did with my current tools. I'm not hung up on the tools I wrote needing to be the way they are, they were just written out of necessity.

[Chainsawkitten]

Having a common format would certainly be very interesting and could enable a bunch of useful tooling to be shared. Writing tools against a common format would allow sharing them instead of everyone having to reimplement the same thing. For instance the aforementioned trimming to trim a capture to a given frame / set of frames. Or a tool to make captures more portable by decompressing block-compressed textures (enabling running desktop captures on mobile and vice versa). Or analysis tools that analyse a capture and warn about non-performant behavior (eg. storing render targets that aren't read again and could be discarded, wasting bandwidth). The possibilities are endless.

Finding a common format that suits the needs of all the tools will be a challenge, though.

Regarding text/binary, I agree a human readable/editable format would be very useful. I went with a binary format for WebGPUReconstruct, though. I want to be able to take long captures and was concerned with performance and size of a text-based format. Storing large binary data (eg. writeBuffer, writeTexture data) separately, as suggested by @stefnotch , might alleviate some of those concerns. That data isn't really human readable/editable anyhow. In fact, it might make it harder to read the file if there's a bunch of huge Base64 encoded strings interspersed with the commands. The downsides would be:

1. Having captures be multiple files instead of a single, self-contained file will be a bit more annoying to deal with for users.
2. If you want to manually trim a file to produce some minimal reproducer, removing commands from the text file wouldn't remove the related data from the binary file. So a "minimal reproducer" might be a tiny .json and a 2GB .bin. It would be trivial to create a tool which removes any unused binary data from the .bin (and adjust the offsets in the .json accordingly) so maybe not a big deal.

It's also a possibility for me to keep using a binary format, and have some tool to convert it to/from the common, text-based interchange format. Or vice versa.

[brendan-duncan]

Small point, but for the JS text format recording, the base64 encoded binary data is stored at the end of the file, so there's a clear division between the readable JS code and the unreadable binary code. Readability of including binary data isn't an issue. But maximum string sizes are.