[AUDIO WORKLET] Support proper shutdown of audio node

[lindell]

Problem

Today, the emscripten_destroy_web_audio_node function only disconnects the AudioNode and removes it from the registry.

This does not guarantee that process() callback wont be called again, and therefore the registered callback in emscripten_create_wasm_audio_worklet_node might continue to be called.

If the userData pointer is used, it might be very hard to free that resource without risking use after free.

How this PR fixes this

When emscripten_destroy_web_audio_node is used, send a message to the Node, that will set a property to ensure that any access to WASM memory will no longer happen.

Since the existing function would not have a way of knowing when it is safe to free any resource connected to the userData pointer. Another function, emscripten_destroy_web_audio_node_async is added, that will get a callback when we can ensure that no more calls to the registered EmscriptenWorkletNodeProcessCallback will happen.

I'm unsure if the callback should happen on the main thread, or in the AudioContext.

Fixes: #25884

[sbc100]

Can we not use a shared memory location here to guarantee that the callback will never fire again once this function returns?

Then we would not need emscripten_destroy_web_audio_node_async at all I think?

[lindell]

But that shared location will then be leaked memory? As it will never be able to tell when the last process callback will be (at least according to the WebAudio spec)

[sbc100]

Hmm, interesting yes. What about this sequence of operations:

1. Main thread sets "shutdown" bit.
2. Main thread blocks/spins until worklet's next "process" callback which consumes the "shutdown" bit and sets a JS flag preventing any future "process" callback. The worklet would then set the "shutdown_complete" bit unblocking the main thread.
3. Main thread is now free to release all shared memory resources.

Assuming the audio worklet always make progress during the emscripten_destroy_web_audio_node I think it should be OK with block like this. WDYT @cwoffenden ?

[cwoffenden]

This type of blocking in the main thread should be fine, since we'd already be in the main thread, waiting on the AW, so at most it should be spinning for 3ms with the default quantum size of 128 samples (as this becomes adjustable this blocking time will increase).

It would need some testing to prove that spinning for 3ms doesn't cause Chrome to start delaying timeouts or frames, so I think in general the async approach is cleaner.

[lindell]

I would very much prefer if we could avoid the callback here.

Assuming process will always be called afterwards, that should work. But is that really guaranteed to happen? Should at least not be when using an Offline Audio context?

Assuming we can guarantee it.
Do we even need the spinlock? If we set the shutdown bit, we can already then guarantee that the process callback wont be called. So there is no need to wait at that point?
As soon as the shutdown bit is set, we should be free to free any resource that is connected to the Process callback?

[sbc100]

Why can't the shutdown bit (and other resources) be synchronously freed on the main thread once it sees that shutdown_competed is set?

I.e. once shutdown_competed is set them main thread has a guarantee that no shared memory resources will be touched by the worker again and all shared memory resources associated with the worker can be free'd?

If you think that blocking the main thread until the not acceptable until the next process callback then I agree we woulsd need some kind of async version.

However, why don't we create the sync version and see if anyone complains? We can always add a async variant later for folks that find such blocking objectionable. Personally I think a 10ms block on the main thread would likely not be noticeable.

Also, are we sure we are going to have many users of this API? IIRC @cwoffenden said they would not used this API, is that right? So maybe keeping it simple for the first iteration and honing as a followup makes sense.

[cwoffenden]

@lindell I've never run timings between process callbacks so will take your word for it that they're unevenly spaced (compounding that the audio worklet's design is terrible 😀). I guess it depends on how often and where closing the audio device is expected, whether a 10ms spin is acceptable.

@sbc100 It's not something I'd think of using, nothing is ever released after acquiring by design (except clean-up of WebGL contexts on page unload, but that's more to do with a long standing issue of browsers keeping them around if the debugger is open). If I look at cross platform shipping code going back 20+ years, some of the close() implementations are empty (which I guess makes me a cowboy, so ye-ha?!).

[lindell]

@cwoffenden https://ui.perfetto.dev is really useful when understanding how the code is running.

Here are some chrome traces of a (non emscripten) audio worklet:

• Between batches when running playback latency hint (23ms between)
• Batch calls of the trace above (each yellow slice is one call to process())
• Between batched calls when running 0.3 (seconds) as latency hint (190ms between)
• Batch calls of the trace above (each yellow slice is one call to process())

Demo page here: https://lindell.me/audio-context-demos/noise-generator.html

[lindell]

@sbc100 Your suggestion would work from a free perspective (being sure you can free when the shutdown_competed bit is set. I was referring to my modified suggestion which will not since the process callback might already be running.

While a predictable 3ms spin might be acceptable, we can see this might take hundreds of milliseconds.

Furthermore, if the Audio Context is suspended, the process callback may never fire, causing the main thread to spinlock indefinitely (deadlock), or if we have a max timeout, cause a use after free when it resumes. Suspended audio contexts are simply no longer requesting audio through the audio graph and thus not calling the process callback on worklet nodes. While MessagePorts are still handled:
https://lindell.me/audio-context-demos/suspended-messages.html

Similarly, with OfflineAudioContexts. Will only call process, when a chunk of audio is actively requested. While MessagePorts will still be processed: https://lindell.me/audio-context-demos/offline-messages.html

Because we cannot guarantee the process scheduler's behavior, we cannot safely block the main thread waiting for it.

Reusing the existing API function without any new functions is definitely to preferred if it can be done without other consequences, which I do unfortunately not think we can do. We do not know

1. How long it might take for schedules (even if we could build the sync API on top of MessagePorts)
2. How performance sensitive users are.
3. How many Worklet nodes the users might need to destroy at once. Since WebAudio is designed to be used with a graph of nodes. There could definitely be scenarios with a lot of different worklet nodes.

If we implement this on top of the existing API, hoping the spinlocking will be fine, and where users can expect the userData will not be used after the destroy. And then some time later realises that this will be unacceptable for some new user. We will not be able to reverse it without breaking existing uses.

[cwoffenden]

if the Audio Context is suspended

A very valid point, suspended either explicitly or simply because the tab is backgrounded.