Add visualizer@v1 role

[maximmaxim345]

Adds the visualizer role, based on the previous visualizer@_draft_r1 proposal.

This alternative version aims to resolve a couple of gaps that were open in the previous PR:

One frame per binary message

Batching multiple frames of mixed types into one WebSocket message, as in the previous proposal, forces the server to either delay frames waiting for siblings (hurting low-latency playback) or send tiny batches anyway. It also makes ordering awkward across batches and pushes sort work onto the client. Per-type messages keep ordering trivial. Also more consistent with how other roles structure their binary messages.

The only concern with this approach is the increased number of messages. We could alternatively batch multiple message types with the same timestamp together.

Added peak and pitch

Both give clients more to work with for more reactive effects. Does not replace the existing types since they are still different: peak fires on any transient independent of the musical grid (where beat is rhythmic), and pitch tracks the perceived fundamental (where f_peak tracks the dominant FFT bin, which strong harmonics can hijack).

Downbeat flag on beat

Lets clients drive bar-aware effects. stream/start advertises tracks_downbeats so clients know whether to trust the bit. Accurate beat detection is hard and often relies on offline analysis, so servers without it omit the beat type entirely. Even when supported, it may be unavailable for some content (live streams, sparse non-percussive material).

Top-level rate_max

Now bounds all periodic types, not just spectrum. beat and peak are event-driven and unthrottled.

Scaling

Pins down what was previously hand-waved as "perceptual weighting" so implementations agree on the numbers.

Version name

Called @v1 here, but we might rename to @_draft_r2 if we find some concerns after implementing a prototype. This is also the reason why this is marked as draft for now.

[jhollowe]

@Aircoookie this looks to be the successor to #28. Does this include all the information you would like available?

[maximmaxim345]

Another potential concern is the amount of messages we are sending.
I don't think the overhead of WebSocket messages is too large though so this just needs testing on more low powered hardware.

There are two reasons why messages are completely split now:

• Consistency with other roles, all other roles already have one message per datum
• Difficulty of defining batching behavior. Requiring batching of multiple messages is difficult since it's always a compromise between latency and message count. But leaving batching open to the server would cause most implementations to never use them, defeating the whole purpose.

[maximmaxim345]

In case this is really a problem, we could also later release a visualizer@v2 if the overhead turns out to be bigger than expected.

This just needs to be tested (with encryption) on a ESP8266 or similar.

[maximmaxim345]

I have a couple concerns with pitch that came up while implementing this into sendspin-cli.

• First of all, the pitch given by aiosendspin isn't too precise and useful. (disabled in Music Assistant for this reason). But thats more of an implementation issue.
• Secondly, how long is a pitch supposed to be valid? If the server stops emitting when there's nothing tonal, the last value just sticks until the next track or so. One could interpret "ignore below your own threshold" as "clear when confidence is below your threshold or 0", but thats not defined in the Specification.
• And then the confidence scale itself: every client picking its own threshold and no defined meaning of the threshold makes behavior between server and client implementations inconsistent.

But if there is no reliable way to get a single useful pitch value, we could also just consider removing pitch from the specification.