Skip to content

docs: Add initial proposal for V2 recording & playback API. (WIP)#791

Draft
microbit-carlos wants to merge 7 commits into
bbcmicrobit:v2-docsfrom
microbit-carlos:docs-recording
Draft

docs: Add initial proposal for V2 recording & playback API. (WIP)#791
microbit-carlos wants to merge 7 commits into
bbcmicrobit:v2-docsfrom
microbit-carlos:docs-recording

Conversation

@microbit-carlos
Copy link
Copy Markdown
Collaborator

@microbit-carlos microbit-carlos commented May 15, 2023

Docs preview:

This initial proposal has been discussed in:

But we have some open question that will likely result and a rework of some of this.

Initial proposal

The initial proposal in this PR was to create a new AudioBuffer class to contain the audio data and sampling rate.
The AudioBuffer.rate property could then be used by microphone.record() and audio.play() to configure recording and playback rates.
This was done to avoid introducing a new parameter to audio.play() to configure the sampling rate, when it could only work with a single type of sound input (as it might not be possible to change the rate of the SoundExpressions or AudioFrames).

Disadvantages

However, changing the rate in a buffer type to change the playback rate in real-time is a bit awkward:

my_recording = audio.AudioBuffer(duration=5000, rate=5500)
microphone.record_into(my_recording)
audio.play(my_recording, wait=False)
while audio.is_playing():
    x = accelerometer.get_x()
    my_recording.rate = scale(x, (-1000, 1000), (2250, 11000))
    sleep(50)

An alternative we considered was to have the playback sampling rate modified via the audio module itself:

audio.play(my_recording, wait=False)
while audio.is_playing():
    x = accelerometer.get_x()
    audio.set_rate(scale(x, (-1000, 1000), (2250, 11000)))
    sleep(50)

However, this would have to set the same rate to everything played via the audio module, and Sound Expression have a different default rate (44K) than recordings (11K). So audio.set_rate(22000) should slow down Sound Expression and speed up recordings.

Alternatively, if we wanted to change the playback rate via the audio module, we could set a ratio instead. Something equivalent to audio.set_speed(100%) (with different semantics). But a disadvantage would be that it's removing some of math/physics learning opportunity to directly relate the sampling rate value with the effects that it has in playback speed.

Alternative proposal: bytearray as the buffer type

In this case a byte array would be returned by microphone.record() and used withmicrophone.record_into().

As this data type does not include info about the rate, we depend on the audio.play() adding an extra argument that might not work with other sound types like Sound Expressions and Audio Frames.

However, we still have the issue of updating the playback rate in real time during playback, which means we might would have to use use a similar approach to the previously mentioned audio.set_speed(100%):

sound_in_byte_array = microphone.record(duration=3000, rate=5500)
audio.play(sound_in_byte_array, rate=5500 wait=False)
while audio.is_playing():
    x = accelerometer.get_x()
    audio.set_speed(scale(x, (-1000, 1000), (50, 200)))
    sleep(50)
DURATION_SECONDS = 3
SAMPLE_RATE = 5500
recording = bytearray(DURATION_SECONDS * SAMPLE_RATE)
microphone.record_into(recording, rate=SAMPLE_RATE)
audio.play(recording, rate=SAMPLE_RATE)

Alternative proposal: AudioFrames as the buffer type

This would be the same as the bytearray proposal, but using the existing AudioFrames instead.

We might need to tweak the AudioFrame class to let us user larger buffers, as the default is 32 samples. As audio.play() can consume an iterable as well, we would need to figure out a good balance between AudioFrame size and number of AudioFrames in a recording buffer.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants