docs: Add initial proposal for V2 recording & playback API. (WIP)#791
Draft
microbit-carlos wants to merge 7 commits into
Draft
docs: Add initial proposal for V2 recording & playback API. (WIP)#791microbit-carlos wants to merge 7 commits into
microbit-carlos wants to merge 7 commits into
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs preview:
This initial proposal has been discussed in:
But we have some open question that will likely result and a rework of some of this.
Initial proposal
The initial proposal in this PR was to create a new
AudioBufferclass to contain the audio data and sampling rate.The
AudioBuffer.rateproperty could then be used bymicrophone.record()andaudio.play()to configure recording and playback rates.This was done to avoid introducing a new parameter to
audio.play()to configure the sampling rate, when it could only work with a single type of sound input (as it might not be possible to change the rate of the SoundExpressions or AudioFrames).Disadvantages
However, changing the rate in a buffer type to change the playback rate in real-time is a bit awkward:
An alternative we considered was to have the playback sampling rate modified via the audio module itself:
However, this would have to set the same rate to everything played via the audio module, and Sound Expression have a different default rate (44K) than recordings (11K). So
audio.set_rate(22000)should slow down Sound Expression and speed up recordings.Alternatively, if we wanted to change the playback rate via the audio module, we could set a ratio instead. Something equivalent to
audio.set_speed(100%)(with different semantics). But a disadvantage would be that it's removing some of math/physics learning opportunity to directly relate the sampling rate value with the effects that it has in playback speed.Alternative proposal: bytearray as the buffer type
In this case a byte array would be returned by
microphone.record()and used withmicrophone.record_into().As this data type does not include info about the rate, we depend on the
audio.play()adding an extra argument that might not work with other sound types like Sound Expressions and Audio Frames.However, we still have the issue of updating the playback rate in real time during playback, which means we might would have to use use a similar approach to the previously mentioned
audio.set_speed(100%):Alternative proposal: AudioFrames as the buffer type
This would be the same as the bytearray proposal, but using the existing AudioFrames instead.
We might need to tweak the AudioFrame class to let us user larger buffers, as the default is 32 samples. As
audio.play()can consume an iterable as well, we would need to figure out a good balance between AudioFrame size and number of AudioFrames in a recording buffer.