The zrc2019 benchmark needs to be adapted as the previous benchmark used humans for the evaluations.
The idea is to replace human evaluations with an ASR, the proposed model to be used is Whisper as it is open source and could be added as a python dependency, and supports the surprise language used in the 2019 benchmark which was Indonesian.
Tasks to-do:
The zrc2019 benchmark needs to be adapted as the previous benchmark used humans for the evaluations.
The idea is to replace human evaluations with an ASR, the proposed model to be used is Whisper as it is open source and could be added as a python dependency, and supports the surprise language used in the 2019 benchmark which was Indonesian.
Tasks to-do:
tts019-benchmark.itemsto use for the ABX metric