Add WER metrics: Faster-Whisper, NeMo ASR, Facebook’s HuBERT-large-finetuned#44
Add WER metrics: Faster-Whisper, NeMo ASR, Facebook’s HuBERT-large-finetuned#44whr-a wants to merge 6 commits intowavlab-speech:mainfrom
Conversation
setup.py
Outdated
| "espnet_model_zoo", | ||
| "discrete-speech-metrics @ git+https://github.com/ftshijt/DiscreteSpeechMetrics.git@v1.0.2", | ||
| "cdpam", | ||
| "nemo_toolkit[asr]" |
There was a problem hiding this comment.
Thanks for fixing this. I'm a bit conservative to put nemo here as they are not always keep the latest related to the other packages version and it is difficult to keep them aligned all the time. Would be better to put it in tools as additional installers
setup.py
Outdated
| "importlib-metadata", | ||
| "kaggle", | ||
| "kaldiio", | ||
| "jamo", |
There was a problem hiding this comment.
If this is a dependency for nemo, we can put it together to installers.
There was a problem hiding this comment.
I built a versa conda environment from scratch, and when I run the test test/testpipeline/test_general.py I encounter this issue. It seems to be a dependency inside espnet2, but it doesn’t appear to be listed in the setup.py at https://github.com/ftshijt/espnet/blob/espnet_inference/setup.py.
There was a problem hiding this comment.
Oh, I see the point. Thanks for brining it up. Sure, it should be fine then (I probably will clean it up later for this hhh)
|
Looks great! I will merge it after the CI test. |
2 participants
This update adds three new WER metrics computed by different ASR models:
faster_whisper_wer): Systran/faster-whisper-large-v3nemo_wer): nvidia/stt_en_conformer_transducer_xlargehubert_wer): facebook/hubert-large-ls960-ft