EAT: Enhanced ASR-TTS for Self-Supervised Speech Recognition

doi:doi:10.57702/24jypnyc

EAT: Enhanced ASR-TTS for Self-Supervised Speech Recognition

Self-supervised ASR-TTS models suffer in out-of-domain data conditions. Here we propose an enhanced ASR-TTS model that incorporates two main features: 1) The ASR→TTS direction is equipped with a language model reward to penalize the ASR hypotheses before forwarding it to TTS. 2) In the TTS→ASR direction, a hyper-parameter is introduced to scale the attention context from synthesized speech before sending it to ASR to handle out-of-domain data.

BibTex:

@dataset{Murali_Karthick_Baskar_and_Lukás_Burget_and_Shinji_Watanabe_and_Ramon_Fernandez_Astudillo_and_Jan_„Honza“_ˇCernocký_2024,
    abstract = {Self-supervised ASR-TTS models suffer in out-of-domain data conditions. Here we propose an enhanced ASR-TTS model that incorporates two main features: 1) The ASR→TTS direction is equipped with a language model reward to penalize the ASR hypotheses before forwarding it to TTS. 2) In the TTS→ASR direction, a hyper-parameter is introduced to scale the attention context from synthesized speech before sending it to ASR to handle out-of-domain data.},
    author = {Murali Karthick Baskar and Lukás Burget and Shinji Watanabe and Ramon Fernandez Astudillo and Jan „Honza“ ˇCernocký},
    doi = {10.57702/24jypnyc},
    institution = {No Organization},
    keyword = {'ASR-TTS', 'Out-of-Domain Data', 'Self-Supervised Learning'},
    month = {dec},
    publisher = {TIB},
    title = {EAT: Enhanced ASR-TTS for Self-Supervised Speech Recognition},
    url = {https://service.tib.eu/ldmservice/dataset/eat--enhanced-asr-tts-for-self-supervised-speech-recognition},
    year = {2024}
}