Authors: Hitesh Tulsiani, David M. Chan, Shalini Ghosh, Garima Lalwani, Prabhat Pandey, Ankish Bansal, Sri Garimella, Ariya Rastrow, Björn Hoffmeister
Abstract: Dialog systems, such as voice assistants, are expected to engage with users
in complex, evolving conversations. Unfortunately, traditional automatic speech
recognition (ASR) systems deployed in such applications are usually trained to
recognize each turn independently and lack the ability to adapt to the
conversational context or incorporate user feedback. In this work, we introduce
a general framework for ASR in dialog systems that can go beyond learning from
single-turn utterances and learn over time how to adapt to both explicit
supervision and implicit user feedback present in multi-turn conversations. We
accomplish that by leveraging advances in student-teacher learning and
context-aware dialog processing, and designing contrastive self-supervision
approaches with Ohm, a new online hard-negative mining approach. We show that
leveraging our new framework compared to traditional training leads to relative
WER reductions of close to 10% in real-world dialog systems, and up to 26% on
public synthetic data.
Source: http://arxiv.org/abs/2409.10515v1
