INJECTING SHORT-TERM SPECTRO-TEMPORAL KNOWLEDGE INTO AUTOMATIC SPEECH RECOGNITION MODELS

원문 (중국어)

Mechanisms are provided for training an Automatic Speech Recognition (ASR) model. An automatic speech recognition (ASR) computer model is trained based on full utterance training data, the ASR computer model having an audio encoder, text predictor, and a joint network which combines outputs of both the audio encoder and the text predictor. Fine-tuning training of the ASR computer model is performed by a knowledge distillation framework at least by: executing a chunking operation on full utterance data to generate a plurality of data chunks corresponding to full utterances in the full utterance data; and executing a knowledge distillation operation with two encoder embeddings. A first encoder embedding is obtained from the full utterance data and a second encoder embedding is obtained from the data chunks. Operational parameters of the trained ASR model are updated based on a loss determined from the first encoder embedding and second encoder embedding.