DNN/CNN Acoustic Model Turbo Fusion for Phoneme Recognition

Abstract:
This work presents phoneme recognition results on the fusion of DNN- and CNN-based acoustic models, both individually trained, using the recently proposed turbo fusion approach. Our contribution is two-fold: For the first time, turbo fusion is applied on a multi-model task, creating the opportunity to perform information fusion in a singlemodality and single-feature representation task. Second, we propose a two-stage optimization procedure to the turbo fusion approach by including both the choice of the optimal start component recognizer and the optimal stop-iteration. Experiments on the classical TIMIT benchmark reveal two insights: Combining DNN and CNN architectures yields performance gains and among all modular reference fusion methods, turbo fusion performs best while achieving a competetive phoneme error rate of 18.8% on the TIMIT core test set.