Investigating Bidirectional Recurrent Neural Network Language Models for Speech Recognition

Keywords

Computer science, Recurrent neural network, Speech recognition, Time delay neural network, Language model

References

• Log-linear interpolation of language modelsDietrich Klakow — 1998
• Recurrent neural network based language modelTomáš Mikolov, Martin Karafiát, Lukáš Burget — 2010
• Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber — 1997
• Two decades of statistical language modeling: where do we go from here?Roni Rosenfeld — 2000
• Lattice-based optimization of sequence classification criteria for neural-network acoustic modelingBrian Kingsbury — 2009
• Bidirectional recurrent neural networksMike Schuster, Kuldip K. Paliwal — 1997
• A survey on the application of recurrent neural networks to statistical language modelingWim De Mulder, Steven Bethard, Marie‐Francine Moens — 2015
• Bidirectional recurrent neural network language models for automatic speech recognitionEbru Arısoy, Abhinav Sethy, Bhuvana Ramabhadran — 2015
• Extensions of recurrent neural network language modelTomáš Mikolov, Stefan Kombrink, Lukáš Burget — 2011
• Fast and Robust Neural Network Joint Models for Statistical Machine TranslationJacob Devlin, Rabih Zbib, Zhongqiang Huang — 2014
• On training bi-directional neural network language model with noise contrastive estimationTianxing He, Yu Zhang, Jasha Droppo — 2016
• Investigation of back-off based interpolation between recurrent neural network and n-gram language modelsXiaoye Chen, Xiaobing Liu, Mark Gales — 2015
• Recurrent Neural Network Based Language Modeling in Meeting RecognitionStefan Kombrink, Tomáš Mikolov, Martin Karafiát — 2011
• Long short-term memory recurrent neural network architectures for large scale acoustic modelingHaşim Sak, Andrew Senior, Françoise Beaufays — 2014
• Two Efficient Lattice Rescoring Methods Using Recurrent Neural Network Language ModelsXunying Liu, Xie Chen, Yongqiang Wang — 2016
• Joint decoding of tandem and hybrid systems for improved keyword spotting on low resource languagesHaipeng Wang, Anton Ragni, Mark Gales — 2015
• CUED-RNNLM — An open-source toolkit for efficient training and evaluation of recurrent neural network language modelsXiaoye Chen, Xiaobing Liu, Yanmin Qian — 2016
• LSTM Neural Networks for Language ModelingMartin Sundermeyer, Ralf Schlüter, Hermann Ney — 2012
• Exploiting the succeeding words in recurrent neural network language modelsYangyang Shi, Martha Larson, Pascal Wiggers — 2013
• Efficient Training and Evaluation of Recurrent Neural Network Language Models for Automatic Speech RecognitionXie Chen, Xunying Liu, Yongqiang Wang — 2016

Cited by

• Recurrent Neural Network Language Model Adaptation for Conversational Speech RecognitionKe Li, Hainan Xu, Yiming Wang — 2018
• Investigation on Estimation of Sentence Probability by Combining Forward, Backward and Bi-directional LSTM-RNNsKazuki Irie, Zhihong Lei, Liuhui Deng — 2018
• Active Memory Networks for Language ModelingOscar Chen, Anton Ragni, Mark Gales — 2018
• Advances in Low Resource ASR: A Deep Learning PerspectiveHardik B. Sailor, Ankur T. Patil, Hemant A. Patil — 2018
• Multiple beamformers with ROVER for the CHiME-5 ChallengeSining Sun, Yangyang Shi, Ching-Feng Yeh — 2018
• Persian Language Modeling Using Recurrent Neural NetworksSeyed Habib Hosseini Saravani, Mohammad Bahrani, Hadi Veisi — 2018
• Unsupervised Pattern Discovery from Thematic Speech Archives Based on Multilingual Bottleneck FeaturesMan-Ling Sung, Siyuan Feng, Tan Lee — 2018
• Improvements to N-gram Language Model Using Text Generated from Neural Language ModelMasayuki Suzuki, Nobuyasu Itoh, Tohru Nagano — 2019
• Effective Sentence Scoring Method using Bidirectional Language Model for Speech RecognitionJoongbo Shin, Yoonhyung Lee, Kyomin Jung — 2019
• Exploiting Future Word Contexts in Neural Network Language Models for Speech RecognitionXie Chen, Xunying Liu, Yu Wang — 2019
• On the use of prior and external knowledge in neural sequence modelsCong Duy Vu Hoang — 2019
• Future word contexts in neural network language modelsXia Chen, Xiaobing Liu, Anton Ragni — 2017
• Improved Training Of Neural Trans-Dimensional Random field Language Models with Dynamic Noise-Contrastive EstimationBin Wang, Zhijian Ou — 2018
• Improved Deep Duel Model for Rescoring N-Best Speech Recognition List Using Backward LSTMLM and Ensemble EncodersAtsunori Ogawa, Marc Delcroix, Shigeki Karita — 2019
• Pseudolikelihood Reranking with Masked Language ModelsJulián Salazar, Davis Liang, Toan Nguyen — 2019
• Effective Sentence Scoring Method Using BERT for Speech RecognitionJoonbo Shin, Yoonhyung Lee, Kyomin Jung — 2019
• Language Modeling Using Part-of-speech and Long Short-Term Memory NetworksSanaz Saki Norouzi, Ahmad Akbari, Babak Nasersharif — 2019
• Generalized Large-Context Language Models Based on Forward-Backward Hierarchical Recurrent Encoder-Decoder ModelsRyo Masumura, Mana Ihori, Tomohiro Tanaka — 2019
• Finnish Language Modeling with Deep Transformer ModelsAbhilash Jain, Ruohe, Aku, Stig-Arne Grönroos — 2020
• Fast and Accurate Deep Bidirectional Language Representations for Unsupervised LearningJoongbo Shin, Yoonhyung Lee, Seunghyun Yoon — 2020

Related papers

• Pattern classification by assembling small neural networksLiang Chen — 2005
• Bidirectional recurrent neural network language models for automatic speech recognitionEbru Arısoy, Abhinav Sethy, Bhuvana Ramabhadran — 2015
• A new recurrent neural network architecture for pattern recognitionHee-Heon Song, Sun-Mee Kang, Seong–Whan Lee — 1996
• The research of digital recognition technology based on bp neural networkXiaofeng Wang — 2013
• 2010 Special Issue: Classification of 2-dimensional array patterns: Assembling many small neural networks is better than using a large oneLiang Chen, Wei Xue, Naoyuki Tokuda — 2010
• Language Identification in Short Utterances Using Long Short-Term Memory (LSTM) Recurrent Neural NetworksRubén Zazo, Alicia Lozano-Díez, Javier Gónzalez-Domínguez — 2016
• Artificial Neural Networks For Pattern Recognize HandwrittenArief Kelik Nugroho, Ipung Permadi, Aini Hanifa — 2020
• New Neural Network Model for Pattern RecognitionZheng Li — 2002