Multimodal Deep Learning

Keywords

Computer science, Modalities, Feature learning, Artificial intelligence, Deep learning

References

• Framing Image Description as a Ranking Task: Data, Models and Evaluation MetricsMicah Hodosh, Peter Young, Julia Hockenmaier — 2013
• See me, hear me: integrating automatic speech recognition and lip-readingPaul Duchnowski, Uwe Meier, Alex Waibel — 1994
• Adaptive multimodal fusion by uncertainty compensationVassilis Pitsikalis, Athanassios Katsamanis, George Papandreou — 2006
• The challenge of multispeaker lip-readingStephen Cox, Richard Harvey, Yuxuan Lan — 2008
• Hearing lips and seeing voicesHarry McGurk, John MacDonald — 1976
• Lipreading and audio-visual speech perception.Quentin Summerfield — 1992
• Extracting and composing robust features with denoising autoencodersPascal Vincent, Hugo Larochelle, Yoshua Bengio — 2008
• CUAVE: A new audio-visual database for multimodal human-computer interface researchEric Patterson, S. Gurbuz, Zekeriya TÜFEKCİ — 2002
• Canonical Correlation Analysis: An Overview with Application to Learning MethodsDavid R. Hardoon, Sándor Szedmák, John Shawe‐Taylor — 2004
• Extraction of Visual Features for LipreadingIain Matthews, T.F. Cootes, Jenny Bangham — 2002
• Training Products of Experts by Minimizing Contrastive DivergenceGeoffrey E. Hinton — 2002
• Self-taught learning: transfer learning from unlabeled dataRajat Raina, Alexis Battle, Honglak Lee — 2007
• Information Theoretic Feature Extraction for Audio-Visual Speech RecognitionMihai Gurban, Jean‐Philippe Thiran — 2009
• Sparse deep belief net model for visual area V2Honglak Lee, Chaitanya Ekanadham, Andrew Y. Ng — 2007
• A Fast Learning Algorithm for Deep Belief NetsGeoffrey E. Hinton, Simon Osindero, Yee‐Whye Teh — 2006
• Adaptive Multimodal Fusion by Uncertainty Compensation With Application to Audiovisual Speech RecognitionGeorge Papandreou, Athanasios Katsamanis, Vassilis Pitsikalis — 2009
• Integration of acoustic and visual speech signals using neural networksBen Yuhas, Moïse H. Goldstein, Terrence J. Sejnowski — 1989
• Adaptive bimodal sensor fusion for automatic speechreadingUwe Meier, Wolfgang Hürst, Paul Duchnowski — 1996
• "Eigenlips" for robust speech recognitionChristoph Bregler, Yochai Konig — 1994
• Histograms of oriented gradients for human detectionNavneet Dalal, Bill Triggs — 2005

Cited by

• A multiple kernel classification approach based on a Quadratic Successive Geometric Segmentation methodology with a fault diagnosis case.Irina Kataeva — 2018
• Stacked Denoising Tensor Auto-Encoder for Action Recognition With Spatiotemporal CorruptionsМакеев Валерий Александрович — 2018
• Learning Two-Branch Neural Networks for Image-Text Matching Tasks賢治 木村, 亜紀子 山形, 喜子 木田 — 2017
• Not all signals are created equal : Dynamic objective auto-encoder for multivariate dataMartin Längkvist, Amy Loutfi — 2012
• An Overview of Deep-Structured Learning for Information ProcessingLi Deng — 2011
• Multimodal learning with deep Boltzmann machinesNitish Srivastava, Ruslan Salakhutdinov — 2012
• Learning joint representation for community question answering with tri-modal DBMBaolin Peng, Wenge Rong, Yuanxin Ouyang — 2014
• Online Multi-Stage Deep Architectures for Feature Extraction and Object RecognitionDerek C. Rose — 2013
• Apprentissage de représentations et robotique développementale : quelques apports de l'apprentissage profond pour la robotique autonome. (Representation learning and developmental robotics : on the use of deep learning for autonomous robots)Alain Droniou — 2015
• Modeling the Variability in Brain Morphology and Lesion Distribution in Multiple Sclerosis by Deep LearningTom Brosch, Youngjin Yoo, David K.B. Li — 2014
• Learning Representations with a Dynamic Objective Sparse AutoencoderMartin Längkvist, Amy Loutfi — 2012
• Combining heterogeneous deep neural networks with conditional random fields for Chinese dialogue act recognitionYucan Zhou, Qinghua Hu, Jie Liu — 2015
• Learning Multiple Tasks with Multilinear Relationship NetworksMingsheng Long, Jianmin Wang, Philip S. Yu — 2015
• Learning Deep Representations : Toward a better new understanding of the deep learning paradigm. (Apprentissage de représentations profondes : vers une meilleure compréhension du paradigme d'apprentissage profond)Ludovic Arnold — 2013
• Cross Modal Distillation for Supervision TransferSaurabh Gupta, Judy Hoffman, Jitendra Malik — 2015
• Correlational Neural NetworksSarath Chandar, Mitesh M. Khapra, Hugo Larochelle — 2015
• Integrating articulatory data in deep neural network-based acoustic modelingLeonardo Badino, Claudia Canevari, Luciano Fadiga — 2016
• Big Data Analytics in Bioinformatics: A Machine Learning PerspectiveHirak J. Kashyap, Hasin A. Ahmed, Nazrul Hoque — 2015
• Effective deep learning-based multi-modal retrievalWei Wang, Xiaoyan Yang, Beng Chin Ooi — 2015
• The Dynamic Brain: Modeling Neural Dynamics and Interactions From Human Electrophysiological RecordingsTimothy Roger Mullen — 2014

Related papers

• Deep LearningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton — 2015
• Multimodal Machine Learning: A Survey and TaxonomyTadas Baltrušaitis, Chaitanya Ahuja, Louis‐Philippe Morency — 2017
• Deep Residual Learning for Image RecognitionKaiming He, Xiangyu Zhang, Shaoqing Ren — 2015
• ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton — 2012
• A Fast Learning Algorithm for Deep Belief NetsGeoffrey E. Hinton, Simon Osindero, Yee‐Whye Teh — 2006
• Gradient-based learning applied to document recognitionYann LeCun, Léon Bottou, Yoshua Bengio — 1998
• ImageNet: A large-scale hierarchical image databaseJia Deng, Wei Dong, Richard Socher — 2009
• A new approach to cross-modal multimedia retrievalNikhil Rasiwasia, José Costa Pereira, Emanuele Coviello — 2010