Layer Normalization

Keywords

Normalization (sociology), Computer science, Artificial neural network, Computation, Artificial intelligence

References

• Path-SGD: Path-Normalized Optimization in Deep Neural NetworksBehnam Neyshabur, Ruslan Salakhutdinov, Nathan Srebro — 2015
• Unifying Visual-Semantic Embeddings with Multimodal Neural Language ModelsRyan Kiros, Ruslan Salakhutdinov, Richard S. Zemel — 2014
• Teaching Machines to Read and ComprehendKarl Moritz Hermann, Tomáš Kočiský, Edward Grefenstette — 2015
• Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading BooksYukun Zhu, Ryan Kiros, Rich Zemel — 2015
• Efficient Estimation of Word Representations in Vector SpaceTomáš Mikolov, Kai Chen, Greg S. Corrado — 2013
• Very Deep Convolutional Networks for Large-Scale Image RecognitionKaren Simonyan, Andrew Zisserman — 2014
• Generating Sequences With Recurrent Neural NetworksBacciu, Davide, Gervasi, Vincenzo, Prencipe, Giuseppe — 2013
• Natural Gradient Works Efficiently in LearningШун-ичи Амари — 1998
• Annotating Expressions of Opinions and Emotions in LanguageJanyce Wiebe, Theresa Wilson, Claire Cardie — 2005
• A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum CutsBo Pang, Lillian Lee — 2004
• The Neural Autoregressive Distribution EstimatorHugo Larochelle, Iain Murray — 2011
• IAM-OnDB - an on-line English sentence database acquired from handwritten text on a whiteboardMarcus Liwicki, Horst Bunke — 2005
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine TranslationKyunghyun Cho, Bart van Merriënboer, Çağlar Gülçehre — 2014
• Mining and summarizing customer reviewsMinqing Hu, Bing Liu — 2004
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research GroupsGeoffrey E. Hinton, Li Deng, Dong Yu — 2012
• Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating ScalesBo Pang, Lillian Lee — 2005
• ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton — 2012
• Large Scale Distributed Deep NetworksJay B. Dean, Greg S. Corrado, Rajat Monga — 2012
• Deep Speech 2 : End-to-End Speech Recognition in English and MandarinDario Amodei, Rishita Anubhai, Eric Battenberg — 2015
• Batch normalized recurrent neural networksCésar Laurent, Gabriel Pereyra, Philémon Brakel — 2015

Cited by

• Learning Two-Branch Neural Networks for Image-Text Matching Tasks賢治 木村, 亜紀子 山形, 喜子 木田 — 2017
• Uncovering Thousands of New Peptides with Sequence-Mask-Search Hybrid De Novo Peptide Sequencing Framework*王高学, 白占涛 — 2019
• Dynamic Neural Turing Machine with Soft and Hard Addressing SchemesÇağlar Gülçehre, Sarath Chandar, Kyunghyun Cho — 2016
• Adjusting for Dropout Variance in Batch Normalization and Weight InitializationDan Hendrycks, Kevin Gimpel — 2016
• Hierarchical Multiscale Recurrent Neural NetworksJun‐Young Chung, Sungjin Ahn, Yoshua Bengio — 2016
• Geometry-based next frame prediction from monocular videoReza Mahjourian, Martin Wicke, Anelia Angelova — 2016
• Deep visual foresight for planning robot motionChelsea Finn, Sergey Levine — 2016
• Video Captioning and Retrieval Models with Semantic AttentionYoungjae Yu, Hyungjin Ko, Jongwook Choi — 2016
• Streaming Normalization: Towards Simpler and More Biologically-plausible Normalizations for Online and Recurrent LearningQianli Liao, Kenji Kawaguchi, Tomaso Poggio — 2016
• Neural Machine Translation in Linear TimeNal Kalchbrenner, Lasse Espeholt, Karen Simonyan — 2016
• Surprisal-Driven ZoneoutKamil Rocki, Tomasz Kornuta, Tegan Maharaj — 2016
• Deep Convolutional Neural Network Design PatternsLeslie N. Smith, Nicholay Topin — 2016
• Recurrent Memory Addressing for Describing VideosArnav Jain, Abhinav Agarwalla, Kumar Krishna Agrawal — 2016
• Multi-task learning with deep model based reinforcement learningAsier Mujika — 2016
• Multi-modal Variational Encoder-DecodersIulian Vlad Serban, Alexander G. Ororbia, Joëlle Pineau — 2017
• Few-Shot Object Recognition from Machine-Labeled Web ImagesZhongwen Xu, Linchao Zhu, Yi Yang — 2016
• An empirical analysis of the optimization of deep network loss surfacesDaniel Jiwoong Im, Michael Tao, Kristin Branson — 2016
• End-to-End Concept Word Detection for Video Captioning, Retrieval, and Question AnsweringYoungjae Yu, Hyungjin Ko, Jongwook Choi — 2016
• Continuous multilinguality with language vectorsRobert Östling, Jörg Tiedemann — 2016
• Normalizing the Normalizers: Comparing and Extending Network Normalization SchemesMengye Ren, Renjie Liao, Raquel Urtasun — 2016

Related papers

• BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingJacob Devlin, Ming‐Wei Chang, Kenton Lee — 2019
• GloVe: Global Vectors for Word RepresentationJeffrey Pennington, Richard Socher, Christopher D. Manning — 2014
• Deep Residual Learning for Image RecognitionKaiming He, Xiangyu Zhang, Shaoqing Ren — 2015
• ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton — 2012
• Dropout: a simple way to prevent neural networks from overfittingNitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky — 2014
• Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber — 1997