Deep Sparse Rectifier Neural Networks

Keywords

Sigmoid function, Rectifier (neural networks), Tangent, Artificial neural network, Hyperbolic function

References

• Recurrent excitation in neocortical circuitsRodney J. Douglas, Christof Koch, Misha Mahowald — 1995
• Active Deep Networks for Semi-Supervised Sentiment ClassificationShusen Zhou, Qingcai Chen, Xiaolong Wang — 2010
• Multiple Aspect Ranking Using the Good Grief AlgorithmBenjamin Snyder, Regina Barzilay — 2007
• Deep Self-Taught Learning for Handwritten Character RecognitionFrédéric Bastien, Yoshua Bengio, Arnaud Bergeron — 2010
• Understanding the difficulty of training deep feedforward neural networksXavier Glorot, Yoshua Bengio — 2010
• A quantitative theory of immediate visual recognition.T. Serre, Gabriel Kreiman, Minjoon Kouh — 2007
• Rectified Linear Units Improve Restricted Boltzmann MachinesVinod Nair, Geoffrey E. Hinton — 2010
• Extracting and composing robust features with denoising autoencodersPascal Vincent, Hugo Larochelle, Yoshua Bengio — 2008
• The Cortical NeuronMichael J. Gutnick, William O. Whetsell — 1995
• The Cortical NeuronDean Johnston — 1996
• On the piecewise analysis of networks of linear threshold neuronsRichard H. R. Hahnloser — 1998
• An Energy Budget for Signaling in the Grey Matter of the BrainDavid Attwell, Simon B. Laughlin — 2001
• Learning Deep Architectures for AIYoshua Bengio — 2007
• The cost of cortical computation.Peter Lennie — 2003
• A model of multiplicative neural responses in parietal cortex.Emilio Salinas, L. F. Abbott — 1996
• Sparse coding with an overcomplete basis set: a strategy employed by V1?Bruno A. Olshausen, David J. Field — 1997
• Measuring Invariances in Deep NetworksIan Goodfellow, Honglak Lee, Quoc V. Le — 2009
• Sparse Feature Learning for Deep Belief NetworksMarc’Aurelio Ranzato, Y-Lan Boureau, Y. Le Cun — 2007
• Gradient-based learning applied to document recognitionYann LeCun, Léon Bottou, Yoshua Bengio — 1998
• A Theoretical Analysis of Robust Coding over Noisy Overcomplete ChannelsEizaburo Doi, Doru C. Balcan, Michael S. Lewicki — 2005

Cited by

• Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning ApproachXavier Glorot, Antoine Bordes, Yoshua Bengio — 2011
• Segmenting Retinal Blood Vessels With Deep Neural NetworksArmando Llanos Ortíz de Landaluze — 2016
• A deep 3D residual CNN for false-positive reduction in pulmonary nodule detection.David G Lazear — 2018
• Realizing private and practical pharmacological collaboration雅敏 内田 — 2018
• Training improvements for ultrasound beamforming with deep neural networksSonia Tapissier Gicquel — 2019
• A Deep Learning Framework for Identifying Essential Proteins by Integrating Multiple Types of Biological InformationGerald Y. Kinro — 2019
• Data‐driven synthetic MRI FLAIR artifact correction via deep neural networkAstri Sulistiani Risnaedi — 2019
• DIMENSION: Dynamic MR imaging with both k‐space and spatial prior knowledge obtained via multi‐supervised network trainingH.A. Lauwerier — 2018
• Unsupervised Feature Learning and Deep Learning: A Review and New PerspectivesYoshua Bengio, Aaron Courville, Pascal Vincent — 2012
• Parallelized Deep Neural Networks for Distributed Intelligent SystemsLance Legel, Herd, Seth, Brown, Timothy — 2013
• An Overview of Deep-Structured Learning for Information ProcessingLi Deng — 2011
• Online Multi-Stage Deep Architectures for Feature Extraction and Object RecognitionDerek C. Rose — 2013
• Identification and Elucidation of Expression Quantitative Trait Loci (eQTL) and their regulating mechanisms using Decodive Deep LearningMenno J. Witteveen — 2014
• Improving language-universal feature extraction with deep maxout and convolutional neural networksYajie Miao, Florian Metze — 2014
• Deep learning of representations and its application to computer visionIan Goodfellow — 2015
• Learning Deep Representations : Toward a better new understanding of the deep learning paradigm. (Apprentissage de représentations profondes : vers une meilleure compréhension du paradigme d'apprentissage profond)Ludovic Arnold — 2013
• Supervised Learning of Semantics-Preserving Hashing via Deep Neural Networks for Large-Scale Image SearchHuei‐Fang Yang, Kevin Lin, Chu‐Song Chen — 2015
• Feature Representation in Convolutional Neural NetworksBen Athiwaratkun, Keegan Kang — 2015
• Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw ImagesManuel Watter, Jost Tobias Springenberg, Joschka Boedecker — 2015
• Foundations and Advances in Deep LearningKyunghyun Cho — 2014

Related papers

• Efficient Implementation of Activation Functions for LSTM acceleratorsYi Sheng Chong, Wang Ling Goh, Yew-Soon Ong — 2021
• Why tanh: choosing a sigmoidal functionBarry L. Kalman, Stan C. Kwasny — 1992
• FPGA Implementation and Comparison of Sigmoid and Hyperbolic Tangent Activation Functions in an Artificial Neural NetworkA Vaisnav, Sandhya Ashok, Shatharajupally Vinaykumar — 2022
• Smish: A Novel Activation Function for Deep Learning MethodsXueliang Wang, Honge Ren, Achuan Wang — 2022
• Fast implementation of neural network classificationGuiwon Seo, Jiheon Ok, Chulhee Lee — 2013
• Choosing a Sigmoidal FunctionBarry L. Kalman, Stan C. Kwasny — 1992
• Backpropagation Neural Network with Combination of Activation Functions for Inbound Traffic PredictionPurnawansyah Purnawansyah, Haviluddin Haviluddin, Herdianti Darwis — 2021
• Revise Saturated Activation FunctionsBing Xu, Ruitong Huang, Mu Li — 2016
• Analog programmable neuron and case study on VLSI implementation of Multi-Layer Perceptron (MLP)Mahsa Heidari, Hossein Shamsi — 2019
• Efficient hardware implementation of the hyperbolic tangent sigmoid functionAshkan Hosseinzadeh Namin, Karl Leboeuf, Roberto Muscedere — 2009