Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Explore this paper's citation graph
Summary
Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin.
- Type
- preprint
- Published
- 2015-02-10
- Cited by
- 47,615
- References
- 54
- Access
- Open access
- OpenAlex
- https://openalex.org/W1836465849
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:5808102
Keywords
Covariate, Normalization (sociology), Computer science, Training (meteorology), Econometrics
References
- On the importance of initialization and momentum in deep learning
- Large Scale Visual Recognition
- You Only Look Once: Unified, Real-Time Object Detection
- Understanding the difficulty of training deep feedforward neural networks
- Deep Image: Scaling up Image Recognition
- Rectified Linear Units Improve Restricted Boltzmann Machines
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Parallel training of Deep Neural Networks with Natural Gradient and Parameter Averaging
- Knowledge Matters: Importance of Prior Information for Optimization
- On the difficulty of training recurrent neural networks
- Critical Points of the Singular Value Decomposition
- Toward a perception-based theory of probabilistic reasoning with imprecise probabilities
- Mean-normalized stochastic gradient for large-scale deep learning
- Dropout: a simple way to prevent neural networks from overfitting
- Going deeper with convolutions
- Gradient-based learning applied to document recognition
- ImageNet Large Scale Visual Recognition Challenge
- Independent component analysis: algorithms and applications
- Nonlinear image representation using divisive normalization
Cited by
- A Deep Learning Based Strategy for Identifying and Associating Mitotic Activity with Gene Expression Derived Risk Categories in Estrogen Receptor Positive Breast Cancers
- Generalizing Pooling Functions in CNNs: Mixed, Gated, and Tree
- Spatiotemporal Recurrent Convolutional Networks for Traffic Prediction in Transportation Networks
- DEEPre: sequence-based enzyme EC number prediction by deep learning
- Deep learning analysis of the myocardium in coronary CT angiography for identification of patients with functionally significant coronary artery stenosis
- Skin Lesion Analysis towards Melanoma Detection Using Deep Learning Network
- Learning Two-Branch Neural Networks for Image-Text Matching Tasks
- Interpreting Deep Visual Representations via Network Dissection
- Prognostic Gene Discovery in Glioblastoma Patients using Deep Learning
- LeukocyteMask: An automated localization and segmentation method for leukocyte in blood smear images using deep neural networks
- Semisupervised Text Classification by Variational Autoencoder
- Convolutional Mixture of Experts Model: A Comparative Study on Automatic Macular Diagnosis in Retinal Optical Coherence Tomography Imaging
- Predicting transgenic markers of a neuron by electrophysiological properties using machine learning.
- A Deep Learning Reconstruction Framework for Differential Phase-Contrast Computed Tomography With Incomplete Data
- Weakly Supervised Learning of Placental Ultrasound Images with Residual Networks
- Deep Learning for Medical Image Segmentation
- Unsupervised Visual Representation Learning by Context Prediction
- Path-SGD: Path-Normalized Optimization in Deep Neural Networks
- Stacked What-Where Auto-encoders
- Correlational Neural Networks
Related papers
- Sparse classification with paired covariates
- An introduction to the full random effects model
- Covariate Adjustment Strategy Increases Power in the Randomized Controlled Trial With Discrete-Time Survival Endpoints
- Survival analysis with coarsely observed covariates
- Compensation and Amplification of Attenuation Bias in Causal Effect Estimates
- Models for time-varying covariates in population pharmacokinetic-pharmacodynamic analysis.
- Covariate Imbalance and Precision in Measuring Treatment Effects
- PNS15 COVARIATE ADJUSTMENT IN INDIRECT TREATMENT COMPARISON (ITC): HOW TO BLEND THE USEFUL WITH THE AGREEABLE
- The Effect of a Covariate on Standard Error and Confidence Interval Width