Weight-averaged consistency targets improve semi-supervised deep learning results
Explore this paper's citation graph
Summary
This work reports state-of-the-art results on semi-supervised SVHN, and proposes a method that averages model weights instead of label predictions that improves test accuracy and enables training with fewer labels than earlier methods.
- Type
- article
- Published
- 2017-03-06
- Cited by
- 1,220
- References
- 28
- Access
- Open access
- OpenAlex
- https://openalex.org/W2592691248
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:2759724
Keywords
Computer science, Consistency (knowledge bases), Artificial intelligence, Residual, Machine learning
References
- Regularization of Neural Networks using DropConnect
- Learning from labeled and unlabeled data with label propagation
- Distilling the Knowledge in a Neural Network
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Auto-Encoding Variational Bayes
- Acceleration of stochastic approximation by averaging
- Dropout: a simple way to prevent neural networks from overfitting
- Deep learning via semi-supervised embedding
- Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks
- Model compression
- Reading Digits in Natural Images with Unsupervised Feature Learning
- Improved Techniques for Training GANs
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Semi-supervised Learning with Ladder Networks
- Swapout: Learning an ensemble of deep architectures
- Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning
- Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
- Rectifier Nonlinearities Improve Neural Network Acoustic Models
- Explaining and Harnessing Adversarial Examples
- Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
Cited by
- Semisupervised Text Classification by Variational Autoencoder
- Stochastic Function Norm Regularization of Deep Networks
- Self-ensembling for domain adaptation
- Function Norms and Regularization in Deep Networks
- VisDA: The Visual Domain Adaptation Challenge
- Self-ensembling for visual domain adaptation
- Virtual Adversarial Ladder Networks For Semi-supervised Learning
- Semi-Supervised and Active Few-Shot Learning with Prototypical Networks
- A Semi-Supervised Two-Stage Approach to Learning from Noisy Labels
- On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks
- Knowledge Distillation in Generations: More Tolerant Teachers Educate Better Students
- Learning Neural Random Fields with Inclusive Auxiliary Generators
- Improving Consistency-Based Semi-Supervised Learning with Weight Averaging
- Manifold Mixup: Encouraging Meaningful On-Manifold Interpolation as a Regularizer
- Semi-Supervised Learning with Uncertainty
- Tangent-Normal Adversarial Regularization for Semi-Supervised Learning
- Semi-Supervised Sequence Modeling with Cross-View Training
- Domain Confusion with Self Ensembling for Unsupervised Adaptation
- Selective Distillation of Weakly Annotated GTD for Vision-Based Slab Identification System
- Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization