Realistic Deep Learning May Not Fit Benignly
Explore this paper's citation graph
Summary
This work found that for tasks such as training a ResNet model on ImageNet dataset, the model does not do benignly, and highlights the importance of understanding implicit bias in underfitting regimes as a future direction.
- Type
- preprint
- Published
- 2022-06-01
- Cited by
- 1
- References
- 53
- Access
- Open access
- OpenAlex
- https://openalex.org/W4362597605
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:249240395
Keywords
Overfitting, Computer science, Artificial intelligence, Noise (video), Set (abstract data type)
References
- Identifying and Eliminating Mislabeled Training Instances
- EM algorithms of Gaussian mixture model and hidden Markov model
- Distilling the Knowledge in a Neural Network
- Eliminating Class Noise in Large Datasets
- The Elements of Statistical Learning: Data Mining, Inference, and Prediction
- Identifying mislabeled training data with the aid of unlabeled data
- Impossibility of successful classification when useful features are rare and weak
- Deep Residual Learning for Image Recognition
- Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection
- Characterizing Implicit Bias in Terms of Optimization Geometry
- A modern maximum-likelihood theory for high-dimensional logistic regression
- Label Refinery: Improving ImageNet Classification through Label Progression
- Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate
- Just Interpolate: Kernel "Ridgeless" Regression Can Generalize
- SURPRISES IN HIGH-DIMENSIONAL RIDGELESS LEAST SQUARES INTERPOLATION
- Harmless interpolation of noisy data in regression
- High Dimensional Classification via Empirical Risk Minimization: Improvements and Optimality
- Benign overfitting in linear regression
- Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Cited by
Related papers
- Overfitting: Causes and Solutions (Seminar Slides)
- An Overview of Overfitting and its Solutions
- Machine Learning Students Overfit to Overfitting
- Measuring Generalization and Overfitting in Machine Learning
- An Optimal Solution to the Overfitting and Underfitting Problem of Healthcare Machine Learning Models
- Overfitting in linear feature extraction for classification of high-dimensional image data
- Hybrid ensemble learning approach for generation of classification rules