Concrete Dropout
Explore this paper's citation graph
Summary
This work proposes a new dropout variant which gives improved performance and better calibrated uncertainties, and uses a continuous relaxation of dropout’s discrete masks to allow for automatic tuning of the dropout probability in large models, and as a result faster experimentation cycles.
- Type
- article
- Published
- 2017-05-22
- Cited by
- 679
- References
- 34
- OpenAlex
- https://openalex.org/W2752013927
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:19840332
Keywords
Dropout (neural networks), Computer science, Reinforcement learning, Range (aeronautics), Bayesian probability
References
- Monte Carlo Methods in Financial Engineering
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Improving neural networks by preventing co-adaptation of feature detectors
- Auto-Encoding Variational Bayes
- Likelihood ratio gradient estimation for stochastic systems
- Going deeper with convolutions
- Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding
- Variational Bayesian Inference with Stochastic Search
- Adaptive dropout for training deep neural networks
- A Theoretically Grounded Application of Dropout in Recurrent Neural Networks
- Deep Gaussian Processes for Regression using Approximate Expectation Propagation
- Doubly Stochastic Variational Bayes for non-Conjugate Inference
- Concrete Problems in AI Safety
- Semantic Segmentation of Small Objects and Modeling of Uncertainty in Urban Remote Sensing Images Using Deep Convolutional Neural Networks
- The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation
- Dropout-based Automatic Relevance Determination
- Uncertainty-Aware Reinforcement Learning for Collision Avoidance
- Categorical Reparameterization with Gumbel-Softmax
Cited by
- Deep and Confident Prediction for Time Series at Uber
- DropoutDAgger: A Bayesian Approach to Safe Imitation Learning
- An Equivalence of Fully Connected Layer and Convolutional Layer
- Bayesian Policy Gradients via Alpha Divergence Dropout Inference
- Uncertainty Estimates for Efficient Neural Network-based Dialogue Policy Optimisation
- Dropout Feature Ranking for Deep Learning Models
- SUDS: System for uncertainty decision support
- Quantifying Uncertainty in Discrete-Continuous and Skewed Data with Bayesian Deep Learning
- Synthesizing Neural Network Controllers with Probabilistic Model-Based Reinforcement Learning
- The Lottery Ticket Hypothesis: Training Pruned Neural Networks
- Safe end-to-end imitation learning for model predictive control
- Regularisation of neural networks by enforcing Lipschitz continuity
- Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection
- MaxGain: Regularisation of Neural Networks by Constraining Activation Magnitudes
- Siamese Capsule Networks
- Excitation Dropout: Encouraging Plasticity in Deep Neural Networks
- Pushing the bounds of dropout
- Deep Gaussian Process autoencoders for novelty detection
- Machine Learning Based Uplink Transmission Power Prediction for LTE and Upcoming 5G Networks Using Passive Downlink Indicators
- Sufficient Conditions for Idealised Models to Have No Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networks
Related papers
- What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
- Deep Residual Learning for Image Recognition
- Dropout: a simple way to prevent neural networks from overfitting
- Variational Dropout and the Local Reparameterization Trick
- Weight Uncertainty in Neural Networks
- Practical Variational Inference for Neural Networks
- Weight Uncertainty in Neural Network