Dropout-based Automatic Relevance Determination

Dmitry Mikhaylov, A. Arseniy, Dmitry Gnatyshak · Neural Information Processing Systems · 2016

Summary

The main idea of Variational dropout is to search for posterior approximation in a specific family of distributions: q(wi) = N (θi, αiθ i ) and the reparameterization trick for this model is equivalent to plain Gaussian dropout.

Type: article
Published: 2016-01-01
Cited by: 2
References: 10
OpenAlex: https://openalex.org/W2565480690
Semantic Scholar: https://api.semanticscholar.org/CorpusID:63280336

Keywords

Dropout (neural networks), Relevance (law), Computer science, Artificial intelligence, Machine learning

References

A Bayesian encourages dropoutShin‐ichi Maeda — 2014
Automatic Relevance Determination For Deep Generative ModelsTheofanis Karaletsos, Gunnar Rätsch — 2015
Auto-Encoding Variational BayesDiederik P. Kingma, Max Welling — 2013
Dropout: a simple way to prevent neural networks from overfittingNitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky — 2014
Gradient-based learning applied to document recognitionYann LeCun, Léon Bottou, Yoshua Bengio — 1998
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep LearningYarin Gal, Zoubin Ghahramani — 2015
Adam: A Method for Stochastic OptimizationDiederik P. Kingma, Jimmy Ba — 2014
Variational Dropout and the Local Reparameterization TrickDiederik P. Kingma, Tim Salimans, Max Welling — 2015
Sparse Bayesian Learning and the Relevan e Ve tor Ma hineG. House, Guildhall StreetCambridge — 2001

Cited by

Concrete DropoutYarin Gal, Jiri Hron, Alex Kendall — 2017
Concrete DropoutYarin Gal, Jiri Hron, Alex Kendall — 2017
Structured Bayesian Pruning via Log-Normal Multiplicative NoiseKirill Neklyudov, Dmitry Molchanov, Arsenii Ashukha — 2017

Related papers

No related papers recorded.