Estimation of Non-Normalized Statistical Models by Score Matching

Keywords

Mathematics, Normalization (sociology), Density estimation, Probability density function, Applied mathematics

References

• On Contrastive Divergence LearningMiguel Á. Carreira-Perpiñán, Geoffrey E. Hinton — 2005
• The theory of statisticsF. Y. Edgeworth, Charles R. McCann — 1996
• Estimating Overcomplete Independent Component Bases for Image WindowsAapo Hyvärinen, Mika Inki — 2002
• Markov Random Field Modeling in Image AnalysisStan Z. Li — 2001
• Efficiency of pseudolikelihood estimation for simple Gaussian fieldsJulian Besag — 1977
• Sparse coding with an overcomplete basis set: a strategy employed by V1?Bruno A. Olshausen, David J. Field — 1997
• Spatial Interaction and the Statistical Analysis of Lattice SystemsJulian Besag — 1974
• Training Products of Experts by Minimizing Contrastive DivergenceGeoffrey E. Hinton — 2002
• A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images.Aapo Hyvärinen, Patrik O. Hoyer — 2001
• Blind separation of mixture of independent sources through a quasi-maximum likelihood approachDinh Tuan Pham, Philippe Garat — 1997
• Information Theory, Inference, and Learning AlgorithmsD. MacKay — 2004
• A generalized Gaussian image model for edge-preserving MAP estimationCharles A. Bouman, K. Sauer — 1993
• Topographic Independent Component AnalysisAapo Hyvärinen, Patrik O. Hoyer, Mika Inki — 2001
• Information theory, inference, and learning algorithms2004

Cited by

• Local Proper Scoring RulesWerner Ehm, Tilmann Gneiting — 2009
• Unsupervised Feature Learning and Deep Learning: A Review and New PerspectivesYoshua Bengio, Aaron Courville, Pascal Vincent — 2012
• Two Distributed-State Models For Generating High-Dimensional Time SeriesGraham W. Taylor, Geoffrey E. Hinton, Sam T. Roweis — 2011
• Probabilistic Models of Phase Variables for Visual Representation and Neural DynamicsCharles F. Cadieu — 2009
• Estimating Dependency Structures for non-Gaussian Components with Linear and Energy CorrelationsHiroaki Sasaki, Hayaru Shouno, Michael U. Gutmann — 2014
• Higher Order Correlations within Cortical Layers Dominate Functional Connectivity in MicrocolumnsUrs Köster, Jascha Sohl‐Dickstein, Charles M. Gray — 2012
• Learning Deep Representations : Toward a better new understanding of the deep learning paradigm. (Apprentissage de représentations profondes : vers une meilleure compréhension du paradigme d'apprentissage profond)Ludovic Arnold — 2013
• Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential FamiliesHeiko Strathmann, Dino Sejdinović, Samuel Livingstone — 2015
• Learning generative models of mid-level structure in natural imagesNicolas Heess — 2012
• High-dimensional inference of graphical models using regularized score matchingLina Lin, Mathias Drton, Ali Shojaie — 2015
• Foundations and Advances in Deep LearningKyunghyun Cho — 2014
• Decomposition Techniques for Learning Graphical ModelsKhaled S. Refaat — 2015
• Provable Tensor Methods for Learning Mixtures of ClassifiersHanie Sedghi, Anima Anandkumar — 2014
• Adaptive Parallel Tempering for Stochastic Maximum Likelihood Learning of RBMsGuillaume Desjardins, Aaron Courville, Yoshua Bengio — 2010
• A Probabilistic Approach to the Primary Visual CortexUrs Köster — 2009
• A review of mean-shift algorithms for clusteringMiguel Á. Carreira-Perpiñán — 2015
• On the Convergence Properties of Contrastive DivergenceIlya Sutskever, Tijmen Tieleman — 2010
• On autoencoder scoringHanna Kamyshanska, Roland Memisevic — 2013
• Herding Dynamic Weights for Partially Observed Random Field ModelsMax Welling — 2009
• Learning Representations by Maximizing CompressionKarol Gregor, Yann LeCun — 2011

Related papers

• Proper local scoring rulesParry, M, Dawid, AP, Lauritzen, S — 2011
• Generative Modeling by Estimating Gradients of the Data DistributionYang Song, Stefano Ermon — 2019
• Bayesian Learning via Stochastic Gradient Langevin DynamicsMax Welling, Yee Whye Teh — 2011
• A Tutorial on Energy-Based LearningYann LeCun, Sumit Chopra, Raia Hadsell — 2006
• Noise-contrastive estimation: A new estimation principle for unnormalized statistical modelsMichael U. Gutmann, Aapo Hyvärinen — 2010
• A Fast Learning Algorithm for Deep Belief NetsGeoffrey E. Hinton, Simon Osindero, Yee‐Whye Teh — 2006
• Training restricted Boltzmann machines using approximations to the likelihood gradientTijmen Tieleman — 2008
• Training Products of Experts by Minimizing Contrastive DivergenceGeoffrey E. Hinton — 2002
• Generative Adversarial Nets柴田 淳司 — 2014