Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC

Keywords

Cross-validation, Bayesian probability, Computer science, Model validation, Variable-order Bayesian network

References

• An Asymptotic Equivalence of Choice of Model by Cross‐Validation and Akaike's CriterionM. Stone — 1977
• Bayesian measures of model complexity and fitDavid Spiegelhalter, Nicola Best, Bradley P. Carlin — 2002
• Comparison of Bayesian predictive methods for model selectionJuho Piironen, Aki Vehtari — 2015
• A New and Efficient Estimation Method for the Generalized Pareto DistributionJin Zhang, Michael A. Stephens — 2009
• Case-deletion importance sampling estimators: Central limit theorems and related resultsIlenia Epifani, Steven N. MacEachern, Mario Peruggia — 2008
• A Predictive Approach to Model SelectionSeymour Geisser, William F. Eddy — 1979
• A survey of cross-validation procedures for model selectionSylvain Arlot, Alain Celisse — 2009
• DIC in variable selectionAngelika van der Linde — 2005
• A survey of Bayesian predictive methods for model assessment, selection and comparisonAki Vehtari, Janne Ojanen — 2012
• Strictly Proper Scoring Rules, Prediction, and EstimationTilmann Gneiting, Adrian E. Raftery — 2007
• On the variability of case-deletion importance sampling weights in the Bayesian linear modelMario Peruggia — 1997
• Predictive likelihood for Bayesian model selection and averagingTomohiro Ando, Ruey S. Tsay — 2010
• Sequential Imputations and Bayesian Missing Data ProblemsAugustine Kong, Jun S. Liu, Wing Hung Wong — 1994
• Truncated Importance SamplingEdward L. Ionides — 2008
• Estimation in Parallel Randomized ExperimentsDonald B. Rubin — 1981
• Bayesian Model Assessment and Comparison Using Cross-Validation Predictive DensitiesAki Vehtari, Jouko Lampinen — 2002
• Testing the assumptions behind importance samplingSiem Jan Koopman, Neil Shephard, Drew Creal — 2009
• The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte CarloMatthew D. Hoffman, Andrew Gelman — 2011
• Understanding predictive information criteria for Bayesian modelsAndrew Gelman, Jessica Hwang, Aki Vehtari — 2013
• A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methodsPrabir Burman — 1989

Cited by

• Metapopulation dynamics in a changing climate: Increasing spatial synchrony in weather conditions drives metapopulation synchrony of a butterfly inhabiting a fragmented landscapeBalbino Pardávila — 2018
• A Bayesian latent process spatiotemporal regression model for areal count data.Vincent Mouysset — 2018
• Readers and their roles: Evidence from readers of contemporary fiction in the NetherlandsL. de las Fuentes, I. Garay, Jorge Mario Noguera Berganza — 2018
• A Mixed-Effects Location Scale Model for Dyadic InteractionsH. Fricke, V. Gerold — 2018
• Hidden cost of disease in a free‐ranging ungulate: brucellosis reduces mid‐winter pregnancy in elkElena Rosangela Dellu — 2018
• How Similarity Between Choice Options Affects Decisions From Experience: The Accentuation-of-Differences ModelGeneral William W. Momyer — 2019
• Heterogeneity Matters: Predicting Self-Esteem in Online Interventions Based on Ecological Momentary Assessment DataKatia Fach Gómez — 2019
• Gaze bias differences capture individual choice behaviourK. Torii, Kazuhiko Okuyama, K Kuzume — 2019
• Incorporating the Q-Matrix Into Multidimensional Item Response Theory Models有礼 森 — 2018
• Using degradation models to assess pipeline lifeQinglong Tian, Shiyao Liu, William Q. Meeker — 2019
• Comparison of Bayesian predictive methods for model selectionJuho Piironen, Aki Vehtari — 2015
• Pareto Smoothed Importance SamplingAki Vehtari, Simpson, Daniel, Andrew Gelman — 2015
• Older fathers' children have lower evolutionary fitness across four centuries and in four populationsRuben C. Arslan, Kai P. Willführ, Emma M. Frans — 2017
• When Too Many Vowels Impede Language Processing: An Eye-Tracking Study of Danish-Learning ChildrenFabio Trecca, Dorthe Bleses, Anders Højen — 2019
• Estimating cross‐validatory predictive p‐values with integrated importance sampling for disease mapping modelsLonghai Li, Cindy X. Feng, Shi Qiu — 2016
• Optimal Group Decision: A Matter of Confidence CalibrationSébastien Massoni, Nicolás Roux — 2017
• Posterior Dispersion IndicesAlp Kucukelbir, David M. Blei — 2016
• Among-site variability in the stochastic dynamics of East African coral reefsKatherine A. Allen, John F. Bruno, Fiona Chong — 2016
• Cross-Validation Without Doing Cross-Validation in Genome-Enabled PredictionDaniel Gianola, Chris‐Carolin Schön — 2016
• Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM PackageWoo‐Young Ahn, Nathaniel Haines, Lei Zhang — 2016

Related papers

• A Large‐Scale Empirical Evaluation of Cross‐Validation and External Test Set Validation in (Q)SARMartin Gütlein, Christoph Helma, Andreas Karwath — 2013
• Procrustes cross-validation - a bridge between cross-validation and independent validation set.Sergey Kucheryavskiy, Sergei Zhilin, Oxana Ye. Rodionova — 2020
• Some Case Studies Using Bayesian Statistical ModelsJuan Sosa, Lina Buitrago — 2021
• Select the Order of AR Models by a Bayesian Factor MethodJian Wang — 2007
• Inferring Subjective Prior Knowledge: An Integrative Bayesian ApproachSean Tauber, Mark Steyvers — 2013
• Analysis of k-Fold Cross-Validation over Hold-Out Validation on Colossal Datasets for Quality ClassificationSanjay Kumar Yadav, Sanyam Shukla — 2016