Improving transparency and replication in Bayesian statistics: The WAMBS-Checklist.
Open accessThe DOI matches its Crossref record; 97 of the 97 references it cites were found in a registry or an index.
Abstract
Bayesian statistical methods are slowly creeping into all fields of science and are becoming ever more popular in applied research. Although it is very attractive to use Bayesian statistics, our personal experience has led us to believe that naively applying Bayesian methods can be dangerous for at least 3 main reasons: the potential influence of priors, misinterpretation of Bayesian features and results, and improper reporting of Bayesian results. To deal with these 3 points of potential danger, we have developed a succinct checklist: the WAMBS-checklist (When to worry and how to Avoid the Misuse of Bayesian Statistics). The purpose of the questionnaire is to describe 10 main points that should be thoroughly checked when applying Bayesian analysis. We provide an account of "when to worry" for each of these issues related to: (a) issues to check before estimating the model, (b) issues to check after estimating the model but before interpreting results, (c) understanding the influence of priors, and (d) actions to take after interpreting results. To accompany these key points of concern, we will present diagnostic tools that can be used in conjunction with the development and assessment of a Bayesian model. We also include examples of how to interpret results when "problems" in estimation arise, as well as syntax and instructions for implementation. Our aim is to stress the importance of openness and transparency of all aspects of Bayesian estimation, and it is our hope that the WAMBS questionnaire can aid in this process. (PsycINFO Database Record
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, the relative novelty of conducting Bayesian analyses in applied fields means that there are no detailed reporting standards, and this in turn may frustrate the broader adoption and proper interpretation of the Bayesian framework.”
From The JASP guidelines for conducting and reporting a Bayesian analysis (2021) · cites this paper as Depaoli & Schoot, 2017
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Stochastic relaxation, gibbs distributions, and the bayesian restoration of images.
Geman, S., & Geman, D. (1984) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1984.4767596 · Cited by 17,949 · cited 1×
“7 The idea behind the Gibbs sampler is that the conditional distribution of one set of parameters given other sets can be used to make random draws of parameter values (see for more information about the Gibbs sampler: Geman & Geman, 1984; Casella & George, 1992).”
Inference from iterative simulation using multiple sequences
Gelman, A., & Rubin, D. B. (1992) · Statistical Science · doi:10.1214/ss/1177011136 · Cited by 16,627 · cited 1×
“Another method that can be used to examine the similarity of results obtained across models with two sets of priors specified is to use the Gelman and Rubin (1992a, 1992b) convergence diagnostic.”
Data Analysis Using Regression and Multilevel/Hierarchical Models
Gelman, A., & Hill, J. (2006) · doi:10.1017/cbo9780511790942 · Cited by 13,597
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,368
JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling
Plummer, M. (2003) · Cited by 4,720
Multilevel and Longitudinal Modeling Using Stata
(2006) · Biometrics · doi:10.1111/j.1541-0420.2006.00588_16.x · Cited by 4,236 · cited 1×
“Further, some models (e.g., mixture or multilevel models) require Bayesian methods to improve convergence issues (Depaoli & Clifton, 2015; Skrondal & Rabe-Hesketh, 2012), aid in model identification (Kim, Suh, Kim, Albanese, & Langer, 2013), and produce more accurate parameter estimates (Depaoli,…”
Gelman, A. (2006) · Bayesian Analysis · doi:10.1214/06-ba117a · Cited by 4,050
Casella, G., & George, E. I. (1992) · The American Statistician · doi:10.1080/00031305.1992.10475878 · Cited by 2,364 · cited 1×
“7 The idea behind the Gibbs sampler is that the conditional distribution of one set of parameters given other sets can be used to make random draws of parameter values (see for more information about the Gibbs sampler: Geman & Geman, 1984; Casella & George, 1992).”
On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion)
Richardson, S., & Green, P. J. (1997) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00095 · Cited by 1,898
A weakly informative default prior distribution for logistic and other regression models
Gelman, A., Jakulin, A., Pittau, M. G., & Su, Y.-S. (2008) · The Annals of Applied Statistics · doi:10.1214/08-aoas191 · Cited by 1,790 · cited 1×
“Essentially, weakly informative priors do not supply any strict information, but yet are still strong enough to avoid inappropriate inferences that can be produced from a noninforma-tive prior (Gelman, Jakulin, Pittau, & Su, 2008).”
Prior distributions for variance parameters in hierarchical modelsPreprint
Gelman, A. (2004) · RePEc: Research Papers in Economics · Cited by 1,632
Bayesian structural equation modeling: A more flexible representation of substantive theory
Muthén, B., & Asparouhov, T. (2012) · Psychological Methods · doi:10.1037/a0026802 · Cited by 1,562 · cited 1×
“First, some complex models simply cannot be estimated using conventional statistics (see, e.g., Muthén & Asparouhov, 2012; Kruschke, 2010, 2011; Wetzels, Matzke, Lee, Rouder, Iverson & Wagenmakers, 2011).”
Uncertain Judgements: Eliciting Experts' Probabilities
O'Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. R., Garthwaite, P. H., Jenkinson, D. J., Oakley, J. E., & Rakow, T. (2006) · doi:10.1002/0470033312 · Cited by 1,550
A WEAKLY INFORMATIVE DEFAULT PRIOR DISTRIBUTION FOR LOGISTIC AND OTHER REGRESSION MODELS
Gelman, A., Jakulin, A., & Grazia, M. (2009) · Cited by 1,352
Simulation Run Length Control in the Presence of an Initial Transient
Heidelberger, P., & Welch, P. D. (1983) · Operations Research · doi:10.1287/opre.31.6.1109 · Cited by 1,297 · cited 1×
“…implemented convergence diagnostics in programs such as R; for example, the Geweke diagnostic (Geweke, 1992), the Heidelberger and Welch diagnostic (Heidelberger & Welch, 1983), and the Raftery and Lewis diagnostic for determining the length of the burn-in and postburn-in portions of the chain…”
BayesFactor: Computation of Bayes Factors for Common Designs
Morey, R. D., & Rouder, J. N. (2012) · CRAN: Contributed Packages · doi:10.32614/cran.package.bayesfactor · Cited by 1,173
Nedelman, J. R. (2005) · Computational Statistics · doi:10.1007/bf02741321 · Cited by 1,078
(1991) · University of Minnesota Digital Conservancy (University of Minnesota) · Cited by 1,077
Bayesian Modeling Using WinBUGS
Ntzoufras, I. (2009) · Wiley eBooks · doi:10.1002/9780470434567 · Cited by 1,001
Statistical Evidence in Experimental Psychology: An Empirical Comparison Using 855 t Tests
Wetzels, R., Matzke, D., Lee, M. D., Rouder, J. N., Iverson, G. J., & Wagenmakers, E.-J. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406923 · Cited by 945
Bayesian Versus Orthodox Statistics: Which Side Are You On?
Dienes, Z. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406920 · Cited by 938 · cited 1×
“Second, many scholars prefer Bayesian statistics because they believe population parameters should be viewed as random (see, e.g., Dienes, 2011; van de Schoot et al., 2011).”
How Many Iterations in the Gibbs Sampler?
Raftery, A. E., & Lewis, S. (1991) · doi:10.21236/ada640705 · Cited by 937 · cited 1×
“…diagnostics in programs such as R; for example, the Geweke diagnostic (Geweke, 1992), the Heidelberger and Welch diagnostic (Heidelberger & Welch, 1983), and the Raftery and Lewis diagnostic for determining the length of the burn-in and postburn-in portions of the chain (Raftery & Lewis, 1992).”
On Bayesian Analysis of Mixtures with an Unknown Number of Components
Richardson, S. T., & Green, P. J. (1997) · Cited by 919
Bayesian Methods for Data Analysis
Carlin, B. P., & Louis, T. A. (2008) · Chapman & Hall/CRC eBooks · doi:10.1201/b14884 · Cited by 917
Casella, G., & George, E. I. (1992) · The American Statistician · doi:10.2307/2685208 · Cited by 863
Eliciting Expert Knowledge in Conservation Science
MARTIN, T. G., BURGMAN, M. A., FIDLER, F., KUHNERT, P. M., LOW‐CHOY, S., MCBRIDE, M., & MENGERSEN, K. (2012) · Conservation Biology · doi:10.1111/j.1523-1739.2011.01806.x · Cited by 829 · cited 1×
“…can ask an expert, or a panel of experts, to provide an estimate for the hyperparameters based on knowledge of the field; see, for example: Bijak and Wisniowski (2010); Fransman et al. (2011); Howard, Maxwell, and Fleming (2000); Martin et al. (2012); and Morris, Oakley, and Crowe (2014).”
Philosophy and the practice of Bayesian statistics
Gelman, A., & Shalizi, C. R. (2013) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.2011.02037.x · Cited by 825 · cited 1×
“…have an impact on parameter estimates and therefore also on substantive findings; for details on the different ways in which priors can adversely impact findings, see: Depaoli (2013); Gelman and Shalizi (2013); Johnson (2013); Seaman, Seaman, and Stamey (2012); and van de Schoot and Depaoli (2014).”
Bayesian Analysis for the Social Sciences
Jackman, S. (2009) · Wiley Series in Probability and Statistics · doi:10.1002/9780470686621 · Cited by 756 · cited 2×
“…novice reader to, among many others: Bolstad (2007); Carlin and Louis (2009); Christensen, Johnson, Branscum, and Hanson (2010); Depaoli and Boyajian, (2014); Gelman and Hill (2007); Kaplan, 2014; Kruschke (2010); Jackman (2009); Lynch (2007); Ntzoufras (2009); or van de Schoot and Depaoli (2014).”
Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
GEMAN, S., & GEMAN, D. (1987) · Readings in Computer Vision · doi:10.1016/b978-0-08-051581-6.50057-x · Cited by 750
StataCorp (2011) · Stata Press/StataCorp eBooks · Cited by 734
The case for objective Bayesian analysis
Berger, J. (2006) · Bayesian Analysis · doi:10.1214/06-ba115 · Cited by 728 · cited 8×
“It is important to note that there are several other commonly implemented convergence diagnostics in programs such as R; for example, the Geweke diagnostic (Geweke, 1992), the Heidelberger and Welch diagnostic (Heidelberger & Welch, 1983), and the Raftery and Lewis diagnostic for determining the length of the burn-in…”
InfluentialPhilosophy and the practice of Bayesian statistics
Gelman, A., & Shalizi, C. R. (2010) · Cited by 709
Introduction to Bayesian Statistics
(2009) · Understanding Computational Bayesian Statistics · doi:10.1002/9780470567371.ch1 · Cited by 660
Structural Equation Modeling: A Bayesian Approach
Lee, S. (2007) · Wiley Series in Probability and Statistics · doi:10.1002/9780470024737 · Cited by 595
Bayesian Methods: A Social and Behavioral Sciences Approach
Gill, J. (2002) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420057478 · Cited by 579 · cited 2×
“3 The term “noninformative prior” refers to the case where researchers supply vague information about the population parameter value; the prior is typically defined with a very wide variance (Gill, 2008).”
Introduction to Applied Bayesian Statistics and Estimation for Social Scientists
(2007) · Springer eBooks · doi:10.1007/978-0-387-71265-9 · Cited by 555 · cited 2×
“…novice reader to, among many others: Bolstad (2007); Carlin and Louis (2009); Christensen, Johnson, Branscum, and Hanson (2010); Depaoli and Boyajian, (2014); Gelman and Hill (2007); Kaplan, 2014; Kruschke (2010); Jackman (2009); Lynch (2007); Ntzoufras (2009); or van de Schoot and Depaoli (2014).”
Lambert, P. C., Sutton, A. J., Burton, P. R., Abrams, K. R., & Jones, D. R. (2005) · Statistics in Medicine · doi:10.1002/sim.2112 · Cited by 511 · cited 1×
“For example, a prior specification that is truly noninformative may have an adverse impact on final parameter estimates via the posterior, especially when sample sizes are small; see Lambert, Sutton, Burton, Abrams, and Jones (2005) for a meta-analytic example of this.”
Convergence assessment techniques for Markov chain Monte Carlo
BROOKS, S. P., & ROBERTS, G. O. (1998) · Statistics and Computing · doi:10.1023/a:1008820505350 · Cited by 463 · cited 1×
“Although Sinharay (2004) and others (see, e.g., Brooks & Roberts, 1998) discuss several diagnostic tools to determine convergence, there is no consensus which statistical criterion can be considered as the “best” one.”
Albert, J. (2009) · Springer eBooks · doi:10.1007/978-0-387-92298-0 · Cited by 462
A systematic review of Bayesian articles in psychology: The last 25 years.
van de Schoot, R., Winter, S. D., Ryan, O., Zondervan-Zwijnenburg, M., & Depaoli, S. (2017) · Psychological Methods · doi:10.1037/met0000100 · Cited by 374
Self-citationPhysiological Pharmacokinetic Analysis Using Population Modeling and Informative Prior Distributions
Gelman, A., Bois, F., & Jiang, J. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476708 · Cited by 353 · cited 1×
“Within the Bayesian framework, “bias” does not take on the same meaning since bias that is directly related to priors will diminish and disappear under large sample sizes given that the impact of the prior will also diminish (Gelman et al., 1996).”
Link, W. A., & Eaton, M. J. (2012) · Methods in Ecology and Evolution · doi:10.1111/j.2041-210x.2011.00131.x · Cited by 351 · cited 2×
“Although it is true that univariate priors used for variances (or standard deviations/ precisions) have some similar complexities (see, e.g., van de Schoot et al., 2015), we highlight some of the issues specific to the multivariate treatment of these priors. In our experience, the multivariate priors used in this situ…”
How many Iterations in the Gibbs Sampler?
Raftery, A. E., & Lewis, S. M. (1992) · Bayesian Statistics 4 · doi:10.1093/oso/9780198522669.003.0053 · Cited by 335
Hypothesis testing and model selection
(1995) · Markov Chain Monte Carlo in Practice · doi:10.1201/b14835-15 · Cited by 311
MCMC Estimation in MLwiN (Version 2.13) Centre for Multilevel Modelling, University of Bristol
Browne, W. J. (2009) · Bristol Research (University of Bristol) · Cited by 295
Bayesian Ideas and Data Analysis
Christensen, R., Johnson, W., Branscum, A., & Hanson, T. E. (2010) · CRC Press eBooks · doi:10.1201/9781439894798 · Cited by 291
van de Schoot, R., Broere, J. J., Perryck, K. H., Zondervan-Zwijnenburg, M., & van Loey, N. E. (2015) · European Journal of Psychotraumatology · doi:10.3402/ejpt.v6.25216 · Cited by 276 · cited 2×
“Although it is true that univariate priors used for variances (or standard deviations/ precisions) have some similar complexities (see, e.g., van de Schoot et al., 2015), we highlight some of the issues specific to the multivariate treatment of these priors.”
Introduction to Bayesian Statistics
Bolstad, W. M. (2007) · Wiley eBooks · doi:10.1002/9780470181188 · Cited by 260
How few countries will do? Comparative survey analysis from a Bayesian perspective
Hox, J. J., van de Schoot, R., & Matthijsse, S. M. (2011) · Survey research methods · doi:10.18148/srm/2012.v6i2.5033 · Cited by 225 · cited 1×
“…hence may produce reasonable results even with small to moderate sample sizes, especially when strong and defensible prior knowledge is available (Hox, van de Schoot, & Matthijsse, 2012; Moore et al., 2015; van de Schoot, Broere, Perryck, Zondervan-Zwijnenburg, & van Loey, 2015; Zhang, Hamagami,…”
van Loey, N. E. E., Maas, C. J. M., Faber, A. W., & Taal, L. A. (2003) · Journal of Traumatic Stress · doi:10.1023/a:1024465902416 · Cited by 202
A Single Series from the Gibbs Sampler Provides a False Sense of Security
Gelman, A., & Rubin, D. B. (1992) · Bayesian Statistics 4 · doi:10.1093/oso/9780198522669.003.0038 · Cited by 201
Bayesian Statistics for the Social Sciences
Kaplan, D. E. (2014) · Cited by 182
A web-based tool for eliciting probability distributions from experts
Morris, D. E., Oakley, J. E., & Crowe, J. A. (2014) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2013.10.010 · Cited by 177 · cited 1×
“…can ask an expert, or a panel of experts, to provide an estimate for the hyperparameters based on knowledge of the field; see, for example: Bijak and Wisniowski (2010); Fransman et al. (2011); Howard, Maxwell, and Fleming (2000); Martin et al. (2012); and Morris, Oakley, and Crowe (2014).”
Bayesian structural equation modeling.
Kaplan, D. E., & Depaoli, S. A. (2012) · Cited by 176
Self-citationSubjective and objective Bayesian statistics : principles, models, and applications
Press, S. J. (2003) · Wiley-Interscience eBooks · Cited by 166
Depaoli, S., & Clifton, J. P. (2015) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2014.937849 · Cited by 163 · cited 1×
“Further, some models (e.g., mixture or multilevel models) require Bayesian methods to improve convergence issues (Depaoli & Clifton, 2015; Skrondal & Rabe-Hesketh, 2012), aid in model identification (Kim, Suh, Kim, Albanese, & Langer, 2013), and produce more accurate parameter estimates (Depaoli,…”
Self-citationBayesian Methods: A Social and Behavioral Sciences Approach
Lee, H. K. H. (2008) · The American Statistician · doi:10.1198/000313008x370915 · Cited by 157
Bayesian Ideas and Data Analysis: An Introduction for Scientists and Statisticians,
Jensen, W. A. (2011) · Journal of Quality Technology · doi:10.1080/00224065.2011.11917854 · Cited by 151
Depaoli, S. (2013) · Psychological Methods · doi:10.1037/a0031609 · Cited by 147 · cited 5×
“…have an impact on parameter estimates and therefore also on substantive findings; for details on the different ways in which priors can adversely impact findings, see: Depaoli (2013); Gelman and Shalizi (2013); Johnson (2013); Seaman, Seaman, and Stamey (2012); and van de Schoot and Depaoli (2014).”
InfluentialSelf-citationBayesian analysis of longitudinal data using growth curve models
Zhang, Z. J., Hamagami, F., Wang, L. L., Nesselroade, J. R., & Grimm, K. J. (2007) · International Journal of Behavioral Development · doi:10.1177/0165025407077764 · Cited by 147
Bayesian analyses : where to start and what to report
van de Schoot, R., & Depaoli, S. A. (2014) · Utrecht University Repository (Utrecht University) · Cited by 144
Self-citationIn-season prediction of batting averages: A field test of empirical Bayes and Bayes methodologies
Brown, L. D. (2008) · The Annals of Applied Statistics · doi:10.1214/07-aoas138 · Cited by 142 · cited 4×
“Finally, data-based priors can be derived based on a variety of methods including maximum likelihood (see, Berger, 2006; Brown, 2008; Candel & Winkens, 2003; van der Linden, 2008) or sample statistics (see, e.”
InfluentialHoward, G., Maxwell, S., & Fleming, K. J. (2000) · Psychological methods · doi:10.1037/1082-989x.5.3.315 · Cited by 141 · cited 1×
“…can ask an expert, or a panel of experts, to provide an estimate for the hyperparameters based on knowledge of the field; see, for example: Bijak and Wisniowski (2010); Fransman et al. (2011); Howard, Maxwell, and Fleming (2000); Martin et al. (2012); and Morris, Oakley, and Crowe (2014).”
Bayesian parametric estimation of stop-signal reaction time distributions.
Matzke, D., Dolan, C. V., Logan, G. D., Brown, S. D., & Wagenmakers, E. (2012) · Journal of Experimental Psychology General · doi:10.1037/a0030543 · Cited by 139 · cited 1×
“There are several key components that must be included in a write-up of Bayesian results and these include information from all of the previous Points 1–9 listed above, see for an example Matzke, Dolan, Logan, Brown, and Wagenmakers (2013).”
Asymptotic Inference for Mixture Models by Using Data-Dependent Priors
Wasserman, L. (2000) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/1467-9868.00226 · Cited by 126 · cited 1×
“Finally, data-based priors can be derived based on a variety of methods including maximum likelihood (see, Berger, 2006; Brown, 2008; Candel & Winkens, 2003; van der Linden, 2008) or sample statistics (see, e.g., Darnieder, 2011; Raftery, 1996; Rich-ardson & Green, 1997; Wasserman, 2000).”
Schuurman, N. K., Grasman, R. P. P. P., & Hamaker, E. L. (2016) · Multivariate Behavioral Research · doi:10.1080/00273171.2015.1065398 · Cited by 123
BAYESIAN OUTPUT ANALYSIS PROGRAM (BOA) VERSION 1.0 USER'S MANUAL
Smith, B. A. (2003) · Cited by 116
Ibrahim, J. G., Chen, M., & Sinha, D. (2005) · Encyclopedia of Biostatistics · doi:10.1002/0470011815.b2a11006 · Cited by 112
Uniformly most powerful Bayesian tests
Johnson, V. E. (2013) · The Annals of Statistics · doi:10.1214/13-aos1123 · Cited by 108 · cited 1×
“…have an impact on parameter estimates and therefore also on substantive findings; for details on the different ways in which priors can adversely impact findings, see: Depaoli (2013); Gelman and Shalizi (2013); Johnson (2013); Seaman, Seaman, and Stamey (2012); and van de Schoot and Depaoli (2014).”
Using Response Times for Item Selection in Adaptive Testing
van der Linden, W. J. (2007) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998607302626 · Cited by 107 · cited 1×
“Finally, data-based priors can be derived based on a variety of methods including maximum likelihood (see, Berger, 2006; Brown, 2008; Candel & Winkens, 2003; van der Linden, 2008) or sample statistics (see, e.g., Darnieder, 2011; Raftery, 1996; Rich-ardson & Green, 1997; Wasserman, 2000).”
Advanced Reach Tool (ART): Development of the Mechanistic Model
Fransman, W., van Tongeren, M., Cherrie, J. W., Tischer, M., Schneider, T., Schinkel, J., Kromhout, H., Warren, N., Goede, H. A., & Tielemans, E. (2011) · The Annals of Occupational Hygiene · doi:10.1093/annhyg/mer083 · Cited by 100 · cited 1×
“…can ask an expert, or a panel of experts, to provide an estimate for the hyperparameters based on knowledge of the field; see, for example: Bijak and Wisniowski (2010); Fransman et al. (2011); Howard, Maxwell, and Fleming (2000); Martin et al. (2012); and Morris, Oakley, and Crowe (2014).”
Bayesian Forecasting of Immigration to Selected European Countries by using Expert Knowledge
Bijak, J., & Wiśniowski, A. (2010) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2009.00635.x · Cited by 91 · cited 5×
“First, the researcher can ask an expert, or a panel of experts, to provide an estimate for the hyperparameters based on knowledge of the field; see, for example: Bijak and Wisniowski (2010); Fransman et al.”
InfluentialHidden Dangers of Specifying Noninformative Priors
Seaman, J. W., Seaman, J. W., & Stamey, J. D. (2012) · The American Statistician · doi:10.1080/00031305.2012.695938 · Cited by 89 · cited 2×
“…have an impact on parameter estimates and therefore also on substantive findings; for details on the different ways in which priors can adversely impact findings, see: Depaoli (2013); Gelman and Shalizi (2013); Johnson (2013); Seaman, Seaman, and Stamey (2012); and van de Schoot and Depaoli (2014).”
Mulder, J., Hoijtink, H., & de Leeuw, C. (2012) · Journal of Statistical Software · doi:10.18637/jss.v046.i02 · Cited by 87
Estimating Item Response Theory Models Using Markov Chain Monte Carlo Methods
Kim, J., & Bolt, D. M. (2007) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.2007.00107.x · Cited by 85 · cited 1×
“The interested reader is referred to Kaplan and Depaoli (2012); Sinharay (2004), or Kim and Bolt (2007) for an overview of additional convergence diagnostics commonly implemented in the Bayesian literature.”
Depaoli, S. A. (2014) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2014.882686 · Cited by 80 · cited 4×
“…novice reader to, among many others: Bolstad (2007); Carlin and Louis (2009); Christensen, Johnson, Branscum, and Hanson (2010); Depaoli and Boyajian, (2014); Gelman and Hill (2007); Kaplan, 2014; Kruschke (2010); Jackman (2009); Lynch (2007); Ntzoufras (2009); or van de Schoot and Depaoli (2014).”
InfluentialSelf-citationExperiences With Markov Chain Monte Carlo Convergence Assessment in Two Psychometric Examples
Sinharay, S. (2004) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986029004461 · Cited by 78 · cited 2×
“Although Sinharay (2004) and others (see, e.g., Brooks & Roberts, 1998) discuss several diagnostic tools to determine convergence, there is no consensus which statistical criterion can be considered as the “best” one.”
UNIFORMLY MOST POWERFUL BAYESIAN TESTS
Johnson, V. E. (2013) · OakTrust (Texas A&M University Libraries) · Cited by 60
Computation of Bayes Factors for Common Designs
Morey, R. D., & Rouder, J. N. (2015) · Cited by 58
Introduction to Special Section on Bayesian Data Analysis
Kruschke, J. K. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406926 · Cited by 54 · cited 1×
“First, some complex models simply cannot be estimated using conventional statistics (see, e.g., Muthén & Asparouhov, 2012; Kruschke, 2010, 2011; Wetzels, Matzke, Lee, Rouder, Iverson & Wagenmakers, 2011).”
van de Schoot, R., Hoijtink, H., Mulder, J., van Aken, M. A. G., de Castro, B. O., Meeus, W., & Romeijn, J. (2011) · Developmental Psychology · doi:10.1037/a0020957 · Cited by 52 · cited 1×
“Second, many scholars prefer Bayesian statistics because they believe population parameters should be viewed as random (see, e.g., Dienes, 2011; van de Schoot et al., 2011).”
Methodological Advances in the Analysis of Individual Growth With Relevance to Education Policy
Kaplan, D. E. (2002) · Peabody Journal of Education · doi:10.1207/s15327930pje7704_9 · Cited by 51 · cited 4×
“If we have the intercept mean hyperparameter specified at 31.37 (based on a previous analysis presented in Kaplan, 2002), then we can start the sensitivity analysis at this point by varying this value upward and downward to see how much of an impact the mean hyperparameter of this prior has on the…”
InfluentialIncorporation of historical data in the analysis of randomized therapeutic trials
Rietbergen, C., Klugkist, I. G., Janssen, K. J. M., Moons, K. G. M., & Hoijtink, H. J. A. (2011) · Contemporary Clinical Trials · doi:10.1016/j.cct.2011.06.002 · Cited by 47 · cited 1×
“Third, they can use the results from a meta-analysis to define hyperparameter values for the prior, where multiple studies are combined to form information about the parameter (Ibrahim, Chen, & Sinha, 2005; Rietbergen et al., 2011).”
Kaplan, D. E., & Depaoli, S. A. (2013) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199934874.013.0020 · Cited by 45
Self-citationSensitivity Analyses for Sparse-Data Problems—Using Weakly Informative Bayesian Priors
Hamra, G., Maclehose, R., & Cole, S. (2013) · Epidemiology · doi:10.1097/ede.0b013e318280db1d · Cited by 45 · cited 1×
“For more examples and details on sensitivity analysis of priors see: Hamra, MacLehose, and Cole (2013); Heydari, Miranda-Moreno, Lord, and Fu (2014); Lopes and Tobias (2011); Millar (2004); and van de Schoot et al. (2015).”
Moore, T. M., Reise, S. P., Depaoli, S. A., & Haviland, M. G. (2015) · Multivariate Behavioral Research · doi:10.1080/00273171.2014.973990 · Cited by 39 · cited 1×
“…even with small to moderate sample sizes, especially when strong and defensible prior knowledge is available (Hox, van de Schoot, & Matthijsse, 2012; Moore et al., 2015; van de Schoot, Broere, Perryck, Zondervan-Zwijnenburg, & van Loey, 2015; Zhang, Hamagami, Wang, Grimm, & Nesselroade, 2007).”
Self-citationMeasurement and Structural Model Class Separation in Mixture CFA: ML/EM Versus MCMC
Depaoli, S. A. (2012) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2012.659614 · Cited by 37 · cited 2×
“For example, as a preliminary analysis Depaoli (2012) conducted a sensitivity analysis for the IW prior in the context of a mixture confirmatory factor analysis model and found that even slightly modifying the hyperparameters of the IW changed final estimates substantially.”
Self-citationHeydari, S., Miranda-Moreno, L., Lord, D., & Fu, L. (2013) · Accident Analysis & Prevention · doi:10.1016/j.aap.2013.11.001 · Cited by 31
Candel, M. J. J. M., & Winkens, B. (2003) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986028002169 · Cited by 26 · cited 1×
“Finally, data-based priors can be derived based on a variety of methods including maximum likelihood (see, Berger, 2006; Brown, 2008; Candel & Winkens, 2003; van der Linden, 2008) or sample statistics (see, e.g., Darnieder, 2011; Raftery, 1996; Rich-ardson & Green, 1997; Wasserman, 2000).”
Bayesian Methods for Data-Dependent Priors
Darnieder, W. F. (2011) · OhioLink ETD Center (Ohio Library and Information Network) · Cited by 25
Lopes, H. F., & Tobias, J. L. (2011) · Annual Review of Economics · doi:10.1146/annurev-economics-111809-125134 · Cited by 23 · cited 1×
“For more examples and details on sensitivity analysis of priors see: Hamra, MacLehose, and Cole (2013); Heydari, Miranda-Moreno, Lord, and Fu (2014); Lopes and Tobias (2011); Millar (2004); and van de Schoot et al. (2015).”
Kim, S.-Y., Suh, Y., Kim, J.-S., Albanese, M. A., & Langer, M. M. (2013) · Multivariate Behavioral Research · doi:10.1080/00273171.2013.802647 · Cited by 18 · cited 1×
“…models (e.g., mixture or multilevel models) require Bayesian methods to improve convergence issues (Depaoli & Clifton, 2015; Skrondal & Rabe-Hesketh, 2012), aid in model identification (Kim, Suh, Kim, Albanese, & Langer, 2013), and produce more accurate parameter estimates (Depaoli, 2013, 2014).”
Linear and nonlinear growth models: Describing a Bayesian perspective.
Depaoli, S. A., & Boyajian, J. G. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0035147 · Cited by 16 · cited 1×
“…we refer the novice reader to, among many others: Bolstad (2007); Carlin and Louis (2009); Christensen, Johnson, Branscum, and Hanson (2010); Depaoli and Boyajian, (2014); Gelman and Hill (2007); Kaplan, 2014; Kruschke (2010); Jackman (2009); Lynch (2007); Ntzoufras (2009); or van de Schoot…”
Self-citationChoosing Priors for Constrained Analysis of Variance: Methods Based on Training Data
van Wesel, F., Hoijtink, H. J. A., & Klugkist, I. G. (2010) · Scandinavian Journal of Statistics · doi:10.1111/j.1467-9469.2010.00719.x · Cited by 15 · cited 1×
“The information embedded in the informative prior can come from a variety of places, which is referred to as prior elicitation (O’Hagan et al., 2006; Van Wesel et al., 2011).”
Depaoli, S. A., van de Schoot, R., Loey, N. E. E. V., & Sijbrandij, M. (2015) · European Journal of Psychotraumatology · doi:10.3402/ejpt.v6.27516 · Cited by 10 · cited 2×
“These specialized topics are beyond the scope of the current article but we refer the reader to more technical sources such as Gelman et al. (2004) and Lee (2007) for details on such topics.”
Self-citationMillar, R. B. (2004) · Biometrics · doi:10.1111/j.0006-341x.2004.00201.x · Cited by 10 · cited 1×
“For more examples and details on sensitivity analysis of priors see: Hamra, MacLehose, and Cole (2013); Heydari, Miranda-Moreno, Lord, and Fu (2014); Lopes and Tobias (2011); Millar (2004); and van de Schoot et al. (2015).”
Cited by
Bayesian statistics and modelling
van de Schoot, R., Depaoli, S., King, R., Kramer, B., Märtens, K., Tadesse, M. G., Vannucci, M., Gelman, A., Veen, D., Willemsen, J., et al. (2021) · Nature Reviews Methods Primers · doi:10.1038/s43586-020-00001-2 · Cited by 1,176
Self-citationThe JASP guidelines for conducting and reporting a Bayesian analysis
van Doorn, J., van den Bergh, D., Böhm, U., Dablander, F., Derks, K., Draws, T., Etz, A., Evans, N. J., Gronau, Q. F., Haaf, J. M., et al. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01798-5 · Cited by 1,114
Brysbaert, M. (2019) · Journal of Cognition · doi:10.5334/joc.72 · Cited by 996
On Using Bayesian Methods to Address Small Sample Problems
McNeish, D. (2016) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2016.1186549 · Cited by 570
“…advantages of Bayesian methods over frequentist maximum likelihood (ML) methods with small samples (e.g., Baldwin & Fellingham, 2013; Depaoli, 2013; Depaoli & van de Schoot, 2015; Dunson, 2000; Gelman, 2006; Hox, van de Schoot, & Matthijsse, 2012; Kadane, 2015; Lambert, Sutton, Burton, Abrams, &…”
MethodBayesian Analysis Reporting Guidelines
Kruschke, J. K. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01177-7 · Cited by 440
Metadata
- Authors
- Depaoli, S., & van de Schoot, R.
- Affiliations
- University of California, Merced (US)
- Corresponding author
- Depaoli, Sarah
- ORCID
- Depaoli, Sarah
- Venue
- Psychological Methods
- Published
- 2017-06
- Volume
- 22
- Issue
- 2
- Pages
- 240-261
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2409860581
- Semantic Scholar
- 232cadce7992b919589966dc1e92e58a4ef4aa43
- Cited by
- 448
- References
- 97
- Citations by year
- 2026: 61 · 2025: 47 · 2024: 54 · 2023: 46 · 2022: 52 · 2021: 57 · 2020: 48 · 2019: 36 · 2018: 18 · 2017: 24
- FWCI
- 13.94
- Type
- journal article
- Language
- en
- Open access
- green
- dspace.library.uu.nl
- Licence
- other-oa
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Funders
- Netherlands Organisation for Scientific Research (NWO-VENI-451-11-008)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- Bayesian analysis, prior sensitivity, model diagnostics, research transparency, replication, Bayesian model checking
- MeSH terms
- Bayes Theorem, Data Interpretation, Statistical, Humans, Checklist
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.