The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Toward a principled Bayesian workflow in cognitive science.
Open accessAbstract
Experiments in research on memory, language, and in other areas of cognitive science are increasingly being analyzed using Bayesian methods. This has been facilitated by the development of probabilistic programming languages such as Stan, and easily accessible front-end packages such as brms. The utility of Bayesian methods, however, ultimately depends on the relevance of the Bayesian model, in particular whether or not it accurately captures the structure of the data and the data analyst's domain expertise. Even with powerful software, the analyst is responsible for verifying the utility of their model. To demonstrate this point, we introduce a principled Bayesian workflow (Betancourt, 2018) to cognitive science. Using a concrete working example, we describe basic questions one should ask about the model: prior predictive checks, computational faithfulness, model sensitivity, and posterior predictive checks. The running example for demonstrating the workflow is data on reading times with a linguistic manipulation of object versus subject relative clause sentences. This principled Bayesian workflow also demonstrates how to use domain knowledge to inform prior distributions. It provides guidelines and checks for valid data analysis, avoiding overfitting complex models to noise, and capturing relevant data structure in a probabilistic model. Given the increasing use of Bayesian methods, we aim to discuss how these methods can be properly employed to obtain robust answers to scientific questions. All data and code accompanying this article are available from https://osf.io/b2vx9/. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Appendix S3 in the online Supporting Information presents the prior predictive checks (Schad, Betancourt, & Vasishth, 2021).”
From Effects of Availability, Contingency, and Formulaicity on the Accuracy of English Grammatical Morphemes in Second Language Writing (2022) · cites this paper as Schad, Betancourt, & Vasishth, 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“We followed a Bayesian workflow for both computational models and conventional analyses [19, 20].”
From Predicting emotional valence in autism: a preregistered study in the Bayesian Brain framework (2026) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
Random effects structure for confirmatory hypothesis testing: Keep it maximal
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013) · Journal of Memory and Language · doi:10.1016/j.jml.2012.11.001 · Cited by 10,328
brms : An R Package for Bayesian Multilevel Models Using Stan
Bürkner, P.-C. (2017) · Journal of Statistical Software · doi:10.18637/jss.v080.i01 · Cited by 9,476 · cited 7×
“p in the literature. It assumes that readers have had some previous exposure to and experience with basic concepts of Bayesian modeling, such as fitting simple Bayesian models (e.g., in brms or Stan) (Bürkner, 2017a; Kruschke, 2014; Vasishth et al., 2018b) , and are interested in how to put together their knowledge abo…”
BackgroundMethodInfluentialMixed-effects modeling with crossed random effects for subjects and items
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008) · Journal of Memory and Language · doi:10.1016/j.jml.2007.12.005 · Cited by 8,540
Stan : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754 · cited 16×
“Much progress has been made in the development of probabilistic programming languages such as Stan (Carpenter et al., 2017), WinBUGS (Lunn, Thomas, Best, & Spiegelhalter, 2000), JASP (JASP Team, 2019), and JAGS (Plummer, 2012).”
BackgroundMethodInfluentialSelf-citationInformation theory, inference, and learning algorithms
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5949 · Cited by 6,536
(2015) · doi:10.1007/s11222-016-9696-4 · Cited by 5,611
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. J. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,397
Kmenta, J. (1997) · University of Michigan Press eBooks · doi:10.3998/mpub.15701 · Cited by 3,240
Flagged first, then by citations
Research works printed but not linked: 5
- Chow, S.-M., & Hoijtink, H. (2017). Bayesian estimation and modeling: Editorial to the second special issue on Bayesian data analysis. Psychological Methods, 22 (4), 609-615.
- Rabe, M. M., Vasishth, S., Hohenstein, S., Kliegl, R., & Schad, D. J. (2020). Hypr: An r package for hypothesis-driven contrast coding. PsyArXiv. Retrieved from 10.31234/osf.io/cqzdx · DOI
- Ratcliff, R., & Rouder, J. N. (2000). A diffusion model account of masking in two-choice letter identification. Journal of Experimental Psychology: Human Perception and Performance, 26 (1), 127.
- Vandekerckhove, J., Rouder, J. N., & Kruschke, J. K. (2018). Bayesian methods for advancing psychological science. Psychonomic Bulletin & Review, 25 (1), 1-4.
- Vehtari, A., Gelman, A., & Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27 (5), 1413-1432.
Cited by
Sorted by citations
Items 201–248 of 248
Reanalysis and Underspecification in Sentence Comprehension: Modelling Eye Movements
Vasishth, S., & Engelmann, F. (2021) · Cambridge University Press eBooks · doi:10.1017/9781316459560.011
Self-citationGuajardo, G. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.815432
A Parsimonious Bayesian Crop Modeling Framework Based on Ordinary Differential EquationsPreprint
Poudel, P., Alderman, P. D., Ochsner, T. E., & Lollato, R. P. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4137895
[PhD Dissertation] Good Research PracticesPreprint
Sarafoglou, A. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/7aj5h
Human exploration strategically balances approaching and avoiding uncertainty
Abir, Y., Shadlen, M. N., & Shohamy, D. (2026) · eLife · doi:10.7554/elife.94231.3
Mouse Tracking for Reading (MoTR): A New Naturalistic Incremental Processing Measurement ToolPreprint
Wilcox, E., Ding, C., Jäger, L. A., & Sachan, M. (2023) · doi:10.31234/osf.io/4ryvs
Arao, H., Suwazono, S., Kimura, A., Asano, H., & Suzuki, H. (2023) · European Journal of Neuroscience · doi:10.1111/ejn.16175
Testing the Obligatory Contour Principle and Tonal Markedness Scale effects in L2 tonal acquisitionPreprint
Zhou, C., & Veríssimo, J. (2024) · doi:10.31234/osf.io/e24zd
Day-to-day fluctuations in motivation drive effort-based choicesPreprint
Hewitt, S., Norbury, A., Huys, Q. J. M., & Hauser, T. U. (2024) · doi:10.31234/osf.io/w3x7d
Dynamic Evaluation of Electoral Preferences in Mexico: A Bayesian Power Ranking Model
Anzarut, M., González, L. F., & Ortiz, M. T. (2024) · Springer proceedings in mathematics & statistics · doi:10.1007/978-3-031-78401-9_4
Farkas, A. H., Escobar, J. C., Gilbert, F. E., Ding, M., & Keil, A. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.02.16.638555
Multi-Stage Product Control Decision-Making Based on Monte Carlo and Bayesian Methods
Tong, H., Li, L., & Lin, S. (2025) · Asia-Europe Conference on Cybersecurity, Internet of Things and Soft Computing (CITSC) · doi:10.1109/citsc64390.2025.00009
An R/brms tutorial on using Regularised Prediction and Poststratification for test normingPreprint
Alsalti, T., Hussey, I., Elson, M., Kurz, A. S., & Arslan, R. C. (2023) · doi:10.31234/osf.io/fcm3n_v1
Hidalgo, A. E., Pearson, L., Oku, T., Kimoto, Y., Furuya, S., & Ruiz, M. H. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.03.31.646422
Toward Dynamical Modeling of Infants' Looking Times
Engbert, R., Funken, J., & Boll‐Avetisyan, N. (2025) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.70006
Similarity-based interference in the processing of classifier-noun dependencies in Mandarin Chinese
Hao, H., Fuchs, Z., & Vasishth, S. (2025) · Journal of Memory and Language · doi:10.1016/j.jml.2025.104669 · Cited by 1
Self-citationUnderstanding the traits underlying vaccine-driven virulence evolution in malaria parasites
Jeong, Y., Kamiya, T., & Mideo, N. (2025) · BMC Biology · doi:10.1186/s12915-025-02366-w
Lam, B., Paskewitz, S., Robbins, H., & Baskin-sommers, A. R. (2025) · PLoS ONE · doi:10.1371/journal.pone.0332189
Bayesian workflow for bias-adjustment model in meta-analysis
Jung, J., & Aloe, A. M. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10050
Eye-tracking evidence for benefits from linguistically-driven text formatting while reading
Dempsey, J., Christianson, K., & Dyke, J. A. V. (2025) · Reading and Writing · doi:10.1007/s11145-025-10716-x
TRACTABLE INVERSE UNCERTAINTY QUANTIFICATION VIA PROBABILISTIC CIRCUITS FOR APPLIED MECHANICS
Herold, M., Jehle, J. S., & Gerdts, M. (2025) · International Journal for Uncertainty Quantification · doi:10.1615/int.j.uncertaintyquantification.2025059325
Theory-driven computational psychiatry and its challenges: An optimistic outlook
Fisher, E. L., Sajid, N., Convertino, L., & Hohwy, J. (2026) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2026.106923
Kang, S., Park, S. H., & Son, Y. J. (2026) · BMC Nursing · doi:10.1186/s12912-025-04279-z
Modulation of sensory attenuation by intensive meditation practice: an active inference perspective
Poublan‐Couzardot, A., Foncelle, A., Koun, É., Rossetti, Y., Abdoun, O., Pagnoni, G., & Antoine Lutz, A. (2025) · Neuroscience of Consciousness · doi:10.1093/nc/niaf051
Hierarchical Bayesian Regression for experimental psychology: a case study of cognitive control
Dudey, T., Braver, T. S., Jackson, J. J., & Cooper, S. R. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1643463 · Cited by 1
Moral injury events, pain intensity, and functional mobility in post-9/11 U.S. combat veterans
Wild, M. G., Coppin, J. D., Greer, D., Mendoza, C., Herbert, M. S., Creech, S. K., & Frankfurt, S. B. (2026) · Journal of Pain · doi:10.1016/j.jpain.2026.106287
Meinhardt, L.-M., Colley, M., BALCI, A., Degenhard, A., Jansen, P., & Rukzio, E. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798420
Johnson, E. C., & Connolly, S. R. (2026) · Ecology Letters · doi:10.1111/ele.70362
Imprecision Can Promote Illusions of Understanding
Haaf, J. M., Cruz, N., & Giacobello, A. (2026) · Computational Brain & Behavior · doi:10.1007/s42113-026-00291-x
Are Semantic Representations Stable? A Bayesian Framework Applied to the Study of Quantifier Meaning
Sarafoglou, A., Giacobello, A., Godmann, H. R., Johnson, T., Visser, I., Haaf, J. M., & Szymanik, J. (2026) · Computational Brain & Behavior · doi:10.1007/s42113-026-00302-x
Anheyer, D., & Cramer, H. (2026) · Journal of Integrative and Complementary Medicine · doi:10.1177/27683605261447142
Non-specific increase in alpha power during a neurofeedback session targeting its downregulation
Maaz, J., Dia, A., Waroquier, L., Paban, V., & Rey, A. (2026) · Imaging Neuroscience · doi:10.1162/imag.a.1258
van Maanen, L., Heathcote, A., Matzke, D., & Tran, N.-H. (2021) · OSF Preprints (OSF Preprints)
Jandrasupalli, K. K., Singh, M., Singh, N., Rajaganesh, S., & Chowdhury, N. (2026) · Acta Cytologica · doi:10.1159/000552516
A four-step simulation-based workflow for ecological analysis and science
Wolkovich, E. M., Davies, T. J., Pearse, W. D., & Betancourt, M. (2026) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2025.0252
Self-citationClosing the gap between statistical and scientific workflows for improved forecasts in ecology
der Meersch, V. V., Regetz, J., Davies, T. J., & Wolkovich, E. M. (2026) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2024.0612
Guitard, D., Jamieson, R. K., Saint‐Aubin, J., & Johns, B. T. (2026) · Journal of Memory and Language · doi:10.1016/j.jml.2026.104814
How Uncertainty Affects Children's Exploration and Exploitation in Statistical Learning
Tong, S. X., Zhang, P., Zhou, M., & Ziegler, J. C. (2026) · Developmental Science · doi:10.1111/desc.70235
Hameed, T., John, L. L. H., Bignell, E., & Tanaka, R. J. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.19.733372
Farkas, A. H., Jia, H., Gardy, S. M., Gehr, M. C., & Sabatinelli, D. (2026) · Psychophysiology · doi:10.1111/psyp.70343
Hansen, M., & Bürki, A. (2026) · Journal of Memory and Language · doi:10.1016/j.jml.2026.104783
Kovač, N., Ratković, K., Simeunović, M., Marovac, U., Farahani, H., Kordić, S., & Zhang, Z. J. (2026) · Springer eBooks · doi:10.1007/978-3-032-18312-5_16
Göttmann, J., Frischkorn, G. T., Oberauer, K., Schaefer, S. B., & Schubert, A. (2026) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2026.103015
Degrees of certainty: beyond threshold thinking in sport and exercise science
Carvalho, H. M. (2026) · British Journal of Sports Medicine · doi:10.1136/bjsports-2026-112159
Mind wandering and executive functions: A conceptual replication of Kam & Handy (2014)
Fazel, M., Verwijk, E., Geurts, H. M., & Filip Van Opstal, A. (2026) · Consciousness and Cognition · doi:10.1016/j.concog.2026.104139
Classical and Bayesian Parameter Estimation under Varying Sample Sizes and Data Conditions
Robiya, R. (2026) · Qubahan Journal of Social Science · doi:10.48161/qjss.v3n2a15
Liss, J. V., von Krause, M., Elsemüller, L., Hunsmann, E. M., & Lerche, V. (2026) · Cognitive Psychology · doi:10.1016/j.cogpsych.2026.101836
Performance and Metacognition Disconnect when Reasoning in Human-AI Interaction
Fernandes, D., Villa, S., Nicholls, S., Haavisto, O., Buschek, D., Schmidt, A., Kosch, T., Shen, C., & Welsch, R. (2024) · arXiv · doi:10.48550/arxiv.2409.16708 · Cited by 71
Metadata
- Authors
- Schad, D. J., Betancourt, M., & Vasishth, S.
- Affiliations
- University of Potsdam (DE)
- Venue
- Psychological Methods
- Published
- 2021-02
- Volume
- 26
- Issue
- 1
- Pages
- 103-126
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2941987401
- Cited by
- 252
- References
- 51
- Citations by year
- 2026: 32 · 2025: 34 · 2024: 40 · 2023: 38 · 2022: 34 · 2021: 58
- FWCI
- 24.54
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- publish.UP (University of Potsdam) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Deutsche Forschungsgemeinschaft (317633480—SFB 1287)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Physical Sciences › Mathematics › Statistics and Probability › Markov Chains and Monte Carlo Methods
- Keywords
- model checking, posterior predictive checks, model sensitivity, prior distributions, reading time, relative clauses, overfitting, probabilistic programming
- MeSH terms
- Bayes Theorem, Humans, Models, Psychological, Models, Statistical, Cognitive Science, Workflow
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
