The Psychological Science Accelerator: Advancing Psychology Through a Distributed Collaborative Network
Open accessPublished version of a preprint
Abstract
Concerns about the veracity of psychological research have been growing. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions or replicate prior research in large, diverse samples. The PSA’s mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time limited), efficient (in that structures and principles are reused for different projects), decentralized, diverse (in both subjects and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside the network). The PSA and other approaches to crowdsourced psychological science will advance understanding of mental processes and behaviors by enabling rigorous research and systematic examination of its generalizability.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Back-translation for cross-cultural research.
Brislin, R. W. (1970) · Journal of Cross-Cultural Psychology · doi:10.1177/135910457000100301 · Cited by 14,091 · see citing works
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641 · see citing works
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · cited 7× · see citing works
“More recent projects such as the Many Labs project series (Klein et al., 2014; Ebersole et al., 2016), Many Babies (Frank et al., 2017), the Reproducibility Project: Psychology (Open Science Collaboration, 2015), the Pipeline Project (Schweinsberg et al., 2016), the Human Penguin Project (IJzerman et al., 2018), and R…”
BackgroundInfluentialPower failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 2× · see citing works
“, 2011) and low statistical power (Button et al., 2013; Cohen, 1962).”
BackgroundSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · see citing works
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200 · see citing works
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824 · see citing works
Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable
Elwert, F., & Winship, C. (2014) · Annual Review of Sociology · doi:10.1146/annurev-soc-071913-043455 · Cited by 1,168 · cited 1× · see citing works
“In particular, the composition and insufficient size of typical samples in psychological research introduces uncertainty about the veracity (Anderson & Maxwell, 2017; Cohen, 1992; Maxwell, 2004) and generalizability of findings (Elwert & Winship, 2014; Henrich, Heine, & Norenzayan, 2010).”
BackgroundConstraints on Generality (COG): A Proposed Addition to All Empirical Papers
Simons, D. J., Shoda, Y., & Lindsay, D. S. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617708630 · Cited by 986 · see citing works
Hartshorne, J. K., & Germine, L. T. (2015) · Psychological Science · doi:10.1177/0956797614567339 · Cited by 788 · see citing works
The Value of Direct Replication
Simons, D. J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613514755 · Cited by 630 · see citing works
Nelson, L. D., Simmons, J., & Simonsohn, U. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011836 · Cited by 548 · see citing works
Fraley, R. C., & Vazire, S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0109019 · Cited by 409 · see citing works
Contextual sensitivity in scientific reproducibility
Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1521897113 · Cited by 398 · see citing works
Registered Replication Report: Schooler and Engstler-Schooler (1990)
Alogna, V. K., Attaya, M. K., Aucoin, P., Bahník, Š., Birch, S., Birt, A. R., Bornstein, B. H., Bouwmeester, S., Brandimonte, M. A., Brown, C., et al. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614545653 · Cited by 223 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Lakens, D. (2022) · Collabra Psychology · doi:10.1525/collabra.33267 · Cited by 1,962 · see citing works
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770 · see citing works
Self-citationOpen science challenges, benefits and tips in early career and beyond
Allen, C., & Mehler, D. M. A. (2019) · PLOS Biology · doi:10.1371/journal.pbio.3000246 · Cited by 608 · see citing works
Yarkoni, T. (2022) · Behavioral and Brain Sciences · doi:10.1017/s0140525x20001685 · Cited by 701 · see citing works
The Sisyphean Cycle of Technology Panics
Orben, A. (2020) · Perspectives on Psychological Science · doi:10.1177/1745691620919372 · Cited by 316 · see citing works
That’s a Lot to Process! Pitfalls of Popular Path Models
Rohrer, J. M., Hünermund, P., Arslan, R. C., & Elson, M. (2022) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221095827 · Cited by 267 · see citing works
Use caution when applying behavioural science to policy
IJzerman, H., Lewis, N. A., Przybylski, A. K., Weinstein, N., DeBruine, L., Ritchie, S. J., Vazire, S., Forscher, P. S., Morey, R. D., Ivory, J. D., et al. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-00990-w · Cited by 262 · see citing works
Self-citationAre specific learning disorders truly specific, and are they disorders?
Peters, L., & Ansari, D. (2019) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2019.100115 · Cited by 151 · see citing works
Self-citationThe Replication Crisis Has Led to Positive Structural, Procedural, and Community Changes
Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., Elsherif, M., Breznau, N., Robertson, O., Kalandadze, T., et al. (2023) · Communications Psychology · doi:10.1038/s44271-023-00003-2 · Cited by 137 · see citing works
Self-citationElsherif, M. M., Middleton, S. L., Phan, J. M., Azevedo, F., Iley, B. J., Grose-Hodge, M., Tyler, S. L., Kapp, S. K., Gourdon-Kanhukamwe, A., Grafton-Clarke, D., et al. (2022) · doi:10.31222/osf.io/k7a9p · Cited by 91 · see citing works
Coles, N. A., Hamlin, J. K., Sullivan, L. L., Parker, T. H., & Altschul, D. (2022) · Nature · doi:10.1038/d41586-022-00150-2 · Cited by 86 · see citing works
Self-citationOpen Science 2.0: Towards a truly collaborative research ecosystem
Thibault, R. T., Amaral, O. B., Argolo, F., Bandrowski, A. E., Davidson, A. R., & Drude, N. I. (2023) · PLOS Biology · doi:10.1371/journal.pbio.3002362 · Cited by 72 · see citing works
Integrating Qualitative Methods and Open Science: Five Principles for More Trustworthy Research*
Humphreys, L., Lewis, N. A., Sender, K., & Won, A. S. (2021) · Journal of Communication · doi:10.1093/joc/jqab026 · Cited by 69 · see citing works
Flake, J. K. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1898962 · Cited by 58 · see citing works
Self-citation
- Authors
- Moshontz, H., Campbell, L., Ebersole, C. R., IJzerman, H., Urry, H. L., Forscher, P. S., Grahe, J. E., McCarthy, R. J., Musser, E. D., Antfolk, J., Castille, C. M., Evans, T. R., Fiedler, S., Flake, J. K., Forero, D. A., Janssen, S. M. J., Keene, J. R., Protzko, J., Aczel, B., Álvarez Solas, S., Ansari, D., Awlia, D., Baskin, E., Batres, C., Borras-Guevara, M. L., Brick, C., Chandel, P., Chatard, A., Chopik, W. J., Clarance, D., Coles, N. A., Corker, K. S., Dixson, B. J. W., Dranseika, V., Dunham, Y., Fox, N. W., Gardiner, G., Garrison, S. M., Gill, T., Hahn, A. C., Jaeger, B., Kačmár, P., Kaminski, G., Kanske, P., Kekecs, Z., Kline, M., Koehn, M. A., Kujur, P., Levitan, C. A., Miller, J. K., Okan, C., Olsen, J., Oviedo-Trespalacios, O., Özdoğru, A. A., Pande, B., Parganiha, A., Parveen, N., Pfuhl, G., Pradhan, S., Ropovik, I., Rule, N. O., Saunders, B., Schei, V., Schmidt, K., Singh, M. M., Sirota, M., Steltenpohl, C. N., Stieger, S., Storage, D., Sullivan, G. B., Szabelska, A., Tamnes, C. K., Vadillo, M. A., Valentova, J. V., Vanpaemel, W., Varella, M. A. C., Vergauwe, E., Verschoor, M., Vianello, M., Voracek, M., Williams, G. P., Wilson, J. P., Zickfeld, J. H., Arnal, J. D., Aydin, B., Chen, S.-C., DeBruine, L. M., Fernandez, A. M., Horstmann, K. T., Isager, P. M., Jones, B., Kapucu, A., Lin, H., Mensink, M. C., Navarrete, G., Silan, M. A., & Chartier, C. R.
- Affiliations
- Duke University (US)
- Western University (CA)
- University of Virginia (US)
- Université Grenoble Alpes (FR)
- Tufts University (US)
- University of Arkansas at Fayetteville (US)
- Pacific Lutheran University (US)
- Northern Illinois University (US)
- Florida International University (US)
- Åbo Akademi University (FI)
- Nicholls State University (US)
- Coventry University (GB)
- Max Planck Institute for Behavioral Economics (DE)
- McGill University (CA)
- Universidad Antonio Nariño (CO)
- University of Nottingham Malaysia Campus (MY)
- Texas Tech University (US)
- University of California, Santa Barbara (US)
- Institute of Psychology, ELTE Eötvös Loránd University
- Engineering in Ecosystems, Universidad Regional Amazónica Ikiam
- Ashland University (US)
- Saint Joseph's University (US)
- Franklin & Marshall College (US)
- University of St Andrews (GB)
- University of Cambridge (GB)
- Pandit Ravishankar Shukla University (IN)
- Centre National de la Recherche Scientifique (FR)
- Michigan State University (US)
- Busara Center for Behavioral Economics (KE)
- University of Tennessee at Knoxville (US)
- Grand Valley State University (US)
- The University of Queensland (AU)
- Vilnius University (LT)
- Yale University (US)
- Rutgers, The State University of New Jersey (US)
- University of California, Riverside (US)
- Vanderbilt University (US)
- Wilfrid Laurier University (CA)
- Cal Poly Humboldt (US)
- Tilburg University (NL)
- University of Pavol Jozef Šafárik (SK)
- Laboratoire Cognition, Langues, Langage, Ergonomie (FR)
- Technische Universität Dresden (DE)
- Department of Psychology, Lund University
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology
- Western Sydney University (AU)
- Occidental College (US)
- Willamette University (US)
- University of Vienna (AT)
- School of Psychology and Counseling, Queensland University of Technology
- Üsküdar University (TR)
- UiT The Arctic University of Norway (NO)
- University of Prešov (SK)
- University of Toronto (CA)
- University of Dundee (GB)
- Norwegian School of Economics (NO)
- Southern Illinois University Carbondale (US)
- University of Essex (GB)
- University of Southern Indiana (US)
- Karl Landsteiner University of Health Sciences (AT)
- University of Illinois Urbana-Champaign (US)
- Queen's University Belfast (GB)
- University of Oslo (NO)
- Universidad Autónoma de Madrid (ES)
- Department of Experimental Psychology, Institute of Psychology, University of São Paulo
- KU Leuven (BE)
- Universidade de São Paulo (BR)
- University of Geneva (CH)
- University of Groningen (NL)
- University of Padua (IT)
- Abertay University (GB)
- Montclair State University (US)
- McDaniel College (US)
- Department of Educational Sciences, RTE University
- Tzu Chi University (TW)
- University of Glasgow (GB)
- Laboratorio de Evolución y Relaciones Interpersonales, Universidad de Santiago de Chile
- Humboldt-Universität zu Berlin (DE)
- Eindhoven University of Technology (NL)
- Ege University (TR)
- University of Wisconsin–Stout (US)
- Adolfo Ibáñez University (CL)
- University of the Philippines Diliman (PH)
- Corresponding author
- Chartier, Christopher R.
- ORCID
- Moshontz, Hannah
- Campbell, Lorne
- Ebersole, Charles R.
- IJzerman, Hans
- Urry, Heather L.
- Forscher, Patrick S.
- Grahe, Jon E.
- McCarthy, Randy J.
- Musser, Erica D.
- Antfolk, Jan
- Castille, Christopher M.
- Evans, Thomas Rhys
- Fiedler, Susann
- Flake, Jessica Kay
- Forero, Diego A.
- Janssen, Steve M. J.
- Keene, Justin Robert
- Protzko, John
- Ansari, Daniel
- Baskin, Ernest
- Batres, Carlota
- Borras-Guevara, Martha Lucia
- Brick, Cameron
- Chandel, Priyanka
- Chatard, Armand
- Chopik, William J.
- Coles, Nicholas A.
- Corker, Katherine S.
- Dixson, Barnaby James Wyld
- Dranseika, Vilius
- Dunham, Yarrow
- Gardiner, Gwendolyn
- Garrison, S. Mason
- Gill, Tripat
- Hahn, Amanda C.
- Jaeger, Bastian
- Kačmár, Pavol
- Kaminski, Gwenaël
- Kanske, Philipp
- Kekecs, Zoltan
- Koehn, Monica A.
- Kujur, Pratibha
- Levitan, Carmel A.
- Miller, Jeremy K.
- Okan, Ceylan
- Olsen, Jerome
- Oviedo-Trespalacios, Oscar
- Özdoğru, Asil Ali
- Pande, Babita
- Parganiha, Arti
- Parveen, Noorshama
- Pfuhl, Gerit
- Pradhan, Sraddha
- Ropovik, Ivan
- Rule, Nicholas O.
- Saunders, Blair
- Schei, Vidar
- Schmidt, Kathleen
- Sirota, Miroslav
- Steltenpohl, Crystal N.
- Stieger, Stefan
- Storage, Daniel
- Sullivan, Gavin Brent
- Szabelska, Anna
- Tamnes, Christian K.
- Vadillo, Miguel A.
- Valentova, Jaroslava V.
- Vanpaemel, Wolf
- Varella, Marco A. C.
- Vergauwe, Evie
- Verschoor, Mark
- Vianello, Michelangelo
- Voracek, Martin
- Williams, Glenn P.
- Wilson, John Paul
- Zickfeld, Janis H.
- Arnal, Jack D.
- Aydin, Burak
- Chen, Sau-Chin
- DeBruine, Lisa M.
- Horstmann, Kai T.
- Isager, Peder M.
- Jones, Benedict
- Kapucu, Aycan
- Lin, Hause
- Mensink, Michael C.
- Navarrete, Gorka
- Silan, Miguel A.
- Chartier, Christopher R.
- Venue
- Advances in Methods and Practices in Psychological Science
- Publisher
- SAGE Publications
- Published
- 2018-10-01
- Volume
- 1
- Issue
- 4
- Pages
- 501-515
- ISSN
- 2515-2459 (ISSN-L)
- 2515-2467
- PMC
- PMC6934079
- OpenAlex
- W2810320295
- Cited by
- 377
- References
- 18
- Citations by year
- 2026: 8 · 2025: 26 · 2024: 35 · 2023: 65 · 2022: 67 · 2021: 63 · 2020: 55 · 2019: 47 · 2018: 11
- FWCI
- 62.17
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Advances in Methods and Practices in Psychological Science (publishedVersion)
- Research portal (Tilburg University) (publishedVersion)
- PubMed (publishedVersion)
- Pure (Coventry University) (publishedVersion, other-oa)
- WU Research (publishedVersion)
- Strathprints: The University of Strathclyde institutional repository (University of Strathclyde) (acceptedVersion, other-oa)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Lirias (KU Leuven) (acceptedVersion)
- Open Access at Essex (University of Essex) (acceptedVersion)
- Sunderland Repository (University of Sunderland) (acceptedVersion)
- Lirias (submittedVersion, other-oa)
- Journal of the Arkansas Academy of Science (submittedVersion, cc-by-nd)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- HAL (submittedVersion)
- HAL (submittedVersion, other-oa)
- Discovery Research Portal (University of Dundee) (submittedVersion)
- Scholarship@Western (Western University) (submittedVersion, cc-by-nc-nd)
- TU/e Research Portal (submittedVersion)
- Lund University Publications (Lund University) (submittedVersion)
- NORA - Norwegian Open Research Archives (submittedVersion, other-oa)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- Åbo Akademi University Research Portal (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- VU Research Portal (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (PZ00P1_154911)
- Fondo Nacional de Desarrollo Científico y Tecnológico (1181114)
- Comunidad de Madrid
- ERC KINSHIP (647910)
- NWO VIDI (452-17-013)
- VCTI
- Division of Graduate Education (1445197)
- Canada Research Chairs
- Agence Nationale de la Recherche (Investissements d’avenir (ANR-15-IDEX-02))
- National Institute of Mental Health (R03MH110812-02)
- National Science Foundation (Graduate Research Fellowship)
- Colciencias
- Gielen-Leyendecker-Stiftung
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Interdisciplinary Research and Collaboration
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- crowdsourced research, replication, statistical power, generalizability, research credibility, large-scale cooperation, inclusive research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
