Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report.
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Following a review of extant reporting standards for scientific publication, and reviewing 10 years of experience since publication of the first set of reporting standards by the American Psychological Association (APA; APA Publications and Communications Board Working Group on Journal Article Reporting Standards, 2008), the APA Working Group on Quantitative Research Reporting Standards recommended some modifications to the original standards. Examples of modifications include division of hypotheses, analyses, and conclusions into 3 groupings (primary, secondary, and exploratory) and some changes to the section on meta-analysis. Several new modules are included that report standards for observational studies, clinical trials, longitudinal studies, replication studies, and N-of-1 studies. In addition, standards for analytic methods with unique characteristics and output (structural equation modeling and Bayesian analysis) are included. These proposals were accepted by the Publications and Communications Board of APA and supersede the standards included in the 6th edition of the Publication Manual of the American Psychological Association (APA, 2010). (PsycINFO Database Record
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A systematic search strategy was implemented (Appelbaum et al., 2018).”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Appelbaum et al., 2018
In the citing paper
“We designed the meta-analyses and report the results in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Moher et al., 2009) and the American Psychological Association's Quantitative Meta-Analysis Article Reporting Standards (Appelbaum et al., 2018).”
From Do growth mindset interventions impact students’ academic achievement? A systematic review and meta-analysis with recommendations for best practices. (2023) · cites this paper as Appelbaum et al., 2018
In the citing paper
“However, the relative novelty of conducting Bayesian analyses in applied fields means that there are no detailed reporting standards, and this in turn may frustrate the broader adoption and proper interpretation of the Bayesian framework.”
From The JASP guidelines for conducting and reporting a Bayesian analysis (2021) · cites this paper as Appelbaum et al., 2018
In the citing paper
“These methods are necessary to produce unbiased and more reliable estimates (Appelbaum et al., 2018).”
From The cognitive and academic benefits of Cogmed: A meta-analysis (2019) · cites this paper as Appelbaum et al., 2018
Citations
Cites
Meta-analysis of Observational Studies in Epidemiology<SUBTITLE>A Proposal for Reporting</SUBTITLE>
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Cook, R. J., Dickens, B. M., & Fathalla, M. F. (2003) · Reproductive Health and Human Rights · doi:10.1093/acprof:oso/9780199241323.003.0025 · Cited by 10,769
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
A Beginner's Guide to Structural Equation Modeling: Fourth Edition
Schumacker, R. E., & Lomax, R. G. (2015) · doi:10.4324/9781315749105 · Cited by 6,604
Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72351 · Cited by 6,277
(2016) · Cited by 3,399
Self-citationStandards for Educational and Psychological Testing
(2007) · Encyclopedia of Measurement and Statistics · doi:10.4135/9781412952644.n432 · Cited by 3,126
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979
Picton, T. W., Bentin, S., Berg, P., Donchin, E., Hillyard, S. A., Johnson, R., Miller, G. A., Ritter, W., Ruchkin, D. S., Rugg, M. D., et al. (2000) · Psychophysiology · doi:10.1111/1469-8986.3720127 · Cited by 2,157
Cited by
Items 201–300 of 1,002
Personality traits, cognitive states, and mortality in older adulthood.
Yoneda, T. B., Graham, E. K., Lozinski, T., Bennett, D. A., Mroczek, D. K., Piccinin, A. M., Hofer, S. M., & Terrera, G. M. (2022) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000418 · Cited by 42
Topor, M. K., Pickering, J. S., Mendes, A. B., Bishop, D., Büttner, F., Elsherif, M. M., Evans, T. R., Henderson, E. L., Kalandadze, T., Nitschke, F., et al. (2023) · Meta-Psychology · doi:10.15626/mp.2021.2840 · Cited by 40
Group Sequential Designs: A TutorialPreprint
Lakens, D., Pahlke, F., & Wassmer, G. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/x4azm · Cited by 46
Deliberate reasoning is not affected by language
Mækelæ, M. J., & Pfuhl, G. (2019) · PLoS ONE · doi:10.1371/journal.pone.0211428 · Cited by 38
Recommendations for Reporting on Rehabilitation Interventions
Whyte, J., Dijkers, M. P. J. M., Fasoli, S. E., Ferraro, M. K., Katz, L. W., Norton, S., Parent, É., Pinto, S. M., Sisto, S. A., Stan, J. H. V., et al. (2020) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/phm.0000000000001581 · Cited by 38
Medina, L. D., Torres-Marrero, S. D., Gioia, A. R., Lopez, A. O., Wang, J., & Cirino, P. T. (2020) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617720001083 · Cited by 42
Hallford, D. J., Austin, D., Takano, K., Fuller‐Tyszkiewicz, M., & Raes, F. (2020) · Behaviour Research and Therapy · doi:10.1016/j.brat.2020.103783 · Cited by 35
Yang, M., Viladrich, C., & Cruz, J. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.965690 · Cited by 45
“We examined first the measurement model using the weighted least squares means and variance adjusted estimator and the internal consistency of each subscale with Cronbach’s alpha and non-linear reliability coefficients (Green and Yang, 2009; Appelbaum et al., 2018).”
MethodWeekers, L. C., Hutsebaut, J., Rovers, J. M. C., & Kamphuis, J. H. (2023) · Personality Disorders Theory Research and Treatment · doi:10.1037/per0000637 · Cited by 39
“We report how the sample size for the present study was determined, how missing data were handled, all manipulations, and all measures in the study and Journal Article Reporting Standards were followed (Appelbaum et al., 2018).”
Warne, E., Oxlad, M., & Best, T. (2023) · Health Psychology Review · doi:10.1080/17437199.2022.2058582 · Cited by 34
“The reporting standards for quantitative research in psychology were followed (Appelbaum et al., 2018; Supplementary Table 1: PRISMA 2020 Checklist).”
Meta-analyzing the multiverse: A peek under the hood of selective reporting.
Olsson-Collentine, A., van Aert, R. C. M., Bakker, M., & Wicherts, J. M. (2025) · Psychological Methods · doi:10.1037/met0000559 · Cited by 52
Metacognition and polyregulation in daily self‐control conflicts
Bürgler, S., & Hennecke, M. (2023) · Scandinavian Journal of Psychology · doi:10.1111/sjop.12964 · Cited by 39
Samea, F., Mortazavi, N., Reimann, G. M., Ebneabbasi, A., Zarei, M., Khazaie, H., Goldstein‐Piekarski, A. N., Spiegelhalder, K., Baglioni, C., Sepehry, A. A., et al. (2025) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2025.102111 · Cited by 39
A conceptual replication of the Strategic Training Initiative in Community Supervision (STICS)
Bonta, J. L., Rugge, T., Bourgon, G., & Wanamaker, K. A. (2019) · Journal of Experimental Criminology · doi:10.1007/s11292-019-09371-4 · Cited by 36
RELEX: An Excel-based software tool for sampling split-half reliability coefficients
Steinke, A., & Kopp, B. (2020) · Methods in Psychology · doi:10.1016/j.metip.2020.100023 · Cited by 34
Implications of the open science era for educational psychology research syntheses
Patall, E. A. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1897009 · Cited by 38
“But, transparent reporting can also be facilitated by researchers utilizing systematic reporting standards and guidelines, such as the American Psychological Association’s (APA) Journal Article Reporting Standards (JARS; Appelbaum et al., 2018; Levitt et al., 2018).”
MethodBasic Empathy Scale: A Systematic Review and Reliability Generalization Meta-Analysis
Cabedo-Peris, J., Vilar, M. M., Merino‐Soto, C., & Morán, M. O. (2021) · Healthcare · doi:10.3390/healthcare10010029 · Cited by 46
“Within the recommendations on psychometry, it is considered essential to introduce reliability and validity tests [99], something that has not always been observed in the studies used in this meta-analysis.”
BackgroundViljoen, J. L., Goossens, I., Monjazeb, S., Cochrane, D. M., Vargen, L. M., Jonnson, M. R., Blanchard, A. J. E., Li, S. M. Y., & Jackson, J. R. (2024) · Behavioral Sciences & the Law · doi:10.1002/bsl.2698 · Cited by 39
Molina-García, A., Galache-Laza, M. T., González-García, V., & Diéguez‐Soto, J. (2025) · Eurasian economic review : · doi:10.1007/s40821-025-00289-8 · Cited by 51
“8 We used the standard psychometric properties thresholds (Appelbaum et al., 2018; Hu & Bentler, 1999; Kline, 2011). supporting H3.”
Berg, H. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02253 · Cited by 36
A modality‐specific dysfunction of pain processing in schizophrenia
Zhou, L., Bi, Y., Liang, M., Kong, Y., Tu, Y., Zhang, X., Song, Y., Du, X., Tan, S., & Hu, L. (2019) · Human Brain Mapping · doi:10.1002/hbm.24906 · Cited by 38
“We obtained a large effect size of the detected group difference (Appelbaum et al., 2018; Cohen, 1992), and confirmed the sufficiency of our sample size (n = 42) with a large statistical power in the EEG experiment (1 − β = .”
BackgroundInfluentialDeters, F., & Schödel, R. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107977 · Cited by 23
Testing the bottom-up and top-down models of self-esteem: A meta-analysis of longitudinal studies.
Dapp, L. C., Krauss, S., & Orth, U. (2023) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000444 · Cited by 35
Deslandes, C., Kaufmann, L. M., & Anderson, J. (2023) · International Journal of Intercultural Relations · doi:10.1016/j.ijintrel.2023.101928 · Cited by 38
Parsons, S. D., Kruijt, A., & Fox, E. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/6ka9z · Cited by 37
Steffens, N. K., Munt, K. A., van Knippenberg, D., Platow, M. J., & Haslam, S. A. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/b94qs · Cited by 33
The inaccuracy of data from online surveys: A cautionary analysis
Agans, J. P., Schade, S. A., Hanna, S. R., Chiang, S., Shirzad, K., & Bai, S. (2023) · Quality & Quantity · doi:10.1007/s11135-023-01733-5 · Cited by 32
“…but although they require a description of the characteristics of the sample, inclusion or exclusion criteria, and sampling procedures, they do not specify the need to describe any strategies used to ensure that the sample does not include fraudulent data from scammers (Appelbaum et al. 2018 ).”
Using within-person change in three large panel studies to estimate personality age trajectories.
Seifert, I. S., Rohrer, J. M., & Schmukle, S. C. (2023) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000482 · Cited by 45
“Advance online publication. https://doi.org/10.1177/19485506221129160 Journal Article Reporting Standards for quantitative research in psychology: The APA Publications and Communications Board task force report.”
MethodInfluentialZygmont, C. S. (2023) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.19.4.p302 · Cited by 40
Automatic implicit motive codings are at least as accurate as humans’ and 99% faster.
Nilsson, A., Runge, J. M., Ganesan, A. V., Lövenstierne, C. V. N. G., Soni, N., & Kjell, O. (2025) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000544 · Cited by 37
The cultural stereotype and cultural bias of ChatGPT
Yuan, H., Che, Z., Zhang, Y., Shao, L. H., Yuan, X., Huang, L., Hu, X., Peng, K., & Luo, S. (2025) · Journal of Pacific Rim Psychology · doi:10.1177/18344909251355673 · Cited by 37
Hom, M. A., Weiss, R. B., Millman, Z. B., Christensen, K., Lewis, E. J., Cho, S., Yoon, S., Meyer, N., Kosiba, J. D., Shavit, E., et al. (2020) · Journal of Psychotherapy Integration · doi:10.1037/int0000212 · Cited by 32
Hoover, L. V., Yu, H. P., Cummings, J. R., Ferguson, S. G., & Gearhardt, A. N. (2022) · Psychology of Addictive Behaviors · doi:10.1037/adb0000870 · Cited by 37
The neurobiological presentation of anxiety in autism spectrum disorder: A systematic review
McVey, A. J. (2019) · Autism Research · doi:10.1002/aur.2063 · Cited by 30
“More recent journal article reporting standards explicitly state that major demographic characteristics, including ethnicity, should be reported [Appelbaum et al., 2018].”
BackgroundSingh, A., & Pathak, G. S. (2020) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2020.102128 · Cited by 38
Steinke, A., Kopp, B., & Lange, F. (2021) · Brain Sciences · doi:10.3390/brainsci11050529 · Cited by 40
“Instead, these split-half reliability estimates should be better conceived as a joint property of the cWCST when self-administered and the studied sample of young volunteers [65,66].”
BackgroundTransparency in Measurement Reporting: A Systematic Literature Review of CHI PLAY
Aeschbach, L. F., Perrig, S. A. C., Weder, L., Opwis, K., & Brühlmann, F. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3474660 · Cited by 35
“To find information about measurements quickly and efficiently, dedicated sections to both the methods and the measurements used aid readers [4].”
MethodThe roots of digital aggression: Exploring cyber-violence through a systematic literature review
Mukred, M., Mokhtar, U. A., Moafa, F. A., Gumaei, A., Sadiq, A. S., & Al-Othmani, A. (2024) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2024.100281 · Cited by 45
A Case For a Study Quality Appraisal in Survey Studies in Psychology
Protogerou, C., & Hagger, M. S. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02788 · Cited by 35
Transformational Leadership and Perceived Overqualification: A Career Development Perspective
Zhang, M., Wang, F., Weng, H., Zhu, T., & Liu, H. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.597821 · Cited by 36
Erbeli, F., & Rice, M. (2022) · Reading & Writing Quarterly · doi:10.1080/10573569.2021.1944830 · Cited by 28
“In addition, systematic review report standards have been updated and revised (e.g., Appelbaum et al., 2018).”
BackgroundMethodInfluentialBehavioral Lifestyles and Survival: A Meta-Analysis
Ballesteros, R. F., Valeriano-Lorenzo, E., Sánchez‐Izquierdo, M., & Botella, J. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.786491 · Cited by 33
“This report follows the guidelines of the APA task force recommendations about reporting standards for quantitative research in Psychology, and especially in meta-analysis articles (Appelbaum et al., 2018).”
MethodAlmaraz, D., Galdós, J. S., Martín, F. M., Sánchez‐Iglesias, I., Fernández, A. J. M., & Goldsby, T. L. (2022) · Healthcare · doi:10.3390/healthcare10081447 · Cited by 32
Participant diversity in Psychophysiology
Kissel, H. A., & Friedman, B. H. (2023) · Psychophysiology · doi:10.1111/psyp.14369 · Cited by 31
The effects of rumination, distraction, and gratitude on positive and negative affect.
Layous, K., Kumar, S. A., Arendtson, M., & Najera, A. (2023) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000440 · Cited by 29
Virtue Ethics and Integration in Evidence-Based Practice in Psychology
Berg, H. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00258 · Cited by 31
“This entails assessing the epistemic quality in different kinds of knowledge (Guba and Lincoln, 1994; Woolfolk, 2015; Appelbaum et al., 2018; Levitt et al., 2018).”
BackgroundIsaacs, T., & Chalmers, H. (2023) · Language Teaching · doi:10.1017/s0261444823000411 · Cited by 29
“The American Psychological Association (APA) published updated reporting ‘ standards ’ for quantitative journal articles in Appelbaum et al. (2018).”
McManus, K. (2024) · Studies in Second Language Acquisition · doi:10.1017/s0272263124000652 · Cited by 38
“At that point, authors were provided with recommendations for reporting replication studies as presented in Appelbaum et al. (2018) and Porte and McManus (2019), including: (cid:129) Report the type of replication in the title and abstract, as follows: exact, close, approximate, or conceptual.”
A scoping study on measures of child-to-parent violence
Arias-Rivera, S., García, M. V. H., & Lara, B. L. (2020) · Aggression and Violent Behavior · doi:10.1016/j.avb.2020.101426 · Cited by 31
Hensel, W. (2020) · European Journal for Philosophy of Science · doi:10.1007/s13194-020-00317-6
Buttlar, B., Pauer, S., Ruby, M. B., Chambon, M., Jimenez-Klingberg, A.-K., Scherf, J., & Scherrer, V. (2023) · Collabra Psychology · doi:10.1525/collabra.73236 · Cited by 28
Mastrothanasis, K., Kladaki, M., & Andreou, A. (2023) · International Journal of Educational Research Open · doi:10.1016/j.ijedro.2023.100243 · Cited by 46
Durgante, H. B., & Dell’Aglio, D. D. (2019) · Psicologia Reflexão e Crítica · doi:10.1186/s41155-019-0119-2 · Cited by 31
Seaman, J., Dettweiler, U., Humberstone, B., Martin, B. C., Prince, H., & Quay, J. (2020) · Journal of Experiential Education · doi:10.1177/1053825920969443 · Cited by 26
“Rather than recapitulating those descriptions here (they are available in the originals; see AERA, 2006; Appelbaum et al., 2018; Levitt et al., 2018; SRCD, 2020), it seems more useful for us to describe our motivations for adopting these guidelines and for approaching this editorial essay collectively.”
BackgroundMethodInfluentialSankar, S. S., & Anandh, K. S. (2024) · Administrative Sciences · doi:10.3390/admsci14090212 · Cited by 34
“This study adhered to the meta-analysis reporting standards outlined by Appelbaum et al. (2018) and followed the PRISMA (Page et al. 2021) guidelines for reporting meta-analysis (Liberati et al. 2009).”
Stigma in substance-based and behavioural addictions: A systematic review
Galanis, C. R., Leske, M., Hamamura, T., Weber, N., Hing, N., Delfabbro, P., & King, D. L. (2025) · Journal of Behavioral Addictions · doi:10.1556/2006.2024.00072 · Cited by 36
“A 10-item quality assessment was conducted based on the methods and results section of the Journal Article Reporting Standards (JARS; Appelbaum et al., 2018).”
A Meta-analysis of the relationship between growth mindset and grit
Lam, K. K. L., & Zhou, M. (2025) · Acta Psychologica · doi:10.1016/j.actpsy.2025.104872 · Cited by 34
Hyperscanning of Interactive Juggling: Expertise Influence on Source Level Functional Connectivity
Stone, D. B., Tamburro, G., Filho, E., di Fronso, S., Robazza, C., Bertollo, M., & Comani, S. (2019) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2019.00321 · Cited by 26
Subjective views of aging in very old age: Predictors of 2-year change in gains and losses.
Kaspar, R., Wahl, H., Diehl, M. K., & Zank, S. (2022) · Psychology and Aging · doi:10.1037/pag0000684 · Cited by 36
Spontaneous Trait Inferences From Behavior: A Systematic Meta-Analysis
Bott, A., Brockmann, L., Denneberg, I., Henken, E., Kuper, N., Kruse, F., & Degner, J. (2022) · Personality and Social Psychology Bulletin · doi:10.1177/01461672221100336 · Cited by 31
“This supports the need for more transparent reporting standards, which are already being implemented more widely in psychology in recent years (e.g., Appelbaum et al., 2018).”
BackgroundBelleville, G., Ouellet, M., Békés, V., Lebel, J., Morin, C. M., Bouchard, S., Guay, S., Bergeron, N., Ghosh, S., Campbell, T. S., et al. (2022) · Behavior Therapy · doi:10.1016/j.beth.2022.08.004 · Cited by 30
Primary Study Quality in Psychological Meta-Analyses: An Empirical Assessment of Recent Practice
Hohn, R. E., Slaney, K. L., & Tafreshi, D. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02667 · Cited by 33
“Similar findings were reported by Harwell and Maeda (2008) and Dieckmann et al. (2009) regarding meta-analyses in general psychological and educational research, respectively. Moreover, Brugha et al. (2012) concluded that reporting practices for psychiatric metaanalyses of observational studies published between 1996…”
ResultInfluentialWrzus, C., & Neubauer, A. B. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/vmr8q · Cited by 28
Bestiary of Questionable Research Practices in Psychology
Nagy, T., Hergert, J., Elsherif, M. M., Wallrich, L., Schmidt, K., Waltzer, T., Payne, J. W., Gjoneska, B., Seetahul, Y., Wang, Y. A., et al. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251348431 · Cited by 37
“Researchers should adopt rigorous and transparent methodologies, adhering to established guidelines and protocols specific to their field (Appelbaum et al., 2018).”
Soto, C. J., Hudson, N. W., Napolitano, C. M., Sewell, M. N., Yoon, H. R., & Roberts, B. W. (2025) · European Journal of Personality · doi:10.1177/08902070251392639 · Cited by 27
“The design, hypotheses, and planned analyses were pre-registered at https://osf.io/cfyq5/, and we follow JARS (Appelbaum et al., 2018).”
Examining Psychological Science Through Systematic Meta-Method Analysis: A Call for Research
Elson, M. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919863296 · Cited by 27
“Although JARS specifically mentions the importance of transparent reporting so that scientific claims can be understood and evaluated, it may not be obvious which information is critical to meaningful research synthesis specifically.”
BackgroundInfluentialLevy, J., Mussack, D., Brunner, M., Keller, U., Cardoso-Leite, P., & Fischbach, A. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.02190 · Cited by 25
Badenes‐Ribera, L., Molla‐Esparza, C., Longobardi, C., Sánchez‐Meca, J., & Fabris, M. A. (2020) · Journal of Traumatic Stress · doi:10.1002/jts.22630 · Cited by 29
Breuer, S., Ortner, T. M., Gruber, F. M., Hofstetter, D., & Scherndl, T. (2022) · Personality and Individual Differences · doi:10.1016/j.paid.2022.111918 · Cited by 24
Heterogeneity in Meta-Analytic Effect Sizes: An Assessment of the Current State of the Literature
Kepes, S., Wang, W., & Cortina, J. M. (2023) · Organizational Research Methods · doi:10.1177/10944281231169942 · Cited by 30
“We note that our recommendations focus on heterogeneity and, with regard to this particular issue, go substantially beyond what others have suggested before (e.g., Appelbaum et al., 2018; DeSimone et al., 2021; Kepes et al., 2013; Moher et al., 2009; Russo, 2007).”
Magalhães, E., & Camilo, C. (2023) · The European Journal of Psychology Applied to Legal Context · doi:10.5093/ejpalc2023a10 · Cited by 32
The effects of social media restriction: Meta-analytic evidence from randomized controlled trials
Burnell, K., Meter, D. J., Andrade, F. C., Slocum, A., & George, M. J. (2025) · SSM - Mental Health · doi:10.1016/j.ssmmh.2025.100459 · Cited by 36
Tang, X., (曾子欣), Z. Z., Huang, H., & Symonds, J. E. (2025) · ECNU Review of Education · doi:10.1177/20965311251371227 · Cited by 25
“As a result, attempts have been made to apply quality appraisal, typically conducted using checklists or tools with multiple quality indicators, to most scienti fi c domains (Appelbaum et al., 2018; Michie et al., 2005).”
Clemens, N. H., Lee, K., Henri, M., Simmons, L. E., Kwok, O., & Otaiba, S. A. (2020) · Journal of School Psychology · doi:10.1016/j.jsp.2020.01.003 · Cited by 28
Gettinger, M., Kratochwill, T. R., Eubanks, A., Foy, A., & Levin, J. R. (2021) · Journal of School Psychology · doi:10.1016/j.jsp.2021.09.004 · Cited by 26
Dillon, D. G., Lazarov, A., Dolan, S. C., Bar‐Haim, Y., Pizzagalli, D. A., & Schneier, F. R. (2022) · Emotion · doi:10.1037/emo0001053 · Cited by 31
Prevalence of Parental Violence Toward Teachers: A Meta-Analysis
Badenes‐Ribera, L., Fabris, M. A., Martinez, A., McMahon, S. D., & Longobardi, C. (2022) · Violence and Victims · doi:10.1891/vv-d-20-00230 · Cited by 32
A Metascientific Review of the Evidential Value of Acceptance and Commitment Therapy for Depression
Williams, A. J., Botanov, Y., Giovanetti, A. K., Perko, V. L., Sutherland, C. L., Youngren, W., & Sakaluk, J. K. (2023) · Behavior Therapy · doi:10.1016/j.beth.2022.06.004 · Cited by 23
Do financial incentives increase mental health treatment engagement? A meta-analysis.
Khazanov, G. K., Morris, P. E., Beed, A. T., Jager‐Hyman, S., Myhre, K., McKay, J. R., Feinn, R., Boland, E. M., & Thase, M. E. (2022) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000737 · Cited by 26
Assessing and Improving Robustness of Psychological Research Findings in Four Steps
Nuijten, M. B. (2022) · Springer eBooks · doi:10.1007/978-3-031-04968-2_17 · Cited by 28
Executive functions, motivation, and children’s academic development in Côte d’Ivoire.
Finch, J. E., Wolf, S., & Lichand, G. (2022) · Developmental Psychology · doi:10.1037/dev0001423 · Cited by 26
Remote Viewing: A 1974-2022 Systematic Review and Meta-Analysis
Tressoldi, P. E., & Katz, D. L. (2023) · Journal of Scientific Exploration · doi:10.31275/20232931 · Cited by 27
“We adopted the APA Meta-Analysis Reporting Standards (MARS, Appelbaum et al., 2018) and the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines (Page et al., 2021).”
MethodContext and education affect the quality of parents' speech to children
Hoff, E., Trecca, F., Højen, A., Laursen, B. P., & Bleses, D. (2024) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2024.101632 · Cited by 32
Baucom, K. J. W., Chen, X. S., Perry, N. S., Revolorio, K. Y., Reina, A., & Christensen, A. (2017) · Family Process · doi:10.1111/famp.12287 · Cited by 25
Differences between young architects' and non-architects' aesthetic evaluation of buildings
Šafárová, K., Pírko, M., Juřík, V., Pavlica, T., & Németh, O. (2019) · Frontiers of Architectural Research · doi:10.1016/j.foar.2019.04.001 · Cited by 23
“In the present study, we conducted a conceptual replication (Bonett, 2012; Appelbaum et al., 2018) of the study by Brown and Gifford (2001). The same procedures were used, that is, we asked participants to evaluate buildings on the scale of 1e10. However, we used stimuli different from those in Brown and Gifford’s stu…”
MethodInfluentialMcClain, M. B., Callan, G. L., Harris, B., Floyd, R. G., Haverkamp, C. R., Golson, M. E., Longhurst, D. N., & Benallie, K. J. (2021) · Journal of School Psychology · doi:10.1016/j.jsp.2020.11.002 · Cited by 25
Benway, N. R., Hitchcock, E. R., McAllister, T., Feeny, G. T., Hill, J., & Preston, J. L. (2021) · American Journal of Speech-Language Pathology · doi:10.1044/2021_ajslp-20-00216 · Cited by 31
Mindfulness for Students in Pre-K to Secondary School Settings: Current Findings, Future Directions
Roeser, R. W., Schussler, D. L., Baelen, R. N., & Galla, B. M. (2023) · Mindfulness · doi:10.1007/s12671-022-02036-w · Cited by 26
“, 2019) and journal reporting standards in psychology and related behavioral science fields (Appelbaum et al., 2018; Hoffmann et al., 2014; Levitt et al., 2018), Baelen et al. (2023) provide a detailed set of reporting recommendations designed to support more systematic and complete assessment and reporting on SBMP im…”
BackgroundInfluentialReliability of structural brain change in cognitively healthy adult samples
Dídac Vidal-piñeiro, A., Sørensen, Ø., Strømstad, M., Amlien, I. K., Anderson, M., Baaré, W. F. C., Bartrés‐Faz, D., Brandmaier, A. M., Bråthen, A. C. S., Garrido, P., et al. (2025) · Imaging Neuroscience · doi:10.1162/imag_a_00547 · Cited by 24
“Finally, reliability is ultimately a property of the measurement, influenced by the measure, but also partially dependent on the sample (e.g., Appelbaum et al., 2018; Parsons et al., 2019), and in neuroimaging, also of the processing stream.”
Single-Case Experimental Designs: Clinical Research and Practice
Vlaeyen, J. W. S., Onghena, P., Vannest, K. J., & Kratochwill, T. R. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-818697-8.00191-6 · Cited by 31
Davis, E. B., Lacey, E. K., Heydt, E. J., LaBouff, J. P., Barker, S. B., van Elk, M., & Ladd, K. L. (2022) · Psychology of Religion and Spirituality · doi:10.1037/rel0000485 · Cited by 26
Mentalizing in and out of awareness: A meta-analytic review of implicit and explicit mentalizing
Kivity, Y., Levy, K. N., Johnson, B. N., Rosenstein, L. K., & LeBreton, J. M. (2024) · Clinical Psychology Review · doi:10.1016/j.cpr.2024.102395 · Cited by 26
Zerna, J., Strobel, A., & Strobel, A. (2024) · Collabra Psychology · doi:10.1525/collabra.92885 · Cited by 32
Peer-Review Guidelines Promoting Replicability and Transparency in Psychological Science
Davis, W. E., Giner‐Sorolla, R., Lindsay, D. S., Lougheed, J. P., Makel, M. C., Meier, M. E., Sun, J., Vaughn, L. A., & Zelenski, J. M. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918806489 · Cited by 24
“One source of inspiration is the American Psychological Association’s Journal Article Reporting Standards ( JARS; Appelbaum et al., 2018).”
BackgroundSakaluk, J. K., Quinn‐Nilas, C., Fisher, A. N., Leshner, C. E., Huber, E., & Wood, J. R. (2020) · Archives of Sexual Behavior · doi:10.1007/s10508-020-01794-9 · Cited by 25
Shin, M., Bryant, D. P., Powell, S. R., Jung, P., Ok, M. W., & Hou, F. (2020) · Remedial and Special Education · doi:10.1177/0741932520964918 · Cited by 25
“To maintain consistency and ensure standardization in reporting the methodology of the current meta-analysis, we followed the Meta-Analysis Reporting Standards for quantitative research syntheses (Appelbaum et al., 2018).”
MethodKeep Your Stats in the Cloud! Evaluating the Use of Google Sheets to Teach Quantitative Methods
Kunicki, Z. J., Zambrotta, N. S., Tate, M. C., Surrusco, A. R., Risi, M. M., & Harlow, L. L. (2019) · Journal of Statistics Education · doi:10.1080/10691898.2019.1665485 · Cited by 27
“Both effect sizes and confidence intervals are an important part of an introductory statistics curriculum, especially within the social sciences (Appelbaum et al. 2018), so instructors using Google Sheets will need to either code these values step by step as well or provide students with prewritten…”
BackgroundCoaches’ influence on team dynamics in sport: A scoping review
Hague, C., McGuire, C. S., Chen, J., Bruner, M. W., Côté, J., Turnnidge, J., & Martin, L. J. (2021) · Sports Coaching Review · doi:10.1080/21640629.2021.1874096 · Cited by 27
“Journal article reporting standards put forth by Appelbaum et al. (2018) highlight the importance of collecting and reporting major demographic participant information to maximise the understanding, replicability, and credibility of results.”
BackgroundSampling between sports and athlete development: a scoping review
Murata, A., Goldman, D. E., Martin, L. J., Turnnidge, J., Bruner, M. W., & Côté, J. (2021) · International Journal of Sport and Exercise Psychology · doi:10.1080/1612197x.2021.1995021 · Cited by 31
“…greater emphasis to be placed on the provision and description of major demographic information has been expressed in other recent publications (i.e., Appelbaum et al., 2018; Robertson et al., 2019) and is an important step to ensure that quality scientific reporting endures within future studies.”
BackgroundContingency management for drug use disorders: Meta-analysis and application of Tolin’s criteria.
Pfund, R. A., Ginley, M. K., Boness, C. L., Rash, C. J., Zajac, K., & Witkiewitz, K. A. (2024) · Clinical Psychology Science and Practice · doi:10.1037/cps0000121 · Cited by 30
Metadata
- Authors
- Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M.
- Affiliations
- University of California San Diego (US)
- Duke University (US)
- Concordia University (US)
- Drexel University (US)
- Cleveland Clinic (US)
- Venue
- American Psychologist
- Published
- 2018-01
- Volume
- 73
- Issue
- 1
- Pages
- 3-25
- ISSN
- 0003-066X (ISSN-L)
- 1935-990X
- OpenAlex
- W2784576540
- Cited by
- 1,449
- References
- 35
- Citations by year
- 2026: 258 · 2025: 246 · 2024: 204 · 2023: 172 · 2022: 154 · 2021: 142 · 2020: 133 · 2019: 89 · 2018: 47
- FWCI
- 78.36
- Type
- journal article
- Language
- en
- Update notices
- correction (2018-10-04)
- Open access
- hybrid
- doi.org
- Locations
- American Psychologist (publishedVersion, other-oa)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- reporting guidelines, meta-analysis reporting, observational studies, clinical trials, longitudinal study, replication study, N-of-1 studies, structural equation modeling, Bayesian analysis, scientific publications
- MeSH terms
- Humans, Periodicals as Topic, Psychology, Research Design, Societies, Scientific, Peer Review, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.