The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report.
Open accessAbstract
Following a review of extant reporting standards for scientific publication, and reviewing 10 years of experience since publication of the first set of reporting standards by the American Psychological Association (APA; APA Publications and Communications Board Working Group on Journal Article Reporting Standards, 2008), the APA Working Group on Quantitative Research Reporting Standards recommended some modifications to the original standards. Examples of modifications include division of hypotheses, analyses, and conclusions into 3 groupings (primary, secondary, and exploratory) and some changes to the section on meta-analysis. Several new modules are included that report standards for observational studies, clinical trials, longitudinal studies, replication studies, and N-of-1 studies. In addition, standards for analytic methods with unique characteristics and output (structural equation modeling and Bayesian analysis) are included. These proposals were accepted by the Publications and Communications Board of APA and supersede the standards included in the 6th edition of the Publication Manual of the American Psychological Association (APA, 2010). (PsycINFO Database Record
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A systematic search strategy was implemented (Appelbaum et al., 2018).”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Appelbaum et al., 2018
In the citing paper
“We designed the meta-analyses and report the results in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Moher et al., 2009) and the American Psychological Association's Quantitative Meta-Analysis Article Reporting Standards (Appelbaum et al., 2018).”
From Do growth mindset interventions impact students’ academic achievement? A systematic review and meta-analysis with recommendations for best practices. (2023) · cites this paper as Appelbaum et al., 2018
In the citing paper
“However, the relative novelty of conducting Bayesian analyses in applied fields means that there are no detailed reporting standards, and this in turn may frustrate the broader adoption and proper interpretation of the Bayesian framework.”
From The JASP guidelines for conducting and reporting a Bayesian analysis (2021) · cites this paper as Appelbaum et al., 2018
In the citing paper
“These methods are necessary to produce unbiased and more reliable estimates (Appelbaum et al., 2018).”
From The cognitive and academic benefits of Cogmed: A meta-analysis (2019) · cites this paper as Appelbaum et al., 2018
Citations
Cites
Meta-analysis of Observational Studies in Epidemiology<SUBTITLE>A Proposal for Reporting</SUBTITLE>
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Cook, R. J., Dickens, B. M., & Fathalla, M. F. (2003) · Reproductive Health and Human Rights · doi:10.1093/acprof:oso/9780199241323.003.0025 · Cited by 10,769
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
A Beginner's Guide to Structural Equation Modeling: Fourth Edition
Schumacker, R. E., & Lomax, R. G. (2015) · doi:10.4324/9781315749105 · Cited by 6,604
Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72351 · Cited by 6,277
(2016) · Cited by 3,399
Self-citationStandards for Educational and Psychological Testing
(2007) · Encyclopedia of Measurement and Statistics · doi:10.4135/9781412952644.n432 · Cited by 3,126
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979
Picton, T. W., Bentin, S., Berg, P., Donchin, E., Hillyard, S. A., Johnson, R., Miller, G. A., Ritter, W., Ruchkin, D. S., Rugg, M. D., et al. (2000) · Psychophysiology · doi:10.1111/1469-8986.3720127 · Cited by 2,157
Cited by
Salk, R. H., Hyde, J. S., & Abramson, L. Y. (2017) · Psychological Bulletin · doi:10.1037/bul0000102 · Cited by 2,622
Cheung, G. W., Cooper-Thomas, H. D., Lau, R. S., & Wang, L. C. (2024) · Asia Pacific Journal of Management · doi:10.1007/s10490-023-09871-y · Cited by 2,262
“The American Psychological Association’s journal article reporting standards (Appelbaum et al., 2018) refer researchers to Kline (2016, Chapter 18), who suggested researchers report chi-square with its degrees of freedom and p -value; RMSEA (Steiger & Lind, 1980) and its 90% confidence interval…”
Latent class analysis: a guide to best practice
Weller, B. E., Bowen, N. K., & Faubert, S. J. (2020) · Journal of Black Psychology · doi:10.1177/0095798420930932 · Cited by 2,223
“To present clear results and justified conclusions, LCA reports need to detail study procedures and results with clarity and coherence (Appelbaum et al., 2018).”
BackgroundExploratory Factor Analysis: A Guide to Best Practice
Watkins, M. W. (2018) · Journal of Black Psychology · doi:10.1177/0095798418771807 · Cited by 2,218
The JASP guidelines for conducting and reporting a Bayesian analysis
van Doorn, J., van den Bergh, D., Böhm, U., Dablander, F., Derks, K., Draws, T., Etz, A., Evans, N. J., Gronau, Q. F., Haaf, J. M., et al. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01798-5 · Cited by 1,114
A perspective on using partial least squares structural equation modelling in data articles
Ringle, C. M., Sarstedt, M., Sinkovics, N., & Sinkovics, R. R. (2023) · Data in Brief · doi:10.1016/j.dib.2023.109074 · Cited by 767
Schäfer, T., & Schwarz, M. A. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00813 · Cited by 774
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770
Effect size guidelines for cross-lagged effects.
Orth, U., Meier, L. L., Bühler, J. L., Dapp, L. C., Krauss, S., Messerli, D., & Robins, R. W. (2024) · Psychological Methods · doi:10.1037/met0000499 · Cited by 720
Eurasian Journal of Educational Research
(2021) · Eurasian Journal of Educational Research · doi:10.14689/ejer · Cited by 638
Methodological guidance paper: High-quality meta- analysis in a systematic review
Pigott, T. D., & Polanin, J. R. (2020) · Review of Educational Research · doi:10.3102/0034654319877153 · Cited by 568
“The MARS (Appelbaum et al., 2018) and PRISMA (Moher et al., 2009) guidelines each provide a helpful framework for meta-analysts to follow when reporting the results of the study.”
BackgroundMethodInfluentialReporting standards for psychological network analyses in cross-sectional data.
Burger, J., Isvoranu, A.-M., Lunansky, G., Haslbeck, J. M. B., Epskamp, S., Hoekstra, R. H. A., Fried, E. I., Borsboom, D., & Blanken, T. F. (2023) · Psychological Methods · doi:10.1037/met0000471 · Cited by 516
Wrzus, C., & Neubauer, A. B. (2023) · Assessment · doi:10.1177/10731911211067538 · Cited by 500
Chiappetta Jabbour, C. J., Seuring, S., Lopes de Sousa Jabbour, A. B., Jugend, D., De Camargo Fiorini, P., Latan, H., & Izeppi, W. C. (2020) · Journal of Environmental Management · doi:10.1016/j.jenvman.2020.110416 · Cited by 346
Bayesian Analysis Reporting Guidelines
Kruschke, J. K. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01177-7 · Cited by 440
Parsons, S., Kruijt, A.-W., & Fox, E. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919879695 · Cited by 425
“Many authors have made this same point (for a few examples, see Appelbaum et al., 2018; Cooper et al., 2017; Hedge et al., 2018; LeBel & Paunonen, 2011; and Wilkinson & Task Force on Statistical Inference, 1999).”
BackgroundMacnamara, B. N., & Burgoyne, A. P. (2023) · Psychological Bulletin · doi:10.1037/bul0000352 · Cited by 415
“…the meta-analyses and report the results in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Moher et al., 2009) and the American Psychological Association ’ s Quantitative Meta-Analysis Article Reporting Standards (Appelbaum et al., 2018).”
MethodPsychotherapists’ attitudes toward online therapy during the COVID-19 pandemic.
Békés, V., & Aafjes-van Doorn, K. (2020) · Journal of Psychotherapy Integration · doi:10.1037/int0000214 · Cited by 393
(2021) · doi:10.1037/amp0000905 · Cited by 392
The potential of immersive virtual reality to enhance learning: A meta-analysis
Coban, M., Bolat, Y. I., & Goksu, I. (2022) · Educational Research Review · doi:10.1016/j.edurev.2022.100452 · Cited by 354
Vafaie, N., & Kober, H. (2022) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2022.1240 · Cited by 342
Trull, T. J., & Ebner-Priemer, U. W. (2020) · Journal of Abnormal Psychology · doi:10.1037/abn0000473 · Cited by 344
O. B. O. T. C.-S. G., Grant, S., Mayo-Wilson, E., Montgomery, P., Macdonald, G., Michie, S., Hopewell, S., & Moher, D. (2018) · Trials · doi:10.1186/s13063-018-2735-z · Cited by 348
“Details about providers should include their professional qualifications and education, expertise or competence with the interventions or area in general, and training and supervision for delivering the interventions [71].”
BackgroundSelf-citationThe Use of Online Panel Data in Management Research: A Review and Recommendations
Porter, C. O. L. H., Outlaw, R., Gale, J. P., & Cho, T. S. (2019) · Journal of Management · doi:10.1177/0149206318811569 · Cited by 338
A Conceptual Introduction to Bayesian Model Averaging
Hinne, M., Gronau, Q. F., van den Bergh, D., & Wagenmakers, E.-J. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919898657 · Cited by 318
“…in the social-sciences literature (for noteworthy exceptions, see Gronau et al., 2017; Kaplan & Lee, 2018; for recommendations and guidelines, see Appelbaum et al., 2018), even though it is regularly employed in other disciplines (see Fragoso, Bertoli, & Louzada, 2018, for a review; see Steel,…”
Harari, M. B., Parola, H. R., Hartwell, C. J., & Riegelman, A. (2020) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2020.103377 · Cited by 291
Principles for the Validation and Use of Personnel Selection Procedures
(2018) · Industrial and Organizational Psychology · doi:10.1017/iop.2018.195 · Cited by 300
“The rules by which the testing professionals categorize the work and jobs studied, the selection procedures used, the job performance criteria used, and other study characteristics that are hypothesized to impact the study results should be fully reported (Appelbaum et al., 2018; Aytug, Rothstein, Zhou, &Kern, 2012; G…”
MethodLåg, T., & Sæle, R. G. (2019) · AERA Open · doi:10.1177/2332858419870489 · Cited by 265
“This is recommended in various reporting standards (American Educational Research Association, 2006; Appelbaum et al., 2018; Schulz, Altman, & Moher, 2010) but is not generally followed.”
BackgroundIntroduction to PRISMA 2020 and implications for research synthesis methodologists
Page, M. J., Moher, D., & McKenzie, J. E. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1535 · Cited by 270
Power to Detect What? Considerations for Planning and Evaluating Sample Size
Giner-Sorolla, R., Montoya, A. K., Reifman, A., Carpenter, T., Lewis, N. A., Aberson, C. L., Bostyn, D. H., Conrique, B. G., Ng, B. W., Schoemann, A. M., et al. (2024) · Personality and Social Psychology Review · doi:10.1177/10888683241228328 · Cited by 221
“Current writing guides (e.g., Appelbaum et al., 2018; APA Publication Manual, American Psychological Association, 2020) leave unclear exactly how power should be reported in manuscripts.”
BackgroundSteffens, N. K., Munt, K. A., van Knippenberg, D., Platow, M. J., & Haslam, S. A. (2021) · Organizational Psychology Review · doi:10.1177/2041386620962569 · Cited by 267
“Following best-practice recommendations, in conducting and reporting the present research, we followed the American Psychological Association’s Meta-Analysis Reporting Standards (Appelbaum et al., 2018) and the Preferred Reporting Items for Systematic reviews and Meta-Analyses (Moher et al., 2009).”
MethodReporting guidelines for meta-analysis in economics
Havránek, T., Stanley, T. D., Doucouliagos, H., Bom, P., Geyer‐Klingeberg, J., Iwasaki, I., Reed, W. R., Rost, K., & van Aert, R. C. M. (2020) · Journal of Economic Surveys · doi:10.1111/joes.12363 · Cited by 271
“Although other guidelines for conducting and reporting meta-analyses have been offered (Stroup et al., 2000; Higgins and Green, 2017; Appelbaum et al., 2018), none other than Stanley et al. (2013) have explicitly considered the type of empirical evidence typically found in applied econometric…”
MethodMartínez-Gregorio, S., Badenes-Ribera, L., & Oliver, A. (2021) · The International Journal of Management Education · doi:10.1016/j.ijme.2021.100545 · Cited by 206
Steffens, N. K., LaRue, C. J., Haslam, C., Walter, Z. C., Cruwys, T., Munt, K. A., Haslam, S. A., Jetten, J., & Tarrant, M. (2021) · Health Psychology Review · doi:10.1080/17437199.2019.1669481 · Cited by 259
“…Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Liberati et al., 2009) and includes the information recommended by the Meta-Analysis Reporting Methods (MARS; Appelbaum et al., 2018), while the reporting of the study protocol follows the PRISMA-P guidelines (Moher et al., 2015).”
MethodUttley, J. (2019) · LEUKOS · doi:10.1080/15502724.2018.1533851 · Cited by 231
“To support this aim, lighting journals should consider adopting existing guidelines for reporting quantitative results provided by expert organizations such as the American Psychological Association (Appelbaum et al. 2018) for studies with outcome measures from the behavioral and social sciences.”
BackgroundPalmer, C. A., Bower, J. L., Cho, K. W., Clementi, M. A., Lau, S., Oosterhoff, B., & Alfano, C. A. (2024) · Psychological Bulletin · doi:10.1037/bul0000410 · Cited by 212
Cognitive and academic benefits of music training with children: A multilevel meta-analysis
Sala, G., & Gobet, F. (2020) · Memory & Cognition · doi:10.3758/s13421-020-01060-2 · Cited by 235
Meaningful associations in the adolescent brain cognitive development study
Dick, A. S., Lopez, D. A., Watts, A. L., Heeringa, S., Reuter, C., Bartsch, H., Fan, C. C., Kennedy, D. N., Palmer, C., Marshall, A., et al. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118262 · Cited by 241
Findley, M. G., Kikuta, K., & Denly, M. (2021) · Annual Review of Political Science · doi:10.1146/annurev-polisci-041719-102556 · Cited by 204
How to conduct a meta-analysis in eight steps: A practical guide
Hansen, C., Steinmetz, H., & Block, J. (2022) · Management Review Quarterly · doi:10.1007/s11301-021-00247-4 · Cited by 218
Best practices for addressing missing data through multiple imputation
Woods, A. D., Gerasimova, D., Van Dusen, B., Nissen, J., Bainter, S., Uzdavines, A., Davis‐Kean, P. E., Halvorson, M., King, K. M., Logan, J. A. R., et al. (2024) · Infant and Child Development · doi:10.1002/icd.2407 · Cited by 208
Use and misuse of corrections for multiple testing
García-Pérez, M. A. (2023) · Methods in Psychology · doi:10.1016/j.metip.2023.100120 · Cited by 194
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T., Mulrow, C. D., Shamseer, L., & Moher, D. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.11.010 · Cited by 211
“meta-analyses (1, 2, 14, 15, 17-19, 21, 23, 24, 26-38, 40-72) (included reports are listed in Table S1 in the Appendix, also available at https://osf.”
Zhang, M. F., Dawson, J. F., & Kline, R. B. (2021) · British Journal of Management · doi:10.1111/1467-8551.12415 · Cited by 183
“…method variance (Podsakoff et al., 2003), specification of alternative or equivalent models (Henley, Shook and Peterson, 2006), nonindependence of nested cases in multilevel data (Appelbaum et al., 2018), measurement invariance in cross-group analysis (Kite, Jorgensen and Chen, 2018) and so on.”
MethodInfluentialSelf-citationAguinis, H., Hill, N. S., & Bailey, J. R. (2021) · Organizational Research Methods · doi:10.1177/1094428119836485 · Cited by 180
“…transparency, many published articles in management and related fields do not include sufficient information on precise steps, decisions, and judgment calls made during a scientific study (Aguinis, Ramani, & Alabduljader, 2018; Aguinis & Solarino, 2019; Appelbaum et al., 2018; Levitt et al., 2018).”
BackgroundReducing bias, increasing transparency and calibrating confidence with preregistration
Hardwicke, T. E., & Wagenmakers, E.-J. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01497-2 · Cited by 179
A meta-analysis of autonomous and controlled forms of teacher motivation
Slemp, G. R., Field, J. G., & Cho, A. S. H. (2020) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2020.103459 · Cited by 179
Gusenbauer, M., & Gauster, S. P. (2024) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2024.123833 · Cited by 212
Life Events and Personality Change: A Systematic Review and Meta-Analysis
Bühler, J. L., Orth, U., Bleidorn, W., Weber, E., Kretzschmar, A., Scheling, L., & Hopwood, C. J. (2024) · European Journal of Personality · doi:10.1177/08902070231190219 · Cited by 159
Single‐case experimental designs: Characteristics, changes, and challenges
Kazdin, A. E. (2021) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.638 · Cited by 166
Richter, E., Brunner, M., & Richter, D. (2021) · Teaching and Teacher Education · doi:10.1016/j.tate.2021.103303 · Cited by 195
Diener, E., Northcott, R., Zyphur, M. J., & West, S. G. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916211037670 · Cited by 176
“Rossi (2013) documented the low statistical power of studies in leading psychology journals; only recently has a priori calculation of statistical power been emphasized as a standard in psychology (Appelbaum et al., 2018).”
BackgroundLarson, A. L., Cycyk, L. M., Carta, J. J., Hammer, C. S., Baralt, M., Uchikoshi, Y., An, Z. G., & Wood, C. L. (2019) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2019.06.001 · Cited by 182
A checklist to assess the quality of survey studies in psychology
Protogerou, C., & Hagger, M. S. (2020) · Methods in Psychology · doi:10.1016/j.metip.2020.100031 · Cited by 183
Orkibi, H., & Feniger-Schaal, R. (2019) · PLOS ONE · doi:10.1371/journal.pone.0212575 · Cited by 151
Muris, P., & Otgaar, H. (2023) · Psychology Research and Behavior Management · doi:10.2147/prbm.s402455 · Cited by 203
“To empirically test these suppositions, we conducted a meta-analysis which has the advantage of combining the results of multiple scientific studies, thereby providing a more accurate look at these topics.(38)”
MethodLeis, O., & Lautenbach, F. (2020) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2020.101738 · Cited by 166
Achievement of learners receiving UDL instruction: A meta-analysis
King‐Sears, M. E., Stefanidis, A., Evmenova, A. S., Rao, K., Mergen, R. L., Owen, L. S., & Strimel, M. M. (2022) · Teaching and Teacher Education · doi:10.1016/j.tate.2022.103956 · Cited by 207
Incorporating feasibility protocols in intervention research
Gadke, D. L., Kratochwill, T. R., & Gettinger, M. (2020) · Journal of School Psychology · doi:10.1016/j.jsp.2020.11.004 · Cited by 175
Easing Into Open Science: A Guide for Graduate Students and Their Advisors
Kathawalla, U., Silverstein, P., & Syed, M. (2021) · Collabra Psychology · doi:10.1525/collabra.18684 · Cited by 158
“, 2020; Wilson et al., 2017), and organizational systems to minimize mistakes (Rouder et al., 2019). One option is to set up a private or public project page on the OSF. Soderberg (2018) provides instructions on how to: (1) create an OSF account, (2) create a project, (3) add collaborators, (4) upload files, and (5) r…”
BackgroundSelecting the “Best” Factor Structure and Moving Measurement Validation Forward: An Illustration
Schmitt, T. A., Sass, D. A., Chappelle, W. L., & Thompson, W. T. (2018) · Journal of Personality Assessment · doi:10.1080/00223891.2018.1449116 · Cited by 161
“The American Psychological Association provides reporting standards for quantitative research in psychology, which includes structural equation modeling standards in Table 7 (Appelbaum et al., 2018).”
BackgroundImproving the reporting quality of reliability generalization meta‐analyses: The REGEMA checklist
Sánchez‐Meca, J., Marín‐Martínez, F., López-López, J. A., Núñez, R. M. N., Rubio‐Aparicio, M., García, J. J. L., Pina, J. A. L., Blázquez-Rincón, D., López‐Ibáñez, C., & López‐Nicolás, R. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1487 · Cited by 143
“The APA Publications and Communications Board Working Group elaborated MARS (Meta-Analysis Reporting Standards), a 74-item guideline aimed at improving the reporting of metaanalyses in psychology.41 More recently, Appelbaum et al.9 have updated the MARS guidelines.”
MethodInfluentialThe cognitive and academic benefits of Cogmed: A meta-analysis
Aksayli, N. D., Sala, G., & Gobet, F. (2019) · Educational Research Review · doi:10.1016/j.edurev.2019.04.003 · Cited by 123
Meyer, J. P., Isaacs, K. R., Shahawy, O. E., Burlew, A. K., & Wechsberg, W. M. (2019) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2019.01.017 · Cited by 135
“, 2018) and quantitative (Appelbaum et al., 2018) research methods updated recently in the American Psychologist provide only limited attention to gender.”
BackgroundThe Reliability of the Wisconsin Card Sorting Test in Clinical Practice
Kopp, B., Lange, F., & Steinke, A. (2019) · Assessment (Odessa, Fla.) · doi:10.1177/1073191119866257 · Cited by 112
Biddlestone, M., Green, R., Douglas, K. M., Azevedo, F., Sutton, R. M., & Cichocka, A. K. (2025) · Psychological Bulletin · doi:10.1037/bul0000463 · Cited by 152
The Way Forward in Mindfulness and Sustainability: a Critical Review and Research Agenda
Thiermann, U. B., & Sheate, W. R. (2020) · Journal of Cognitive Enhancement · doi:10.1007/s41465-020-00180-6 · Cited by 145
“In all cases, the reporting of the qualitative and mixed methods research process should follow standard procedures of qualitative research (Levitt et al. 2018).”
MethodReviewer Resources: Confirmatory Factor Analysis
Nye, C. D. (2022) · Organizational Research Methods · doi:10.1177/10944281221120541 · Cited by 154
Biddlestone, M., Azevedo, F., & van der Linden, S. (2022) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2022.101390 · Cited by 166
Telepsychotherapy for generalized anxiety disorder: Impact on the working alliance.
Watts, S., Marchand, A., Bouchard, S., Gosselin, P., Langlois, F., Belleville, G., & Dugas, M. J. (2020) · Journal of Psychotherapy Integration · doi:10.1037/int0000223 · Cited by 117
Schoeps, K., Mónaco, E., Cotolí, A., & Montoya‐Castilla, I. (2020) · PLoS ONE · doi:10.1371/journal.pone.0227627 · Cited by 163
Schoeps, K., Villanueva, L., Prado‐Gascó, V., & Montoya‐Castilla, I. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02050 · Cited by 154
“The results of the study are presented following the indications of APA for quantitative research in psychology (Appelbaum et al., 2018).”
BackgroundWu, H., & Shen, J. (2021) · Educational Research Review · doi:10.1016/j.edurev.2021.100423 · Cited by 172
Choi-Lundberg, D. L., Butler-Henderson, K. A., Harman, K., & Crawford, J. (2023) · Australasian Journal of Educational Technology · doi:10.14742/ajet.7615 · Cited by 157
“Authors can improve the quality of their studies and reporting through following published guidelines for designing and reporting studies, for example, “ Checklist for the Rigor of Education-Experiment Designs ” and others cited by its authors (Sung et al., 2019), Journal Article Reporting Standards for Quantitative R…”
Diversity in randomized clinical trials of depression: A 36-year review
Polo, A. J., Makol, B. A., Castro, A. S., Colón-Quintana, N., Wagstaff, A. E., & Guo, S. (2019) · Clinical Psychology Review · doi:10.1016/j.cpr.2018.09.004 · Cited by 114
Sun, C., Hon, C. K. H., Way, K. A., Jimmieson, N. L., & Xia, B. (2021) · Safety Science · doi:10.1016/j.ssci.2021.105485 · Cited by 128
Single-case design standards: An update and proposed upgrades
Kratochwill, T. R., Horner, R. H., Levin, J. R., Machalicek, W., Ferron, J. M., & Johnson, A. H. (2021) · Journal of School Psychology · doi:10.1016/j.jsp.2021.10.006 · Cited by 132
Howard, M. C., & Zandt, E. C. V. (2021) · Virtual Reality · doi:10.1007/s10055-021-00524-3 · Cited by 134
“We followed the Meta-Analysis Reporting Standards (MARS) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to perform and report our meta-analysis (Appelbaum et al. 2018; Kepes et al. 2013; Levitt et al. 2018; Moher et al. 2009, 2015).”
MethodCorrecting for publication bias in a meta-analysis with the p-uniform* method
van Aert, R. C. M., & van Assen, M. A. L. M. (2026) · Psychonomic Bulletin & Review · doi:10.3758/s13423-025-02812-4 · Cited by 124
“This is in agreement with the Meta-Analytic Reporting Standards (MARS; Appelbaum et al., 2018) that recommends assessing the impact of publication bias in any meta-analysis.”
Strategic Considerations in Auditing.
Fellingham, J. C., & Newman, D. P. (1985) · The Accounting Review · doi:10.2308/tar-4491649 · Cited by 107
Youths’ and Adults’ Identity in STEM: a Systematic Literature Review
Simpson, A. M., & Bouhafa, Y. (2020) · Journal for STEM Education Research · doi:10.1007/s41979-020-00034-y · Cited by 118
Isbell, D. R., BROWN, D., Chen, M., Derrick, D. J., Ghanem, R., Arvizu, M. N. G., Schnur, E., Zhang, M., & Plonsky, L. (2022) · Modern Language Journal · doi:10.1111/modl.12760 · Cited by 108
Affect and cannabis use in daily life: a review and recommendations for future research
Wycoff, A. M., Metrik, J., & Trull, T. J. (2018) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2018.07.001 · Cited by 117
A Systematic Review of Momentary Assessment Designs for Mood and Anxiety Symptoms
Hall, M., Scherner, P. V., Kreidel, Y., & Rubel, J. A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.642044 · Cited by 120
“We therefore strongly advise future EMA researchers to provide item phrasing, psychometric properties and further materials relevant to the design (Appelbaum et al., 2018; Vachon et al., 2019; Trull and Ebner-Priemer, 2020).”
BackgroundEffectiveness of conceptual change strategies in science education: A meta‐analysis
Paçacı, Ç., Üstün, U., & Özdemir, Ö. F. (2023) · Journal of Research in Science Teaching · doi:10.1002/tea.21887 · Cited by 113
Study quality as an intellectual and ethical imperative: A proposed framework
Plonsky, L. (2024) · Annual Review of Applied Linguistics · doi:10.1017/s0267190524000059 · Cited by 101
“Of course, well thought-out guidelines exist from other disciplines, such as education and psychology (e.g., Appelbaum et al., 2018), but we are not education or psychology.”
Kratochwill, T. R., Horner, R. H., Levin, J. R., Machalicek, W., Ferron, J. M., & Johnson, A. H. (2023) · Journal of School Psychology · doi:10.1016/j.jsp.2022.12.002 · Cited by 101
Latan, H., Jabbour, C. J. C., Jabbour, A. B. L. D. S., Fiorini, P. D. C., & Foropon, C. R. H. (2019) · International Journal of Production Economics · doi:10.1016/j.ijpe.2019.107526 · Cited by 100
“Robustness checks In the final step, we performed several complementary analyses to maintain the robustness of our main results, which has become a mandatory step in reporting the results of SEM analysis (Appelbaum et al. 2018; Ketokivi and McIntosh 2017; Latan 2018).”
BackgroundMethodInfluentialNew Recommendations on the Use of R-Squared Differences in Multilevel Model Comparisons
Rights, J. D., & Sterba, S. K. (2019) · Multivariate Behavioral Research · doi:10.1080/00273171.2019.1660605 · Cited by 109
“The use of effect size measures has been widely recommended to help social scientists move beyond exclusive reliance on statistical significance when fitting and comparing models (e.g., APA, 2008, 2009; Appelbaum et al., 2018; Harlow, Mulaik, & Steiger, 1997; Kelley & Preacher, 2012; Panter & Sterba, 2011).”
BackgroundAdverse Childhood Experiences and Body Dysmorphic Symptoms: A Meta-Analysis
Longobardi, C., Badenes-Ribera, L., & Fabris, M. A. (2022) · Body Image · doi:10.1016/j.bodyim.2022.01.003 · Cited by 86
Riegler, L. J., Raj, S. P., Moscato, E. L., Narad, M. E., Kincaid, A., & Wade, S. L. (2020) · Journal of Psychotherapy Integration · doi:10.1037/int0000220 · Cited by 92
Self-reported cognitive biases in depression: A meta-analysis
Nieto, I., Robles, E., & Vázquez, C. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101934 · Cited by 99
Mindfulness as a way to cope with COVID‐19‐related stress and anxiety.
Weis, R., Ray, S. D., & Cohen, T. A. (2021) · Counselling and Psychotherapy Research · doi:10.1002/capr.12375 · Cited by 82
Making sense of learning gain in higher education
Evans, C., Kandiko Howson, C., & Forsythe, A. (2018) · Higher Education Pedagogies · doi:10.1080/23752696.2018.1508360 · Cited by 81
“Detailed standards for reporting are now commonplace in fields that education science draws upon, (see, for example, APA, 2008; Appelbaum et al., 2018; Levitt et al., 2018) but those standards are not necessarily so transparent in pedagogical research.”
BackgroundBadenes-Ribera, L., Rubio-Aparicio, M., Sánchez-Meca, J., Fabris, M. A., & Longobardi, C. (2019) · Journal of Behavioral Addictions · doi:10.1556/2006.8.2019.44 · Cited by 80
Ferrero, M., Vadillo, M. A., & León, S. P. (2021) · Intelligence · doi:10.1016/j.intell.2021.101566 · Cited by 99
Паолини, С., Gibbs, M., Sales, B., Anderson, D., & McIntyre, K. (2024) · Psychological Bulletin · doi:10.1037/bul0000439 · Cited by 94
Meta-analysis of Big Five personality traits in autism spectrum disorder
Lodi‐Smith, J., Rodgers, J. D., Cunningham, S. A., Lopata, C. J., & Thomeer, M. L. (2018) · Autism · doi:10.1177/1362361318766571 · Cited by 101
Predictors of turnover amongst volunteers: A systematic review and meta‐analysis
Forner, V. W., Holtrop, D., Boezeman, E. J., Slemp, G. R., Kotek, M., Kragt, D., Askovic, M., & Johnson, A. M. (2023) · Journal of Organizational Behavior · doi:10.1002/job.2729 · Cited by 91
Does co-teaching improve academic achievement for students with disabilities? A meta-analysis
King‐Sears, M. E., Stefanidis, A., Berkeley, S. L., & Strogilos, V. (2021) · Educational Research Review · doi:10.1016/j.edurev.2021.100405 · Cited by 115
Metadata
- Authors
- Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M.
- Affiliations
- University of California San Diego (US)
- Duke University (US)
- Concordia University (US)
- Drexel University (US)
- Cleveland Clinic (US)
- Venue
- American Psychologist
- Published
- 2018-01
- Volume
- 73
- Issue
- 1
- Pages
- 3-25
- ISSN
- 0003-066X (ISSN-L)
- 1935-990X
- OpenAlex
- W2784576540
- Cited by
- 1,449
- References
- 35
- Citations by year
- 2026: 258 · 2025: 246 · 2024: 204 · 2023: 172 · 2022: 154 · 2021: 142 · 2020: 133 · 2019: 89 · 2018: 47
- FWCI
- 78.36
- Type
- journal article
- Language
- en
- Update notices
- correction (2018-10-04)
- Open access
- hybrid
- doi.org
- Locations
- American Psychologist (publishedVersion, other-oa)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- reporting guidelines, meta-analysis reporting, observational studies, clinical trials, longitudinal study, replication study, N-of-1 studies, structural equation modeling, Bayesian analysis, scientific publications
- MeSH terms
- Humans, Periodicals as Topic, Psychology, Research Design, Societies, Scientific, Peer Review, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.