Why the resistance to statistical innovations? Bridging the communication gap.
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
While quantitative methodologists advance statistical theory and refine statistical methods, substantive researchers resist adopting many of these statistical innovations. Traditional explanations for this resistance are reviewed, specifically a lack of awareness of statistical developments, the failure of journal editors to mandate change, publish or perish pressures, the unavailability of user friendly software, inadequate education in statistics, and psychological factors. Resistance is reconsidered in light of the complexity of modern statistical methods and a communication gap between substantive researchers and quantitative methodologists. The concept of a Maven is introduced as a means to bridge the communication gap. On the basis of this review and reconsideration, recommendations are made to improve communication of statistical innovations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.6.1173 · Cited by 91,532
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971
Evaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“Paradoxically, those same underpowered studies sometimes produce statistically significant results, albeit spurious results that fail to replicate (Button et al., 2013).”
ResultSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
On the practice of dichotomization of quantitative variables.
MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.19 · Cited by 3,538 · cited 1×
“MacCallum et al. (2002) suggested that the practice of dichotomization of continuous variables reflects researchers’ and graduate students’ greater familiarity with ANOVA over regression.”
BackgroundStatistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 1×
“Researchers can cook results to make them statistically significant, mine data looking for statistically significant results, and selectively publish to support preexisting hypotheses (Fanelli, 2009).”
BackgroundPiderit, S. K. (2000) · The Academy of Management Review · doi:10.5465/amr.2000.3707722 · Cited by 1,873
Best practices for missing data management in counseling psychology
Schlomer, G. L., Bauman, S., & Card, N. A. (2010) · Journal of Counseling Psychology · doi:10.1037/a0018082 · Cited by 1,835
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Practical significance: a concept whose time has come
Kirk, R. E. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056005002 · Cited by 1,778 · cited 11×
“The growth in popularity of structural equation modeling begat other structural equation software such as EQS, Amos, and Mplus (see Byrne, 2012). Similarly, power analysis was facilitated by the presence of readily accessible software such as G Power (Faul, Erdfelder, Lang, & Buchner, 2007). A bad news software story…”
BackgroundMethodInfluentialNegative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248 · cited 1×
“Competition for academic jobs and research funding demands frequent publication in the top ranked journals, those journals with the highest rejection rates, the most challenging reviewers, the most selective editors, and the most compelling, hypothesis supporting findings (Fanelli, 2012).”
BackgroundWhen small effects are impressive
Prentice, D. A., & Miller, D. T. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.112.1.160 · Cited by 1,008 · cited 1×
“A small effect can be important when the independent variable is manipulated minimally or the dependent variable is difficult to influence (Prentice & Miller, 1992).”
BackgroundModern robust statistical methods: An easy way to maximize the accuracy and power of your research.
Erceg-Hurn, D. M., & Mirosevich, V. M. (2008) · American Psychologist · doi:10.1037/0003-066x.63.7.591 · Cited by 959 · cited 2×
“Indeed, Erceg-Hurn and Mirosevich (2008) authored a recent publication in American Psychologist with the explicit goal of raising awareness of robust statistics.”
BackgroundMethodBem, D. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0021524 · Cited by 893 · cited 2×
“Recent developments including controversial finding of psi (Bem, 2011), admissions of data fraud and data fudging in social psychology (Ferguson, 2012), and perplexing failures to replicate established research findings (Yong, 2012) serve to highlight the vulnerability of and our dependence on…”
BackgroundDo studies of statistical power have an effect on the power of studies?
Sedlmeier, P., & Gigerenzer, G. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.2.309 · Cited by 748 · cited 2×
“Cohen found the psychological literature to be riddled with underpowered studies, a finding replicated a quarter century later by Sedlmeier and Gigerenzer (1989).”
BackgroundStepwise Regression and Stepwise Discriminant Analysis Need Not Apply here: A Guidelines Editorial
Thompson, B. (1995) · Educational and Psychological Measurement · doi:10.1177/0013164495055004001 · Cited by 434
Jaccard, J., & Guilamo-Ramos, V. (2002) · Journal of Clinical Child & Adolescent Psychology · doi:10.1207/s15374424jccp3101_15 · Cited by 248 · cited 2×
“A clear description of how to calculate contrasts following a factorial ANOVA is found in an appendix of an article in the Journal of Clinical Child and Adolescent Psychology (Jaccard & Guilamo-Ramos, 2002); Osborne (2010b) provided a state of the art tutorial on data cleaning in Newborn and Infant Nursing Reviews; Du…”
MethodStatistical Reform in Psychology: Is Anything Changing?
Cumming, G., Fidler, F., Leonard, M., Kalinowski, P., Christiansen, A., Kleinig, A., Lo, J., McMenamin, N., & Wilson, S. (2007) · Psychological Science · doi:10.1111/j.1467-9280.2007.01881.x · Cited by 122 · cited 9×
“A clear description of how to calculate contrasts following a factorial ANOVA is found in an appendix of an article in the Journal of Clinical Child and Adolescent Psychology (Jaccard & Guilamo-Ramos, 2002); Osborne (2010b) provided a state of the art tutorial on data cleaning in Newborn and Infant Nursing Reviews; Du…”
BackgroundMethodInfluentialStatistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial
Cobo, E., Selva-O'Callagham, A., Ribera, J.-M., Cardellach, F., Dominguez, R., & Vilardell, M. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000332 · Cited by 112
Cited by
Thanks coefficient alpha, we’ll take it from here.
McNeish, D. (2018) · Psychological Methods · doi:10.1037/met0000144 · Cited by 2,533
Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications
Wagenmakers, E.-J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., et al. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1343-3 · Cited by 1,627
A Review of Key Likert Scale Development Advances: 1995–2019
Jebb, A. T., Ng, V., & Tay, L. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.637547 · Cited by 770
The fickle P value generates irreproducible results
Halsey, L. G., Curran-Everett, D., Vowler, S. L., & Drummond, G. B. (2015) · Nature Methods · doi:10.1038/nmeth.3288 · Cited by 765
On the unnecessary ubiquity of hierarchical linear modeling.
McNeish, D., Stapleton, L. M., & Silverman, R. D. (2017) · Psychological Methods · doi:10.1037/met0000078 · Cited by 742
Metadata
- Authors
- Sharpe, D.
- Venue
- Psychological Methods
- Published
- 2013
- Volume
- 18
- Issue
- 4
- Pages
- 572-582
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2102599529
- Cited by
- 100
- References
- 21
- FWCI
- 15.31
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.