The DOI matches its Crossref record; 132 of the 132 references it cites were found in a registry or an index.
Facts Are More Important Than Novelty: Replication in the Education Sciences
Abstract
Despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not on the merit of replicating important results. The present study analyzed the complete publication history of the current top 100 education journals ranked by 5-year impact factor and found that only 0.13% of education articles were replications. Contrary to previous findings in medicine, but similar to psychology, the majority of education replications successfully replicated the original studies. However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. The results emphasize the importance of third-party, direct replications in helping education research improve its ability to shape education policy and practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Novel or innovative results are more likely to be published than replications or incremental extensions of existing findings 13-15 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [15]
Citations
Cites
2 retracted
Wakefield, A., Murch, S., Anthony, A., Linnell, J., Casson, D., Malik, M., Berelowitz, M., Dhillon, A., Thomson, M., Harvey, P., et al. (1998) · The Lancet · doi:10.1016/s0140-6736(97)11096-0 · Cited by 3,006
RetractedLancet, T. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60175-4 · Cited by 500
Retracted(1963) · Medical Entomology and Zoology · Cited by 15,085
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
An Introduction to Categorical Data Analysis
Agresti, A. (2006) · Wiley series in probability and statistics · doi:10.1002/0470114754 · Cited by 6,775
Journal of social behavior and personality
(1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90025-5 · Cited by 6,751
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Moffitt, R. (2009) · American Economic Review · doi:10.1257/aer.99.2.660 · Cited by 4,888
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 5×
“Using health care research as an example, Ioannidis (2005a), in a review of highly cited medical publications (i.e., those cited more than 1,000 times), found only 44% of replications produced results similar to the original study.”
ResultInfluential
Research works printed but not linked: 14
- 10.1037//0003–066x.49.12.997 · DOI · Paper page
- Collins H. M., 1985, Changing order
- Kane E. J., 1984, Quarterly Journal of Business and Economics, 23, 3
- Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 1
- Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 21
- Perspectives on Psychological Science. (2012). Special section on replicability in psychological science: A crisis of confidence? Retrieved from http://pps.sagepub.com/content/7/6.toc · DOI · Paper page
- Powell K., 2007, PLoS Biology, 12, 2748
- Rhine J. B., 1997, Extra-sensory perception
- Ritchie S. J., 2012, Psychologist, 25, 346
- 10.1037//0033–2909.86.3.638 · DOI · Paper page
- Rotton J., 1995, Journal of Social Behavior and Personality, 10, 1
- Simons D. J., 2012, Psychologist, 25, 349
- Simons D. J., 2014, Perspectives on Psychological Science. APS Observer, 27, 16
- Williams S. N., 2012, Science, 336, 801
Cited by
Sorted by citations
A systematic review of learning computational thinking through Scratch in K-9
Zhang, L., & Nouri, J. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.103607 · Cited by 538
Steenbergen-Hu, S., Makel, M. C., & Olszewski-Kubilius, P. (2016) · Review of Educational Research · doi:10.3102/0034654316675417 · Cited by 429
Self-citationAssessment rubrics: towards clearer and more replicable design, research and practice
Dawson, P. (2017) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2015.1111294 · Cited by 368
“Compared to other disciplines, educational researchers publish few replication studies (Makel and Plucker 2014).”
BackgroundHow school leadership influences student learning: A test of “The four paths model”
Leithwood, K., Sun, J., & Schumacker, R. (2020) · Educational Administration Quarterly · doi:10.1177/0013161x19878772 · Cited by 329
“DISCUSSION AND CONCLUSIONS Replication studies are relatively rare in the educational sciences, in general, and educational leadership studies, in particular ((Makel and Plucker, 2014).”
BackgroundChallenges for assessing replicability in preclinical cancer biology
Errington, T. M., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.67995 · Cited by 345
Metadata
- Authors
- Makel, M. C., & Plucker, J. A.
- Affiliations
- Duke University (US)
- University of Connecticut (US)
- Venue
- Educational Researcher
- Published
- 2014-08
- Volume
- 43
- Issue
- 6
- Pages
- 304-316
- ISSN
- 0013-189X (ISSN-L)
- 1935-102X
- OpenAlex
- W2139319950
- Semantic Scholar
- 493a2e93117a876871ac446b26acf47c50ea2cef
- Cited by
- 609
- References
- 131
- Citations by year
- 2026: 17 · 2025: 37 · 2024: 40 · 2023: 40 · 2022: 66 · 2021: 47 · 2020: 59 · 2019: 81 · 2018: 73 · 2017: 52 · 2016: 51 · 2015: 37 · 2014: 8
- FWCI
- 68.61
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Educational Researcher (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Education › Educational Research and Methods
- Keywords
- replication, direct replication, educational research, publishing history, research credibility, methodological quality, replication success, educational periodicals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
