Cited by 609Psychology Research and BibliometricsMeta-analysis and systematic reviewsEducational Research and Methods

The DOI matches its Crossref record; 132 of the 132 references it cites were found in a registry or an index.

Facts Are More Important Than Novelty: Replication in the Education Sciences

Makel, Matthew C., Plucker, Jonathan A.·2014·Educational Researcher

Open this paper in CoChat — free account

Abstract

Despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not on the merit of replicating important results. The present study analyzed the complete publication history of the current top 100 education journals ranked by 5-year impact factor and found that only 0.13% of education articles were replications. Contrary to previous findings in medicine, but similar to psychology, the majority of education replications successfully replicated the original studies. However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. The results emphasize the importance of third-party, direct replications in helping education research improve its ability to shape education policy and practice.

Integrity signals

Registry

Crossref · 10.3102/0013189x14545513

Cites

132 of 132 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

2 retracted

RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children
Wakefield, A., Murch, S., Anthony, A., Linnell, J., Casson, D., Malik, M., Berelowitz, M., Dhillon, A., Thomson, M., Harvey, P., et al. (1998) · The Lancet · doi:10.1016/s0140-6736(97)11096-0 · Cited by 3,006
Retracted
Retraction—Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children
Lancet, T. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60175-4 · Cited by 500
Retracted
Untitled work
(1963) · Medical Entomology and Zoology · Cited by 15,085
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
An Introduction to Categorical Data Analysis
Agresti, A. (2006) · Wiley series in probability and statistics · doi:10.1002/0470114754 · Cited by 6,775
Journal of social behavior and personality
(1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90025-5 · Cited by 6,751
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
American Economic Review
Moffitt, R. (2009) · American Economic Review · doi:10.1257/aer.99.2.660 · Cited by 4,888
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 5×
“Using health care research as an example, Ioannidis (2005a), in a review of highly cited medical publications (i.e., those cited more than 1,000 times), found only 44% of replications produced results similar to the original study.”
ResultInfluential

Show all 131 references

Research works printed but not linked: 14

10.1037//0003–066x.49.12.997 · DOI · Paper page
Collins H. M., 1985, Changing order
Kane E. J., 1984, Quarterly Journal of Business and Economics, 23, 3
Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 1
Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 21
Perspectives on Psychological Science. (2012). Special section on replicability in psychological science: A crisis of confidence? Retrieved from http://pps.sagepub.com/content/7/6.toc · DOI · Paper page
Powell K., 2007, PLoS Biology, 12, 2748
Rhine J. B., 1997, Extra-sensory perception
Ritchie S. J., 2012, Psychologist, 25, 346
10.1037//0033–2909.86.3.638 · DOI · Paper page
Rotton J., 1995, Journal of Social Behavior and Personality, 10, 1
Simons D. J., 2012, Psychologist, 25, 349
Simons D. J., 2014, Perspectives on Psychological Science. APS Observer, 27, 16
Williams S. N., 2012, Science, 336, 801

Sorted by citations

Items 401–500 of 608

Interpretations of reproducibility crisis in medical education research: a qualitative study
Ahmady, S., Kohan, N., Hamidi, H., Vahednasiri, A., & Hooshmandja, M. (2026) · Scientific Reports · doi:10.1038/s41598-025-34640-w · Cited by 4
Replicability: 21st Century Crisis of the Positivist Social Sciences
Kalelioğlu, U. B. (2021) · Uluslararası Sosyal Bilgilerde Yeni Yaklaşımlar Dergisi (IJONASS) · doi:10.38015/sbyy.1003103 · Cited by 4
Can social cognitive theory explain breakfast frequency in workplace institutional feeding populations?
Harris, J., Carins, J., Rundle-Thiele, S. R., & David, P. (2022) · Journal of Social Marketing · doi:10.1108/jsocm-09-2021-0214 · Cited by 4
A Framework for Approaching Mixed Methods Intervention Research to Address the Emotional and Behavioral Health Needs of Children
Houchins, D. E., Hitchcock, J. H., & Conroy, M. A. (2022) · Behavioral Disorders · doi:10.1177/01987429221131279 · Cited by 9
“Replication refers to degree to which intervention effects replicate across different participants, settings, and time to either reject or confirm theories under dissimilar or similar conditions (Makel & Plucker, 2014).”
Background
Supporting the replication of your research
Forbes, H. J., Travers, J. C., & Johnson, J. V. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-90969-3.00003-7 · Cited by 4
Teachers’ judgment accuracy: A replication check by psychometric meta-analysis
Kaufmann, E. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307594 · Cited by 5
Cross-cultural application of positive psychology therapy for children and adolescents in low- and middle-income countries: A scoping review
Suemith-Fajardo, A., Taylor, A. M., & Roberts, R. (2025) · The Journal of Positive Psychology · doi:10.1080/17439760.2025.2542224 · Cited by 4
“The scarcity of replication studies compromises the robustness of findings in diverse cultural contexts (Makel & Plucker, 2014).”
School-Based Team Research to Address Grand Challenges Through P–12 Physical Education Programs
Metzler, M. W. (2016) · Research Quarterly for Exercise and Sport · doi:10.1080/02701367.2016.1234284 · Cited by 4
“In our ﬁ rst 40-plus years of research in physical education, we have not promoted or valued new studies based on planned variations of previous studies — but we are not alone among educational researchers (Makel & Plucker, 2014).”
From Achievement to Non-Test Outcomes in PISA: National Trends in STEM Career Expectations
Han, S. W. (2017) · Emerald Group Publishing Limited eBooks · doi:10.1108/s1479-367920160000031002 · Cited by 4
The Role of Theory in Quantitative Data Analysis
Pigott, T. (2017) · SAGE Publications Ltd eBooks · doi:10.4135/9781473983953.n35 · Cited by 4
“In a sense, using theory to connect our work to others’ work is a form of conceptual replication (Makel and Plucker, 2014).”
Background
Repetita Iuvant? A Repeated Randomized Controlled Trial on the Effectiveness of an At-Scale Teacher Professional Development Program
Abbiati, G., Argentin, G., Caputo, A., & Pennisi, A. (2021) · Evaluation Review · doi:10.1177/0193841x211055354 · Cited by 4
Reconsidering Achievement Goal Orientation for Undergraduate Honors Students
Miller, A. L. (2022) · Journal of Advanced Academics · doi:10.1177/1932202x221086139 · Cited by 5
“Replicating results within different contexts and populations is an important part of generating scientific knowledge (Makel & Plucker, 2014), as is the publication of results that are not statistically significant (Whitley, 2002).”
Result
Open Science: Recommendations for Research on School Bullying
Noret, N. F., Hunter, S. C., Pimenta, S. M., Taylor, R., & Johnson, R. (2022) · International Journal of Bullying Prevention · doi:10.1007/s42380-022-00130-0 · Cited by 7
Digitale Lehre nachhaltig gestalten
:null (2022) · Waxmann Verlag GmbH eBooks · doi:10.31244/9783830996330 · Cited by 5
Unfortunately, Journals in Industrial, Work, and Organizational Psychology Still Fail to Support Open Science Practices
Hüffmeier, J., & Mertes, M. (2023) · Meta-Psychology · doi:10.15626/mp.2022.3271 · Cited by 4
“…for replication studies that is well-documented for most journals (Tipu and Ryan, 2021; see also Evanschitzky et al., 2007; Tenney et al., 2021) limits our collective ability to establish a cumulative knowledge base and to “differentiate ‘truth from nonsense’” (Kidwell et al., 2014, p. 304).”
Background
Tablet-based intervention to foster music-related hand responses and positive engagement in people with advanced Alzheimer’s disease
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Laporta, D., Scordamaglia, A., & Pinto, K. (2019) · Journal of Enabling Technologies · doi:10.1108/jet-06-2018-0027 · Cited by 4
Open Science in the Field of Emotional and Behavioral Disorders
Cook, B. G., Lloyd, J. W., & Therrien, W. J. (2019) · Education and Treatment of Children · doi:10.1353/etc.2019.0027 · Cited by 4
How Best to Quantify Replication Success? A Simulation Study on the Comparison of Replication Success MetricsPreprint
Muradchanian, J., Hoekstra, R., Kiers, H. A. L., & van Ravenzwaaij, D. (2020) · doi:10.31222/osf.io/wvdjf · Cited by 4
Assessing the replication landscape in experimental linguisticsPreprint
Kobrock, K., & Roettger, T. B. (2022) · doi:10.31234/osf.io/fzngs · Cited by 4
“Researchers from diverse fields such as psychology (Makel et al., 2012), educational science (Makel & Plucker, 2014), ecology (Kelly, 2019), criminology (McNeeley & Warner, 2015), and economics (Mueller-Langer et al., 2019) report on very low numbers of published replications, ranging from 0.02% in…”
BackgroundResultInfluential
A Call for Open Science in Giftedness ResearchPreprint
McBee, M. T., Makel, M. C., Peters, S. J., & Matthews, M. S. (2017) · doi:10.31234/osf.io/nhuv3 · Cited by 4
Self-citation
Trendlines in research methodology and topics in teacher education: big data analysis of theses in Swedish teacher education between 2005 and 2023
Skau, S., Börjesson, M., Nyman, R., & Thompson, W. H. (2025) · Scandinavian Journal of Educational Research · doi:10.1080/00313831.2025.2506378 · Cited by 3
“…al., 2021; Maniadis et al., 2017; Open-Science-Collaboration, 2015) with only a few attempts to evaluate its prevalence in the educational sciences (Makel & Plucker, 2014; Perry et al., 2022), it is problematic that such a low proportion of the teachers has had any training evaluating such…”
The Design of Current Replication Studies: A Systematic Literature Review on the Variation of Study Characteristics
Hoffmann, J., Twardawski, M., Höhs, J. M., Gast, A., Pohl, S., & Sengewald, M.-A. D.-P. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251328273 · Cited by 5
“Furthermore, overlap of the authors of the primary and replication studies is associated with more successful replications (e.g., Lemons et al., 2016; Makel et al., 2012; Makel & Plucker, 2014), maybe because study characteristics can be more readily kept constant in replication studies.”
Relationships between the inclusion of students with special educational needs and their classmates’ school outcomes – a replication study
Kocaj, A. (2025) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2025.102426 · Cited by 9
Evidenz, Signifikanz und das kleine p
Passon, O., & von der Twer, T. (2020) · Zeitschrift für Bildungsforschung · doi:10.1007/s35834-020-00282-3 · Cited by 3
Same as It Ever Was: An Updated Review of Replication Studies in Special Education Journals
Cook, B. G., Therrien, W. J., Waterfield, D. A., McClain, S. S., Fleming, J. I., Robinson, H. E., Watson, L., & Boyle, J. J. (2024) · Remedial and Special Education · doi:10.1177/07419325241248766 · Cited by 9
“Perry et al. (2022) updated Makel and Plucker’s (2014) review of replication research in the education sciences and found that the rate of replication studies had increased from 0.13% (reported by Makel & Plucker, 2014) to 0.20% of publications in education journals from 2011 to 2020.”
Influential
Conducting replication in translation and interpreting studies
Han, C., & Wang, Y. (2025) · Target International Journal of Translation Studies · doi:10.1075/target.23164.han · Cited by 3
Persons with Mild and Moderate Alzheimer’s Disease Use Simple Technology to Support Their Leisure Engagement
Lancioni, G. E., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Pinto, K., Vanna, F. D., & Scordamaglia, A. (2016) · Advances in Neurodevelopmental Disorders · doi:10.1007/s41252-016-0002-4 · Cited by 3
“Given this situation, additional research efforts seem to be warranted to determine the strength of the data available and investigate possible upgrades/extensions of the program so as to make it suitable for a wider number of persons (Godwin et al. 2013; Kazdin 2011; Makel and Plucker 2014).”
Background
Dental training in evidence-based dentistry: A replication study
Gerhardt-, S. S. (2017) · Journal of Dental Problems and Solutions · doi:10.17352/2394-8418.000052 · Cited by 3
“13% of education articles were replications [1].”
BackgroundInfluential
Who Wants to Learn More Science? The Role of Elementary School Science Experiences and Science Self-Perceptions
Aschbacher, P. R., & Ing, M. (2017) · Teachers College Record · doi:10.1177/016146811711900808
“Of course, future research replicating this relationship with different samples and measures is encouraged (Makel & Plucker, 2014).”
Background
Instructor Caring: Using Self-Determination Theory to Understand Perceptions, Measurement, and Impact of Instructor Caring on Motivation and Learning in Online Contexts
Lawrence, A. J. (2018) · UKnowledge (University of Kentucky) · doi:10.13023/etd.2018.249 · Cited by 4
“Replication is an important and sometimes undervalued possibility (Kaufmann & Tatum, 2017; Makel & Plucker, 2014; McElreath & Smaldino, 2015).”
Background
Collaboration in Giftedness and Talent Development Research
Makel, M. C., Smith, K. N., Miller, E. M., Peters, S. J., & McBee, M. T. (2020) · journal for the education of the gifted · doi:10.1177/0162353220912019 · Cited by 8
“Similarly, when researchers attempt to replicate their own previous findings, their success rate tends to be higher than when independent research teams attempt to replicate those findings (Makel & Plucker, 2014).”
ResultSelf-citation
Reproducibility literature analysis - a federal information professional perspective
Antognoli, E., Avila, R. L., Sears, J., Christiansen, L. L., Tieman, J., & Hart, J. (2020) · IASSIST Quarterly · doi:10.29173/iq967 · Cited by 2
The Unsuccessful Self-treatment of a Case of 'Writer’s Block': A Replication in Science Education
Ampatzidis, G. (2021) · Journal of Trial and Error · doi:10.36850/e7 · Cited by 3
An Investigation of Replication Practice in Nursing Education Research
Abbott, M., & Nininger, J. M. (2023) · Nursing Education Perspectives · doi:10.1097/01.nep.0000000000001116 · Cited by 3
A replication study on the intuitiveness of programming language syntax
Lappi, V., Tirronen, V., & Itkonen, J. (2023) · Software Quality Journal · doi:10.1007/s11219-023-09631-7 · Cited by 6
“Furthermore, a big portion of the replications are conducted by the same author as the original study (Makel & Plucker, 2014; Hao et al., 2018).”
BackgroundResult
Faculty beliefs and the need for teaching improvement: a conceptual replication study
Kanuka, H., Smith, E. E., & Luth, R. W. (2023) · Teaching in Higher Education · doi:10.1080/13562517.2023.2244887 · Cited by 3
“…and procedures (including variation with the data collection instruments, participants, time and/or place) but also tests the generalizability of the original study and considers the conditions under which the prior findings may or may not hold true (Makel and Plucker 2014; Polio and Gass 1997).”
Groundbreaking issue of the Cambodian Journal of Educational and Social Sciences (CJESS)
Em, S. (2024) · Cambodian journal of educational and social sciences. · doi:10.69496/cjess.v1i1.4 · Cited by 3
Giving Away the Science of School Psychology
Ash, T. L. (2024) · School Psychology Review · doi:10.1080/2372966x.2024.2417234 · Cited by 3
“Editors should reward good science—rather than flashy find-ings—in their review process (Makel & Plucker, 2014).”
How to do research in classroom settings
Goudeau, S., Easterbrook, M. J., & Fayant, M. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02027-y · Cited by 3
Module-Phase-Dependent Development of Pedagogical Content Knowledge: Replicating a Role-Change Approach in Pre-Service Teacher Education in an Outreach Lab
Scharfenberg, F., & Bogner, F. X. (2019) · Research in Science Education · doi:10.1007/s11165-019-09887-9 · Cited by 3
Instructional Design Methods and Practice
Spector, J. M. (2015) · Lecture notes in educational technology · doi:10.1007/978-3-662-47956-8_3 · Cited by 3
How Special are Teachers of Specialized Schools? Assessing Self-Confidence Levels in the Technology Domain
Çorlu, M. S., & Çatma, Z. (2016) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.12973/eurasia.2016.1245a · Cited by 3
The Effects of Mathematics Placement on Successful Completion of an Engineering Degree and How One Student Beat the Odds.
Dyken, V., & Elizabeth, J. (2016) · ProQuest LLC eBooks · Cited by 3
Estimating the Standard Deviation From the Range: a Replication of Analysis of Demographic Data Reported in Marriage & Family Review, 2016-2017
Schumm, W. R., Crawford, D. W., Higgins, M., Lockett, L., AlRashed, A., & bin Ateeq, A. (2018) · Marriage & Family Review · doi:10.1080/01494929.2018.1500128 · Cited by 3
“…do not replicate previous findings (Bohannon, 2015; Earp & Trafimow, 2016; Fiedler & Praeger, 2018; Francis, 2012; Hunter, 2001; King, 1995; Makel & Plucker, 2014; Makel, Plucker, Freeman, Lombardi, Simonsen, & Coyne, 2016; Maxwell, Lau, & Howard, 2015; McEwan, Carpenter, & Westerman, 2018;…”
The Report-Research Continuum: A Differentiation Decision-Making Tool for Teachers to Design Rigorous Student Projects
Gilson, C. M., & Brigandi, C. B. (2020) · Gifted Child Today · doi:10.1177/1076217520916769 · Cited by 3
Learning, Technology, and Technique
Dron, J. (2022) · Canadian Journal of Learning and Technology · doi:10.21432/cjlt28315 · Cited by 3
Potential for Meta-Scientific Inquiry to Improve the Usefulness of HRD Research Outcomes for Practice
Passmore, D. L., & Chae, C. (2019) · Advances in Developing Human Resources · doi:10.1177/1523422319870173 · Cited by 4
“For instance, Makel and Plucker (2014b) found that replications comprised only 0.13% of the approximately 412 Advances in Developing Human Resources 21(4) 165,000 articles published over a 5-year period in the top 100 refereed journals in education.”
BackgroundInfluential
Interdisciplinary Open Science: What are the implications for educational technology research?Preprint
Alhadad, S. S. J., Searston, R. A., & Lodge, J. M. (2019) · doi:10.35542/osf.io/w2bg9 · Cited by 3
Replication study in education
Perry, T. W., & See, B. H. (2022) · Educational Research and Evaluation · doi:10.1080/13803611.2021.2022307 · Cited by 3
“Looking at the literature up to January 2013, Makel and Plucker (2014) analysed articles in the top-100 education journals and found that only 0.13% of articles were replications.”
BackgroundInfluential
Biomedical researchers’ perspectives on the reproducibility of research: a cross-sectional international surveyPreprint
Cobey, K. D., Ebrahimzadeh, S., Page, M. J., Thibault, R. T., Nguyen, P., Abu-Dalfa, F., & Moher, D. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.09.18.558287 · Cited by 3
Applying generalized theory to optimize the quality of high-stakes objective structured clinical examinations for undergraduate medical students: experience from the French medical school
Feigerlová, E. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07255-y · Cited by 5
“The problem of generalizing findings in the field of medical education has previously been addressed, with only 0.13% of studies being replications [26].”
Influential
The PRSIST Intervention to Improve Self‐Regulation in Preschoolers: An Adaptation to the Italian Context
Ruffini, C., Magni, E., Pecini, C., & Howard, S. J. (2025) · Infant and Child Development · doi:10.1002/icd.70032 · Cited by 2
Design research in between the past and future: linking the epistemic, ontological, and axiological dimensions of generative projects
Penuel, W. R., Bakker, A., & Akkerman, S. F. (2025) · Mathematical Thinking and Learning · doi:10.1080/10986065.2025.2595715 · Cited by 2
“To still others (e.g., Cai et al., 2018; Makel & Plucker, 2014), one key problem lies in our inability to replicate findings, a challenge that has confronted the human and natural sciences in recent years.”
Cataloguing and Theorising Open Research Practices in the Arts, Humanities and Social Sciences: Problematising and Diversifying ‘Open Science’
Adams, J., Barnes, M., Moore, S., & Pinfield, S. (2026) · F1000Research · doi:10.12688/f1000research.178353.1 · Cited by 2
Criterion Validity of a Computer Adaptive Universal Screener to an End-of-Year State Mathematics Assessment
Peltier, C. J., Vannest, K. J., Tomaszewski, B. R., Morin, K. L., Sallese, M. R., & Pulos, J. M. (2020) · Exceptionality · doi:10.1080/09362835.2020.1727337 · Cited by 2
“Second, replication in social science research has received increased attention and the field has addressed the need to increase replication studies (Makel & Plucker, 2014; Makel et al., 2016) to examine the generalization of findings.”
Background
Programas de intervención y replicabilidad: consideraciones sobre su evaluación en psicología
Romo, J. F. M. (2021) · Revista Iberoamericana de Psicología · doi:10.33881/2027-1786.hrip.14108 · Cited by 4
Responsible Educational Technology Research: From Open Science and Open Data to Ethics and Trustworthy Learning Analytics
Hernández‐Leo, D., Amarasinghe, I., Beardsley, M., Hakami, E., Ruiz-García, A., & Santos, P. (2023) · Higher education dynamics · doi:10.1007/978-3-031-24193-2_7 · Cited by 4
Ciencias de la educación abiertas
Sierra, R. H. V., Morillo, J. P., & Tunjano-Huertas, W. (2023) · Revista Científica · doi:10.14483/23448350.20686 · Cited by 2
Till death do us part. Do customers cheat on brands?
Klepek, M., & Toth, L. (2024) · E+M Ekonomie a Management · doi:10.15240/tul/001/2024-5-008 · Cited by 2
Evidential Value of Intervention Studies in Two School Psychology Journals: A p -Curve Analysis of Research from 2011 to 2021
Ni, H., Price, P. C., & Jones, C. J. (2024) · School Psychology Review · doi:10.1080/2372966x.2024.2365618 · Cited by 2
Counseling Research Trends: A Content Analysis 2016–2020
Marshall, D. T., Elmer, J. W., Hebert, C. C., & Barringer, K. E. (2024) · Journal of Professional Counseling Practice Theory & Research · doi:10.1080/15566382.2024.2368647 · Cited by 2
“While we do not doubt the results of the studies we reviewed, that fewer than four percent of intervention articles yielded null findings leads us to wonder if there is a file drawer problem in this corner of the literature (Franco et al., 2010; Makel & Plucker, 2014).”
The Impact of Setting, Time of Day, and Giftedness on Divergent Thinking Test Scores
Alabbasi, A. M. A., Acar, S., Runco, M. A., Alsuqer, S. A., AL-Jasim, F. A., & Sultan, Z. M. (2025) · SAGE Open · doi:10.1177/21582440251320442 · Cited by 2
“Aside from these improvements, replication studies are very informative, especially when conducted in different cultures (Makel & Plucker, 2014).”
The Reproducibility Crisis and Kinesiology: Confronting the Human & Statistical Limitations
Twietmeyer, G. (2025) · International Journal of Kinesiology in Higher Education · doi:10.1080/24711616.2025.2468469 · Cited by 2
“Given these disincentives, far too few scientific findings are ever put to the test of replication (Makel & Plucker, 2014; Martin & Clarke, 2017).”
Practical methods for incorporating open science into your speech-language-hearing science research workflow across study designs
Ford, A. L. B., Elmquist, M., Finestack, L. H., Pfeiffer, D. L., Rountrey, C. E., & Sterling, A. M. (2025) · Evidence-Based Communication Assessment and Intervention · doi:10.1080/17489539.2025.2592070 · Cited by 2
“OS practices are currently used across several fields, including psychology, education, biology, and SLHS (Baggerly & Coombes, 2009; El Amin et al., 2023; Fraser et al., 2018; Makel & Plucker, 2014).”
Publishing Journal Articles : A Scientific Guide for New Authors Worldwide
Yan, Z., & Snow, C. E. (2020) · Cited by 2
BRICS' scientific excellence and the search for relevance and replicability
Vílchez‐Román, C. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23654 · Cited by 2
“Requirements for publishing replicable studies are reported in the literature (Fanelli, 2012; Makel & Plucker, 2014; Makel, Plucker, & Hegarty, 2012; Schmidt, 2009).”
Background
A Revised Replication Study Typology and A Call for Participation: Replication Studies involving Technology and Teacher Education
Milman, N. B., Christensen, R. W., Spector, J. M., Branch, R. M., Schmidt-Crawford, D. A., Hodges, C. B., Borthwick, A. C., Shoffner, M., Knezek, G., Rutledge, D. W., et al. (2016) · Society for Information Technology & Teacher Education International Conference · Cited by 2
Making a Case for a Blended Approach
Deaton, C. C. M., & Malloy, J. A. (2017) · International Journal of Adult Vocational Education and Technology · doi:10.4018/ijavet.2017040106 · Cited by 2
Developing and Sustaining a Research Agenda in Special Education Teacher Education
Maheady, L. (2017) · Teacher Education and Special Education The Journal of the Teacher Education Division of the Council for Exceptional Children · doi:10.1177/0888406417725798 · Cited by 2
“Makel and Plucker (2014) noted that educational research has focused heavily on experimental design but not much on replicating important results.”
Background
Reviews of research funded by U.S. Institute of Educational Sciences: A case of reading development and instruction
Lee, S., Kuo, L., Moody, S. M., & Chen, Z. (2017) · Cogent Education · doi:10.1080/2331186x.2017.1401444 · Cited by 2
“This finding is inconsistent with that of Makel and Plucker (2014), who determined that only 0.13% of articles in the top 100 education journals were replication studies.”
BackgroundResultInfluential
Level 1 Evidence: A Prospective Randomized Controlled Study
Ekhtiari, S., Mundi, R., Khanna, V., & Bhandari, M. (2019) · Springer eBooks · doi:10.1007/978-3-662-58254-1_29 · Cited by 2
The quest for novelty
Reio, T. G. (2020) · Human Resource Development Quarterly · doi:10.1002/hrdq.21409 · Cited by 4
Development and Validation of the “Tübingen Inventory to Measure Teachers’ Profession-Specific Value Orientations” (TIVO)
Merk, S., Drahmann, M., & Cramer, C. (2021) · AERA Open · doi:10.1177/23328584211033595 · Cited by 2
Crafting the Methods to Test Hypotheses
Hiebert, J., Cai, J., Hwang, S., Morris, A., & Hohensee, C. (2022) · Research in mathematics education · doi:10.1007/978-3-031-19078-0_4 · Cited by 2
Qualitative Research as the Cornerstone Methodology for Understanding Leadership Studies
Coker, D. C. (2024) · Springer studies on populism, identity politics and social justice · doi:10.1007/978-3-031-56415-4_23 · Cited by 3
Long-term effects of first-grade teachers on students’ achievement: a replication study
Vanwynsberghe, G., Vanlaar, G., Damme, J. V., & Fraine, B. D. (2018) · School Effectiveness and School Improvement · doi:10.1080/09243453.2018.1556164 · Cited by 2
“Makel and Plucker (2014) studied the frequency of replication studies in the complete publication history of the top-100 education journals with the highest 5-year impact factor.”
Background
Why Fast and Effortless Reading Is Indispensable for Comprehension
Abadzi, H., & Centanni, T. M. (2020) · Comparative Education Review · doi:10.1086/708300 · Cited by 2
The Utility of Dynamic Assessment of Phonological Awareness for Bilingual Children in Singapore
Yap, D. F. (2018) · eScholarship (California Digital Library) · Cited by 2
Actions for Increasing the Credibility of Educational ResearchPreprint
Carvalho, P. F., Cook, B. G., de Leeuw, J. R., Fyfe, E. R., Gehlbach, H., Goldstone, R. L., Makel, M. C., Mellor, D. T., Montoya, A. K., Motz, B. A., et al. (2019) · EdArXiv (OSF Preprints) · doi:10.35542/osf.io/tv7rn · Cited by 1
Self-citation
Same as it Ever Was: An Updated Review of Replication Studies in Special Education JournalsPreprint
Cook, B. G., Therrien, W. J., Waterfield, D. A., McClain, S. S., Fleming, J. I., Robinson, H. E., Watson, L., & Boyle, J. R. (2023) · doi:10.35542/osf.io/27nvt · Cited by 2
Data for Psychological Research in the Educational Field: Spotlights, Data Infrastructures, and Findings from Research
Bayer, S., Blask, K., Gnambs, T., Jansen, M., Maehler, D. B., Meyermann, A., & Neuendorf, C. (2023) · Journal of Open Psychology Data · doi:10.5334/jopd.105 · Cited by 2
Mentors Matter Recruitment Replication & Extension: Investigating Effects Across Implementation Years
Ronfeldt, M., Bardelli, E., Truwit, M., Schaaf, K., & Baker, J. C. (2024) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2023.2287615 · Cited by 1
“Replication under both the same and different experimental designs allows for a clearer delineation between spurious one-off effects and concrete causal relationships that scale up beyond the original study (Makel & Plucker, 2014 ; Steiner et al., 2019 ; Simpson, 2022; Wong & Steiner, 2018).”
Towards conceptual clarity in pedagogical research: the case of the ‘imposter phenomenon’ jingle/jangle fallacy
Pownall, M., Blundell-Birtill, P., & Harris, R. J. (2025) · Higher Education Research & Development · doi:10.1080/07294360.2025.2470821 · Cited by 4
“…to improve the credibility and precision of pedagogical research, scholars have advocated for replication studies within educational literature (Makel & Plucker, 2014), refined the precision of measurement in pedagogical research (Evans et al., 2021), and worked to improve links between…”
Conditions Under Which College Students Can Be Responsive to Text-Based Nudging
Page, L. C., Meyer, K. E., Lee, J. J., & Gehlbach, H. (2025) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2025.2481219 · Cited by 3
“We situate this work within ongoing conversations regarding the importance of replication studies for guiding policy development around consistent empirical trends (Makel & Plucker, 2014; Plucker & Makel, 2021).”
A Step-by-Step Guide to Evidence-Based Research With National Datasets: A QuantCrit Perspective
Bryan, J. A., Kim, C.-D., & Kim, J. (2025) · Professional School Counseling · doi:10.1177/2156759x251335523 · Cited by 1
“As many educational secondary datasets often collect similar data on participants over time, it is bene ﬁ cial to replicate studies with other available datasets to test the consistency of the ﬁ ndings (Makel & Plucker, 2014).”
Integrating Open Science Principles into Quasi-Experimental Social Science Research
Heller, B., & Robinson, C. D. (2025) · Social Sciences · doi:10.3390/socsci14080499 · Cited by 2
“…replication is a cornerstone of the scientific process, replication studies represent a miniscule share of published research in most fields, including psychology (1%; Makel, Plucker & Hagerty, 2012), education (0.13%; Makel & Plucker, 2014), and economics (0.1%; Mueller-Langer et al., 2019).”
Influential
Comparing early-stage L2 processing of derived and inflected words: a conceptual replication of Jacob et al. (2018) with Chinese learners of L2 English
Wu, Z., & Fricke, M. (2025) · Studies in Second Language Acquisition · doi:10.1017/s0272263125101368 · Cited by 1
“…in nearly every methodological aspect of the original study (McManus, 2023) to ascertain the generalizability and understand the limits of the theory in the original study and to potentially develop or build upon the original theory (Makel & Plucker, 2014; Porte & McManus, 2019; Schmidt, 2009).”
‘Don’t hate the players, hate the game’: qualitative insights from education researchers on questionable and open research practices
Makel, M. C., Caroleo, S. A., Meyer, M. S., Pei, M. A., Fleming, J. I., Hodges, J., Cook, B. G., & Plucker, J. A. (2026) · Royal Society Open Science · doi:10.1098/rsos.251888 · Cited by 1
Self-citation
What educational researchers present: Trends in research topics and methods based on conference abstracts
Dekker, I., Smixioti, E., & Meeter, M. (2026) · European Educational Research Journal · doi:10.1177/14749041261459565 · Cited by 1
Cognitive capability and behavioral exposure in generative AI use: a dual-pathway model of perceived facilitation and integrity-related risk
Mu, S., Wang, B., Xiong, J., Xie, Y., & Hu, W. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1876096 · Cited by 1
“…to distinguish between measurement stability and structural consistency across samples, which is particularly relevant in rapidly evolving GenAI contexts where institutional guidance, assessment expectations, and student learning practices may differ across environments (Makel and Plucker, 2014).”
A Critical Response to “The Psychology of Creativity: A Critical Reading”
Plucker, J. A. (2014) · Creativity Theories – Research – Applications · doi:10.15290/ctra.2014.01.02.09 · Cited by 1
“This is both reasonable and an important and necessary step for advancing the science of creativity (see Makel, Plucker & Hegarty, 2012; Makel & Plucker, 2014a, 2014b).”
BackgroundSelf-citation
Inaugural Editorial
Warschauer, M., Duncan, G. J., & Eccles, J. S. (2015) · AERA Open · doi:10.1177/2332858415574841 · Cited by 1
“Indeed, one recent study indicated that only 0.13% of articles published in the top 100 education journals are replications (Makel & Plucker, 2014).”
BackgroundInfluential
A Multifaceted Consideration of Motivation and Learning within ASSISTments
Ostrow, K. (2015) · Digital WPI · Cited by 1
Design-Based Case Study
Deaton, C. C. M., & Malloy, J. A. (2018) · Advances in library and information science (ALIS) book series · doi:10.4018/978-1-5225-5164-5.ch004 · Cited by 1
Bridging the Achievement Gap for Low-Performing Students Using Computer-Adaptive Instruction
Shamir, H., Pocklington, D., Feehan, K., & Yoder, E. (2019) · International Journal of Information and Education Technology · doi:10.18178/ijiet.2019.9.3.1198 · Cited by 1
Řízení třídy: studenti učitelství a jejich provázející učitelé
Vlčková, K., Lojdová, K., Lukas, J., Mareš, J., Škarková, L., Kohoutek, T., Květoň, P., & Ježek, S. (2020) · doi:10.5817/cz.muni.m210-9652-2020 · Cited by 1
Evaluating the Reporting Quality of Researcher-Developed Alphabet Knowledge Measures: How Transparent and Replicable Is It?
Horner, S. L., & Shaffer, S. A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.601849 · Cited by 1
“Several researchers (e.g., Slavin, 2002, 2005; Asendorpf et al., 2013; Makel, 2014; Tressoldi and Giofrè, 2015) have made recommendations to authors, editors, and reviewers related to various elements of published documentation of research.”
Background
Medyalararası Gündem Belirleme Çalışmalarında R Stüdyo Kullanımı: Rozelle-Campbell Taban Hizası ile Çapraz Gecikmeli Korelasyonun Bilişimleştirilmesi
ALİKILIÇ, İ., & Alikılıç, Ö. (2019) · DergiPark (Istanbul University) · Cited by 1
Designing a Replicable Data Infrastructure for Education Research
Machita, J., Rohrich, T., Jiang, Y., & Zheng, Y. (2021) · Systems and Information Engineering Design Symposium · doi:10.1109/sieds52267.2021.9483729 · Cited by 1
Supporting Teachers in Argument Writing Instruction at Scale: A Replication Study of the College, Career, and Community Writers Program (C3WP)
Arshan, N., Park, C. J., & Goetz, R. (2023) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2023.2179953 · Cited by 1
“A 2013 study found that only 0.13% of articles published in top education journals were replication studies, one eighth the rate of replications in psychology journals (Makel & Plucker, 2014).”
BackgroundInfluential

Authors: Makel, M. C., & Plucker, J. A.
Affiliations: Duke University (US)
University of Connecticut (US)
ORCID: Makel, Matthew C.
Plucker, Jonathan A.
Venue: Educational Researcher
Publisher: American Educational Research Association (AERA)
Published: 2014-08
Volume: 43
Issue: 6
Pages: 304-316
ISSN: 0013-189X (ISSN-L)
1935-102X
DOI: 10.3102/0013189x14545513
OpenAlex: W2139319950
Semantic Scholar: 493a2e93117a876871ac446b26acf47c50ea2cef
Cited by: 609
References: 131
Citations by year: 2026: 17 · 2025: 37 · 2024: 40 · 2023: 40 · 2022: 66 · 2021: 47 · 2020: 59 · 2019: 81 · 2018: 73 · 2017: 52 · 2016: 51 · 2015: 37 · 2014: 8
FWCI: 68.61
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Educational Researcher (publishedVersion)
Topics: Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Education › Educational Research and Methods
Keywords: replication, direct replication, educational research, publishing history, research credibility, methodological quality, replication success, educational periodicals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Facts Are More Important Than Novelty: Replication in the Education Sciences

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks