Cited by 609Psychology Research and BibliometricsMeta-analysis and systematic reviewsEducational Research and Methods

The DOI matches its Crossref record; 132 of the 132 references it cites were found in a registry or an index.

Facts Are More Important Than Novelty: Replication in the Education Sciences

Makel, Matthew C., Plucker, Jonathan A.·2014·Educational Researcher

Open this paper in CoChat — free account

Abstract

Despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not on the merit of replicating important results. The present study analyzed the complete publication history of the current top 100 education journals ranked by 5-year impact factor and found that only 0.13% of education articles were replications. Contrary to previous findings in medicine, but similar to psychology, the majority of education replications successfully replicated the original studies. However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. The results emphasize the importance of third-party, direct replications in helping education research improve its ability to shape education policy and practice.

Integrity signals

Registry

Crossref · 10.3102/0013189x14545513

Cites

132 of 132 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

2 retracted

RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children
Wakefield, A., Murch, S., Anthony, A., Linnell, J., Casson, D., Malik, M., Berelowitz, M., Dhillon, A., Thomson, M., Harvey, P., et al. (1998) · The Lancet · doi:10.1016/s0140-6736(97)11096-0 · Cited by 3,006
Retracted
Retraction—Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children
Lancet, T. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60175-4 · Cited by 500
Retracted
Untitled work
(1963) · Medical Entomology and Zoology · Cited by 15,085
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
An Introduction to Categorical Data Analysis
Agresti, A. (2006) · Wiley series in probability and statistics · doi:10.1002/0470114754 · Cited by 6,775
Journal of social behavior and personality
(1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90025-5 · Cited by 6,751
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
American Economic Review
Moffitt, R. (2009) · American Economic Review · doi:10.1257/aer.99.2.660 · Cited by 4,888
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 5×
“Using health care research as an example, Ioannidis (2005a), in a review of highly cited medical publications (i.e., those cited more than 1,000 times), found only 44% of replications produced results similar to the original study.”
ResultInfluential

Show all 131 references

Research works printed but not linked: 14

10.1037//0003–066x.49.12.997 · DOI · Paper page
Collins H. M., 1985, Changing order
Kane E. J., 1984, Quarterly Journal of Business and Economics, 23, 3
Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 1
Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 21
Perspectives on Psychological Science. (2012). Special section on replicability in psychological science: A crisis of confidence? Retrieved from http://pps.sagepub.com/content/7/6.toc · DOI · Paper page
Powell K., 2007, PLoS Biology, 12, 2748
Rhine J. B., 1997, Extra-sensory perception
Ritchie S. J., 2012, Psychologist, 25, 346
10.1037//0033–2909.86.3.638 · DOI · Paper page
Rotton J., 1995, Journal of Social Behavior and Personality, 10, 1
Simons D. J., 2012, Psychologist, 25, 349
Simons D. J., 2014, Perspectives on Psychological Science. APS Observer, 27, 16
Williams S. N., 2012, Science, 336, 801

Sorted by citations

Items 201–300 of 608

MORF: A Framework for Predictive Modeling and Replication At Scale With Privacy-Restricted MOOC Data
Gardner, J., Brooks, C. A., Andres, J. M. L., & Baker, R. S. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8621874 · Cited by 22
“A survey of the top 100 education journals [12] demonstrated that only 0.”
Background
Changing research culture toward more use of replication research: a narrative review of barriers and strategies
Vachon, B., Curran, J., Karunananthan, S., Brehaut, J., Graham, I. D., Moher, D., Sales, A. E., Straus, S. E., Fiander, M., Paprica, P. A., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.09.027 · Cited by 27
Towards reproducible systematic reviews in Open, Distance, and Digital Education—An umbrella mapping review
Zawacki‐Richter, O., Cefa, B., & Bai, J. Y. H. (2025) · Review of Education · doi:10.1002/rev3.70031 · Cited by 30
Effect of the Reread-Adapt and Answer-Comprehend Intervention on the Reading Achievement of Middle and High School Readers Who are Deaf
Schirmer, B. R., Schaffer, L., Therrien, W. J., & Schirmer, T. N. (2015) · Reading Psychology · doi:10.1080/02702711.2015.1105338 · Cited by 21
“Given the risks of relying on results that have not been replicated, we have systematically varied aspects of the sample and procedure in line with constructive replications that are intended to test the targeted construct (Makel & Plucker, 2014).”
Background
On the benefits of ‘doing science’: Does integrative writing about scientific controversies foster epistemic beliefs?
Rosman, T., Mayer, A., Merk, S., & Kerwer, M. (2019) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2019.02.007 · Cited by 21
How to trust a scientist
de Ridder, J. (2022) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2022.02.003 · Cited by 24
Advanced Reading Comprehension Expectations in Secondary School
Ciullo, S. P., Ortiz, M. B., Otaiba, S. A., & Lane, K. L. (2015) · Journal of Disability Policy Studies · doi:10.1177/1044207315604365 · Cited by 21
“Although several strategies show promise and will be highlighted in the following sections, further replication studies are needed to establish that these strategies meet criteria for evidence-based practices (Cook & Odom, 2013; Horner & Kratochwill, 2012; Makel & Plucker, 2014).”
Replicable Services for Reproducible Research: A Model for Academic Libraries
Sayre, F., & Riegelman, A. (2019) · College & Research Libraries · doi:10.5860/crl.80.2.260 · Cited by 25
“В последние годы в ряде дисциплин, включая психологию [1], биологию [2], биомедицину [3], неврологию [4], разработку лекарств [5], химию [6], климатологию [7], экономику [8, 9] и образование [10], появились свидетельства того, что наука стоит перед лицом кризиса воспроизводимости.”
What do beginning teachers make of task-based language teaching? A comparative re-production of East (2014)
East, M. (2020) · Language Teaching · doi:10.1017/s026144481900048x · Cited by 21
“Makel and Plucker (2014) noted that just over 48% of the replications they identified were carried out by the same researchers who published the original research.”
Background
Chapter 6: Preservice Teachers’ Perceptions of Twitter for Health and Physical Education Teacher Education: A Self-Determination Theoretical Approach
Hyndman, B., & Harvey, S. (2020) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.2019-0278 · Cited by 21
Teacher Beliefs and Responses Toward Student Misbehavior: Influence of Cognitive Skill Deficits
Hart, S. C., & DiPerna, J. C. (2016) · Journal of Applied School Psychology · doi:10.1080/15377903.2016.1229705 · Cited by 21
“Although the current results require replication (e.g., Makel & Plucker, 2014), school psychologists and other support professionals may consider sharing CSD information with teachers who are working with students displaying challenging behavior.”
Background
Improving reading rates and comprehension? Benefits and limitations of the reading acceleration approach
Korinth, S. P., & Nagler, T. (2021) · Language and Linguistics Compass · doi:10.1111/lnc3.12408 · Cited by 23
“In light of the current replication crisis (Ioannidis, 2005), however, it would strengthen the status of the RAP approach, if independent groups would replicate its major findings (Makel & Plucker, 2014).”
The Practical Difficulties for Early Educators Who Tried to Address Children’s Realities in Their High-Stakes Teaching Context
Brown, C. P., Weber, N. B., & Yoon, Y. (2015) · Journal of Early Childhood Teacher Education · doi:10.1080/10901027.2014.996925 · Cited by 20
“…further amplify the potential impact of such reform initiatives as Common Core or Race to the Top that seek to standardize and/or align ECE with elementary school programs and limit the space for teachers to examine issues central to children’s lives in and out of school (Makel & Plucker, 2014).”
Background
Persons with multiple disabilities manage positive leisure and communication engagement through a technology-aided program
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Boccasini, A., Perilli, V., & Spagnuolo, C. (2016) · International Journal of Developmental Disabilities · doi:10.1080/20473869.2016.1187462 · Cited by 20
“In light of this situation, new research studies with additional participants seem warranted (Makel and Plucker 2014).”
Background
A technology-aided program for helping persons with Alzheimer’s disease perform daily activities
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Pinto, K., Vanna, F. D., & Caffò, A. O. (2017) · Journal of Enabling Technologies · doi:10.1108/jet-03-2017-0011 · Cited by 20
A Path to Greater Credibility: Large-Scale Collaborative Education Research
Makel, M. C., Smith, K. N., McBee, M. T., Peters, S. J., & Miller, E. M. (2019) · AERA Open · doi:10.1177/2332858419891963 · Cited by 27
Self-citation
The relationship of working memory and inhibition with different number knowledge skills in preschool children
Traverso, L., Tonizzi, I., Usai, M. C., & Viterbori, P. (2020) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2020.105014 · Cited by 20
Different measures of holistic face processing tap into distinct but partially overlapping mechanisms
Boutet, I., Nelson, E. A., Watier, N. N., Cousineau, D., Béland, S., & Collin, C. A. (2021) · Attention Perception & Psychophysics · doi:10.3758/s13414-021-02337-7 · Cited by 21
Making the Visible Invisible
Powers, J. M., Fischman, G. E., & Berliner, D. C. (2016) · Review of Research in Education · doi:10.3102/0091732x16663703 · Cited by 23
“This is striking because there are few replication studies in education research (Makel & Plucker, 2014; Schneider, 2004).”
Background
A systematic review of transparency in Lesson Study research: how do we report on the observation and reflection stages?
Kager, K., Kalinowski, E., Jurczok, A., & Vock, M. (2024) · Frontiers in Education · doi:10.3389/feduc.2024.1322624 · Cited by 21
Similarities and differences in program registers: A case study
Zack, M. K., Karre, J. K., Olson, J. R., & Perkins, D. F. (2019) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2019.101676 · Cited by 19
Open Science Practices in Communication Sciences and Disorders: A Survey
Amin, M. E., Borders, J. C., Long, H. L., Keller, M. A., & Kearney, E. (2022) · Journal of Speech Language and Hearing Research · doi:10.1044/2022_jslhr-22-00062 · Cited by 26
Evaluating the effects of repeated reading and continuous reading using a standardized dosage of words read
Hammerschmidt‐Snidarich, S. M., Maki, K. E., & Adams, S. R. (2019) · Psychology in the Schools · doi:10.1002/pits.22241 · Cited by 18
A systematic review of CDIO knowledge library publications (2010 – 2020): An Overview of trends and recommendations for future research
O’Connor, S. M., Power, J., & Blom, N. (2023) · Australasian journal of engineering education · doi:10.1080/22054952.2023.2220265 · Cited by 18
“‘If education research is to be relied upon to develop sound policy and practice, then conducting replications on important findings is essential to moving towards a more reliable and trust-worthy understanding of educational environments’ (Makel and Plucker 2014).”
Background
AIED: From Cognitive Simulations to Learning Engineering, with Humans in the Middle
McNamara, D. S. (2023) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00349-y · Cited by 18
“…and interventions have either not been tested at scale, or have failed to replicate beyond their initial, constrained contexts and populations (e.g., laboratory experiments and targeted classroom studies; Kizilcec et al., 2020 ; Lortie-Forgues and Inglis, 2019 ; Makel and Plucker, 2014 ).”
Registered reports and replications: An ongoing Journal of School Psychology initiative
Braden, J. P. (2024) · Journal of School Psychology · doi:10.1016/j.jsp.2024.101294 · Cited by 18
Replicating rockets and feathers
Cook, S., & Fosten, J. (2018) · Energy Economics · doi:10.1016/j.eneco.2017.12.021 · Cited by 18
Replication Research Series-Paper 1 : A concept analysis and meta-narrative review established a comprehensive theoretical definition of replication research to improve its use
Vachon, B., Curran, J., Karunananthan, S., Brehaut, J., Graham, I. D., Moher, D., Sales, A. E., Straus, S. E., Fiander, M., Paprica, P. A., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.07.006 · Cited by 19
Replicating 21 findings on student success in online learning
Andres, J. M. L., Baker, R. S., Gašević, D., Siemens, G., & Spann, C. A. (2017) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 18
Promoting Functional Activity Engagement in People with Multiple Disabilities through the Use of Microswitch-Aided Programs
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Alberti, G., Perilli, V., & Campodonico, F. (2017) · Frontiers in Public Health · doi:10.3389/fpubh.2017.00205 · Cited by 19
Research Commentary: On Replications
Schoenfeld, A. H. (2018) · Journal for Research in Mathematics Education · doi:10.5951/jresematheduc.49.1.0091 · Cited by 18
A Replication of the Internal Validity Structure of Three Major Teaching Rating Scales
Peters, S. J., & Pereira, N. (2017) · Journal of Advanced Academics · doi:10.1177/1932202x17701940 · Cited by 20
“To quote a seminal paper by Makel and Plucker (2014), “facts are more important than novelty” (p. 304), and facts require replication.”
Background
The impact of English-medium instruction on university student performance
Barrigüete, J. L. A., Sánchez, J. I. L., Contreras, M. F. M., & Soffritti, M. (2022) · Journal of Multilingual and Multicultural Development · doi:10.1080/01434632.2022.2047193 · Cited by 17
“As Makel and Plucker (2014) point out, there is a shortage of replication in the Education Sciences, and this work aims precisely to ﬁ ll this gap, extending the work of Dafouz, Camacho-Mill-ano, and Urquia (2014), and Dafouz and Camacho-Miñano (2016).”
Interactions among dimensions of divergent thinking as predictors of creative activity and accomplishment
Runco, M. A., & Alabbasi, A. M. A. (2024) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2024.101583 · Cited by 17
Examining the Relationship Between College Advising and Student Outputs: A Content Analysis of the NACADA Journal
Alvarado, A. R., & Olson, A. B. (2020) · NACADA Journal · doi:10.12930/nacada-19-33 · Cited by 17
“Replication of empirical studies is considered one of the ‘‘ basic building blocks of science ’’ (Makel & Plucker, 2014, p. 305), but studies cannot be replicated unless sufﬁcient details are provided about the study’s design, sample, and methods.”
Background
Incremental Benefits of a Daily Report Card Over Time for Youth with Disruptive Behavior: Replication and Extension
Holdaway, A. S., Hustus, C. L., Owens, J. S., Evans, S. W., Coles, E. K., Egan, T. E., Himawan, L. K., Zoromski, A. K., Dawson, A. E., & Mixon, C. S. (2020) · School Mental Health · doi:10.1007/s12310-020-09375-w · Cited by 17
Is There a Replication Crisis in Medical Education Research?
Hope, D., Dewar, A., & Hay, C. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004063 · Cited by 16
Assessing the replication landscape in experimental linguistics
Kobrock, K., & Roettger, T. B. (2023) · Glossa Psycholinguistics · doi:10.5070/g6011135 · Cited by 18
Parent-implemented interventions in Chinese families of children with autism spectrum disorder
Gao, X., & Drani, S. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-02710-5 · Cited by 16
Project-based learning for middle school students monitoring standby power: replication of impact on stem knowledge and dispositions
Knezek, G., & Christensen, R. W. (2019) · Educational Technology Research and Development · doi:10.1007/s11423-019-09674-3 · Cited by 18
Large-Scale Estimates of LGBQ-Heterosexual Disparities in the Presence of Potentially Mischievous Responders: A Preregistered Replication and Comparison of Methods
Cimpian, J. R., & Timmer, J. D. (2019) · AERA Open · doi:10.1177/2332858419888892 · Cited by 18
“Education researchers, and indeed the broader field of social scientists, are under increasing pressure to replicate and preregister studies to ensure the robustness of published findings (Fanelli & Ioannidis, 2013; Gehlbach & Robinson, 2018; Makel & Plucker, 2014; Open Science Collaboration, 2015).”
Background
College Students’ Development of Badminton Skills and Tactical Competencies Following Play Practice
Liu, H., Wang, W., Zhang, C., & Hastie, P. A. (2020) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.2019-0292 · Cited by 18
Open science reforms: Strengths, challenges, and future directions
Wentzel, K. R. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1901709 · Cited by 18
“Studies that have documented low replication rates have included only a very small number of educational psychology journals (e.g., Makel et al., 2012; Makel & Plucker, 2014).”
Background
Failed Warnings
Moss, B. G., & Yeaton, W. H. (2015) · Evaluation Review · doi:10.1177/0193841x15610192 · Cited by 16
The Importance of Replication in Measurement Research: Using Curriculum-Based Measures With Postsecondary Students With Developmental Disabilities
Hosp, J. L., Ford, J. W., Huddle, S. M., & Hensley, K. (2017) · Assessment for Effective Intervention · doi:10.1177/1534508417727489 · Cited by 16
“It is designed to examine the accuracy of specific results and data from the original study (Makel & Plucker, 2014).”
BackgroundMethod
Non-ambulatory People with Intellectual Disabilities Practice Functional Arm, Leg or Head Responses Via a Smartphone-Based Program
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Alberti, G., Campodonico, F., Tedone, R., Quaranta, S., & Caffò, A. O. (2018) · Journal of Developmental and Physical Disabilities · doi:10.1007/s10882-018-9636-7 · Cited by 16
From CREATE Workshop to Course Implementation: Examining Downstream Impacts on Teaching Practices and Student Learning at 4-Year Institutions
Kenyon, K. L., Cosentino, B. J., Gottesman, A. J., Onorato, M. E., Hoque, J., & Hoskins, S. G. (2018) · BioScience · doi:10.1093/biosci/biy145 · Cited by 15
Developing pre-service teachers’ professional vision with video interventions: a divergent replication
Simpson, A. P., & Vondrová, N. (2019) · Journal of Education for Teaching International Research and Pedagogy · doi:10.1080/02607476.2019.1674563 · Cited by 16
“Replication studies are rare in most fields: Makel and Plucker (2014) suggest only around 1 in 750 education articles is a replication. In studies of the development of professional vision, there appears to be only one: Star, Lynch, and Perova's (2011)”
BackgroundInfluential
Balancing Rigor, Replication, and Relevance: A Case for Multiple-Cohort, Longitudinal Experiments
Blazar, D., & Kraft, M. A. (2019) · AERA Open · doi:10.1177/2332858419876252 · Cited by 16
“A recent review of the top 100 education journals as measured by impact factor found that 0.13% of all published studies over a 5-year period were replications (Makel & Plucker, 2014).”
BackgroundInfluential
Discovering Empirically-Based Best Practices in Computing Education Through Replication, Reproducibility, and Meta-Analysis Studies
McGill, M. M. (2019) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3364510.3364528 · Cited by 17
Effectiveness Research for Teacher Education
Hill, H. C., Mancenido, Z., & Loeb, S. (2024) · Educational Researcher · doi:10.3102/0013189x241260393 · Cited by 15
“Doing so would essentially test the durability of findings and potentially build understanding of where content and pedagogy are most and least effective (for an example, see Cohen et al., 2024; for discussions of replication, see Makel and Plucker, 2014; Star, 2021).”
The Efficacy of Digital Media Resources in Improving Children’s Ability to Use Informational Text: An Evaluation of Molly of Denali From PBS KIDS
Kennedy, J. L., Christensen, C., Maxon, T., Gerard, S. N., Garcia, E. B., Kook, J. F., Hupert, N., Vahey, P., & Pasnik, S. (2022) · American Educational Research Journal · doi:10.3102/00028312221113326 · Cited by 15
“Although replication is an essential part of the scientific process to confirm the efficacy of an intervention, it is uncommon in education research for study findings to be replicated with a second group of participants (Makel & Plucker, 2014).”
Background
A Technology-Aided Program to Support Basic Occupational Engagement and Mobility in Persons with Multiple Disabilities
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Alberti, G., Campodonico, F., Perilli, V., Chiariello, V., & Zimbaro, C. (2017) · Frontiers in Public Health · doi:10.3389/fpubh.2017.00338 · Cited by 15
“Obviously, new studies with additional participants are warranted to verify the suitability of the program and the reliability of the findings (19, 47).”
Background
Authoritative school climate, number of parents at home, and academic achievement.
Huang, F. L., Eklund, K. R., & Cornell, D. G. (2016) · School Psychology Quarterly · doi:10.1037/spq0000182 · Cited by 18
“Recent articles in education sciences and psychology have highlighted the need to replicate studies in order for research to be able to influence educational policy and practice (Makel & Plucker, 2014) and “replications in psychology reflect a growing trend in science” (Bohannon, 2014, p. 789).”
Background
Investigating a Tier 1 Intervention Focused on Proportional Reasoning: A Follow-Up Study
Jitendra, A. K., Harwell, M. R., Karl, S. R., Simonson, G. R., & Slater, S. C. (2017) · Exceptional Children · doi:10.1177/0014402917691017 · Cited by 15
“Several authors have pointed out that replications are a staple in methods textbooks and are badly needed (Makel & Plucker, 2014) but rarely appear in practice (Duncan, Engel, Claessens, & Dowsett, 2015).”
Method
Mainstream technology to support basic communication and leisure in people with neurological disorders, motor impairment and lack of speech
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Buonocunto, F., Lanzilotti, C., Alberti, G., & Navarro, J. (2020) · Brain Injury · doi:10.1080/02699052.2020.1763462 · Cited by 15
“Further studies with additional participants may help clarify the effectiveness and robustness of the new technology solution and determine the reliability of the present data (34,35).”
Background
Blocking Vs. Interleaving: Examining Single-Session Effects Within Middle School Math Homework
Ostrow, K., Heffernan, N., Heffernan, C., & Peterson, Z. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-19773-9_34 · Cited by 23
“13% of publications were replications [8].”
Conceptual replications, research, and the “what works” agenda in education
Morrison, K. (2022) · Educational Research and Evaluation · doi:10.1080/13803611.2021.2022314 · Cited by 15
“For example: Chhin et al. (2018), Coyne et al. (2013, 2016), Gersten et al. (2015), Q. Hao et al. (2019), Hawe et al. (2015), Hendrick (1991), Makel and Plucker (2014), Melhuish (2018), Morrison (2019), National Science Foundation and Institute of Education Sciences (2018), Rosenthal (1991),…”
Background
Noncognitive skills and critical thinking predict undergraduate academic performance
Martín‐Raugh, M. P., Kell, H. J., Ling, G., Fishtein, D., & Yang, Z. (2022) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2022.2073964 · Cited by 19
“We underscore the importance of the first component of this study, as the need to replicate prior research in education (Makel and Plucker 2014) and psychology (Maxwell, Lau, and Howard 2015) has become increasingly important over the last decade.”
Method
Surfacing Inequities and Their Broader Implications in the CS Education Research Community
McGill, M. M., Davis, S., & Reyes, J. (2022) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3501385.3543969 · Cited by 14
A Targeted Review of Open Practices in Special Education Publications
Cook, B. G., van Dijk, W., Vargas, I., Aigotti, S. M., Fleming, J. I., McDonald, S. D., Richmond, C. L., Griendling, L. M., McLucas, A. S., & Johnson, R. M. (2023) · Exceptional Children · doi:10.1177/00144029221145195 · Cited by 18
“…studies represent approximately 0.5% of the published special education literature (Lemons et al., 2016; Makel et al., 2016), which—though a small proportion of the literature base—is larger than the proportion of replications found in education journals generally (0.1%; Makel & Plucker, 2014).”
ResultInfluential
Technology-Aided Programs to Support Positive Verbal and Physical Engagement in Persons with Moderate or Severe Alzheimer’s Disease
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Renna, C., & Pinto, K. (2016) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2016.00087 · Cited by 15
“…research efforts seem to be warranted to determine the strength of the data available and conceivably, investigate upgrades of the programs so as to make them more functional and/or respectful of the participants (Barlow et al., 2009; Kazdin, 2011; Godwin et al., 2013; Makel and Plucker, 2014).”
BackgroundInfluential
Smartphone-Based Interventions to Foster Simple Activity and Personal Satisfaction in People With Advanced Alzheimer’s Disease
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Turnone, B., Laporta, D., Scordamaglia, A., & Pinto, K. (2019) · American Journal of Alzheimer s Disease & Other Dementias® · doi:10.1177/1533317519844144 · Cited by 15
Training teachers' diagnostic competence with problem-based learning: A pilot and replication study
Wedel, A., Müller, C. R., Pfetsch, J. S., & Ittel, A. (2019) · Teaching and Teacher Education · doi:10.1016/j.tate.2019.102909 · Cited by 17
Australian and Italian Psychologists’ View of Replication
Agnoli, F., Fraser, H. S., Thorn, F. S., & Fidler, F. M. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211039218 · Cited by 13
“Similar studies in other fields have also shown very low prevalence of published replication studies, including rates of only 0.13% in education research (Makel & Plucker, 2014) and 1.2% in marketing research (Evanschitzky et al., 2007).”
ResultInfluential
The influence of prior experiences on early childhood education students' anticipated work with families
Winder, C. M., & Corter, C. M. (2016) · Teaching and Teacher Education · doi:10.1016/j.tate.2016.01.005 · Cited by 17
Measuring teacher authenticity: Criteria students use in their perception of teacher authenticity
Bruyckere, P. D., & Kirschner, P. A. (2017) · Cogent Education · doi:10.1080/2331186x.2017.1354573 · Cited by 22
“There is at present a major discussion in both psychology and educational sciences about replication (Makel & Plucker, 2014).”
Background
Language and Literacy: Issues and Considerations
Hartman, M. C., Nicolarakis, O. D., & Wang, Y. (2019) · Education Sciences · doi:10.3390/educsci9030180 · Cited by 14
“In regards to replication research, a recent study published in Educational Researcher (ER), a peer-reviewed journal of the American Educational Research Association (AERA), reported that although replicating is essential for helping education research improve its usefulness to policymakers and practitioners, less tha…”
Background
Over Diagnosed or Over Looked? The Effect of Age at Time of School Entry on Students Receiving Special Education Services
Shapiro, A. (2022) · Exceptional Children · doi:10.1177/00144029221108735 · Cited by 13
“This finding adds to the literature, given the relative lack of replication evidence in education research (Makel & Plucker, 2014; Travers et al., 2016).”
Result
Effects of Reading Instruction on Learning Outcomes in Social Studies: A Synthesis of Quantitative Research
McCulley, L. V., & Osman, D. J. (2015) · The Journal of Social Studies Research · doi:10.1016/j.jssr.2015.06.002 · Cited by 13
“As researchers attempt to understand “What works” for students in social studies education, they might consider the importance of rigorous replication studies (Makel & Plucker, 2014).”
Background
The Dual Commodification of College-Going: Individual and Institutional Influences on Access and Choice
Hughes, R., Kimball, E. W., & Koricich, A. (2019) · Higher education · doi:10.1007/978-3-030-03457-3_10 · Cited by 15
Prospective Elementary Teachers’ Beliefs About Mathematical Myths: A Historical and Qualitative Examination
Phelps-Gregory, C. M., Frank, M. L., & Spitzer, S. M. (2019) · The Teacher Educator · doi:10.1080/08878730.2019.1618423 · Cited by 13
“Given the passage of time and limitations of the previous research, we conducted a conceptual follow-up replication for the current study instead of a direct replication (Makel & Plucker, 2014).”
Method
Improving the reproducibility of findings by updating research methodology
Klein, J. (2021) · Quality & Quantity · doi:10.1007/s11135-021-01196-6 · Cited by 19
A conceptual replication study of a self-affirmation intervention to improve the academic achievement of low-income pupils in England
See, B. H., Morris, R., Gorard, S. A. C., Siddiqui, N., Easterbrook, M. J., Nieuwenhuis, M., Fox, K. J., Harris, P. R., & Banerjee, R. A. (2022) · Educational Research and Evaluation · doi:10.1080/13803611.2021.2022317 · Cited by 13
“Makel and Plucker (2014) noted that only 13% of around 16,000 studies in the top-100 education journals were replications.”
Background
Generalizability, transferability, and the practice-to-practice gap
de Leeuw, J. R., Motz, B. A., Fyfe, E. R., Carvalho, P. F., & Goldstone, R. L. (2022) · Behavioral and Brain Sciences · doi:10.1017/s0140525x21000406 · Cited by 23
Building the field’s knowledge of teaching and learning: Centering the socio-cultural contexts of observation systems to ensure valid score interpretation
Bell, C. A., & Gitomer, D. (2023) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2023.101278 · Cited by 13
Rating Teachers Cheaper, Faster, and Better
Good, T. L., & Lavigne, A. L. (2015) · Journal of Teacher Education · doi:10.1177/0022487115574292 · Cited by 13
“Although we agree with Makel and Plucker (2014) that replication is an important activity and that often too much reliance is placed upon a single or few studies, GS’ work would have more impact if they reported that their research was part of the continuing history of research on teaching and not…”
Supporting leisure and functional activity engagement in people with multiple disabilities via a technology-aided program
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Alberti, G., Perilli, V., Zimbaro, C., Chiariello, V., & Campodonico, F. (2018) · Technology and Disability · doi:10.3233/tad-170182 · Cited by 13
Prospective elementary teachers’ conceptions of multidigit number: exemplifying a replication framework for mathematics education
Jacobson, E. D., & Simpson, A. M. (2018) · Mathematics Education Research Journal · doi:10.1007/s13394-018-0242-x · Cited by 14
“However, replication studies of education studies are rare; only about 1 in 1000 studies published in major education journals is a replication study (Makel and Plucker 2014).”
Background
Assistive technology to help persons in a minimally conscious state develop responding and stimulation control: Performance assessment and social rating
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Buonocunto, F., Navarro, J., Lanzilotti, C., Fiore, P., Megna, M., et al. (2015) · Neurorehabilitation · doi:10.3233/nre-151269 · Cited by 13
Effectiveness of Flipped Classrooms for K–12 Students: Evidence From a Three-Level Meta-Analysis
Li, S., Fu, W., Liu, X., & Hwang, G. (2024) · Review of Educational Research · doi:10.3102/00346543241261732 · Cited by 34
“This will not only benefit knowledge accumulation and lead to more thorough evaluations but also support the replication and generalization of FCs across diverse populations (Hedges, 2017; Makel & Plucker, 2014).”
Questionable and Improved Research Practices in Single-Case Experimental Design: Initial Investigation and Findings
Tincani, M., Travers, J. C., Dowdy, A., Slocum, T. A., & Deitrich, R. (2025) · Perspectives on Behavior Science · doi:10.1007/s40614-025-00441-9 · Cited by 17
“…findings less reliable or robust than initially conceived, or they have noted a general lack of published replication studies (e.g., Evanschitzky & Armstrong, 2010; Hubbard & Armstrong, 1994; Ioannidis, 2005, 2012; Open Science Collaboration, 2015; Makel & Plucker, 2014; Pashler & Harris, 2012).”
Reproducibility: A Primer on Semantics and Implications for Research
Pellizzari, E. D., Lohr, K. N., Blatecky, A. R., & Creel, D. V. (2017) · doi:10.3768/rtipress.2017.bk.0020.1708 · Cited by 12
Registered Reports, Replication, and the Canadian Journal of School Psychology : Improving the Evidence in Evidence-Based School Psychology
Shaw, S. R., D'Intino, J., & Lysenko, E. (2019) · Canadian Journal of School Psychology · doi:10.1177/0829573519843027 · Cited by 12
Nicht replizieren: publizieren!?
Rost, D. H., & Bienefeld, M. A. (2019) · Zeitschrift für Pädagogische Psychologie · doi:10.1024/1010-0652/a000253 · Cited by 13
A Formative Experiment to Align Middle-School History Instruction with Literacy Goals
Colwell, J., & Reinking, D. (2016) · Teachers College Record The Voice of Scholarship in Education · doi:10.1177/016146811611801201 · Cited by 12
“Replication, which has been documented to be minimal in published education research (Makel & Plucker, 2014), is essential to this methodological approach, because replication enables the development of more generalizable pedagogical theory (Reinking & Bradley, 2008).”
Background
Collaborative research and use of Q methodology to understand technology infusion in teacher preparation
Clausen, J., Borthwick, A. C., & Rutledge, D. W. (2021) · Educational Technology Research and Development · doi:10.1007/s11423-021-10018-3 · Cited by 15
Positivity bias in higher education research
Tight, M. (2022) · Higher Education Quarterly · doi:10.1111/hequ.12388 · Cited by 11
Systematic reviewers' perspectives on replication of systematic reviews: A survey
Nguyen, P., McKenzie, J. E., Hamilton, D. G., Moher, D., Tugwell, P., Fidler, F. M., Haddaway, N., Higgins, J. P. T., Kanukula, R., Karunananthan, S., et al. (2023) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.12009 · Cited by 12
“Moreover, many participants were convinced that not enough replication was being conducted, echoing similar conclusions from opinion surveys of researchers using other study designs [24, 25], as well as the actual occurrence of replication studies in the literature [10 – 12].”
Eine Analyse der Informationssuche bei der Erstellung der Übergangsempfehlung: Welcher Urteilsregel folgen Lehrkräfte?
Böhmer, I., Hörstermann, T., Gräsel, C., Krolak‐Schwerdt, S., & Glock, S. (2015) · Pedocs (German Institute for International Educational Research) · doi:10.25656/01:11490 · Cited by 12
Rigor, Replication, and Reproducibility: Increasing the Relevance of Behavioral Disorders Research
Gage, N. A., & Stevens, R. N. (2018) · Education and Treatment of Children · doi:10.1353/etc.2018.0029 · Cited by 12
“For example, Makel and Plucker (2014) reviewed the top 100 education journals to identify how many published studies were true replications.”
Background
A revalidation of the school participant empowerment scale amongst science and mathematics teachers
Rangel, V. S., Suskavcevic, M., Kapral, A., & Dominey, W. (2018) · Educational Studies · doi:10.1080/03055698.2018.1541131 · Cited by 12
“Additionally, there is a need for replication work within education research to ensure that the instruments we use and the work we produce are meaningful across contexts (Makel and Plucker 2014).”
Background
A Systematic Review Exploring the Differences in Reported Data for Pre-College Educational Activities for Computer Science, Engineering, and Other STEM Disciplines
Decker, A., & McGill, M. M. (2019) · Education Sciences · doi:10.3390/educsci9020069 · Cited by 12
Leveraging the opportunities of mixed methods in research synthesis: Key decisions in systematic mixed studies review methodology
Ferguson, S. L., Kerrigan, M. R., & Hovey, K. A. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1436 · Cited by 13
Persons with intellectual and multiple disabilities activate via non-verbal responses a smartphone’s Google Assistant to access preferred stimulation
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Alberti, G., Campodonico, F., Acquaviva, G., Chiariello, V., & Desideri, L. (2020) · International Journal of Developmental Disabilities · doi:10.1080/20473869.2020.1827213 · Cited by 12
“Yet, this information may have relevant practical implications (i.e. may help determine the overall usability of the program and ways to implement it within daily contexts) and thus would need to be gathered (Kazdin 2011, Makel and Plucker 2014).”
Background
Reviews in Research on Adolescence: Genres, Trends, and Challenges
Levesque, R. J. R. (2015) · Adolescent Research Review · doi:10.1007/s40894-015-0022-8 · Cited by 12
“…a finding supported by reviews of disciplines that identify very few published replications (e.g., the complete publication history of the current top 100 education journals ranked by 5-year impact factor found that only 0.13 % of education articles were replications; see Makel and Plucker 2014).”
BackgroundInfluential
Case series of technology-aided interventions to support leisure and communication in extensive disabilities
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Alberti, G., Perilli, V., Chiariello, V., Grillo, G., & Turi, C. (2018) · International Journal of Developmental Disabilities · doi:10.1080/20473869.2018.1533062 · Cited by 12
“Such limitation imposes a certain level of caution in making general statements about the results reported and asks for direct and systematic replication efforts to determine the dependability of the program and its usability across situations and participants (Kazdin 2011, Makel and Plucker 2014).”
Background
Graded sensitivity to structure and meaning throughout the human language networkPreprint
Shain, C., Kean, H., Casto, C., Lipkin, B., Affourtit, J., Siegelman, M., Mollica, F., & Fedorenko, E. G. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.11.12.467812 · Cited by 12
The need to develop tailored tools for improving the quality of thematic bibliometric analyses: Evidence from papers published in Sustainability and Scientometrics
Cabezas-Clavijo, Á., Milanés‐Guisado, Y., Alba-Ruiz, R., & Delgado-Vázquez, Á. M. (2023) · Journal of Data and Information Science · doi:10.2478/jdis-2023-0021 · Cited by 15
Exploring the scientific impact of negative results
Tian, D., Hu, X., Qian, Y., & Li, J. (2024) · J. Informetrics · doi:10.1016/j.joi.2023.101481 · Cited by 7

Authors: Makel, M. C., & Plucker, J. A.
Affiliations: Duke University (US)
University of Connecticut (US)
ORCID: Makel, Matthew C.
Plucker, Jonathan A.
Venue: Educational Researcher
Publisher: American Educational Research Association (AERA)
Published: 2014-08
Volume: 43
Issue: 6
Pages: 304-316
ISSN: 0013-189X (ISSN-L)
1935-102X
DOI: 10.3102/0013189x14545513
OpenAlex: W2139319950
Semantic Scholar: 493a2e93117a876871ac446b26acf47c50ea2cef
Cited by: 609
References: 131
Citations by year: 2026: 17 · 2025: 37 · 2024: 40 · 2023: 40 · 2022: 66 · 2021: 47 · 2020: 59 · 2019: 81 · 2018: 73 · 2017: 52 · 2016: 51 · 2015: 37 · 2014: 8
FWCI: 68.61
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Educational Researcher (publishedVersion)
Topics: Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Education › Educational Research and Methods
Keywords: replication, direct replication, educational research, publishing history, research credibility, methodological quality, replication success, educational periodicals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Facts Are More Important Than Novelty: Replication in the Education Sciences

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks