Cited by 609Psychology Research and BibliometricsMeta-analysis and systematic reviewsEducational Research and Methods

The DOI matches its Crossref record; 132 of the 132 references it cites were found in a registry or an index.

Facts Are More Important Than Novelty: Replication in the Education Sciences

Makel, Matthew C., Plucker, Jonathan A.·2014·Educational Researcher

Open this paper in CoChat — free account

Abstract

Despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not on the merit of replicating important results. The present study analyzed the complete publication history of the current top 100 education journals ranked by 5-year impact factor and found that only 0.13% of education articles were replications. Contrary to previous findings in medicine, but similar to psychology, the majority of education replications successfully replicated the original studies. However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. The results emphasize the importance of third-party, direct replications in helping education research improve its ability to shape education policy and practice.

Integrity signals

Registry

Crossref · 10.3102/0013189x14545513

Cites

132 of 132 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

2 retracted

RETRACTED: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children
Wakefield, A., Murch, S., Anthony, A., Linnell, J., Casson, D., Malik, M., Berelowitz, M., Dhillon, A., Thomson, M., Harvey, P., et al. (1998) · The Lancet · doi:10.1016/s0140-6736(97)11096-0 · Cited by 3,006
Retracted
Retraction—Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children
Lancet, T. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60175-4 · Cited by 500
Retracted
Untitled work
(1963) · Medical Entomology and Zoology · Cited by 15,085
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
An Introduction to Categorical Data Analysis
Agresti, A. (2006) · Wiley series in probability and statistics · doi:10.1002/0470114754 · Cited by 6,775
Journal of social behavior and personality
(1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90025-5 · Cited by 6,751
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
American Economic Review
Moffitt, R. (2009) · American Economic Review · doi:10.1257/aer.99.2.660 · Cited by 4,888
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 5×
“Using health care research as an example, Ioannidis (2005a), in a review of highly cited medical publications (i.e., those cited more than 1,000 times), found only 44% of replications produced results similar to the original study.”
ResultInfluential

Show all 131 references

Research works printed but not linked: 14

10.1037//0003–066x.49.12.997 · DOI · Paper page
Collins H. M., 1985, Changing order
Kane E. J., 1984, Quarterly Journal of Business and Economics, 23, 3
Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 1
Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 21
Perspectives on Psychological Science. (2012). Special section on replicability in psychological science: A crisis of confidence? Retrieved from http://pps.sagepub.com/content/7/6.toc · DOI · Paper page
Powell K., 2007, PLoS Biology, 12, 2748
Rhine J. B., 1997, Extra-sensory perception
Ritchie S. J., 2012, Psychologist, 25, 346
10.1037//0033–2909.86.3.638 · DOI · Paper page
Rotton J., 1995, Journal of Social Behavior and Personality, 10, 1
Simons D. J., 2012, Psychologist, 25, 349
Simons D. J., 2014, Perspectives on Psychological Science. APS Observer, 27, 16
Williams S. N., 2012, Science, 336, 801

Sorted by citations

Items 301–400 of 608

Transforming mathematics education in Kazakhstan: evaluating the impact of innovative teaching methods on student outcomes in technical universities
Saparbayeva, E., Abdualiyeva, M., Torebek, Y., Kostangeldinova, A., Tursynbayev, A., Takibayeva, G., & Sabalakhova, A. (2025) · Cogent Education · doi:10.1080/2331186x.2025.2461978 · Cited by 11
“This replication of positive effects aligns with Makel and Plucker ’ s (2014) emphasis on reproducibility in educational research and Zhanys et al. ’ s (2020) findings on the importance of integrated approaches to mathematics education in Kazakhstan.”
Science Teachers’ Conceptions of Atomic Models
Wiener, J. (2020) · European Journal of Mathematics and Science Education · doi:10.12973/ejmse.1.2.67 · Cited by 11
Open Game Data: A Technical Infrastructure for Open Science with Educational Games
Gagnon, D., & Swanson, L. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44751-8_1 · Cited by 11
Incongruence Between Native and Test Administration Languages: Towards Equal Opportunity in International Literacy Assessment
Smith, P., Frazier, P., Lee, J., & Chang, R. (2017) · International Journal of Testing · doi:10.1080/15305058.2017.1407767 · Cited by 11
“…the PISA reading literacy test is used as a basis for numerous comparisons of youth across the globe, this previous and parallel examination of 2009 PISA reading literacy (Smith, Cheema, & Kumi-Yeboah, 2016) to arrive at more definitive, accurate and representative results (Makel & Plucker, 2014).”
Method
Do Computer Science Students Understand Their Programming Task?—A Case Study of Solving the Josephus Variant Problem
Febrian, A., & Lawanto, O. (2018) · International Education Studies · doi:10.5539/ies.v11n12p26 · Cited by 11
A Program Based on Common Technology to Support Communication Exchanges and Leisure in People With Intellectual and Other Disabilities
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Alberti, G., Chiariello, V., Campanella, C., Grillo, G., & Tagliente, V. (2019) · Behavior Modification · doi:10.1177/0145445519850747 · Cited by 11
Realizing the promises of replication studies in education
Morrison, K. (2019) · Educational Research and Evaluation · doi:10.1080/13803611.2020.1838300 · Cited by 11
Considerations for Evidence Frameworks in Education Research
Taylor, J. A., Davis, E. B., & Michaelson, L. (2021) · Review of Research in Education · doi:10.3102/0091732x20985077 · Cited by 11
“Studies replicated without researcher independence are more likely than independent replications to report favorable findings, suggesting that conflict of interest can have an effect on the results of a study (Makel & Plucker, 2014).”
BackgroundResult
In Pursuit of a Replication Culture in Mathematics Education
Star, J. R. (2021) · Implementation and Replication Studies in Mathematics Education · doi:10.1163/26670127-01010003 · Cited by 13
Experimental and Quasi-Experimental Research in CSCL
Janssen, J. J. H. M., & Kollar, I. (2021) · Springer eBooks · doi:10.1007/978-3-030-65291-3_27 · Cited by 15
“4 The Future: Reproducibility, Replications, and Open Science Over the last decade, concerns have grown that a large number of reported social research ﬁ ndings may be dif ﬁ cult to replicate by independent researchers (Makel and Plucker 2014) or may even be false (Ioannidis 2005).”
Influential
Further Effects of Phylogenetic Tree Style on Student Comprehension in an Introductory Biology Course
Dees, J. A., Bussard, C., & Momsen, J. L. (2018) · CBE—Life Sciences Education · doi:10.1187/cbe.17-03-0058 · Cited by 10
“the same institution, which limits the robustness of the claims due to potential experimenter bias (Makel and Plucker, 2014).”
Background
Is the educational ‘what works’ agenda working? Critical methodological developments
Pampaka, M., Williams, J. S., & Homer, M. (2016) · International Journal of Research & Method in Education · doi:10.1080/1743727x.2016.1205817 · Cited by 11
“How many study findings would replicate (see Makel and Plucker 2014)?”
Education policy evaluation
Golden, G. (2020) · OECD education working papers · doi:10.1787/9f127490-en · Cited by 25
Examining the Impact of a First Grade Whole Number Intervention by Group Size
Clarke, B., Doabler, C. T., Sutherland, M., Kosty, D. B., Turtura, J. E., & Smolkowski, K. (2022) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2093299 · Cited by 10
“The importance of systematic replication is garnering increased focus (Chhin et al., 2018; National Science Foundation et al., 2018) to address the current lack of replication research (Cook et al., 2015) and it’s perceived value (Makel & Plucker, 2014).”
Background
Teachers’ stated beliefs and practices regarding L2 motivational strategies: A mixed-methods study of misalignment and contributing factors
Ye, X. S., & Hu, G. (2024) · System · doi:10.1016/j.system.2024.103236 · Cited by 13
A Modified Smartphone-Based Program to Support Leisure and Communication Activities in People with Multiple Disabilities
Lancioni, G. E., O’Reilly, M. F., Sigafoos, J., Campodonico, F., Perilli, V., Alberti, G., Ricci, C., & Miglino, O. (2017) · Advances in Neurodevelopmental Disorders · doi:10.1007/s41252-017-0047-z · Cited by 10
“New studies with additional participants will have to determine the reliability of the present data and, based on the outcome, the possibility of recommending the program reported for use within regular education, rehabilitation, and care contexts (Kazdin 2011; Makel and Plucker 2014).”
Background
Structural Validity and Reliability of Social, Academic, and Emotional Behavior Risk Screener–Student Rating Scale Scores: A Replication Study
Kilgus, S. P., Eklund, K. R., von der Embse, N. P., Weist, M., Barber, A., Kaul, M., & Dodge, S. (2020) · Assessment for Effective Intervention · doi:10.1177/1534508420909527 · Cited by 11
“…psychology has several far-reaching implications, it is particularly problematic for applied disciplines, which rely upon replication research to identify those practices in which schools can place confidence regarding their capacity to enhance student educational outcomes (Makel & Plucker, 2014).”
Background
An Integrative Qualitative Framework: Improving Research Through Strategic Mapping
Coker, D. C. (2023) · International Research in Education · doi:10.5296/ire.v11i1.20921 · Cited by 18
Defining and assessing FoI in a large-scale randomized trial: Core components of values affirmation
Bradley, D. N., Crawford, E., & Dahill-Brown, S. E. (2016) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2016.04.002 · Cited by 9
Supporting Simple Activity Engagement in Persons With Moderate to Severe Alzheimer’s Disease Through a Technology-Aided Program
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Pinto, K., & Chiapparino, C. (2017) · American Journal of Alzheimer s Disease & Other Dementias® · doi:10.1177/1533317517698793 · Cited by 10
“Fifth, the positive activity engagement outcomes described earlier, although highly encouraging as to the potential impact of a technology-aided program in daily contexts, could hardly be considered sufficient to guarantee its adoption within those contexts.(32,42) In fact, such an adoption might be based not only on…”
BackgroundResult
Identifying and Using Secondary Datasets to Answer Policy Questions Related to School-Based Counseling: A Step-by-Step Guide
Bryan, J. A., Kim, J., & Shi, Q. (2017) · Springer eBooks · doi:10.1007/978-3-319-58179-8_11 · Cited by 10
The Road to Uncreative Science Is Paved With Good Intentions: Ideas, Implementations, and Uneasy Balances
Kaufman, J. C., & Glăveanu, V. P. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691617753947 · Cited by 11
Journal for Research in Mathematics Education: Practical Guides for Promoting and Disseminating Significant Research in Mathematics Education
Cai, J., Hwang, S., & Robison, V. (2019) · ICME-13 monographs · doi:10.1007/978-3-030-15636-7_21 · Cited by 10
Self-Regulation Strategies in an Engineering Design Project
Lawanto, O., Febrian, A., Butler, D. L., & Mina, M. (2019) · International Education Studies · doi:10.5539/ies.v12n5p133 · Cited by 10
“Although replication studies are currently underappreciated, they are “essential to moving toward a more reliable and trustworthy understanding of educational environments” (Makel & Plucker, 2014, p. 313).”
Background
Smartphone-Based Technology to Support Functional Occupation and Mobility in People with Intellectual Disability and Visual Impairment
Lancioni, G. E., O’Reilly, M. F., Sigafoos, J., Desideri, L., Alberti, G., Campodonico, F., & Chiariello, V. (2019) · Advances in Neurodevelopmental Disorders · doi:10.1007/s41252-019-00120-2 · Cited by 10
Editors’ Introduction: Randomized Controlled Trials Meet the Real World: The Nature and Consequences of Null Findings
Herrington, C. D., & Maynard, R. A. (2019) · Educational Researcher · doi:10.3102/0013189x19891441 · Cited by 10
Structuring First-Year Retention at a Regional Public Institution: Validating and Refining the Structure of Bowman’s SEM Analysis
Collier, D., Fitzpatrick, D., Brehm, C., Hearit, K. M., & Beach, A. L. (2020) · Research in Higher Education · doi:10.1007/s11162-020-09612-w · Cited by 14
Supporting leisure and communication in people with visual and intellectual disabilities via a smartphone-based program
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Alberti, G., Perilli, V., Zimbaro, C., & Chiariello, V. (2017) · British Journal of Visual Impairment · doi:10.1177/0264619617715497 · Cited by 10
“In light of this limitation, new studies with additional participants seem warranted to determine the strength and generality of the evidence available (Kazdin, 2011; Makel & Plucker, 2014).”
Background
Improving Learning
Higgins, S. (2018) · Cambridge University Press eBooks · doi:10.1017/9781139519618 · Cited by 26
Low Research-Data Availability in Educational-Psychology Journals: No Indication of Effective Research-Data Policies
Huff, M., & Bongartz, E. C. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459231156419 · Cited by 11
“…Glasziou et al., 2014; Ioannidis et al., 2014) and the replication crises in psychology (Open Science Collaboration, 2015), the educational sciences (Makel & Plucker, 2014; Plucker & Makel, 2021; Shaver & Norton, 1980), the social sciences (Camerer et al., 2018), cancer research (Errington…”
Background
Adolescent Exposure to Restorative Practices and Their Perceptions of Support, Structure, and Bullying in the School Climate
Gregory, A., Huang, F. L., & Ward-Seidel, A. R. (2024) · AERA Open · doi:10.1177/23328584241288525 · Cited by 31
“Recent articles in psychology and education sciences have highlighted the importance of replicating studies in order for research to influence educational policy and practice (Makel & Plucker, 2014).”
Promoting Occupational Engagement and Personal Satisfaction in People with Neurodevelopmental Disorders via a Smartphone-Based Intervention
Lancioni, G. E., O’Reilly, M. F., Sigafoos, J., Alberti, G., Campodonico, F., & Chiariello, V. (2019) · Advances in Neurodevelopmental Disorders · doi:10.1007/s41252-019-00102-4 · Cited by 9
Assessing the Fairness of Course Success Prediction Models in the Face of (Un)equal Demographic Group Distribution
Deho, O. B., Joksimovíc, S., Liu, L., Li, J., Zhan, C., & Liu, J. (2023) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3573051.3593381 · Cited by 9
A basic technology-aided programme for leisure and communication of persons with advanced amyotrophic lateral sclerosis: performance and social rating
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Ferlisi, G., Zullo, V., Denitto, F., Lauta, E., Abbinante, C., et al. (2015) · Disability and Rehabilitation Assistive Technology · doi:10.3109/17483107.2015.1104561 · Cited by 9
Emergent themes from recent research syntheses in science education and their implications for research design, replication, and reporting practices
Taylor, J. A., Furtak, E. M., Kowalski, S. M., Martinez, A., Slavin, R. E., Stuhlsatz, M. A. M., & Wilson, C. D. (2016) · Journal of Research in Science Teaching · doi:10.1002/tea.21327 · Cited by 10
An analysis of grammatical patterns in generation 1.5, L1 and L2 students’ writings: A replication study
de Kleine, C., & Lawton, R. (2018) · Journal of Second Language Writing · doi:10.1016/j.jslw.2018.10.003 · Cited by 9
Comparative Efficacy and Generalization of Two Word-Reading Interventions with English Learners in Elementary School
Klingbeil, D. A., January, S. A., & Ardoin, S. P. (2019) · Journal of Behavioral Education · doi:10.1007/s10864-019-09331-y · Cited by 9
The Role of A/B Tests in the Study of Large-Scale Online LearningPreprint
Savi, A. O., Williams, J. J., Maris, G., & van der Maas, H. L. J. (2017) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/83jsg · Cited by 9
Replication Studies Using Secondary or Nonexperimental Datasets
Huang, F. L., & Huang, A. B. (2024) · School Psychology Review · doi:10.1080/2372966x.2024.2346781 · Cited by 8
“Whereas replications may at times be thought of as redundant, several authors have discussed the importance of replication studies (Makel & Plucker, 2014; Nosek et al., 2015).”
Is Sexual Racism Still Really Racism? Revisiting Callander et al. (2015) in the USA
Smith, J. G., Sage, A. J., McGlenn, M., Robbins, J., & Garmon, S. L. (2022) · Archives of Sexual Behavior · doi:10.1007/s10508-022-02351-2 · Cited by 9
“Makel and Plucker (2014) found that only 0.13% of the papers in top education journals were replication studies.”
BackgroundInfluential
Why replicate? Systematic review of calls for replication in Language Teaching
Yamashita, T., & Neiriz, R. (2024) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100091 · Cited by 8
Structured Reporting Guidelines for Classroom Intervention Research
Hill, H. C., Lynch, K. S., & Polanin, J. R. (2025) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2025.2470322 · Cited by 11
“These features and characteristics may be necessary for successful replication (e.g., Makel & Plucker, 2014).”
Zusammenhänge zwischen einem gemeinsamen Unterricht und kognitiven und non-kognitiven Outcomes von Kindern ohne sonderpädagogischen Förderbedarf
Kuhl, P., Kocaj, A., & Stanat, P. (2020) · Zeitschrift für Pädagogische Psychologie · doi:10.1024/1010-0652/a000283 · Cited by 9
Telepractice parent-implemented training and coaching in a rural area in the UK: Impact on mothers and their children with autism
Kossyvaki, L., Meadan, H., Cristescu, L., Alharbi, H., Sands, M. M., & Tuite, D. (2022) · Research in Autism · doi:10.1016/j.rasd.2022.102044 · Cited by 9
A cautionary note on aggregation in educational psychology and beyond
Krammer, G. (2023) · Theory & Psychology · doi:10.1177/09593543231172495 · Cited by 8
“This lack has been criticized in educational psychology (Makel & Plucker, 2014; Plucker & Makel, 2021; Rost & Bienefeld, 2019; Travers et al., 2016), and unfortunately ties into psychol-ogy’s tradition of preferring novel and statistically significant findings over thoroughly researched findings…”
AP Calculus and Science Coursetaking: Their Relationships With Choosing a STEM Major and Expecting to Be in a STEM Occupation
Bohrnstedt, G. W., Ogut, B., Yee, D., & Bai, Y. (2023) · AERA Open · doi:10.1177/23328584231184426 · Cited by 7
“As noted by Makel and Plucker (2014): “If education research is to be relied upon to develop sound policy and practice, then conducting replications on important findings is essential to moving toward a more reliable and trustworthy understanding of educational environments” (p. 313).”
Background
Reviewing School District and Caregiver Engagement: Toward Building Capacity to Serve English Learners
Howell, E., Kaminski, R., & Pirolla, T. (2024) · Language and Literacy · doi:10.20360/langandlit29664 · Cited by 7
“We hope this project method will serve as a model for future research on how to be inclusive and responsive to participants and address needed elements of sustainability and replication (Makel & Plucker, 2014).”
Method
Will Mentoring a Student Teacher Harm My Evaluation Scores? Effects of Serving as a Cooperating Teacher on Evaluation Metrics
Ronfeldt, M., Bardelli, E., Brockman, S. L., & Mullman, H. (2019) · American Educational Research Journal · doi:10.3102/0002831219872952 · Cited by 9
“In keeping with recent calls for more replication studies in educational research (Makel & Plucker, 2014), the present study replicates and extends the Goldhaber et al. study in a different teacher labor market and state context.”
Result
Open Science als Beitrag zur Qualität in der Bildungsforschung
Krammer, G., & Svecnik, E. (2020) · Zeitschrift für Bildungsforschung · doi:10.1007/s35834-020-00286-z · Cited by 11
Reproducible Analyses in Education Research
LeBeau, B. C., Ellison, S., & Aloe, A. M. (2021) · Review of Research in Education · doi:10.3102/0091732x20985076 · Cited by 9
“…stronger topics of interest in education research and feel the field will be enhanced by this discussion in a similar way that discussion of replication has enhanced the educational literature (Hedges & Schauer, 2019a, 2019b; IES, U.S. Department of Education, & NSF, 2018; Makel & Plucker, 2014).”
BackgroundMethodInfluential
A Framework for Classifying Replication Studies in Educational Technologies Research
Christensen, R. W., Hodges, C. B., & Spector, J. M. (2021) · Technology Knowledge and Learning · doi:10.1007/s10758-021-09532-3 · Cited by 8
“2 Background Makel and Plucker (2014) defined replication as “the purposeful repetition of previous results to corroborate or disconfirm the previous results” (p. 305).”
BackgroundResultInfluential
Agricultural Leadership Development Program Participant Personality and Demographic Characteristics: An Empirical Analysis
Lamm, K. W., Holder, M., Randall, N. L., Edgar, D. W., & Lamm, A. J. (2021) · SAGE Open · doi:10.1177/21582440211061577 · Cited by 7
“As Makel and Plucker (2014) state, “If education research is to be relied upon to develop sound policy and practice, then conducting replications on important findings is essential to moving toward a more reliable and trustworthy understanding of educational environments” (p. 313).”
Background
Alternating treatments designs: Interpretation challenges and design solutions for validating and comparing interventions
Skinner, C. H., McClurg, V. M., Crewdson, M. A., Coleman, M. B., Bennett, J., Fowler, K., & Killion, J. B. (2021) · Psychology in the Schools · doi:10.1002/pits.22638 · Cited by 7
Perceived aggressive monetization: why some mobile gamers won’t spend any money on in-app purchases
Salehudin, I., & Alpert, F. H. (2022) · Electronic Commerce Research · doi:10.1007/s10660-022-09603-2 · Cited by 11
Replication and Reproduction in Canadian Policing Research: A Note
Huey, L., & Bennell, C. (2016) · Canadian Journal of Criminology and Criminal Justice/La Revue canadienne de criminologie et de justice pénale · doi:10.3138/cjccj.2016.e09 · Cited by 8
“One recent study that examined education research found that, out of the 164,589 studies examined, only 0.13% of them were replication attempts (Makel and Plucker 2014).”
BackgroundInfluential
Inspecting the foundations of claims about cognitive demand and student learning: A citation analysis of Stein and Lane (1996)
Otten, S., Webel, C., & de Araujo, Z. (2017) · The Journal of Mathematical Behavior · doi:10.1016/j.jmathb.2016.12.008 · Cited by 7
Replikationsstudie zur Testgüte der Professionsunabhängigen Einstellungsskala zum Inklusiven Schulsystem – ein Beitrag zur Validierung
Schulze, S., Lüke, T., Schröter, A., Krause, K., & Kühl, J. (2019) · Unterrichtswissenschaft · doi:10.1007/s42010-018-00034-3 · Cited by 7
Price adjustment in the London housing market
Webb, R., Watson, D., & Cook, S. (2019) · Urban Studies · doi:10.1177/0042098019878349 · Cited by 7
“While perhaps most closely associated with the ‘replication crisis’ in psychology (see Open Science Collaboration, 2015), calls for replication have been witnessed in numerous disciplines (see, inter alia, Chang et al., 2017; Frank and Saxe, 2012; Makel and Plucker, 2014; McNeeley and Warner, 2015; Sukhtankar, 2017; W…”
Background
Japanese physical education graduate students’ learning about research skill development
Sato, T., & McKay, C. (2019) · Curriculum Studies in Health and Physical Education · doi:10.1080/25742981.2019.1685898 · Cited by 7
“According to Makel and Plucker (2014), many graduate students try to duplicate the theoretical application, sampling procedures, test condition using an education model (e.g. sport education or teaching games for understanding), and method of analysis from past studies.”
Background
Self-Regulated Versus Staff-Regulated Stimulation for Promoting Indices of Satisfaction in Persons with Severe/Profound and Multiple Disabilities
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Resta, E., Campodonico, F., Alberti, G., & Chiariello, V. (2020) · Journal of Developmental and Physical Disabilities · doi:10.1007/s10882-020-09737-x · Cited by 7
Evidence-informed practice in assessment: forging the nexus between research, policy & practice
Davari-Torshizi, M. (2020) · Assessment in Education Principles Policy and Practice · doi:10.1080/0969594x.2020.1815647 · Cited by 7
“Most importantly, in order to rely on evidence as to develop valid policy and practice, doing replications on important findings is necessary and likely to lead to a more solid understanding of educational environments (Makel & Plucker, 2014).”
A Further Evaluation of Microswitch-Aided Intervention for Fostering Responding and Stimulation Control in Persons in a Minimally Conscious State
Lancioni, G. E., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Buonocunto, F., Devalle, G., Trimarchi, P. D., Navarro, J., & Lanzilotti, C. (2018) · Advances in Neurodevelopmental Disorders · doi:10.1007/s41252-018-0064-6 · Cited by 7
“Indeed, study 1 was to evaluate the effectiveness of a microswitchaided program in helping ten new participants increase functional responding to control preferred stimulation events (Kazdin 2011; Makel and Plucker 2014).”
Background
Implementation and effects of a schoolwide data-based decision making intervention: a large-scale study
Geel, V., & Keuning, T. (2016) · doi:10.3990/1.9789036541190 · Cited by 14
20 years of interactive tasks in large‐scale assessments: Process data as a way towards sustainable change?
Stadler, M. J., Brandl, L., & Greiff, S. (2023) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12847 · Cited by 6
Estimating the Impact of Integrated Student Support on Elementary School Achievement: A Natural Experiment
Lawson, J. L., O’Dwyer, L. M., Dearing, E., Raczek, A. E., Foley, C., Khanani, N., Walsh, M. E., & Leigh, Y. R. (2024) · AERA Open · doi:10.1177/23328584241292072 · Cited by 6
“…a significant contribution to a growing evidence base on integrated student support in that it serves as a conceptual replication and extends the cumulative knowledge about integrated student support (Chhin et al., 2018; Derksen & Morawski, 2022; Duncan et al., 2014; Makel & Plucker, 2014).”
Comment on “The Psychology of Creativity: A Critical Reading” by Vlad Petre Glăveanu
Hennessey, B. A. (2015) · Creativity Theories – Research – Applications · doi:10.1515/ctra-2015-0004 · Cited by 6
“In August of this year, Makel and Plucker (2014) reported that only 13% of education articles published in the field’s top 100 journals are replications and argued convincingly for a turn-around of this trend; while Beghetto (2014) expanded upon recent critiques of the state of social science…”
Background
Primer on Reproducible Research in R: Enhancing Transparency and Scientific Rigor
Siraji, M. A., & Rahman, M. (2023) · Clocks & Sleep · doi:10.3390/clockssleep6010001 · Cited by 6
“How to attain this balance is an issue of high debate [10,11].”
Background
Academic Culture: Its Meaning, Measure and Contribution to Student Learning
Leithwood, K. A., Sun, J., Zhang, S., & Hua, C. (2024) · Journal of School Leadership · doi:10.1177/10526846241258199 · Cited by 11
Re-conceptualising and accounting for examiner (cut-score) stringency in a ‘high frequency, small cohort’ performance test
Homer, M. (2020) · Advances in Health Sciences Education · doi:10.1007/s10459-020-09990-x · Cited by 6
A Conceptual Replication of a Kindergarten Math Intervention Within the Context of a Research-Based Core
Clarke, B., Turtura, J. E., Lesner, T. A., Cook, M. A., Smolkowski, K., Kosty, D. B., & Doabler, C. T. (2022) · Exceptional Children · doi:10.1177/00144029221088938 · Cited by 6
“…research supports the generalizability of hypotheses generated from initial research (Schmidt, 2009) and serves not only to affirm scientific understanding of the findings replicated but also to expose methodological biases and insufficiencies within an initial study (Makel & Plucker, 2014).”
BackgroundResultInfluential
Social and Emotional Learning and Early Literacy Skills: A Quasi-Experimental Study of RULER
Bailey, C. S., Martinez, O., & DiDomizio, E. E. (2023) · Education Sciences · doi:10.3390/educsci13040397 · Cited by 12
Promoting low-income preschoolers’ school readiness through a text messaging intervention
Baroody, A. E., Ferretti, C. K., & Larsen, R. (2018) · Journal of Children and Poverty · doi:10.1080/10796126.2018.1517939 · Cited by 9
“Yet replication is an important aspect of the scientific method and is essential to providing trustworthy evidence to guide education policies and practices (Makel and Plucker 2014).”
Background
Replicating a study about children’s drawings concerning radiation
Plötz, T., & Hollenthoner, F. (2019) · Multidisciplinary Journal for Education Social and Technological Sciences · doi:10.4995/muse.2019.10390 · Cited by 6
“Makel and Plucker (2014) stated the importance of replication studies in education research.”
Background
Procedural versus content‐related hints for word problem solving: an exploratory study
de Kock, W. D., & Harskamp, E. G. (2016) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12148 · Cited by 6
Developing Prospective Teachers’ Ability to Diagnose Evidence of Student Thinking: Replicating a Classroom Intervention
Phelps-Gregory, C. M., & Spitzer, S. M. (2017) · Springer eBooks · doi:10.1007/978-3-319-66327-2_12 · Cited by 7
Basic smartphone-aided communication and leisure for people with extensive neuro-motor impairment and absence of speech
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Vincenti, A., Buonocunto, F., Susco, V., Lanzilotti, C., & Navarro, J. (2019) · Neurorehabilitation · doi:10.3233/nre-192811 · Cited by 6
“New studies with additional participants are needed to assess the effectiveness of the new upgraded technology version as well as its adaptation and determine the reliability of the data obtained in this study (Kazdin, 2011; Makel & Plucker, 2014).”
Replication Studies in Educational Technology
(2015) · TechTrends · doi:10.1007/s11528-015-0862-2 · Cited by 6
Gender Differences and Inequality? A 20-Year Retrospective Analysis Based on 39,980 Students’ Perceptions of Physical Education in Sweden
Jansson, A., Sundblad, G. B., Lundvall, S., Bjärsholm, D., & Norberg, J. R. (2022) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.2021-0270 · Cited by 5
Evidence of the Internal Structure and Measurement Invariance of the BASC-3 Behavioral and Emotional Screening System Teacher Form
Eklund, K. R., Kilgus, S. P., Willenbrink, J. B., Collins, B., Gill, N., Weist, M., Porter, J., Lewis, T. J., Mitchell, B. S., & Wills, H. P. (2022) · Journal of Psychoeducational Assessment · doi:10.1177/07342829221116807 · Cited by 5
“…crisis within psychology has several far-reaching implications, it is particularly problematic for applied disciplines (e.g., school psychology) that rely on replication research to identify tools and practices that schools can confidently use to best serve children (Makel & Plucker, 2014).”
Background
Pivoting Training and Coaching for an Early Numeracy Intervention During COVID-19
Fainstein, D., Clarke, B., Kosty, D. B., Cycyk, L. M., Turtura, J. E., & Doabler, C. T. (2024) · Prevention Science · doi:10.1007/s11121-024-01713-7 · Cited by 5
“Author overlap presents an opportunity for unintentional bias (Makel & Plucker, 2014).”
“1-800-Help-Me-With-Open-Science-Stuff”: A Qualitative Examination of Open Science Practices in Communication Sciences and Disorders
Pfeiffer, D. L., Thompson, A. R., Ciullo, B., Hirsch, M. E., Amin, M. E., Ford, A. L. B., Riccardi, J. S., & Kearney, E. (2024) · Journal of Speech Language and Hearing Research · doi:10.1044/2024_jslhr-24-00378 · Cited by 5
The GPT Surprise: Offering Large Language Model Chat in a Massive Coding Class Reduced Engagement But May Increase Adopters' Exam Performances
Nie, A., Chandak, Y., Suzara, M., Malik, A., Woodrow, J., Peng, M., Sahami, M., Brunskill, E., & Piech, C. (2024) · ACM Conference on Learning @ Scale · doi:10.1145/3698205.3733960 · Cited by 47
Two Additional Suggested Reforms to Encourage Replication Studies in Educational Research
Warne, R. T. (2014) · Educational Researcher · doi:10.3102/0013189x14562294 · Cited by 5
Does Test Item Performance Increase with Test-to-Standards Alignment?
Traynor, A. (2017) · Educational Assessment · doi:10.1080/10627197.2017.1344092 · Cited by 5
“We intended the item difficulty model for the TIMSS item response data to serve as a partial operational replication (e.g., Makel & Plucker, 2014) of the NAEP data analysis.”
Method
Disrupting (Mis)Representation in the Literacy Achievement of “(Under)Performing” Youth
Smith, P., Warrican, S. J., Yeboah, A. K., Cheema, J., & Alleyne, M. L. (2019) · Journal of Education and Development in the Caribbean · doi:10.46425/j518021775 · Cited by 5
Smartphone-Based Technology to Help Individuals with Intellectual Disability and Blindness Manage Basic Indoor Travel
Lancioni, G. E., O’Reilly, M. F., Sigafoos, J., Desideri, L., Alberti, G., Chiariello, V., & Nozzolillo, A. (2020) · Advances in Neurodevelopmental Disorders · doi:10.1007/s41252-020-00169-4 · Cited by 5
Regional differences in educational achievement: A replication study of municipality data
Boman, B. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.854342 · Cited by 7
“This article underlines the importance of replication in educational research (Makel and Plucker, 2014) as it contributes with a replication of the relationships found in Boman (2022b) by utilizing similar data sources, analytical strategies, dependent, and independent variables.”
Method
Method considerations for school psychology from longitudinal research on gifted students
Wai, J., Kell, H. J., & Worrell, F. C. (2023) · Journal of School Psychology · doi:10.1016/j.jsp.2023.101269 · Cited by 8
Breakdowns in Scientific Practices: How and Why Some Accepted Scientific Claims May Have Little Actual Support
White, M., & Stovner, R. B. (2024) · Collabra Psychology · doi:10.1525/collabra.121436 · Cited by 7
Assessing Novices’ Game Performance in Rugby Union: The Rugby Attack Assessment Instrument (RAAI)
Llobet-Martí, B., López‐Ros, V., Barrera‐Gómez, J., & Comino, J. (2016) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.2014-0134 · Cited by 5
“Although a recent research shows very low repeatability of studies (Makel & Plucker, 2014), we have constructed the IP in a way that provides objective measures of the RAAI, which favours replication, even in other contexts of educational research.”
Result
Diversified occupation and communication program versions for persons with acquired neurological damage and multiple disabilities
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Buonocunto, F., Navarro, J., Lanzilotti, C., & Alberti, G. (2016) · International Journal on Disability and Human Development · doi:10.1515/ijdhd-2016-0022 · Cited by 5
Fostering Indoor Ambulation and Object Transportation as a Form of Physical Exercise for Persons with Multiple Disabilities
Lancioni, G. E., O’Reilly, M. F., Sigafoos, J., Alberti, G., Perilli, V., Zimbaro, C., Chiariello, V., & Campodonico, F. (2017) · Advances in Neurodevelopmental Disorders · doi:10.1007/s41252-017-0032-6 · Cited by 5
“Obviously, new studies with additional participants are needed to determine the strength of the present findings (Kazdin 2011; Makel and Plucker 2014).”
Background
Replicable quantitative psychological and educational research: Possibility or pipe dream?
Cantley, I. (2022) · Educational Philosophy and Theory · doi:10.1080/00131857.2022.2090926 · Cited by 5
“In an analysis of the full publication history of the top 100 education journals according to 2011 5-year impact factors, Makel and Plucker (2014) reported that just 0.13% of the 164,589 articles that had been published (i.e. 221 articles) involved replication studies (which were direct…”
BackgroundResultInfluential
The treatment adherence of paraprofessional behavior training programs: a systematic literature review
Zarate, K., & Barcus, C. L. (2022) · European Journal of Psychology of Education · doi:10.1007/s10212-022-00646-5 · Cited by 5
“Moreover, trainings must provide clear objectives with replicable methods, so that learners gain the necessary information to implement procedures accurately and future researchers can confirm results through replication (Makel & Plucker, 2014).”
Background
Quackery in Educational Research
Robinson, D. H., & Levin, J. R. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108235631.003 · Cited by 5
Exploring the connection between gamification and student engagement in computer-supported collaboration.
Knutas, A., Hynninen, T., Wolff, A., & Khakurel, J. (2019) · Cited by 5
A new tablet-based program to support leisure and video calls in people with intellectual and motor disabilities
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Grillo, G., Desideri, L., Alberti, G., & Campodonico, F. (2020) · Technology and Disability · doi:10.3233/tad-200268 · Cited by 5
Promoting conceptual replications in educational research
Cai, J. (2022) · Educational Research and Evaluation · doi:10.1080/13803611.2021.2022313 · Cited by 5
The effects of reflection on seventh grade social studies student learning and perception
Denton, D. W., & Ellis, A. K. (2023) · Educational Studies · doi:10.1080/03055698.2020.1835616 · Cited by 4
“Indeed, Makel and Plucker (2014) have provided sound justification for replication research, such as noting that replication studies are useful for developing sound practice.”
Background
Bridging cognitive skills and environmental awareness: critical and creative thinking as predictors of digital ecoliteracy
Hasibuan, S., Yustina, Y., & Wahyuni, R. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1705676 · Cited by 13

Authors: Makel, M. C., & Plucker, J. A.
Affiliations: Duke University (US)
University of Connecticut (US)
ORCID: Makel, Matthew C.
Plucker, Jonathan A.
Venue: Educational Researcher
Publisher: American Educational Research Association (AERA)
Published: 2014-08
Volume: 43
Issue: 6
Pages: 304-316
ISSN: 0013-189X (ISSN-L)
1935-102X
DOI: 10.3102/0013189x14545513
OpenAlex: W2139319950
Semantic Scholar: 493a2e93117a876871ac446b26acf47c50ea2cef
Cited by: 609
References: 131
Citations by year: 2026: 17 · 2025: 37 · 2024: 40 · 2023: 40 · 2022: 66 · 2021: 47 · 2020: 59 · 2019: 81 · 2018: 73 · 2017: 52 · 2016: 51 · 2015: 37 · 2014: 8
FWCI: 68.61
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Educational Researcher (publishedVersion)
Topics: Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Education › Educational Research and Methods
Keywords: replication, direct replication, educational research, publishing history, research credibility, methodological quality, replication success, educational periodicals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Facts Are More Important Than Novelty: Replication in the Education Sciences

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks