The DOI matches its Crossref record; 132 of the 132 references it cites were found in a registry or an index.
Facts Are More Important Than Novelty: Replication in the Education Sciences
Abstract
Despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not on the merit of replicating important results. The present study analyzed the complete publication history of the current top 100 education journals ranked by 5-year impact factor and found that only 0.13% of education articles were replications. Contrary to previous findings in medicine, but similar to psychology, the majority of education replications successfully replicated the original studies. However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. The results emphasize the importance of third-party, direct replications in helping education research improve its ability to shape education policy and practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Novel or innovative results are more likely to be published than replications or incremental extensions of existing findings 13-15 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [15]
Citations
Cites
2 retracted
Wakefield, A., Murch, S., Anthony, A., Linnell, J., Casson, D., Malik, M., Berelowitz, M., Dhillon, A., Thomson, M., Harvey, P., et al. (1998) · The Lancet · doi:10.1016/s0140-6736(97)11096-0 · Cited by 3,006
RetractedLancet, T. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60175-4 · Cited by 500
Retracted(1963) · Medical Entomology and Zoology · Cited by 15,085
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
An Introduction to Categorical Data Analysis
Agresti, A. (2006) · Wiley series in probability and statistics · doi:10.1002/0470114754 · Cited by 6,775
Journal of social behavior and personality
(1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90025-5 · Cited by 6,751
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Moffitt, R. (2009) · American Economic Review · doi:10.1257/aer.99.2.660 · Cited by 4,888
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 5×
“Using health care research as an example, Ioannidis (2005a), in a review of highly cited medical publications (i.e., those cited more than 1,000 times), found only 44% of replications produced results similar to the original study.”
ResultInfluential
Research works printed but not linked: 14
- 10.1037//0003–066x.49.12.997 · DOI · Paper page
- Collins H. M., 1985, Changing order
- Kane E. J., 1984, Quarterly Journal of Business and Economics, 23, 3
- Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 1
- Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 21
- Perspectives on Psychological Science. (2012). Special section on replicability in psychological science: A crisis of confidence? Retrieved from http://pps.sagepub.com/content/7/6.toc · DOI · Paper page
- Powell K., 2007, PLoS Biology, 12, 2748
- Rhine J. B., 1997, Extra-sensory perception
- Ritchie S. J., 2012, Psychologist, 25, 346
- 10.1037//0033–2909.86.3.638 · DOI · Paper page
- Rotton J., 1995, Journal of Social Behavior and Personality, 10, 1
- Simons D. J., 2012, Psychologist, 25, 349
- Simons D. J., 2014, Perspectives on Psychological Science. APS Observer, 27, 16
- Williams S. N., 2012, Science, 336, 801
Cited by
Sorted by citations
Items 101–200 of 608
Sharing successes and hiding failures: ‘reporting bias’ in learning and teaching research
Dawson, P., & Dawson, S. L. (2016) · Studies in Higher Education · doi:10.1080/03075079.2016.1258052 · Cited by 58
“Given the low rate of replication studies in educational research compared to other subject areas, and the low success rate when educational research replications are conducted by researchers not on the original study (Makel and Plucker 2014), approaches similar to Psych File Drawer may be needed.”
BackgroundChhin, C. S., Taylor, K. A., & Wei, W. S. (2018) · Educational Researcher · doi:10.3102/0013189x18788047 · Cited by 49
“…© 2018 AERA. http://er.aera.net Recent publications have highlighted the limited number of replication studies in the social sciences and education (e.g., Lemons et al., 2016; Makel & Plucker, 2014) and have helped sound an alarm for more replication studies to inform education policy and practice.”
BackgroundMethodInfluentialReplication in computing education research: researcher attitudes and experiences
Ahadi, A., Hellas, A., Ihantola, P., Korhonen, A., & Petersen, A. (2016) · Proceedings of the 16th Koli Calling International Conference on Computing Education Research · doi:10.1145/2999541.2999554 · Cited by 48
Educational Technology Research in a VUCA World.
Reeves, T. C., & Reeves, P. M. (2015) · Educational Technology archive · Cited by 47
Exploring profiles of academic help seeking: A mixture modeling approach
Finney, S. J., Barry, C. L., Horst, S. J., & Johnston, M. M. (2017) · Learning and Individual Differences · doi:10.1016/j.lindif.2017.11.011 · Cited by 47
An Investigation Into the Reasons Physical Education Professionals Use Twitter
Harvey, S., & Hyndman, B. (2018) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.2017-0188 · Cited by 46
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Alberti, G., Perilli, V., Chiariello, V., Grillo, G., & Turi, C. (2018) · Disability and Rehabilitation Assistive Technology · doi:10.1080/17483107.2018.1508515 · Cited by 44
Bauer, E., Sailer, M., Niklas, F., Greiff, S., Sarbu‐Rothsching, S., Zottmann, J., Kiesewetter, J., Stadler, M. J., Fischer, M. R., Seidel, T., et al. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.13123 · Cited by 63
Forty Years of BIRGing: New Perspectives on Cialdini’s Seminal Studies
Jensen, J. A., Turner, B. A., James, J., McEvoy, C. D., Seifried, C. S., Delia, E. B., Greenwell, T. C., Ross, S., & Walsh, P. (2016) · Journal of Sport Management · doi:10.1123/jsm.2015-0340 · Cited by 45
Replication crisis or an opportunity to improve scientific production?
Navarro, M. D. F., Pascual‐Llobell, J., Pascual‐Soler, M., Perezgonzalez, J. D., & Berríos-Riquelme, J. (2020) · European Journal of Education · doi:10.1111/ejed.12417 · Cited by 55
Merle, J. L., Thayer, A. J., Larson, M. F., Pauling, S., Cook, C. R., Rios, J. A., McGinnis, J. L., & Sullivan, M. M. (2021) · Journal of School Psychology · doi:10.1016/j.jsp.2021.11.005 · Cited by 58
Reproducible research practices and transparency across linguistics
Bochyńska, A., Keeble, L., Halfacre, C. A. E., Casillas, J. V., Champagne, I.-A., Chen, K., Röthlisberger, M., Buchanan, E. M., & Roettger, T. B. (2023) · Glossa Psycholinguistics · doi:10.5070/g6011239 · Cited by 53
“…2023), as well as previous estimates in psychology (5%, Hardwicke et al., 2022; 1.6%, Makel et al., 2012), social science (1%, Hardwicke et al., 2020), biomedicine (5%, Wallach et al., 2018), educational science (0.1%, Makel & Plucker, 2014), and economics (0.1%, Mueller-Langer et al., 2019).”
InfluentialWord-Problem-Solving Strategy for Minority Students at Risk for Math Difficulties
Kong, J. E., & Orosco, M. J. (2015) · Learning Disability Quarterly · doi:10.1177/0731948715607347 · Cited by 44
“The literature denotes the value of replicating results in educational research, particularly for single-subject research design (Horner et al., 2005; Makel & Plucker, 2014).”
BackgroundAn Inadvertent Concurrent Replication
Lemons, C. J., King, S. A., Davidson, K. A., Berryessa, T. L., Gajjar, S. A., & Sacks, L. H. (2016) · Remedial and Special Education · doi:10.1177/0741932516631116 · Cited by 43
“Similar to Makel and Plucker (2014), team members recorded the type of replication, authorship and journal overlap, and replication success for each study.”
BackgroundMethodResultInfluentialRyan, J. C., & Tipu, S. A. A. (2021) · Research Policy · doi:10.1016/j.respol.2021.104408 · Cited by 43
Camburn, E. M., & Han, S. W. (2017) · School Effectiveness and School Improvement · doi:10.1080/09243453.2017.1302968 · Cited by 45
“Replication studies like this are important because they provide insight about whether empirical results hold across different populations, settings, or conditions (Lindsay & Ehrenberg, 1993; Makel & Plucker, 2014).”
BackgroundAnalysing discussion forum data
Farrow, E., Moore, J. D., & Gašević, D. (2019) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3303772.3303779 · Cited by 42
“We observe that many of these models were developed on small data sets, and there have been few replication studies [1] – in common with other areas of the learning sciences and psychology [19, 20].”
BackgroundArtificial Intelligence in Special Education
Harkins-Brown, A. R., Carling, L. Z., & Peloff, D. C. (2025) · Encyclopedia · doi:10.3390/encyclopedia5010011 · Cited by 41
“At its core, replication confirms that the results from any individual study are reproducible and not due merely to chance—a central tenet of trustworthy scientific results [34–36].”
Nelson, G., Carter, H., Boedeker, P., Knowles, E., Buckmiller, C. W., & Eames, J. (2023) · Review of Educational Research · doi:10.3102/00346543231156182 · Cited by 52
“Replication is an essential component of the research cycle, especially when research in this field is still emerging (Makel & Plucker, 2014).”
BackgroundA Causal Replication Framework for Designing and Assessing Replication Efforts
Steiner, P. M., Wong, V. C., & Anglin, K. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000385 · Cited by 47
“Combined, these results contribute to a growing sense of a "replication crisis" occurring in multiple domains of science, including education (Makel & Plucker, 2014), prevention science (Valentine et al., 2011), and economics (Duvendack, Palmer-Jones, & Reed, 2017).”
BackgroundPreventive healthcare policies in the US: solutions for disease management using Big Data Analytics
Batarseh, F. A., Ghassib, I. H., Chong, D., & Su, P.-H. (2020) · Journal Of Big Data · doi:10.1186/s40537-020-00315-8 · Cited by 44
Nielsen, J. D., Thompson, J. A., Wadsworth, L. L., & Vallett, J. D. (2020) · Journal of Organizational Behavior · doi:10.1002/job.2469 · Cited by 45
Evaluating Article Search and Selection Procedures in Special Education Literature Reviews
King, S. A., Davidson, K. A., Chitiyo, A., & Apple, D. (2018) · Remedial and Special Education · doi:10.1177/0741932518813142 · Cited by 43
“We used the WOS categories as the basis for inclusion due to their prominence and use in previous systematic reviews of education literature (e.g., Makel & Plucker, 2014).”
MethodMaking Every Study Count: Learning From Replication Failure to Improve Intervention Research
Kim, J. S. (2019) · Educational Researcher · doi:10.3102/0013189x19891428 · Cited by 50
“Direct replications that duplicate the methods of an original study are both rare and difficult to implement in the education sciences because the district, school, and classroom context for experimentation changes over time (Makel & Plucker, 2014; Nosek et al., 2015; Van Bavel et al., 2016).”
MethodA Brief Summary of Teacher Recruitment and Retention in the Smallest Illinois Rural Schools
Ulferts, J. D. (2018) · The rural educator · doi:10.35608/ruraled.v37i1.292 · Cited by 38
Studying MOOC completion at scale using the MOOC replication framework
Andres, J. M. L., Baker, R. S., Gašević, D., Siemens, G., Crossley, S. A., & Joksimovíc, S. (2018) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3170358.3170369 · Cited by 38
“within MOOCs has led to several contradictory findings of this nature, duplicating the “crisis of replication” seen in the social psychology community [10].”
BackgroundBecker, S., Klein, P., Gößling, A., & Kühn, J. (2020) · Zeitschrift für Didaktik der Naturwissenschaften · doi:10.1007/s40573-020-00116-9 · Cited by 43
Vanlaar, G., Kyriakides, L. E., Panayiotou, A., Vandecandelaere, M., McMahon, L., Fraine, B. D., & Damme, J. V. (2015) · Research Papers in Education · doi:10.1080/02671522.2015.1027724 · Cited by 40
The Trouble With Binaries: A Perspective on the Science of Reading
Yaden, D. B., Reinking, D., & Smagorinsky, P. (2021) · Reading Research Quarterly · doi:10.1002/rrq.402 · Cited by 53
“In a recent analysis of 100 top- tier, peer- reviewed research journals in education, including journals reporting reading research, Makel and Plucker (2014) found that only 13 published studies in 10,000 were replications and that only six journals had published between one and five in 100.”
BackgroundCreativity is more than novelty: Reconsidering replication as a creativity act.
Makel, M. C., & Plucker, J. (2014) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/a0035811 · Cited by 35
Self-citationBaker, D. L., Santoro, L. E., Biancarosa, G., Baker, S. K., Fien, H., & Otterstedt, J. (2020) · Reading and Writing · doi:10.1007/s11145-020-10060-2 · Cited by 76
An editorial on replication studies and scaling up efforts
Spector, J. M., Johnson, T. E., & Young, P. A. (2014) · Educational Technology Research and Development · doi:10.1007/s11423-014-9364-3 · Cited by 37
“Moreover, replication studies can identify potential biases in the original study and serve as a basis for confirming or disconfirming prior findings (Makel and Plucker 2014).”
InfluentialSantagata, R., & Lee, J. (2019) · Journal of Mathematics Teacher Education · doi:10.1007/s10857-019-09447-y · Cited by 39
Experimental Estimates of the Student Attendance Production Function
Tran, L., & Gershenson, S. (2018) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373720984463 · Cited by 45
“Replicating these findings in other contexts and with experimental data is important, as replication is a key element of the scientific method that is often missing in education research (Duncan et al., 2014; Makel & Plucker, 2014).”
MethodCreative self-efficacy and teacher ratings of student creativity in Chinese elementary classrooms
Hartley, K. A., Plucker, J. A., & Long, H. (2016) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2016.10.001 · Cited by 38
Self-citationCarolan, B. V. (2015) · British Journal of Sociology of Education · doi:10.1080/01425692.2015.1013085 · Cited by 35
“These measures and their use in a recursive path model also highlight the importance of replication studies that use different measurement and modeling specifications in order to better and more accurately assess the utility of theoretical propositions (Makel and Plucker 2014; Schmidt 2009).”
MethodLancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Alberti, G., Chiariello, V., & Carrella, L. (2020) · Developmental Neurorehabilitation · doi:10.1080/17518423.2020.1737590 · Cited by 41
Pedersen, M. J., & Stritch, J. M. (2018) · Public Administration Review · doi:10.1111/puar.12910 · Cited by 36
Replication Studies in Mathematics Education: What Kind of Questions Would Be Productive to Explore?
Aguilar, M. S. (2020) · International Journal of Science and Mathematics Education · doi:10.1007/s10763-020-10069-7 · Cited by 40
“Makel and Plucker (2014) analyzed the complete publication history of the top 100 education journals—according to the 5-year impact factor issued by Journal Citations Reports—and found that only 0.13% of education articles were replications.”
BackgroundInfluentialThe Effect of Replications on Citation Patterns: Evidence From a Large-Scale Reproducibility Project
Schafmeister, F. (2021) · Psychological Science · doi:10.1177/09567976211005767 · Cited by 35
“Makel and Plucker (2014), Makel et al. (2016), Evanschitzky et al. (2007), and Hubbard and Armstrong (1994) present similar results for educational sciences, special education, and marketing.”
ResultEvidence-based education: Objections and future directions
Dekker, I., & Meeter, M. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.941410 · Cited by 57
“13% of studies of the articles in leading educational journals (Makel and Plucker, 2014)], would further consolidate the reliability of the findings.”
BackgroundReplication is relevant to qualitative research
Makel, M. C., Meyer, M. S., Simonsen, M. A., Roberts, A. M., & Plucker, J. A. (2022) · Educational Research and Evaluation · doi:10.1080/13803611.2021.2022310 · Cited by 38
“This comes on the heels of prominent instances of data fabrication (National Academies of Sciences, Engineering, and Medicine, 2017) and estimates that few studies attempt to replicate previous findings (Makel & Plucker, 2014).”
BackgroundSelf-citationThe empirical march: Making science better at self-correction.
Makel, M. C. (2014) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/a0035803 · Cited by 35
Self-citationPolicy Development in Physical Education … The Last Best Chance?
van der Mars, H. (2018) · Quest · doi:10.1080/00336297.2018.1439391 · Cited by 33
“Makel and Plucker (2014) found that of the research published in the top 100 education research journals, only 0.13% of published research constituted replications.”
BackgroundInfluentialA Call for Open Science in Giftedness Research
McBee, M. T., Makel, M. C., Peters, S. J., & Matthews, M. S. (2018) · Gifted Child Quarterly · doi:10.1177/0016986218784178 · Cited by 36
“As Makel and Plucker (2014) argued, facts are more important than novelty.”
BackgroundInfluentialSelf-citationImplications of the open science era for educational psychology research syntheses
Patall, E. A. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1897009 · Cited by 37
“After reviewing the publication history of the top 100 education journals by impact factor, Makel and Plucker (2014) found that a mere 0.13% of articles were replication attempts.”
BackgroundInfluentialUsing Smartphones to Help People with Intellectual and Sensory Disabilities Perform Daily Activities
Lancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Alberti, G., Zimbaro, C., & Chiariello, V. (2017) · Frontiers in Public Health · doi:10.3389/fpubh.2017.00282 · Cited by 33
An Introduction to Replication Research in Gifted Education
Makel, M. C., & Plucker, J. A. (2015) · Gifted Child Quarterly · doi:10.1177/0016986215578747 · Cited by 31
“Moreover, replications get cited at relatively high rates (Makel & Plucker, 2014b; Makel, Plucker, & Hegarty, 2012).”
BackgroundResultInfluentialSelf-citationSpillane, J. P., & Shirrell, M. (2017) · Educational Administration Quarterly · doi:10.1177/0013161x17696557 · Cited by 32
“Replication is essential if we are to amass an empirical body of knowledge to inform educational policy and practice (Makel & Plucker, 2014).”
BackgroundWhat Values and Traits Do Parents Teach to Their Children? New Data from 54 Countries
Minkov, M., Dutt, P., Schachner, M., Jandosova, J., Khassenbekov, Y., Morales, O., Sanchez, C. J., & Mudd, B. (2018) · Comparative Sociology · doi:10.1163/15691330-12341456 · Cited by 37
The Use of Concrete Examples Enhances the Learning of Abstract Concepts: A Replication Study
Micallef, A., & Newton, P. M. (2022) · Teaching of Psychology · doi:10.1177/00986283211058069 · Cited by 31
“Replication of results from key studies in education research is essential to con fi rm, or not, the generalisability of fi ndings (Simons, 2014), perhaps even more important than the generation of new fi ndings (Makel & Plucker, 2014).”
Foster, M. E. (2023) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2023.2174919 · Cited by 37
“Given the replication crisis in psychological science (see Pashler & Wagenmakers, 2012) and the lack of replication studies in education research (Makel & Plucker, 2014), there is a need to replicate intervention studies to determine what works for whom, under what conditions (Schneider, 2018,…”
BackgroundMethodProfessional Development in the Use of Data: From Data to Knowledge in Data Teams
Schildkamp, K., Smit, M., & Blossing, U. (2019) · Scandinavian Journal of Educational Research · doi:10.1080/00313831.2017.1376350 · Cited by 31
“Replication studies are often viewed as lacking originality, prestige, and excitementand are not seen as contributing very much to the field (Makel & Plucker, 2014).”
BackgroundMethodResultInfluentialLancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., Alberti, G., Perilli, V., Chiariello, V., & Buono, S. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00234 · Cited by 31
Effective strategies to attract crowdfunding investment based on the novelty of business ideas
Jung, E., Lee, C., & Hwang, J. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.121558 · Cited by 31
Issues of Teacher Performance Stability are Not New: Limitations and Possibilities
Good, T. L., & Lavigne, A. L. (2015) · Education Policy Analysis Archives · doi:10.14507/epaa.v23.1916 · Cited by 31
“The argument for replication has been made exceedingly well by Makel and Plucker (2014).”
BackgroundMalouf, D. B., & Taymans, J. M. (2016) · Educational Researcher · doi:10.3102/0013189x16678417 · Cited by 34
“Randomized designs have relative advantages in causal validity, but they also have limitations (Makel & Plucker, 2014), and the costs and difficulties of conducting experimental research in education settings have long been recognized (National Research Council, 2002).”
BackgroundHart, S. C., DiPerna, J. C., Lei, P., & Cheng, W. (2020) · Educational Researcher · doi:10.3102/0013189x19898721 · Cited by 31
“The current study reports findings from one sample and one program, and results require replication (e.g., Makel & Plucker, 2014).”
BackgroundLongitudinal Research on Music Education and Child Development: Contributions and Challenges
Ilari, B. S. (2020) · Music & Science · doi:10.1177/2059204320937224 · Cited by 48
“Some have even gone as far as to state that Randomized Con- trol Trials (or RCTSs) are the gold standard of education studies (see Makel & Plucker, 2014).”
BackgroundTaylor, J. A., Kowalski, S. M., Polanin, J. R., Askinas, K., Stuhlsatz, M. A. M., Wilson, C. D., Tipton, E., & Wilson, S. J. (2018) · AERA Open · doi:10.1177/2332858418791991 · Cited by 32
“From this result, it follows naturally that replications of rigorous impact studies are also rare (Makel & Plucker, 2014; Taylor et al., 2016).”
BackgroundTherrien, W. J., Mathews, H. M., Hirsch, S. E., & Solis, M. R. (2016) · Remedial and Special Education · doi:10.1177/0741932516646081 · Cited by 28
Ricci, C., Miglino, O., Alberti, G., Perilli, V., & Lancioni, G. E. (2017) · International Journal of Developmental Disabilities · doi:10.1080/20473869.2017.1288888 · Cited by 29
“New research needs to verify the reliability of these findings with additional participants and address the aforementioned limitations of the present study (Callahan et al. 2008, Barlow et al. 2009, Kazdin 2011, Lamontagne et al. 2013, Makel and Plucker 2014).”
BackgroundHensel, P. G. (2018) · European Management Journal · doi:10.1016/j.emj.2018.03.004 · Cited by 29
Continuous-time deconvolutional regression for psycholinguistic modeling
Shain, C., & Schuler, W. (2021) · Cognition · doi:10.1016/j.cognition.2021.104735 · Cited by 28
Reliability analysis of the objective structured clinical examination using generalizability theory
Trejo-Mejía, J. A., Mendiola, M. S., Ramírez, I. M., & Martínez-González, A. (2016) · Medical Education Online · doi:10.3402/meo.v21.31650 · Cited by 28
“The reproducibility problem has been studied in education research (30), where only 0.”
BackgroundSearching for Evidence-Based Practice
Missett, T. C., & Foster, L. H. (2015) · Journal of Advanced Academics · doi:10.1177/1932202x15577206 · Cited by 27
“This will also facilitate the implementation of replication studies to further deepen the understanding of evidence-based practices for gifted learners (Makel & Plucker, 2014).”
BackgroundSmart but Evil? Student-Teachers’ Perception of Educational Researchers’ Epistemic Trustworthiness
Merk, S., & Rosman, T. (2019) · AERA Open · doi:10.1177/2332858419868158 · Cited by 28
“In fact, preregistration has been shown to lower the likelihood of false-positive findings (Nelson, Simmons, & Simonsohn, 2018), with calls for replications becoming more pronounced in recent years (Makel & Plucker, 2014).”
BackgroundThe Curious Case of Centrality Measures: A Large-Scale Empirical Investigation
Saqr, M., & López‐Pernas, S. (2022) · Journal of Learning Analytics · doi:10.18608/jla.2022.7415 · Cited by 33
“Makel and Plucker (2014) estimated that only 0.13% of educational research articles were replications of other studies, concluding that “despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not on the merit of…”
BackgroundInfluentialFischman, G. E., Anderson, K. T., Tefera, A., & Zuiker, S. J. (2018) · AERA Open · doi:10.1177/2332858417750133 · Cited by 39
“…the structural challenges highlighted earlier is the well-documented and growing public indifference to expert judgment in education research (Makel & Plucker, 2014; Miller, 2013).3 In part to combat the structural changes and seeming loss of public faith in education research, a central…”
BackgroundKeuning, T., Geel, M., Visscher, A., & Fox, J. (2019) · Journal of Educational Measurement · doi:10.1111/jedm.12236 · Cited by 25
Hyndman, B., & Harvey, S. (2019) · The Australian journal of teacher education · doi:10.14221/ajte.2018v44n2.3 · Cited by 26
“Further replicative work with a larger, more diverse sample can enhance the generalisability of the findings to other settings (Hill, 1998; Makel & Plucker, 2014).”
BackgroundHow accurately do teachers’ judge students? Re-analysis of Hoge and Coladarci (1989) meta-analysis
Kaufmann, E. (2020) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2020.101902 · Cited by 33
Muradchanian, J., Hoekstra, R., Kiers, H. A. L., & van Ravenzwaaij, D. (2021) · Royal Society Open Science · doi:10.1098/rsos.201697 · Cited by 38
Replicating MOOC predictive models at scale
Gardner, J., Brooks, C. A., Andres, J. M., & Baker, R. S. (2018) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3231644.3231656 · Cited by 26
“The learning sciences field has also been criticized for poor efforts toward replicability [30].”
BackgroundMethodInfluentialRisk Preference Theory and Gender Differences in Religiousness: A Replication and Extension
Hoffmann, J. P. (2018) · Journal for the Scientific Study of Religion · doi:10.1111/jssr.12578 · Cited by 25
Pieschl, S., & Sivyer, D. (2020) · Computers & Education · doi:10.1016/j.compedu.2020.104038 · Cited by 27
Rasooli, A., Zandi, H., & DeLuca, C. (2022) · School Psychology Review · doi:10.1080/2372966x.2021.2000843 · Cited by 28
“Given the concern over a replication crisis (Makel & Plucker, 2014), the transparent report of the outlined recommendations in future studies on fairness or justice in classrooms can enhance the replication and reproducibility of the findings and add to the explanatory power of predictive validity…”
Reproducibility in Computing Research: An Empirical Study
Raghupathi, W., Raghupathi, V., & Ren, J. (2022) · IEEE Access · doi:10.1109/access.2022.3158675 · Cited by 33
“The fields of psychology [18], biology [19], [20], biomedicine [9], neuroscience [21], drug development [22], chemistry [23], climate science [24], economics [25] and education [26] among others, have reported reproducibility problems [20].”
BackgroundQuantifying “Promising Trials Bias” in Randomized Controlled Trials in Education
Sims, S., Anders, J., Inglis, M., & Lortie‐Forgues, H. (2022) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2090470 · Cited by 30
“Then we set out our empirical approach to estimating the probability and severity of erroneous findings, paying particular attention to our use and selection of assumed effect sizes.”
Cobey, K. D., Fehlmann, C. A., Franco, M. C., Ayala, A. P., Sikora, L., Rice, D. B., Xu, C., Ioannidis, J. P. A., Lalu, M. M., Ménard, A., et al. (2023) · eLife · doi:10.7554/elife.78518 · Cited by 26
Data collection and measurement assessment in behavioral research: 1958–2013.
Kostewicz, D. E., King, S. A., Datchuk, S. M., Brennan, K. M., & Casey, S. D. (2016) · Behavior Analysis Research and Practice · doi:10.1037/bar0000031 · Cited by 25
Why We Blog: Homeschooling Mothers of Gifted Children
Jolly, J. L., & Matthews, M. S. (2017) · Roeper Review · doi:10.1080/02783193.2017.1289579 · Cited by 25
“The rapid turnover in blog publishing is another potential cause for concern; in light of the growing emphasis on replication of research (e.g., Makel & Plucker, 2014) it seems that there is ample room for additional work on this topic, yet future studymay be sampling a completely different…”
BackgroundLancioni, G. E., Singh, N. N., O’Reilly, M. F., Sigafoos, J., D’Amico, F., Laporta, D., Cattaneo, M. G., Scordamaglia, A., & Pinto, K. (2018) · American Journal of Alzheimer s Disease & Other Dementias® · doi:10.1177/1533317518775038 · Cited by 25
A Systematic Review of Eye-Tracking Technology in Second Language Research
Hu, X., & Aryadoust, V. (2024) · Languages · doi:10.3390/languages9040141 · Cited by 30
“Researchers can opt to duplicate the experimental procedure of the original study (direct replication), change specific facets of the methodology (partial replication) or adopt different methods (conceptual replication) (Makel and Plucker 2014; Marsden et al. 2018).”
Hodgen, J., Taylor, B., Francis, B., Craig, N., Bretscher, N., Tereshchenko, A., Connolly, P., & Mazenod, A. (2022) · British Educational Research Journal · doi:10.1002/berj.3838 · Cited by 24
“Ireson et al.’s (2005) study reports data that are now more than 20 years old and, aside from the need to replicate individual studies (Makel & Plucker, 2014), there is a need to provide up-to-date evidence about the effects of setting.”
BackgroundReplication and the Manufacture of Scientific Inferences: A Formal Approach
García, F. M. (2016) · International Studies Perspectives · doi:10.1093/isp/ekv011 · Cited by 23
Jitendra, A. K., Harwell, M. R., Im, S., Karl, S. R., & Slater, S. C. (2019) · Journal of Educational Psychology · doi:10.1037/edu0000335 · Cited by 23
Brief Research Report: The Message Behind Teacher Emotions
Taxer, J. L., & Frenzel, A. C. (2019) · The Journal of Experimental Education · doi:10.1080/00220973.2019.1588699 · Cited by 25
“Also, in light of the replication crisis in educational research (Makel & Plucker, 2014), the aim of the present research was to reexamine the message that teacher anger versus pity sends to students following failure in a higher education learning context.”
BackgroundA Comparison of Two Flashcard Interventions for Teaching Sight Words to Early Readers
January, S. A., Lovelace, M. E., Foster, T. E., & Ardoin, S. P. (2016) · Journal of Behavioral Education · doi:10.1007/s10864-016-9263-2 · Cited by 23
Atroszko, P. A. (2018) · Psychiatria i Psychologia Kliniczna · doi:10.15557/pipk.2018.0034 · Cited by 23
“…introducing novelty rather Paweł Andrzej Atroszko 278 DOI: 10.15557/PiPK.2018.0034 © PSYCHIATR PSYCHOL KLIN 2018, 18 (3), p. 276–282 than systematic investigation of phenomena in the collaborative elaboration of theoretical and methodological problems (Makel and Plucker, 2014; Nosek et al., 2012).”
BackgroundInfluentialCoaching Parents to Use Higher Level Questioning With Their Twice-Exceptional Children
Ritchotte, J. A., & Zaghlawan, H. Y. (2019) · Gifted Child Quarterly · doi:10.1177/0016986218817042 · Cited by 23
Göhner, M. F., & Krell, M. (2020) · Zeitschrift für Didaktik der Naturwissenschaften · doi:10.1007/s40573-020-00111-0 · Cited by 25
“Like Upgrading From a Typewriter to a Computer”: Registered Reports in Education Research
Reich, J., Gehlbach, H., & Albers, C. J. (2020) · AERA Open · doi:10.1177/2332858420917640 · Cited by 22
“These issues range from a lack of replication studies to “p-hacking” to a lack of transparency in analytic decision making (Gelman & Loken, 2014; Makel & Plucker, 2014; Simmons et al., 2011).”
BackgroundOpen Science and Single-Case Design Research
Cook, B. G., Johnson, A. H., Maggin, D. M., Therrien, W. J., Barton, E. E., Lloyd, J. W., Reichow, B., Talbott, E., & Travers, J. C. (2021) · Remedial and Special Education · doi:10.1177/0741932521996452 · Cited by 23
“Evidence suggests relatively few replication studies are conducted, and when independent replications are conducted, they often fail to reproduce the results reported in the original studies (Makel et al., 2016; Makel & Plucker, 2014).”
ResultSchool equity, marketisation and access to the Australian senior secondary curriculum
Dean, J., Roberts, P., & Perry, L. B. (2021) · Educational Review · doi:10.1080/00131911.2021.1909537 · Cited by 27
Cobern, W. W., Adams, B. A., Pleasants, B. A.-S., Bentley, A., & Kagumba, R. (2022) · Science & Education · doi:10.1007/s11191-021-00292-1 · Cited by 26
Teachers trust educational science - Especially if it confirms their beliefs
Schmidt, K. A., Rosman, T., Cramer, C., Besa, K.-S., & Merk, S. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.976556 · Cited by 22
A guide for social science journal editors on easing into open science
Silverstein, P., Elman, C., Montoya, A. K., McGillivray, B. C., Pennington, C. R., Harrison, C. H., Steltenpohl, C. N., Röer, J. P., Corker, K. S., Charron, L., et al. (2024) · Research Integrity and Peer Review · doi:10.1186/s41073-023-00141-5 · Cited by 28
“…by open science, including psychology [2, 14–16], genetics [17], biomedicine [18], animal behavior [4, 19], economics [20–24], education [21, 25–29], political science [30], public health [31, 32], science and technology studies [33], scientometrics [34], and sociology [35, 36], among…”
BackgroundPredicting Faculty Integration of Faith and Learning
Kaul, C. R., Hardin, K. A., & Beaujean, A. A. (2017) · Christian Higher Education · doi:10.1080/15363759.2016.1250684 · Cited by 22
“Moreover, it is important to determine if a particular study’s results are generalizable or if they are overly influenced by idiosyncrasies in sampling, methods, or data analysis (Asendorpf et al., 2013; Makel & Plucker, 2014; Open Science Collaboration, 2015).”
BackgroundThanheiser, E. (2018) · Journal for Research in Mathematics Education · doi:10.5951/jresematheduc.49.1.0039 · Cited by 22
Metadata
- Authors
- Makel, M. C., & Plucker, J. A.
- Affiliations
- Duke University (US)
- University of Connecticut (US)
- Venue
- Educational Researcher
- Published
- 2014-08
- Volume
- 43
- Issue
- 6
- Pages
- 304-316
- ISSN
- 0013-189X (ISSN-L)
- 1935-102X
- OpenAlex
- W2139319950
- Semantic Scholar
- 493a2e93117a876871ac446b26acf47c50ea2cef
- Cited by
- 609
- References
- 131
- Citations by year
- 2026: 17 · 2025: 37 · 2024: 40 · 2023: 40 · 2022: 66 · 2021: 47 · 2020: 59 · 2019: 81 · 2018: 73 · 2017: 52 · 2016: 51 · 2015: 37 · 2014: 8
- FWCI
- 68.61
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Educational Researcher (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Education › Educational Research and Methods
- Keywords
- replication, direct replication, educational research, publishing history, research credibility, methodological quality, replication success, educational periodicals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
