The DOI matches its Crossref record; 132 of the 132 references it cites were found in a registry or an index.
Facts Are More Important Than Novelty: Replication in the Education Sciences
Abstract
Despite increased attention to methodological rigor in education research, the field has focused heavily on experimental design and not on the merit of replicating important results. The present study analyzed the complete publication history of the current top 100 education journals ranked by 5-year impact factor and found that only 0.13% of education articles were replications. Contrary to previous findings in medicine, but similar to psychology, the majority of education replications successfully replicated the original studies. However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. The results emphasize the importance of third-party, direct replications in helping education research improve its ability to shape education policy and practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Novel or innovative results are more likely to be published than replications or incremental extensions of existing findings 13-15 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [15]
Citations
Cites
2 retracted
Wakefield, A., Murch, S., Anthony, A., Linnell, J., Casson, D., Malik, M., Berelowitz, M., Dhillon, A., Thomson, M., Harvey, P., et al. (1998) · The Lancet · doi:10.1016/s0140-6736(97)11096-0 · Cited by 3,006
RetractedLancet, T. (2010) · The Lancet · doi:10.1016/s0140-6736(10)60175-4 · Cited by 500
Retracted(1963) · Medical Entomology and Zoology · Cited by 15,085
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
An Introduction to Categorical Data Analysis
Agresti, A. (2006) · Wiley series in probability and statistics · doi:10.1002/0470114754 · Cited by 6,775
Journal of social behavior and personality
(1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90025-5 · Cited by 6,751
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Moffitt, R. (2009) · American Economic Review · doi:10.1257/aer.99.2.660 · Cited by 4,888
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 5×
“Using health care research as an example, Ioannidis (2005a), in a review of highly cited medical publications (i.e., those cited more than 1,000 times), found only 44% of replications produced results similar to the original study.”
ResultInfluentialLangenberg, D. N. (2000) · Cited by 4,305
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 3×
“The concern over replication exists in many research domains, including advertising (Madden et al., 1995), biology (Begley & Ellis, 2012; Powell, 2007), economics (Anderson, Greene, McCullough, & Vinod, 2005; Dewald, Thursby, & Anderson, 1986; Kane, 1984), library sciences (Winters, 1996),…”
ResultInfluentialMisinformation and Its Correction: Continued Influence and Successful Debiasing
Lewandowsky, S., Ecker, U. K. H., Seifert, C. M., Schwarz, N., & Cook, J. (2012) · Psychological Science in the Public Interest · doi:10.1177/1529100612451018 · Cited by 2,920
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Plucker, J. A., Beghetto, R. A., & Dow, G. T. (2004) · Educational Psychologist · doi:10.1207/s15326985ep3902_1 · Cited by 1,794
Self-citationExperimenter Effects in Behavioral Research
McGuigan, F. J., & Rosenthal, R. (1968) · The American Journal of Psychology · doi:10.2307/1420816 · Cited by 1,633
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Reproducible research in computational science
Peng, R. D. (2011) · Science · doi:10.1126/science.1213847 · Cited by 1,449
Statistical significance in psychological research.
Lykken, D. T. (1968) · Psychological bulletin · doi:10.1037/h0026141 · Cited by 1,363
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248
Experimenter Effects in Behavioral Research.
Titchener, J. L. (1967) · Archives of Internal Medicine · doi:10.1001/archinte.1967.00300050109027 · Cited by 1,201
Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences
Schmidt, S. (2009) · Review of General Psychology · doi:10.1037/a0015108 · Cited by 1,125
Peer-review practices of psychological journals: The fate of published articles, submitted again
Peters, D. P., & Ceci, S. J. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00011183 · Cited by 1,079 · cited 1×
“…& Canada, 2013; Sterling, 1959; Sterling, Rosenbaum, & Weinkam, 1995), low reliability among peer reviewers (e.g., Cicchetti, 1991; Cole, 1992; D. Peters & Ceci, 1982), hypothesizing after the results are known (e.g., N. Kerr, 1998; Makel, 2014; Maxwell, 2004), misuse of statistical tests and…”
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022790 · Cited by 977
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964 · cited 4×
“…(e.g., Bakker, van Dijk, & Wicherts, 2012; Bozarth & Roberts, 1972; Fanelli, 2010, 2012; Pigott, Valentine, Polanin, Williams, & Canada, 2013; Sterling, 1959; Sterling, Rosenbaum, & Weinkam, 1995), low reliability among peer reviewers (e.g., Cicchetti, 1991; Cole, 1992; D. Peters & Ceci,…”
BackgroundInfluentialThe test of significance in psychological research.
Bakan, D. (1966) · Psychological Bulletin · doi:10.1037/h0020412 · Cited by 957
Bem, D. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0021524 · Cited by 893
The demon-haunted world: science as a candle in the dark
(1996) · Choice Reviews Online · doi:10.5860/choice.34-0277 · Cited by 825
Revised standards for statistical evidence
Johnson, V. E. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1313476110 · Cited by 808
The rules of the game called psychological science
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459060 · Cited by 800
The Replication Recipe: What makes for a convincing replication?
Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner‐Sorolla, R., Grange, J. A., Perugini, M., Spies, J. R., & Veer, A. V. '. (2013) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2013.10.005 · Cited by 758
Autism and measles, mumps, and rubella vaccine: no epidemiological evidence for a causal association
Taylor, B., Miller, E., Farrington, C., Petropoulos, M.-C., Favot-Mayaud, I., Li, J., & Waight, P. A. (1999) · The Lancet · doi:10.1016/s0140-6736(99)01239-8 · Cited by 753
Replications in Psychology Research: How Often Do They Really Occur?
Makel, M. C., Plucker, J. A., & Hegarty, B. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612460688 · Cited by 750
Self-citationThe fallacy of the null-hypothesis significance test.
Rozeboom, W. W. (1960) · Psychological Bulletin · doi:10.1037/h0042040 · Cited by 740
“Positive” results increase down the Hierarchy of the Sciences
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010068 · Cited by 734
Is the Replicability Crisis Overblown? Three Arguments Examined
Pashler, H., & Harris, C. R. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463401 · Cited by 697
Maxwell, S. E. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.2.147 · Cited by 655
An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science
Nosek, B. A., & Lakens, D. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612462588 · Cited by 592
Why Science Is Not Necessarily Self-Correcting
Ioannidis, J. P. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612464056 · Cited by 563
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.2307/2282137 · Cited by 524
Cicchetti, D. V. (1991) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00065675 · Cited by 496
The Design of Replicated Studies
Lindsay, R. M., & Ehrenberg, A. S. C. (1993) · The American Statistician · doi:10.1080/00031305.1993.10475983 · Cited by 491
Replication in Empirical Economics: The Journal of Money, Credit and Banking Project
Dewald, W. G., Thursby, J. G., & Anderson, R. G. (2016) · American Economic Review · Cited by 477
King, G. (1995) · PS Political Science & Politics · doi:10.1017/s1049096500057607 · Cited by 473
An Introduction to Categorical Data Analysis
Bilder, C. R., & Tebbs, J. M. (2008) · Journal of the American Statistical Association · doi:10.1198/jasa.2008.s251 · Cited by 458
The ironic effect of significant results on the credibility of multiple-study articles.
Schimmack, U. (2012) · Psychological Methods · doi:10.1037/a0029487 · Cited by 455
Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995) · The American Statistician · doi:10.1080/00031305.1995.10476125 · Cited by 453
Replications and extensions in marketing: Rarely published but quite contrary
Hubbard, R., & Armstrong, J. S. (1994) · International Journal of Research in Marketing · doi:10.1016/0167-8116(94)90003-5 · Cited by 417
Financial Incentives and Student Achievement: Evidence from Randomized Trials *
Fryer, R. G. (2011) · The Quarterly Journal of Economics · doi:10.1093/qje/qjr045 · Cited by 414
Yong, E. (2012) · Nature · doi:10.1038/485298a · Cited by 406
Making science: between nature and society
(1993) · Choice Reviews Online · doi:10.5860/choice.30-3240 · Cited by 359
The (mis)reporting of statistical results in psychology journals
Bakker, M., & Wicherts, J. M. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0089-5 · Cited by 356
Unpublished results hide the decline effect
Schooler, J. W. (2011) · Nature · doi:10.1038/470437a · Cited by 335
Science publishing: The trouble with retractions
Van Noorden, R. (2011) · Nature · doi:10.1038/478026a · Cited by 328
Analyzing data: Sanctification or detective work?
Tukey, J. W. (1969) · American Psychologist · doi:10.1037/h0027108 · Cited by 321
Priming, Replication, and the Hardest Science
Cesario, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613513470 · Cited by 320
Eden, D. (2002) · Academy of Management Journal · doi:10.5465/amj.2002.7718946 · Cited by 318
Replication research's disturbing trend
Evanschitzky, H., Baumgarth, C., Hubbard, R., & Armstrong, J. S. (2007) · Journal of Business Research · doi:10.1016/j.jbusres.2006.12.003 · Cited by 271
Eich, E. (2014) · Psychology Science · doi:10.1177/0956797613512465 · Cited by 267
Improving Validation Practices in “Omics” Research
Ioannidis, J., & Khoury, M. (2011) · Science · doi:10.1126/science.1211811 · Cited by 251
Jasny, B. R., Chin, G. J., Chong, L. D., & VIGNIERI, S. N. (2011) · Science · doi:10.1126/science.334.6060.1225 · Cited by 248
Manuscript Characteristics Which Influence Acceptance for Management and Social Science Journals
Kerr, S., Tolliver, J. T., & Petree, D. (1977) · Academy of Management Journal · doi:10.2307/255467 · Cited by 242
Teacher Incentives and Student Achievement: Evidence from New York City Public Schools
Fryer, R. G. (2013) · Journal of Labor Economics · doi:10.1086/667757 · Cited by 236
Teacher Incentives and Student Achievement: Evidence from New York City Public Schools
Fryer, R. G. (2011) · National Bureau of Economic Research · doi:10.3386/w16850 · Cited by 233
Hubbard, R., & Vetter, D. E. (1996) · Journal of Business Research · doi:10.1016/0148-2963(95)00084-4 · Cited by 230
Frank, M. C., & Saxe, R. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612460686 · Cited by 175
Sterling, T. D., Rosenbaum, W. L., & Jj, W. (1995) · The American Statistician · doi:10.2307/2684823 · Cited by 168
Replication in prevention science
Valentine, J. C., Biglan, A., Boruch, R. F., Castro, F. G., Collins, L. M., Flay, B. R., Kellam, S., Mościcki, E. K., & Schinke, S. P. (2011) · Prevention Science · doi:10.1007/s11121-011-0217-6 · Cited by 166
Signifying significant significance.
Bozarth, J. D., & Roberts, R. R. (1972) · American Psychologist · doi:10.1037/h0038034 · Cited by 164
LeBel, E. P., & Peters, K. R. (2011) · Review of General Psychology · doi:10.1037/a0025172 · Cited by 163
Lack of Association between Measles Virus Vaccine and Autism with Enteropathy: A Case-Control Study
Hornig, M., Briese, T., Buie, T. M., Bauman, M. L., Lauwers, G. Y., Siemetzki, U., Hummel, K., Rota, P. A., Bellini, W. J., O’Leary, J. J., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003140 · Cited by 150
King, G. (1995) · PS Political Science & Politics · doi:10.2307/420301 · Cited by 145
Editorial bias against replication research.
Neuliep, J. W., & Crandall, R. (1990) · Journal of social behavior and personality · Cited by 139
Outcome-Reporting Bias in Education Research
Pigott, T. D., Valentine, J. C., Polanin, J. R., Williams, R. T., & Canada, D. D. (2013) · Educational Researcher · doi:10.3102/0013189x13507104 · Cited by 127
How Journal Editors View Replication Research
Madden, C. S., Easley, R. W., & Dunn, M. G. (1995) · Journal of Advertising · doi:10.1080/00913367.1995.10673490 · Cited by 126
Concepts of Relative Importance in Recent Scientific Literature
Kruskal, W., & Majors, R. (1989) · The American Statistician · doi:10.1080/00031305.1989.10475596 · Cited by 115
Covert communication in the psychological experiment
Rosenthal, R. (1968) · PsycEXTRA Dataset · doi:10.1037/e535742008-002 · Cited by 114
Covert communication in the psychological experiment.
Rosenthal, R. (1967) · Psychological Bulletin · doi:10.1037/h0024529 · Cited by 113
Reviewer bias against replication research.
Neuliep, J. W., & Crandall, R. (1993) · Journal of social behavior and personality · Cited by 113
Tracking Replicability as a Method of Post-Publication Open Evaluation
Hartshorne, J. K., & Schachner, A. (2012) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2012.00008 · Cited by 107
Furchtgott, E. (1984) · American Psychologist · doi:10.1037/0003-066x.39.11.1315.b · Cited by 106
Incentive Pay Programs Do Not Affect Teacher Motivation or Reported Practices
Yuan, K., Le, V., McCaffrey, D. F., Marsh, J. A., Hamilton, L. S., Stecher, B. M., & Springer, M. G. (2012) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373712462625 · Cited by 102
Carpenter, S. (2012) · Science · doi:10.1126/science.335.6076.1558 · Cited by 101
Psychology’s Woes and a Partial Cure: The Value of Replication
Roediger, H. L., III (2012) · APS observer · Cited by 101
Reproducibility: The risks of the replication drive
Bissell, M. (2013) · Nature · doi:10.1038/503333a · Cited by 99
(1956) · New England Journal of Medicine · doi:10.1056/nejm195611012551809 · Cited by 96
Methodological Challenges in the Study of Primate Cognition
Tomasello, M., & Call, J. (2011) · Science · doi:10.1126/science.1213443 · Cited by 93
Crocker, J., & Cooper, M. L. (2011) · Science · doi:10.1126/science.1216775 · Cited by 86
Why Selective Publication of Statistically Significant Results Can Be Effective
de Winter, J., & Happee, R. (2013) · PLoS ONE · doi:10.1371/journal.pone.0066463 · Cited by 69
The Institute of Education Sciences: New Wine, New Bottles.
Whitehurst, G. J. (2003) · Cited by 68
Making Science: Between Nature and Society.
Fuchs, S., & Cole, S. R. (1994) · Social Forces · doi:10.2307/2579806 · Cited by 68
Harnessing the Undiscovered Resource of Student Research Projects
Grahe, J. E., Reifman, A. S., Hermann, A. D., Walker, M., Oleson, K. C., Nario‐Redmond, M. R., & Wiebe, R. P. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459057 · Cited by 65
REPLICATION IN EXPERIMENTAL COMMUNICATION RESEARCH: AN ANALYSIS
Kelly, C. W., Chase, L. J., & Tucker, R. K. (1979) · Human Communication Research · doi:10.1111/j.1468-2958.1979.tb00646.x · Cited by 59
Introduction to the Special Section: Data, Data, Everywhere . . . Especially in My File Drawer
Spellman, B. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691611432124 · Cited by 58
Replication as a Verification Technique in Survey Research: A Paradigm
Sorte, M. A. L. (1972) · Sociological Quarterly · doi:10.1111/j.1533-8525.1972.tb00805.x · Cited by 43
Replication in Field Biology: The Case of the Frog-Eating Bat
Ryan, M. J. (2011) · Science · doi:10.1126/science.1214532 · Cited by 43
An Investigation of the Prevalence of Replication Research in Human Factors
Jones, K. S., Derby, P. L., & Schmidlin, E. A. (2010) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810384394 · Cited by 41
Publication practices and the file drawer problem: A survey of published authors.
Rotton, J., Foos, P. W., Meek, L. V., & Levitt, M. J. (1995) · Journal of social behavior and personality · Cited by 41
Concepts of Relative Importance in Recent Scientific Literature
Kruskal, W., & Majors, R. (1989) · The American Statistician · doi:10.2307/2685157 · Cited by 36
Creativity is more than novelty: Reconsidering replication as a creativity act.
Makel, M. C., & Plucker, J. (2014) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/a0035811 · Cited by 35
Self-citationThe empirical march: Making science better at self-correction.
Makel, M. C. (2014) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/a0035803 · Cited by 35
Self-citationReplication, replication, replication
Ritchie, S. J., Wiseman, R. J., & French, C. C. (2012) · University of Hertfordshire Research Archive (University of Hertfordshire) · Cited by 33
Disproportionate referrals for child academic/behavior problems: Replication and extension.
Tarnowski, K. J., Anderson, D. F., Drabman, R. S., & Kelly, P. A. (1990) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.58.2.240 · Cited by 29
Replicability as a publication criterion.
Lubin, A. (1957) · American Psychologist · doi:10.1037/h0039746 · Cited by 24
Powell, K. (2007) · PLoS Biology · doi:10.1371/journal.pbio.0050338 · Cited by 18
Replication and Non-Quantitative Research
Golden, M. A. (1995) · PS Political Science & Politics · doi:10.2307/420313 · Cited by 18
The Role of Data & Program Code Archives in the Future of Economic Research
Anderson, R. G., Vinod, H. D., Greene, W. H., & McCullough, B. D. (2005) · doi:10.20955/wp.2005.014 · Cited by 17
On not so replicated experiments and not so null results.
Rosenthal, R. (1969) · Journal of Consulting and Clinical Psychology · doi:10.1037/h0027231 · Cited by 17
Social desirability in children: An extension and replication.
Klein, E. B., Gould, L. J., & Corey, M. (1969) · Journal of Consulting and Clinical Psychology · doi:10.1037/h0027339 · Cited by 15
Hansen, W. B. (2011) · Prevention Science · doi:10.1007/s11121-011-0218-5 · Cited by 12
Welcoming Quality in Non-Significance and Replication Work, but Moving Beyond the p -Value
McBee, M. T., & Matthews, M. S. (2014) · Journal of Advanced Academics · doi:10.1177/1932202x14532177 · Cited by 12
The Role of Data and Program Code Archives in the Future of Economic Research
Anderson, R. G., Greene, W. H., & McCullough, B. D. (2005) · Cited by 12
Reflections on replications and the experimenter bias effect.
Levy, L. H. (1969) · Journal of Consulting and Clinical Psychology · doi:10.1037/h0027230 · Cited by 10
A modest proposal for encouraging replication.
Ahlgren, A. (1969) · American Psychologist · doi:10.1037/h0037798 · Cited by 10
Loevinger, J. (1968) · American Psychologist · doi:10.1037/h0020800 · Cited by 9
Change starts with journal editors: In response to Makel (2014).
McBee, M. T., & Matthews, M. S. (2014) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/a0035801 · Cited by 8
Pizzello, S. (2023) · University Press of Mississippi eBooks · doi:10.2307/jj.7193915.8 · Cited by 8
Repeating fundamental experiments.
Schlosberg, H. (1951) · American Psychologist · doi:10.1037/h0056148 · Cited by 7
Ostrom, T. H. (1971) · American Psychologist · doi:10.1037/h0020338 · Cited by 4
Contradictions in Highly Cited Clinical Research—Reply
(2005) · JAMA · doi:10.1001/jama.294.21.2696-a · Cited by 3
An Example of Replication of an Experiment for Increased Reliability
Peters, C. C. (1938) · The Journal of Educational Research · doi:10.1080/00220671.1938.10880802 · Cited by 2
Publishing Replications: OK, Let's Try It
Meier, K. J. (1995) · PS Political Science & Politics · doi:10.2307/420515 · Cited by 2
But i don't have anything to write about
Winters, B. A. (1996) · Library Acquisitions Practice & Theory · doi:10.1016/s0364-6408(96)00068-3 · Cited by 2
Commentaries on Replication in Prevention Science: A Rejoinder
Valentine, J. C., Biglan, A., Boruch, R. F., Castro, F. G., Collins, L. M., Flay, B. R., Kellam, S. G., Mościcki, E. K., & Schinke, S. P. (2011) · Prevention Science · doi:10.1007/s11121-011-0220-y · Cited by 2
Replication Initiative: Prioritize Publication
Williams, S. N. (2012) · Science · doi:10.1126/science.336.6083.801-c · Cited by 2
Research works printed but not linked: 14
- 10.1037//0003–066x.49.12.997 · DOI · Paper page
- Collins H. M., 1985, Changing order
- Kane E. J., 1984, Quarterly Journal of Business and Economics, 23, 3
- Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 1
- Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 21
- Perspectives on Psychological Science. (2012). Special section on replicability in psychological science: A crisis of confidence? Retrieved from http://pps.sagepub.com/content/7/6.toc · DOI · Paper page
- Powell K., 2007, PLoS Biology, 12, 2748
- Rhine J. B., 1997, Extra-sensory perception
- Ritchie S. J., 2012, Psychologist, 25, 346
- 10.1037//0033–2909.86.3.638 · DOI · Paper page
- Rotton J., 1995, Journal of Social Behavior and Personality, 10, 1
- Simons D. J., 2012, Psychologist, 25, 349
- Simons D. J., 2014, Perspectives on Psychological Science. APS Observer, 27, 16
- Williams S. N., 2012, Science, 336, 801
Cited by
Sorted by citations
A systematic review of learning computational thinking through Scratch in K-9
Zhang, L., & Nouri, J. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.103607 · Cited by 538
Steenbergen-Hu, S., Makel, M. C., & Olszewski-Kubilius, P. (2016) · Review of Educational Research · doi:10.3102/0034654316675417 · Cited by 429
Self-citationAssessment rubrics: towards clearer and more replicable design, research and practice
Dawson, P. (2017) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2015.1111294 · Cited by 368
“Compared to other disciplines, educational researchers publish few replication studies (Makel and Plucker 2014).”
BackgroundHow school leadership influences student learning: A test of “The four paths model”
Leithwood, K., Sun, J., & Schumacker, R. (2020) · Educational Administration Quarterly · doi:10.1177/0013161x19878772 · Cited by 329
“DISCUSSION AND CONCLUSIONS Replication studies are relatively rare in the educational sciences, in general, and educational leadership studies, in particular ((Makel and Plucker, 2014).”
BackgroundChallenges for assessing replicability in preclinical cancer biology
Errington, T. M., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.67995 · Cited by 345
Metadata
- Authors
- Makel, M. C., & Plucker, J. A.
- Affiliations
- Duke University (US)
- University of Connecticut (US)
- Venue
- Educational Researcher
- Published
- 2014-08
- Volume
- 43
- Issue
- 6
- Pages
- 304-316
- ISSN
- 0013-189X (ISSN-L)
- 1935-102X
- OpenAlex
- W2139319950
- Semantic Scholar
- 493a2e93117a876871ac446b26acf47c50ea2cef
- Cited by
- 609
- References
- 131
- Citations by year
- 2026: 17 · 2025: 37 · 2024: 40 · 2023: 40 · 2022: 66 · 2021: 47 · 2020: 59 · 2019: 81 · 2018: 73 · 2017: 52 · 2016: 51 · 2015: 37 · 2014: 8
- FWCI
- 68.61
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Educational Researcher (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Education › Educational Research and Methods
- Keywords
- replication, direct replication, educational research, publishing history, research credibility, methodological quality, replication success, educational periodicals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
