Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences
The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
Replication is one of the most important tools for the verification of facts within the empirical sciences. A detailed examination of the notion of replication reveals that there are many different meanings to this concept and the relevant procedures, but hardly any systematic literature. This paper analyzes the concept of replication from a theoretical point of view. It demonstrates that the theoretical demands are scarcely met in everyday work within the social sciences. Some demands are just not feasible, whereas others are constricted by restrictions relating to publication. A new classification scheme based on a functional approach that distinguishes between different types of replication is proposed. Next, it will be argued that replication addresses the important connection between existing and new knowledge. To do so it has to be applied explicitly and systematically. The paper ends with a description of procedures how this could be done and a set of recommendations how to handle the concept of replication in the future to exploit its potential to the full.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Novel or innovative results are more likely to be published than replications or incremental extensions of existing findings 13-15 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [14]
In the citing paper
“RRs provide a mechanism for publishing replication research”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [14]
In the citing paper
“which are rarely published in the standard model 13,14 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [14]
In the citing paper
“Although sometimes replication efforts can reveal poorly designed studies and irreproducible results, this is more the exception than the rule [48].”
From The natural selection of bad science (2016) · cites this paper as [48]
Citations
Cites
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
The Logic of Scientific Discovery
(2013) · A Realist Theory of Science · doi:10.4324/9780203090732-9 · Cited by 7,731
Journal of Consciousness Studies
(2022) · Journal of Consciousness Studies · doi:10.53765/20512201 · Cited by 6,832
Journal of social behavior and personality
(1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90025-5 · Cited by 6,706
Design and Analysis: A Researcher's Handbook
Beaton, A. E., & Keppel, G. (1975) · American Educational Research Journal · doi:10.2307/1162588 · Cited by 5,822
Scientific Explanation and the Causal Structure of the World
Salmon, W. C. (1985) · Princeton University Press eBooks · doi:10.1515/9780691221489 · Cited by 2,459
Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life
Cohen, I. B., Shapin, S., & Schaffer, S. (1987) · The American Historical Review · doi:10.2307/1869945 · Cited by 2,151
Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life
Cohen, I. B., Shapin, S., & Schaffer, S. (1987) · The American Historical Review · doi:10.1086/ahr/92.3.658 · Cited by 2,151
Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life
Traynham, J. G., Shapin, S., & Schaffer, S. (1987) · Journal of Interdisciplinary History · doi:10.2307/204299 · Cited by 2,045
Design and Analysis: A Researcher's Handbook
Aaker, D. A., & Keppel, G. (1976) · Journal of Marketing Research · doi:10.2307/3150757 · Cited by 2,011
Research works printed but not linked: 12
- American Psychological Association, 2001, Publication Manual of the American Psychological Association
- Beloff J., 1985, The repeatability problem in parapsychology, 1
- Collins H. M., 1985, Changing order
- CookT. D. (1990). Rersearch methodology: Strengthening causal interpretations of nonexperimental data. In SechrestL., PerrinE. & BunkerJ., US Department of health and human services (pp. 9–30). Public Health Service, Agency for Health Care Policy and Research.
- Edge H., 1985, The repeatability problem in parapsychology, 53
- Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 21
- Rao K. R., 1981, Journal of Parapsychology, 45, 311
- Rosenthal R., 1991, Replication research in the social sciences, 1
- Sargent C. L., 1981, European Journal of Parapsychology, 3, 423
- Schacter D. L., 2000, Wir sind Erinnerung. Gedächtnis und Persönlichkeit
- Walach H., 2000, Neuroendocrinology Letters, 21, 221
- Weizsäcker E. U. v., 1986, Offene Systeme I: Beiträge zur Zeitstruktur von Information, Entropie und Evolution, 82
Cited by
Items 501–600 of 1,001
Denissen, J. J. A., & Sijtsma, K. (2022) · Personality Science · doi:10.5964/ps.9227 · Cited by 13
“Conducting a conceptual replication means systematically varying elements of the meth odology—including the used measures (e.g., LeBel et al., 2018; Schmidt, 2009).”
MethodMorgan, M. J., Schreiber, K. M., & Solomon, J. A. (2016) · Attention Perception & Psychophysics · doi:10.3758/s13414-016-1160-1 · Cited by 12
“Since our experiment was a conceptual (Schmidt, 2009) rather than an exact replication, we cannot be certain why our results are different.”
An Examination of Multiple Factors Influencing Support for Sex Offender Policy
Manchak, S. M., & Fisher, L. R. (2017) · Criminal Justice Policy Review · doi:10.1177/0887403417715684 · Cited by 13
“First, because replication is central to building an evidence base (Schmidt, 2009), this study provides an additional test of the importance of several factors previously examined in the literature.”
BackgroundBanerjee, R., Movahedazarhouligh, S., Millen, K., & Luckner, J. L. (2018) · Topics in Early Childhood Special Education · doi:10.1177/0271121417748197 · Cited by 15
“Although validity of previous studies is not directly examined through conceptual replications, they can produce results related to identifying the generalizability of prior findings regarding population, settings, and outcomes of an effective and ineffective intervention (Earp & Trafimow, 2015; Schmidt, 2009).”
BackgroundResultInfluentialBuehler, E. M., Peterson, A., & High, A. C. (2018) · Communication Studies · doi:10.1080/10510974.2018.1463275 · Cited by 12
“Our study seeks to replicate and advance the research conducted by Oh and LaRose (2016), and in so doing we address four of the reasons Schmidt (2009) articulated for conducting replication studies in the social sciences: to control for sampling error, to control for artifacts, to recognize…”
MethodDidion, L. A., Toste, J. R., & Benz, S. A. (2020) · Learning Disabilities Research and Practice · doi:10.1111/ldrp.12234 · Cited by 13
“Conceptual replications vary in one or more features from the original study (Schmidt, 2009); they support the accumulation of scientific knowledge (Francis, 2012), and can inform whether the effects of an intervention generalize when study components are altered (Brandt et al., 2014).”
BackgroundThe Design of Replication Studies
Hedges, L. V., & Schauer, J. M. (2021) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12688 · Cited by 17
“Replications intended to be the same are often referred to as direct replications (see Schmidt, 2009), and these have been prominent in replication research in the social sciences.”
BackgroundWissman, K. T., & Rawson, K. A. (2017) · Memory · doi:10.1080/09658211.2017.1350717 · Cited by 12
“…the field on the importance of independent replication of novel findings (e.g., Braver et al., 2014; Cumming, 2008; Ferguson & Heene, 2012; Francis, 2012; LeBel & Peters, 2011; Maner, 2014; Pashler & Harris, 2012; Roediger, 2012; Schimmack, 2012; Schmidt, 2009; Simmons et al., 2011; Simons, 2014).”
Feng, X., Wang, J., & Rost, D. H. (2018) · Zeitschrift für Pädagogische Psychologie · doi:10.1024/1010-0652/a000218 · Cited by 18
Kendler, K. S., Ohlsson, H., Sundquist, J., & Sundquist, K. (2019) · American Journal of Medical Genetics Part B Neuropsychiatric Genetics · doi:10.1002/ajmg.b.32720 · Cited by 13
“…and conceptual replications which try to test the same theoretical process as a prior study but importantly use at least partially different methods with recent authors suggesting that conceptual replications are ultimately of more value to the field (Crandall & Sherman, 2016; Schmidt, 2009).”
BackgroundResultDecker, A., & McGill, M. M. (2019) · Education Sciences · doi:10.3390/educsci9020069 · Cited by 13
“Replication is needed in order to determine whether the results of a study are robust or whether they were merely an anomaly [15].”
BackgroundThe Open Science Framework: Improving Science by Making It Open and Accessible
Spies, J. R. (2013) · doi:10.18130/v35c6x · Cited by 18
Measurement Issues Relevant to Questionnaire Data
Hunter, S. C., Noret, N. F., & Boyle, J. M. E. (2021) · Wiley eBooks · doi:10.1002/9781118482650.ch10 · Cited by 17
Insufficient Effort Responding in Management Research: A Critical Review and Future Directions
Huang, J. L. W., Wang, Z., Huang, R., Wu, D., & Shi, H. (2025) · Journal of Management · doi:10.1177/01492063251330268 · Cited by 16
“To the extent that the removal of IER cases can affect research results, inconsistent application of IER detection methods and cutoffs can exacerbate challenges in replicating existing research (Schmidt, 2009).”
Cohen, N. L., Becker-Hébly, I., & Štulhofer, A. (2020) · Archives of Sexual Behavior · doi:10.1007/s10508-020-01713-y · Cited by 17
“…two reasonably large independent panels, which enabled replication of findings—a practice sorely neglected in social sciences (Freese, 2007; Koole & Lakens, 2012; Schmidt, 2009)—the use of a robust analytical approach (LGCM), and a relatively lengthy period under observation (16 and 24 months).”
BackgroundThe Complexity of Comparative Judgments in Assessing Argumentative Writing: An Eye Tracking Study
Gijsen, M., van Daal, T., Lesterhuis, M., Gijbels, D., & Mæyer, S. D. (2021) · Frontiers in Education · doi:10.3389/feduc.2020.582800 · Cited by 13
“This study will conceptually replicate the findings of van Daal et al. (2017), meaning that the findings of the 2017 will be reexamined using a different set-up (Schmidt, 2009), and address both limitations.”
BackgroundMethodResultRisky choice framing with various problem descriptions: A replication and extension study
Zhou, L., Liu, N., Liao, Y.-Q., & Aimei, L. I. (2021) · Judgment and Decision Making · doi:10.1017/s1930297500008615 · Cited by 16
Wang, G. G., Doty, D. H., & Yang, S. (2021) · Advances in Developing Human Resources · doi:10.1177/15234223211054457 · Cited by 12
“Such cases would be similar to replication which differentiates belief from knowledge, and thus has the function of establishing and confirming the stability of knowledge claims (Schmidt, 2016).”
BackgroundTen simple rules for designing and conducting undergraduate replication projects
Moreau, D., & Wiebels, K. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010957 · Cited by 14
“conduct a direct replication, as these are often better suited to undergraduate research projects given that they tend to be less complex and require fewer resources [20], but see [27] for a discussion of conceptual replication.”
BackgroundInvestigation of collaborative inhibition for key‐term definitions
Wissman, K. T. (2019) · Applied Cognitive Psychology · doi:10.1002/acp.3606 · Cited by 12
Rivera, M. A., Fahey, T. D., Lopez-Taylor, J. R., & Martínez, J. L.-Q. (2020) · Sports Medicine - Open · doi:10.1186/s40798-020-00243-0 · Cited by 12
Design-Based Approaches to Causal Replication Studies
Wong, V. C., Anglin, K., & Steiner, P. M. (2021) · Prevention Science · doi:10.1007/s11121-021-01234-7 · Cited by 18
“The goal is to test and identify potential sources of effect variation based on subject matter theory, often for the purpose of generalizing effects for broader target populations (Clemens, 2017; Schmidt, 2009; Simons et al., 2017).”
BackgroundRealizing the promises of replication studies in education
Morrison, K. (2019) · Educational Research and Evaluation · doi:10.1080/13803611.2020.1838300 · Cited by 15
Ahmad, S. I., Meza, J. I., Posserud, M.-B. R., Brevik, E. J., Hinshaw, S. P., & Lundervold, A. J. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.609789 · Cited by 15
“On the other hand, the lack of precise matching of the two sample and studies with respect to variables like timing of longitudinal intervals, ages, and exact measures can be considered a strength, in terms of a variant of systematic replication (see, for example, Schmidt, 2009).”
BackgroundRegenwetter, M., Robinson, M. M., & Wang, C. (2022) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221074653 · Cited by 13
“Relatedly, for arguments on the relative merits (e.g., of direct and conceptual replication), see also Carpenter (2012), Nosek et al. (2012), Pashler and Harris (2012), Schmidt (2009), and Simons (2014).”
BackgroundVermeulen, I., Batenburg, A., Beukeboom, C. J., & Smits, T. (2014) · Social Psychology · doi:10.1027/1864-9335/a000182 · Cited by 12
What Crisis? Management Researchers’ Experiences with and Views of Scholarly Misconduct
Hopp, C., & Hoover, G. A. (2017) · Science and Engineering Ethics · doi:10.1007/s11948-018-0079-4 · Cited by 14
Avoidance Learning Across the Menstrual Cycle: A Conceptual Replication
Diekhof, E. K., Korf, S., Ott, F., Schädlich, C., & Holtfrerich, S. K. C. (2020) · Frontiers in Endocrinology · doi:10.3389/fendo.2020.00231 · Cited by 11
“This can be accomplished by testing a larger repetition sample with the same experimental setup as used in the first study [in our example this would have been equivalent to a within-subject design simply applied to a bigger sample; see Schmidt (13) for discussion and overview].”
MethodCorrespondence measures for assessing replication success.
Steiner, P. M., Sheehan, P., & Wong, V. C. (2023) · Psychological Methods · doi:10.1037/met0000597 · Cited by 13
Abstractness impacts conversational dynamics
Mazzuca, C., Villani, C., Lamarra, T., Bolognesi, M., & Borghi, A. M. (2025) · Cognition · doi:10.1016/j.cognition.2025.106084 · Cited by 14
Fleming, P. J. S. (2019) · Forest Ecology and Management · doi:10.1016/j.foreco.2019.04.011 · Cited by 10
The replicability of the negative testing effect: differences across participant populations
Mulligan, N. W., Rawson, K. A., Peterson, D. J., & Wissman, K. T. (2018) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0000490 · Cited by 10
Murphy, J., Mesquida, C., & Warne, J. P. (2023) · Communications in Kinesiology · doi:10.51224/cik.2023.53 · Cited by 12
Millender, L., Bisel, R. S., & Zanin, A. C. (2023) · Communication Research Reports · doi:10.1080/08824096.2023.2236926 · Cited by 13
“Additionally, replication of these findings is desirable (Schmidt, 2009).”
Backgroundda Silva, F. Q. B., Suassuna, M., Lopes, R. F., Gouveia, T. B., Franca, A. C. A., Oliveira, J. P., de Oliveira, L. F. M., & Santos, A. L. M. (2011) · International Workshop on Replication in Empirical Software Engineering Research (RESER) · doi:10.1109/reser.2011.14 · Cited by 11
“' [33], and 'what are all types of replication and their respective roles'.”
Influentialvan der Staay, F. J., Nordquist, R. E., & Arndt, S. S. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-809468-6.00003-6 · Cited by 11
“…are based on a wider notion of replication, namely the repetition of a test of a hypothesis or a result of earlier work with different methods (schmidt, 2009). in “partial” replications (slight) procedural modifications are introduced whereas all other aspects closely mimic the original study.”
The effect of layout and pacing on learning from diagrams with unnecessary text
Rop, G., Schüler, A., Verkoeijen, P. P. J. L., Scheiter, K., & van Gog, T. (2018) · Applied Cognitive Psychology · doi:10.1002/acp.3445 · Cited by 15
“We set out to replicate and extend the results from Chandler and Sweller (1991). They showed that it is better to leave out unnecessary text (diagram only) than to present it integrated into the diagram or separated from the diagram with the instruction to integrate it. They also demonstrated that a separated presenta…”
BackgroundResultInfluentialReplication research in instructed SLA
McManus, K. (2022) · Research methods in applied linguistics · doi:10.1075/rmal.3.05mcm · Cited by 11
Derksen, M., & Rietzschel, E. F. (2013) · Industrial and Organizational Psychology · doi:10.1111/iops.12053 · Cited by 12
“Which is why some people prefer the term ‘‘direct replication’’ (Schmidt, 2009) or ‘‘close replication’’ (LeBel & Peters, 2011).”
Taylor, J. A., Furtak, E. M., Kowalski, S. M., Martinez, A., Slavin, R. E., Stuhlsatz, M. A. M., & Wilson, C. D. (2016) · Journal of Research in Science Teaching · doi:10.1002/tea.21327 · Cited by 13
Becoming Critical Consumers of Research: Understanding Replication
Mathews, H. M., Hirsch, S. E., & Therrien, W. J. (2017) · Intervention in School and Clinic · doi:10.1177/1053451217736863 · Cited by 10
“…purposeful research regarding a strategy or practice Synthesis Summary of the evidence related to a particular strategy or practice; includes multiple studies but does not calculate statistical effects and as a way to demonstrate the applicability of research findings in practice (Schmidt, 2009).”
BackgroundPépin, M., Kwakkenbos, L., Carrier, M., Peláez, S., El‐Baalbaki, G., Malcarne, V. L., & Thombs, B. D. (2019) · Journal of Scleroderma and Related Disorders · doi:10.1177/2397198319849806 · Cited by 10
Can You Repeat That? Exploring the Definition of a Successful Model Replication in Health Economics
McManus, E., Turner, D., & Sach, T. (2019) · PharmacoEconomics · doi:10.1007/s40273-019-00836-y · Cited by 13
“Considerable research has also been conducted looking at what constitutes a replication, with several taxonomies about the different types and definitions of replication having been developed [6, 7, 21].”
BackgroundAn Interactionist Approach to Cognitive Debiasing
Bland, S. (2020) · Episteme · doi:10.1017/epi.2020.9 · Cited by 13
“Granting agencies and academic journals express, implicitly and explicitly, a strong preference to fund and publish positive, original findings (Schmidt 2009; Fanelli 2012).”
BackgroundComparative clinical trials in psychotherapy: Have large effects been replicated?
Frost, N. D., Baskin, T. W., & Wampold, B. E. (2020) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796020000402 · Cited by 14
Replication of design theories: Reflections on function, outcome, and impact
Brendel, B., & Muntermann, J. (2022) · Information Systems Journal · doi:10.1111/isj.12387 · Cited by 12
“F IGURE 5 Truth theory and reality F IGURE 6 Functions of replication Based on Schmidt (2009), the outcome of a replication study that leads to a decrease in confidence can have various causes ranging from sample errors to missing internal validity, and even fraud.”
BackgroundDirect and conceptual replications of the taxometric analysis of type a behavior.
Wilmot, M. P., Haslam, N., Tian, J., & Öneş, D. S. (2018) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000195 · Cited by 11
Experiment Replication: A Proposed Solution for Developing Psychological Research in Indonesia
Shadiqi, M. A., Muluk, H., & Milla, M. N. (2019) · ANIMA Indonesian Psychological Journal · doi:10.24123/aipj.v33i4.1795 · Cited by 10
“From a philosophical perspective, Schmidt (2009) has placed replication studies as the heart of any science because they contribute enormously to science.”
BackgroundValidating the Workplace Dignity Scale
Scott-Campbell, C., & Williams, M. N. (2020) · Collabra Psychology · doi:10.1525/collabra.337 · Cited by 10
Zhu, X., Shek, D. T. L., Gong, Z., & Tang, Y. (2025) · Applied Research in Quality of Life · doi:10.1007/s11482-024-10416-8 · Cited by 11
“…as a process of continually accumulating knowledge and evidence, increase the generalizability and scientific credibility of original findings (Schmidt, 2009), and are widely emphasized in academia as a cor - nerstone of social science research, particularly disciplines such as developmental…”
Liu, T., Aryadoust, V., & Foo, S. W. L. (2021) · Language Testing · doi:10.1177/02655322211018139 · Cited by 11
“In a systematic review of replication studies in second language (L2) learning, Marsden et al. (2018) recommended three subtypes of replication research from Lykken (1968), Makel et al. (2012), and Schmidt (2009).”
BackgroundSubjective Outcome Evaluation of Instructional Videos in Leadership Education
Li, X., Shek, D. T. L., Wong, T., & Yu, L. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20010367 · Cited by 9
“As pointed out by Schmidt [69], replication is much needed in social science research to control for errors or to generalize research findings to a different or a larger population.”
BackgroundThompson, P. S., Bolino, M. C., Norris, K. R., & Kuo, S.-T. (2023) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2023.104275 · Cited by 9
A Survey of Researcher Perceptions of Replication in Geography
Kedron, P., Holler, J., & Bardin, S. (2024) · Annals of the American Association of Geographers · doi:10.1080/24694452.2024.2415695 · Cited by 10
“Within these taxonomies, replication studies are also often classified as either direct or conceptual depending on which of these aspects a researcher is changing and which objective they are pursuing (see Sargent 1981; Schmidt 2009; Plesser 2017).”
InfluentialAn Inception-Cohort Study Quantifying How Many Registered Studies Are Publicly Shared
Ensinck, E. N. F., & Lakens, D. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459241296031 · Cited by 16
“If researchers do not report failed replications, it is impossible to correct false-positive claims in the scientific literature (Schmidt, 2009), and if null results remain unreported, effect-size estimates in meta-analyses will be inflated (Egger et al., 1997).”
Mazzuca, C., Arcovito, M., Falcinelli, I., Fini, C., & Borghi, A. M. (2025) · Cognitive Science · doi:10.1111/cogs.70094 · Cited by 9
Abbott, B. R. (2017) · International Journal of Law and Psychiatry · doi:10.1016/j.ijlp.2017.03.003 · Cited by 9
“Models for replication of the VRAG-R revised standardization sample are described by Schmidt (2009) and Rossegger et al. (2014).”
MethodReplication of Experimental Research: Implications for the Study of Public Management
Walker, R. M., Lee, M. J., & James, O. (2017) · Cambridge University Press eBooks · doi:10.1017/9781316676912.022 · Cited by 9
Tilden, T., Theisen, M., Wampold, B. E., Johnson, S. U., & Hoffart, A. (2019) · Psychotherapy Research · doi:10.1080/10503307.2019.1645369 · Cited by 10
“…2”), investigating whether the Cohort 1-results could be replicated.1 “Replication” is here understood in the narrow definition as “direct replication,” meaning that we used the same procedure (identical intake criteria, instruments, statistical analyses) in Cohort 2 as in Cohort 1 (Schmidt, 2009).”
MethodA General Theory of Objectivity: Contributions from the Reformational Philosophy Tradition
Gunton, R. M., Stafleu, M. D., & Reiss, M. J. (2021) · Foundations of Science · doi:10.1007/s10699-021-09809-x · Cited by 10
Student understanding of functions of two variables: A reproducibility study
Borji, V., Martínez-Planell, R., & Trigueros, M. (2022) · The Journal of Mathematical Behavior · doi:10.1016/j.jmathb.2022.100950 · Cited by 9
Butt, I., Ul‐Haq, S., Shareef, M. A., Chowdhury, A. H., & Ahmed, J. U. (2022) · International Journal of Bank Marketing · doi:10.1108/ijbm-03-2021-0104 · Cited by 9
Walach, H. (2011) · Studies in neuroscience, consciousness and spirituality · doi:10.1007/978-94-007-2079-4_1 · Cited by 9
Cook, B. G. (2016) · Behavioral Disorders · doi:10.17988/0198-7429-41.3.161 · Cited by 9
“Yet despite its generally recognized importance, replication research tends to be held in low regard and is seldom conducted (Schmidt, 2009).”
BackgroundKim, K., & Clariana, R. B. (2018) · Educational Technology Research and Development · doi:10.1007/s11423-018-9607-9 · Cited by 9
Cranmer, G. A., Gagnon, R. J., & Mazer, J. P. (2019) · Communication & Sport · doi:10.1177/2167479518824868 · Cited by 9
“The replication and extension of CT within the coaching context is warranted given the uniqueness of Cranmer et al.’s findings, the retrospective nature of their sample, and the restricted understanding of coaching effectiveness (Schmidt, 2009).”
BackgroundConditional effects of gaze on automatic imitation: the role of autistic traits
Trilla, I., Wnendt, H., & Dziobek, I. (2020) · Scientific Reports · doi:10.1038/s41598-020-72513-6 · Cited by 12
Grahe, J. E., & Cuccolo, K. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009000796.014 · Cited by 9
Huang, Y., & Snedeker, J. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/huve3 · Cited by 9
The Open Science Framework: Improving Science by Making It Open and AccessiblePreprint
Spies, J. R. (2017) · Thesis Commons (OSF Preprints) · doi:10.31237/osf.io/t23za · Cited by 9
Charting the Constellation of Science Reform
Field, S. M. (2022) · Liberate Science GmbH eBooks · doi:10.53962/meeb-2ktd · Cited by 11
How and Why to Conduct a Replication Study
McManus, K. (2023) · Wiley eBooks · doi:10.1002/9781394259670.ch15 · Cited by 10
Jankvist, U. T., Aguilar, M. S., Misfeldt, M., & Koichu, B. (2021) · Implementation and Replication Studies in Mathematics Education · doi:10.1163/26670127-01010015 · Cited by 8
Scientific Truth in a Post-Truth Era: A Review*
Hagiopol, C., & Leru, P. M. (2024) · Science & Education · doi:10.1007/s11191-024-00527-x · Cited by 12
“Confidence in scientific data serves as the differentiating factor between hard and soft sciences (as seen in the replicability within social science) (Romero, 2019; Schmidt, 2009).”
Strength is negatively associated with depression and accounts for some of the sex difference
Smith, C. B., Rosenström, T. H., & Hagen, E. H. (2022) · Evolution Medicine and Public Health · doi:10.1093/emph/eoac007 · Cited by 12
Is Sexual Racism Still Really Racism? Revisiting Callander et al. (2015) in the USA
Smith, J. G., Sage, A. J., McGlenn, M., Robbins, J., & Garmon, S. L. (2022) · Archives of Sexual Behavior · doi:10.1007/s10508-022-02351-2 · Cited by 11
“Replication, although an often-undervalued practice, is extremely important to assuring the validity of scientific claims (Funder et al., 2014; Schmidt, 2009).”
BackgroundInfluentialSustainability and the Social Representation of the COVID-19 Pandemic: A Missing Link
Jóia, L. A., Michelotto, F. D. P., & Lorenzo, M. F. (2022) · Sustainability · doi:10.3390/su141710527 · Cited by 11
Testing Replicability and Generalizability of the Time on Task Effect
Krämer, R. J., Koch, M., Levacher, J., & Schmitz, F. (2023) · Journal of Intelligence · doi:10.3390/jintelligence11050082 · Cited by 9
“It ensures that knowledge can be separated from the context in which it was gained (i.e., a specific result pattern concerning the effects of response time on successful completion of a task is not limited to a particular sample), therefore it operationalizes objectivity (Schmidt 2009).”
BackgroundReplication Studies Using Secondary or Nonexperimental Datasets
Huang, F. L., & Huang, A. B. (2024) · School Psychology Review · doi:10.1080/2372966x.2024.2346781 · Cited by 10
“Replications have been referred to as “one of the most important tools for the verification of facts within the empirical sciences” (Schmidt, 2009, p. 90).”
Sign-Congruence, External Validity, and Replication
Slough, T. L., & Tyson, S. A. (2025) · Political Analysis · doi:10.1017/pan.2024.26 · Cited by 11
“These features map directly onto a replication classification, shown in Table 1, that expands on common expositions of replication, including exact, direct, and conceptual replication (Collins, 1992; Schmidt, 2009; Nosek and Errington, 2017).”
On the failure to replicate past findings regarding positive affirmations and self-esteem
Flynn, M. K., & Bordieri, M. J. (2020) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2020.03.003 · Cited by 9
Daley, N., & Rawson, K. A. (2020) · Educational Psychology Review · doi:10.1007/s10648-020-09553-x · Cited by 8
“Experiment 2 thus included a direct replication to establish this effect further, given recent emphasis in the field on the importance of direct replication of novel outcomes (e.g., Pashler and Harris 2012; Schmidt 2009; Simons 2014).”
BackgroundOpen Science als Beitrag zur Qualität in der Bildungsforschung
Krammer, G., & Svecnik, E. (2020) · Zeitschrift für Bildungsforschung · doi:10.1007/s35834-020-00286-z · Cited by 14
“In einzelnen Teildisziplinen scheinen Replikationsstudien jedenfalls bereits zunehmendes Interesse zu genießen (z.B. Coyne et al. 2016; Travers et al. 2016; Schmidt 2009).”
BackgroundSuls, J. M. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00872 · Cited by 8
“To be clear, a direct replication refers to repetition of an experimental procedure, in contrast to “repetition of a test of a hypothesis or a result of earlier research work with different methods,” which is a conceptual replication (Schmidt, 2009).”
ResultReplication and Reproduction in Canadian Policing Research: A Note
Huey, L., & Bennell, C. (2016) · Canadian Journal of Criminology and Criminal Justice/La Revue canadienne de criminologie et de justice pénale · doi:10.3138/cjccj.2016.e09 · Cited by 8
Screen Recordings as a Tool to Document Computer Assisted Data Collection Procedures
Heycke, T., & Spitzer, L. (2019) · Psychologica Belgica · doi:10.5334/pb.490 · Cited by 10
Saavedra, R., Barros, A., Silva, A. J., Leitão, J. C., Narciso, J., Costa, A. M., & Tallon, J. M. (2020) · Health Education Research · doi:10.1093/her/cyaa021 · Cited by 8
“In fact, for an intervention to be considered evidence based and with generalized results, the study needs to be replicated with adapted methods or, as is more usual, by modifying the content to the target population in a different setting [44].”
MethodThe Epistemology of Evolutionary Psychology Offers a Rapprochement to Cultural Psychology
Saad, G. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.579578 · Cited by 11
“…psychology that has been written about extensively, nomological networks amass cumulative evidence stemming from direct and conceptual replications (Schmidt, 2009) but more importantly they uncover what I coin as metareplications (i.e., replications of a grand effect across a very broad range of…”
BackgroundLi, M., Wei, D., Yang, W., Zhang, J., & Qiu, J. (2019) · Neuroreport · doi:10.1097/wnr.0000000000001306 · Cited by 10
Rucklidge, J. J., Afzali, M. U., Kaplan, B. J., Bhattacharya, O., Blampied, F. M., Mulder, R., & Blampied, N. M. (2021) · International Perspectives in Psychology · doi:10.1027/2157-3891/a000003 · Cited by 10
Openness and Reproducibility: Insights from a Model-Centric ApproachPreprint
Baumgaertner, B. O., Devezer, B., Buzbas, E. O., & Nardin, L. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.04525 · Cited by 11
“As the thinking goes, the building blocks of a reproducible science are replication studies (Earp and Trafimow, 2015; LeBel and Peters, 2011; Schmidt, 2009), and transparency makes replication studies possible by making research materials and processes explicit and accessible.”
BackgroundOpen Science Collaboration (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2195999 · Cited by 10
Liu, H., Shen, W., An-yi, H., Wang, W., Li, W., & Hastie, P. A. (2022) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.2022-0058 · Cited by 7
Facing Fear with Courage: Psychometric and Behavioral Evidence of the Courage Measure (CM) in Poland
Pajestka, G. (2023) · Roczniki Psychologiczne · doi:10.18290/rpsych2023.0015 · Cited by 8
“The present study was therefore a conceptual replication rather than a direct one (e.g., Schmidt, 2009).”
BackgroundMethodGood, D. J., Lyddy, C. J., Thompson, P. S., & Becker, W. J. (2024) · The Journal of Applied Behavioral Science · doi:10.1177/00218863241261071 · Cited by 9
“This was to increase con fi dence that our fi ndings are robust and generalizable (Hochwarter et al., 2011; Schmidt, 2009), which is particularly important during a crisis (Bruno et al., 2020; Skar et al., 2023).”
Deitzer, J. R., Frankenhuis, W. E., Jacobsen, W. C., Ribeaud, D., Eisner, M., Feinberg, M. E., & van Gelder, J. (2024) · Journal of Research in Crime and Delinquency · doi:10.1177/00224278241275573 · Cited by 9
Krause, M. S. (2016) · Psychotherapy Research · doi:10.1080/10503307.2016.1222459 · Cited by 7
“Doing so must be because these all are true replicates of the same RCTjk (see, e.g., Ioannidis, 2014; Schmidt, 2009; van der Veer, van Ijzendoorn, & Valsiner, 1994, on the concept and feasibility of such replication), which would mean that they all have the same statistically expected outcome…”
BackgroundJansen, C., & van der Kroef, G. (2018) · Journal of Intercultural Communication Research · doi:10.1080/17475759.2018.1555673 · Cited by 10
Białobrzeska, O., Parzuchowski, M., & Wojciszke, B. (2019) · PLoS ONE · doi:10.1371/journal.pone.0213183 · Cited by 10
Hoehle, H., Wei, J., Schuetz, S. W., & Venkatesh, V. (2021) · Internet Research · doi:10.1108/intr-02-2020-0105 · Cited by 8
Metadata
- Authors
- Schmidt, S.
- Affiliations
- University Medical Center Freiburg (DE)
- Corresponding author
- Schmidt, Stefan
- ORCID
- Schmidt, Stefan
- Venue
- Review of General Psychology
- Publisher
- SAGE Publications
- Published
- 2009-06
- Volume
- 13
- Issue
- 2
- Pages
- 90-100
- ISSN
- 1089-2680 (ISSN-L)
- 1939-1552
- OpenAlex
- W2021458971
- Cited by
- 1,125
- References
- 55
- Citations by year
- 2026: 42 · 2025: 95 · 2024: 88 · 2023: 85 · 2022: 75 · 2021: 138 · 2020: 94 · 2019: 105 · 2018: 93 · 2017: 74 · 2016: 84 · 2015: 52 · 2014: 35 · 2013: 26 · 2012: 23
- FWCI
- 16.48
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Review of General Psychology (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Keywords
- replication, social sciences, fact-checking, publication practices, research methodology, epistemology of science
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.