Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences
The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
Replication is one of the most important tools for the verification of facts within the empirical sciences. A detailed examination of the notion of replication reveals that there are many different meanings to this concept and the relevant procedures, but hardly any systematic literature. This paper analyzes the concept of replication from a theoretical point of view. It demonstrates that the theoretical demands are scarcely met in everyday work within the social sciences. Some demands are just not feasible, whereas others are constricted by restrictions relating to publication. A new classification scheme based on a functional approach that distinguishes between different types of replication is proposed. Next, it will be argued that replication addresses the important connection between existing and new knowledge. To do so it has to be applied explicitly and systematically. The paper ends with a description of procedures how this could be done and a set of recommendations how to handle the concept of replication in the future to exploit its potential to the full.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Novel or innovative results are more likely to be published than replications or incremental extensions of existing findings 13-15 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [14]
In the citing paper
“RRs provide a mechanism for publishing replication research”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [14]
In the citing paper
“which are rarely published in the standard model 13,14 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [14]
In the citing paper
“Although sometimes replication efforts can reveal poorly designed studies and irreproducible results, this is more the exception than the rule [48].”
From The natural selection of bad science (2016) · cites this paper as [48]
Citations
Cites
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
The Logic of Scientific Discovery
(2013) · A Realist Theory of Science · doi:10.4324/9780203090732-9 · Cited by 7,731
Journal of Consciousness Studies
(2022) · Journal of Consciousness Studies · doi:10.53765/20512201 · Cited by 6,832
Journal of social behavior and personality
(1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90025-5 · Cited by 6,706
Design and Analysis: A Researcher's Handbook
Beaton, A. E., & Keppel, G. (1975) · American Educational Research Journal · doi:10.2307/1162588 · Cited by 5,822
Scientific Explanation and the Causal Structure of the World
Salmon, W. C. (1985) · Princeton University Press eBooks · doi:10.1515/9780691221489 · Cited by 2,459
Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life
Cohen, I. B., Shapin, S., & Schaffer, S. (1987) · The American Historical Review · doi:10.2307/1869945 · Cited by 2,151
Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life
Cohen, I. B., Shapin, S., & Schaffer, S. (1987) · The American Historical Review · doi:10.1086/ahr/92.3.658 · Cited by 2,151
Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life
Traynham, J. G., Shapin, S., & Schaffer, S. (1987) · Journal of Interdisciplinary History · doi:10.2307/204299 · Cited by 2,045
Design and Analysis: A Researcher's Handbook
Aaker, D. A., & Keppel, G. (1976) · Journal of Marketing Research · doi:10.2307/3150757 · Cited by 2,011
Research works printed but not linked: 12
- American Psychological Association, 2001, Publication Manual of the American Psychological Association
- Beloff J., 1985, The repeatability problem in parapsychology, 1
- Collins H. M., 1985, Changing order
- CookT. D. (1990). Rersearch methodology: Strengthening causal interpretations of nonexperimental data. In SechrestL., PerrinE. & BunkerJ., US Department of health and human services (pp. 9–30). Public Health Service, Agency for Health Care Policy and Research.
- Edge H., 1985, The repeatability problem in parapsychology, 53
- Neuliep J. W., 1993, Journal of Social Behavior and Personality, 8, 21
- Rao K. R., 1981, Journal of Parapsychology, 45, 311
- Rosenthal R., 1991, Replication research in the social sciences, 1
- Sargent C. L., 1981, European Journal of Parapsychology, 3, 423
- Schacter D. L., 2000, Wir sind Erinnerung. Gedächtnis und Persönlichkeit
- Walach H., 2000, Neuroendocrinology Letters, 21, 221
- Weizsäcker E. U. v., 1986, Offene Systeme I: Beiträge zur Zeitstruktur von Information, Entropie und Evolution, 82
Cited by
Items 401–500 of 1,001
Bickley, S. J., Chan, H. F., Dao, B., Torgler, B., Tran, S., & Zimbatu, A. (2024) · Journal of Services Marketing · doi:10.1108/jsm-11-2023-0441 · Cited by 27
Similarities and differences in program registers: A case study
Zack, M. K., Karre, J. K., Olson, J. R., & Perkins, D. F. (2019) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2019.101676 · Cited by 19
When a Crisis Becomes an Opportunity: The Role of Replications in Making Better Theories
Lavelle, J. S. (2021) · The British Journal for the Philosophy of Science · doi:10.1086/714812 · Cited by 25
“Repeated sampling from the same population?” A critique of Neyman and Pearson’s responses to Fisher
Rubin, M. (2020) · European Journal for Philosophy of Science · doi:10.1007/s13194-020-00309-6 · Cited by 26
“First, Fisher would agree that “there is no such thing as an exact replication” (Schmidt, 2009, p. 92; see also Nosek & Errington, 2020), that exact replications are “an illusion” (Stroebe & Strack, 2014, p. 59), and that “it is impossible to conduct exact replications” (Zwaan et al., 2018, p. 6).”
BackgroundReplication, Uncertainty and Progress in Comparative Cognition
Boyle, A. (2021) · Animal Behavior and Cognition · doi:10.26451/abc.08.02.15.2021 · Cited by 23
“…aim to reproduce an experiment as closely as possible, holding all relevant factors fixed, with a view to determining whether the original experiment was reliable – that its result was not the result of non-directional error, questionable research practices and so on (e.g., Schmidt, 2009).”
BackgroundStimulus Sampling and Research Integrity
Cummings, J. J., & Reeves, B. B. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780190938550.003.0008 · Cited by 24
Back to the basics: how feelings of anger affect cooperation
Motro, D., Kugler, T., & Connolly, T. (2016) · International Journal of Conflict Management · doi:10.1108/ijcma-10-2015-0068 · Cited by 21
“Conceptual replications refer to repetitions of a test of a hypothesis or a result of earlier research work with different methods (Schmidt, 2009).”
ResultInter-study and intra-study replications in leading marketing journals: a longitudinal analysis
Kwon, E. S., Yan, S., Lee, J. S., & Reid, L. N. (2017) · European Journal of Marketing · doi:10.1108/ejm-07-2015-0450 · Cited by 22
Hedges, L. V. (2019) · Methodology · doi:10.1027/1614-2241/a000173 · Cited by 24
Vachon, B., Curran, J., Karunananthan, S., Brehaut, J., Graham, I. D., Moher, D., Sales, A. E., Straus, S. E., Fiander, M., Paprica, P. A., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.07.006 · Cited by 25
Hopp, C., & Pruschak, G. (2020) · The Leadership Quarterly · doi:10.1016/j.leaqua.2020.101475 · Cited by 26
Advancing the Cognitive Science of Religion through Replication and Open Science
Hoogeveen, S., & van Elk, M. (2021) · Journal for the Cognitive Science of Religion · doi:10.1558/jcsr.39039 · Cited by 23
Family reciprocity of older Singaporeans
Verbrugge, L. M., & Ang, S. (2018) · European Journal of Ageing · doi:10.1007/s10433-017-0452-1 · Cited by 22
“Do they come to the same conclusion about the shape of family ties?5 Replication is not boring! It is a fundamental goal of scientific research, and often an explicit purpose of research in the biological and physical sciences, but seldom in the social sciences (Bronstein 1990; Earp and Trafimow 2015; Hüffmeier et al.…”
BackgroundBalafoutas, L., Celse, J., Karakostas, A., & Umashev, N. (2024) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2024.102327 · Cited by 24
Financial performance of hospitals in Europe – a scoping review
Dubas‐Jakóbczyk, K., Ndayishimiye, C., Szetela, P., Sowada, C., & Kocot, E. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13080-2 · Cited by 18
“It emphasizes the problem of low reproducibility of empirical results which among other consequences, undermines the credibility of the theoretical background used [105].”
Cloudy, J., Banks, J., & Bowman, N. D. (2022) · Technology Mind and Behavior · doi:10.1037/tmb0000083 · Cited by 20
When identities collide: organizational and professional identity conflict and employee outcomes
Ostermeier, K. K., Anzollitto, P., Cooper, D., & Hancock, J. I. (2023) · Management Decision · doi:10.1108/md-07-2022-0971 · Cited by 27
Kedron, P., Bardin, S., Holler, J., Gilman, J., Grady, B. W., Seeley, M. M., Wang, X., & Yang, W. (2023) · Geographical Analysis · doi:10.1111/gean.12370 · Cited by 20
“…principle, reproducible research publicly discloses the evidence base for claims from prior work not only to improve the transparency of scientific communication but also to facilitate the independent verification of those claims (Schmidt 2009; Nosek, Spies, and Motyl 2012; Earp and Trafimow 2015).”
Estimating the Expected Replicability of a Pattern of Correlations and Other Measures of Association
Sherman, R. A., & Wood, D. (2014) · Multivariate Behavioral Research · doi:10.1080/00273171.2013.822785 · Cited by 18
“According to Schmidt (2009), replication “is the proof that the experiment reflects knowledge that can be separated from the specific circumstances (such as time, place, or persons) under which it was gained” (p. 90).”
BackgroundResultInfluentialVariation factors in the design and analysis of replicated controlled experiments
Runeson, P., Stefik, A., & Andrews, A. K. A. (2013) · Empirical Software Engineering · doi:10.1007/s10664-013-9262-z · Cited by 18
Stereotype Threat Reduces the Positivity of Older Adults’ Recall
Barber, S. J., Seliger, J., Yeh, N., & Tan, S. C. (2018) · The Journals of Gerontology Series B · doi:10.1093/geronb/gby026 · Cited by 22
Changing Police Officers’ Attitudes in Sexual Offense Cases: A 12-Month Follow-Up Study
Tidmarsh, P., Hamilton, G., & Sharman, S. J. (2020) · Criminal Justice and Behavior · doi:10.1177/0093854820921201 · Cited by 23
“…replication of a particular set of findings, namely Darwinkel et al.’s, along with a “conceptual” replication in which we sought to determine a boundary condition; that is, how long these changes in attitudes lasted (see Schmidt, 2009, for a discussion of direct and conceptual replication).”
BackgroundCollege Students’ Development of Badminton Skills and Tactical Competencies Following Play Practice
Liu, H., Wang, W., Zhang, C., & Hastie, P. A. (2020) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.2019-0292 · Cited by 19
Grasley‐Boy, N. M., Gage, N. A., Reichow, B., MacSuga‐Gage, A. S., & Lane, H. B. (2021) · School Psychology Review · doi:10.1080/2372966x.2020.1853486 · Cited by 18
“Conceptual replications are far more common and require fewer study characteristics be replicated exactly (Coyne et al., 2016; Schmidt, 2009).”
BackgroundAccounting research and the significance test crisis
Johnstone, D. J. (2021) · Critical Perspectives on Accounting · doi:10.1016/j.cpa.2021.102296 · Cited by 22
“(Schmidt, 2009 p.95) In what is now a vast literature on what’s wrong with significance tests, and how so many research programs have arrived at a replication crisis, most discussion on how results can be made more reliable is aimed at (i) the families of statistical methods used, and (ii) better…”
BackgroundMethodInfluentialDon't characterize replications as successes or failures
Gelman, A. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x18000638 · Cited by 20
Yamada, Y., Xue, J., Li, P., Fernández, S. R., Özdoğru, A. A., Sarı, Ş., Cervera‐Torres, S., Hinojosa, J. A., Montoro, P. R., AlShebli, B., et al. (2024) · Memory & Cognition · doi:10.3758/s13421-024-01637-1 · Cited by 20
“…theories at the intersection of embodied cognition and social psychology is also supported from a more general perspective, as not many influential papers in the aforementioned area have been sufficiently replicated (cf. Schmidt, 2009; Wiggins & Christopherson, 2019; Zwaan et al., 2017).”
Replication and methodological robustness in quantitative typology
Becker, L., & Naranjo, M. G. (2025) · Linguistic Typology · doi:10.1515/lingty-2023-0076 · Cited by 19
“It has long been recognized as a research standard across di ff erent research disciplines (e.g. Donoho 2010; Gelman 2018; Goodman et al. 2016; Schmidt2009) and has become a more prominent issue in linguistics as well (e.g. Aguilar-Sánchez 2014; Berez-Kroeker et al. 2018; Bisang 2011; Gawne and…”
Slaughter, J. E., Christian, M. S., Podsakoff, N. P., Sinar, E. F., & Lievens, F. (2013) · Personnel Psychology · doi:10.1111/peps.12056 · Cited by 24
“Given the importance of replication for understanding new phenomena in the social sciences (Schmidt, 2009; Tsang & Kwan, 1999), we conducted a third study.”
BackgroundThe Testing Effect and Far Transfer: The Role of Exposure to Key Information
van Eersel, G. G., Verkoeijen, P. P. J. L., Povilenaite, M., & Rikers, R. M. J. P. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01977 · Cited by 23
“…of replication in (psychological) science (e.g., Cartwright, 1991; Ioannidis, 2005; Open Science Collaboration, 2012; Pashler and Wagenmakers, 2012; Klein et al., 2014), the purpose of our first experiment was to conduct an exact replication (see Schmidt, 2009) of Butler’s third experiment.”
MethodLet’s do it together! The role of interaction in false belief understanding
Białecka‐Pikul, M., Kosno, M., Białek, A., & Szpak, M. (2018) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2018.07.018 · Cited by 18
Pharmacogenetic mechanisms underlying unanticipated drug responses.
Carlquist, J. F., & Anderson, J. L. (2011) · PubMed · Cited by 17
Landry, S. H., Zucker, T. A., Montroy, J. J., Hsu, H.-Y. M., Assel, M. A., Varghese, C. A., Crawford, A. D., & Feil, E. G. (2021) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2021.03.007 · Cited by 21
Beste, C. (2021) · Communications Biology · doi:10.1038/s42003-021-02634-9 · Cited by 18
Taylor, J. A., Adams, C. T., Westbrook, A. L., Gee, J. C., Spybrook, J. K., Kowalski, S. M., Gardner, A. L., & Bloom, M. (2021) · Journal of Research in Science Teaching · doi:10.1002/tea.21733 · Cited by 21
Data Resource Profile: Melbourne Children’s LifeCourse initiative (LifeCourse)
O’Connor, M., Moreno‐Betancur, M., Goldfeld, S. R., Wake, M. A., Patton, G. C., Dwyer, T., Tang, M. L. K., Saffery, R., Craig, J. M., Loke, J., et al. (2022) · International Journal of Epidemiology · doi:10.1093/ije/dyac086 · Cited by 17
“13 Cohorts can also be brought together to enhance confidence in findings through replication 14 and to address questions spanning different age periods or constructs.”
Engineering Open Innovation – Towards a Framework for Fostering Open Innovation
Wnuk, K., & Runeson, P. (2013) · Lecture notes in business information processing · doi:10.1007/978-3-642-39336-5_6 · Cited by 21
“Building synthesized knowledge from the empirical observation needs replication of studies [61], synthesis of findings from several empirical studies [62], as well as work in theory building [63].”
MethodThe Meaning of Failed Replications: A Review and ProposalPreprint
Clemens, M. A. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2601686 · Cited by 23
Roisman, G. I. (2021) · Child Development · doi:10.1111/cdev.13538 · Cited by 18
Heterogeneity of Beliefs and Trading Behavior — A Reexamination
Füllbrunn, S., Huber, C., Eckel, C. C., & Weitzel, U. (2022) · Journal of Financial and Quantitative Analysis · doi:10.1017/s002210902300011x · Cited by 19
Revisiting the relationship between age, employment, and recidivism
Nguyen, H., Thomas, K. J., & Tostlebe, J. J. (2023) · Criminology · doi:10.1111/1745-9125.12338 · Cited by 19
“According to Schmidt (2009), direct replication of an earlier study is not sufficient to verify underlying hypotheses.”
Are Provided Examples or Faded Examples More Effective for Declarative Concept Learning?
Zamary, A., & Rawson, K. A. (2018) · Educational Psychology Review · doi:10.1007/s10648-018-9433-y · Cited by 21
Seeking reproducibility: Assessing a multimodal study of the testing effect
Beardsley, M., Hernández‐Leo, D., & Ramírez, R. (2018) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12265 · Cited by 19
“(2011) that attempts to validate the experimental setup, materials and methods as the contextual variables such as language, profiles of students, equipment used, and data processing techniques applied (Schmidt, 2009) differ from the original study.”
BackgroundInfluentialLohse, B., Prescott, M. P., & Cunningham-Sabo, L. D. (2019) · Nutrients · doi:10.3390/nu11071501 · Cited by 18
“Usually, little support is given for this type of endeavor, especially for publication and promotion credit [31,45], and the consequences of failure to replicate challenge this undertaking.”
BackgroundStorme, M., Myszkowski, N., Baron, S., & Bernard, D. (2019) · Journal of Intelligence · doi:10.3390/jintelligence7030017 · Cited by 17
“In conceptual replications, comparability to the original study is limited to the aspects that are considered theoretically relevant [28,29].”
BackgroundOh, T. M., Graham, S., Ng, P., Yeh, I. B., Chan, B., & Edwards, A. M. (2019) · Frontiers in Communication · doi:10.3389/fcomm.2019.00039 · Cited by 19
“…direct and conceptual replication studies have a crucial role to play in increasing the confidence in neuroimaging findings (Open Science Collaboration, 2015), with the latter referring to a study in which a previous result or hypothesis is tested with different methods (Schmidt, 2009; Evans, 2017).”
BackgroundMatos, P., & Albuquerque, P. B. (2021) · Cognitive Processing · doi:10.1007/s10339-021-01016-7 · Cited by 17
“To elucidate this issue, we conducted a conceptual replication with precise manipulation of theoretically relevant parameters (see Schmidt 2009).”
MethodToward replication study types for design science research
Brendel, B., Lembcke, T.-B., Muntermann, J., & Kolbe, L. M. (2021) · Journal of Information Technology · doi:10.1177/02683962211006429 · Cited by 23
“…conduct replication studies with at least one of five intentions: (1) finding sample errors, (2) controlling for a lack of internal validity, (3) uncovering fraud, (4) expanding or generalizing results to cover a larger or different context, and (5) verifying original hypotheses (Schmidt, 2009).”
BackgroundTeaching on video as an instructional strategy to reduce confirmation bias—a pre-registered study
van Brussel, S., Timmermans, M., Verkoeijen, P. P. J. L., & PAAS, F. G. W. C. (2021) · Instructional Science · doi:10.1007/s11251-021-09547-4 · Cited by 19
“For future research, it is of importance to replicate our social presence and arousal measurements in order to determine the robustness of our findings (Lakens et al., 2012; Schmidt, 2009; Simmons et al., 2011).”
BackgroundDemocracy is good for the poor: A procedural replication of Ross (2006)
García, F. M. (2014) · Research & Politics · doi:10.1177/2053168014559094 · Cited by 19
“One reason for this is that social scientists disagree on what constitutes a replication study, how they are done, and for what purpose (Anderson et al., 2008; Berthon et al., 2002; Bueno de Mesquita et al., 2003a; Burman et al., 2010; Gelman, 2013; Herrnson, 1995; Ishiyama, 2014; King, 1995; McCullough et al., 2008;…”
BackgroundHosp, J. L., Ford, J. W., Huddle, S. M., & Hensley, K. (2017) · Assessment for Effective Intervention · doi:10.1177/1534508417727489 · Cited by 17
“the impact of those threats across studies (Schmidt, 2009).”
BackgroundInfluentialSimpson, A. P., & Vondrová, N. (2019) · Journal of Education for Teaching International Research and Pedagogy · doi:10.1080/02607476.2019.1674563 · Cited by 16
“Schmidt (2009) lists five functions of replication: controlling fraud, controlling sampling error, controlling artefacts, generalising to different populations and assessing hypotheses in the existing literature.”
BackgroundA High School Replication of Targeted Professional Development for Classroom Management
Freeman, J., Kowitt, J., Simonsen, B., Wei, Y., Dooley, K., Gordon, L., & Maddock, E. (2018) · Remedial and Special Education · doi:10.1177/0741932517719547 · Cited by 19
“Therefore, replication studies serve many important functions, from controlling for coincidental results to generalizing results to different settings or populations (Schmidt, 2009).”
BackgroundEvaluating meta-analysis as a replication success measure
Muradchanian, J., Hoekstra, R., Kiers, H. A. L., & van Ravenzwaaij, D. (2024) · PLoS ONE · doi:10.1371/journal.pone.0308495 · Cited by 9
Conceptual replications, research, and the “what works” agenda in education
Morrison, K. (2022) · Educational Research and Evaluation · doi:10.1080/13803611.2021.2022314 · Cited by 17
“…(2019), Hawe et al. (2015), Hendrick (1991), Makel and Plucker (2014), Melhuish (2018), Morrison (2019), National Science Foundation and Institute of Education Sciences (2018), Rosenthal (1991), Schmidt (2009), Shanahan (2017), Therrien et al. (2016), Tyson (2014), What Works Clearinghouse (2018).”
BackgroundMethodInfluentialFrontline employee work engagement and customer service evaluations: a conceptual replication
Vaerenbergh, Y. V., & Hazée, S. (2024) · Journal of service management · doi:10.1108/josm-07-2024-0304 · Cited by 19
An Integrative Review of Moderators Attenuating or Exacerbating Negative Outcomes of Job Insecurity
Jiang, L. (2025) · Academy of Management Annals · doi:10.5465/annals.2024.0040 · Cited by 18
The Wisdom of Some: Do We Always Need High Consensus to Shape Consumer Behavior?
Sciandra, M. R., Lamberton, C. P., & Reczek, R. W. (2016) · Journal of Public Policy & Marketing · doi:10.1509/jppm.14.123 · Cited by 16
Learning about Learning at Scale: Methodological Challenges and Recommendations
van der Sluis, F., van der Zee, T., & Ginn, J. (2017) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3051457.3051461 · Cited by 17
“Here, hypotheses or results from earlier research are tested with a different experimental setup [37].”
ResultKrell, M. (2017) · Zeitschrift für Didaktik der Naturwissenschaften · doi:10.1007/s40573-017-0069-0 · Cited by 24
Heinrich, A., & Knight, S. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01590 · Cited by 15
“We acknowledge that a lively discussion exists within the wider field of behavioral science regarding the theoretical question of what exactly constitutes “reproducibility” and how best to assess it (Schmidt, 2009; Goodman et al., 2016).”
BackgroundMethodInfluentialReplication target selection in clinical psychology: A Bayesian and qualitative reevaluation.
Pittelkow, M.-M., Hoekstra, R., Karsten, J., & van Ravenzwaaij, D. (2021) · Clinical Psychology Science and Practice · doi:10.1037/cps0000013 · Cited by 17
User Interface Factors of Mobile UX: A Study with an Incident Reporting Application
Einfeldt, L., & Degbelo, A. (2021) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0010325302450254 · Cited by 19
How (not) to measure replication
Fletcher, S. C. (2021) · European Journal for Philosophy of Science · doi:10.1007/s13194-021-00377-2 · Cited by 19
“In this essay, these terms will not mark distinct concepts, although I will attempt to use only terms derived from “replicable”; I am in particular only concerned here with the kind, adumbrated above, sometimes known as direct replication (Schmidt 2009).”
BackgroundAdams, K. D., & Cook, A. M. (2016) · Disability and Rehabilitation Assistive Technology · doi:10.3109/17483107.2016.1151954 · Cited by 15
“[17] The study presented in this paper was performed to determine if the same benefits in the study above [16] could be found when the students performed Level 2 mathematics measurement lessons.”
ResultInvestigating a Tier 1 Intervention Focused on Proportional Reasoning: A Follow-Up Study
Jitendra, A. K., Harwell, M. R., Karl, S. R., Simonson, G. R., & Slater, S. C. (2017) · Exceptional Children · doi:10.1177/0014402917691017 · Cited by 16
“234) to determine the extent to which findings generalize across participants, settings, and conditions (Schmidt, 2009).”
BackgroundResultInfluentialSame Question, Different World: Replicating an Open Access Research Impact Study
Arendt, J., Peacemaker, B., & Miller, H. (2019) · College & Research Libraries · doi:10.5860/crl.80.3.303 · Cited by 24
ter Kuile, H., van der Lippe, T., & Kluwer, E. S. (2021) · Family Relations · doi:10.1111/fare.12546 · Cited by 25
“Replication is not only essential to empirical science in general (e.g., Schmidt, 2009), but, more specifically, the results of group-based modeling require replication among larger and diverse samples (Don & Mickelson, 2014).”
BackgroundMethodTeixeira, C. P., Spears, R., Iyer, A., & Leach, C. W. (2023) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2023.104454 · Cited by 15
Impact of Lesson Study and Fractions Resources on Instruction and Student Learning
Schoen, R. C., Lewis, C., Rhoads, C. H., Lai, K., & Riddell, C. M. (2023) · The Journal of Experimental Education · doi:10.1080/00220973.2023.2183374 · Cited by 18
“The study described here is a conceptual replication (Schmidt, 2009) of the Lewis and Perry (2017) randomized trial.”
BackgroundPeña, A. R. (2023) · Journal of International Entrepreneurship · doi:10.1007/s10843-023-00338-1 · Cited by 20
“As a result, we strengthen the confirmatory power of EO to FP and CV to FP by implementing different types of replications (Schmidt 2009).”
Are goal states represented during kinematic imitation?
Cole, G. G., Atkinson, M. A., D'Souza, A. D. C., Welsh, T. N., & Skarratt, P. A. (2017) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/xhp0000429 · Cited by 15
Najavits, L. M., Enggasser, J. L., Brief, D. J., & Federman, E. J. (2018) · American Journal on Addictions · doi:10.1111/ajad.12709 · Cited by 14
Regulatory Focus and Individual Sales Performance of Field Marketers: A Constructive Replication
Hamstra, M. R. W., McCabe, K. O., Klekamp, B., & Rietzschel, E. F. (2018) · Basic and Applied Social Psychology · doi:10.1080/01973533.2018.1526086 · Cited by 16
VRDOLJAK, J. E., Sánchez, K. I., Arreola-Ramos, R., Huesa, E. G. D., Villagra, A., Ávila, L. J., & Morando, M. M. (2020) · Biological Journal of the Linnean Society · doi:10.1093/biolinnean/blaa079 · Cited by 14
“However, most researchers rarely attempt to replicate results, possibly because—motivated by the ‘publish or perish’ dogma—most scientific journals have explicit policies against publishing replication studies (Schmidt, 2009).”
BackgroundHorster, I., Nickel, K., Holovics, L., Schmidt, S., Endres, D., van Elst, L. T., Zeeck, A., Maier, S. J., & Joos, A. A. B. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00777 · Cited by 17
“Furthermore, there is no general agreement on the definition or directives of replication procedures (9, 16, 18, 19).”
BackgroundSelf-citationA Systematic Review of Expressive and Receptive Prosody in People With Dementia
Oh, C., Morris, R. J., & Wang, X. (2021) · Journal of Speech Language and Hearing Research · doi:10.1044/2021_jslhr-21-00013 · Cited by 19
Sibunruang, H., Capezio, A., & Restubog, S. L. D. (2013) · Journal of Career Assessment · doi:10.1177/1069072713514821 · Cited by 20
“Indeed, recent research by Schmidt (2009) stresses the importance—and largely neglected—role of replication in the behavioral sciences.”
BackgroundWhen does practice testing promote transfer on deductive reasoning tasks?
Wissman, K. T., Zamary, A., & Rawson, K. A. (2018) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2018.03.002 · Cited by 16
Jacobson, E. D., & Simpson, A. M. (2018) · Mathematics Education Research Journal · doi:10.1007/s13394-018-0242-x · Cited by 14
“Because replication studies are so rare in the social sciences (Schmidt 2009) and mathematics education researchers may not have had training to conduct such studies, this article contributes to the field of education reseach in part by describing how we utilized and adapted a framework for…”
BackgroundPfabigan, D. M., & Han, S. (2019) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-019-00737-9 · Cited by 19
“…published research, although reproducibility is a basic and indispensable requirement of empirical science that should be more strongly acknowledged also in psychological science (GinerSorolla, 2012; Schmidt, 2009) to foster recent replication efforts (Open Science, 2015; Pashler & Harris, 2012).”
BackgroundThe Status and Value of Replications in Animal Behavior Science
Brecht, K. F., Legg, E. W., Nawroth, C., Fraser, H. S., & Ostojić, L. (2021) · Animal Behavior and Cognition · doi:10.26451/abc.08.02.01.2021 · Cited by 15
How to True Psychology’s Objects
Morawski, J. G. (2021) · Review of General Psychology · doi:10.1177/10892680211046518 · Cited by 17
“Commentators hold that dynamic, complex, and contextsensitive objects are largely confined to in certain specialties, notably social psychology, because unlike cognitions and neural processes, social life and society are ever changing (Schmidt, 2009).”
BackgroundOsborne, C., & Norris, E. (2022) · Cogent Psychology · doi:10.1080/23311908.2022.2066304 · Cited by 15
“Non-replicable findings are those that cannot be independently re-derived using the same or similar methods (Schmidt, 2009) and may arise from helpful or unhelpful sources.”
BackgroundA Large Scale Test of the Effect of Social Class on Prosocial BehaviorPreprint
Korndörfer, M., Egloff, B., & Schmukle, S. C. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2704303 · Cited by 17
Futterleib, H., Thomm, E., & Bauer, J. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.1006766 · Cited by 14
“Study 2 was a follow-up (Schmidt, 2009) that aimed to differentiate the assessments of the perceived potency and pertinence of educational research as facets of devaluation, and to increase the external validity of prior findings.”
Kedron, P., Holler, J., & Bardin, S. (2023) · Annals of the American Association of Geographers · doi:10.1080/24694452.2023.2276115 · Cited by 15
“eproducible research publicly discloses the evidence used to support claims made in prior work, facilitates the independent verification of those claims, and enables the extension of that work by the broader research community (Schmidt 2009; Nosek, Spies, and Motyl 2012; Earp and Trafimow 2015).”
MethodNazmy, M. S., Noor, N. A., Mohammed, F. F., Khadrawy, Y. A., & Radwan, N. M. (2025) · Naunyn-Schmiedeberg s Archives of Pharmacology · doi:10.1007/s00210-024-03767-4 · Cited by 13
Cooke, D. J., & Michie, C. (2014) · Journal of Threat Assessment and Management · doi:10.1037/tam0000004 · Cited by 18
Australian and Italian Psychologists’ View of Replication
Agnoli, F., Fraser, H. S., Thorn, F. S., & Fidler, F. M. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211039218 · Cited by 13
“Despite the essential role of replicability in establishing the reliability of findings, remarkably few replication studies have been published in psychology and other social science journals (see Schmidt, 2009).”
BackgroundClausen, J., Borthwick, A. C., & Rutledge, D. W. (2021) · Educational Technology Research and Development · doi:10.1007/s11423-021-10018-3 · Cited by 15
“Replication studies often follow the processes and protocols that a prior study used to establish previous findings (Schmidt, 2009).”
ResultConvergence in the Bilingual Lexicon: A Pre-registered Replication of Previous Studies
White, A., Malt, B. C., & Storms, G. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.02081 · Cited by 20
“Given the current crisis of confidence in psychology, replication studies in general are important to provide evidence for the credibility of psychological research and to support earlier obtained results (Schmidt, 2009).”
BackgroundResultInfluentialChung, S., & Fink, E. L. (2018) · Annals of the International Communication Association · doi:10.1080/23808985.2018.1425100 · Cited by 13
“…for theoretical development through confirmation and disconfirmation of results (Brandt et al., 2014; Lamal, 1990; Neuliep, 1991; Nosek, Spies, & Motyl, 2012; Schmidt, 2009) and imperative when there are significant discrepancies in existing studies (Benoit & Holbert, 2008; Collins, 1985).”
BackgroundResultInfluentialIn Pursuit of a Replication Culture in Mathematics Education
Star, J. R. (2021) · Implementation and Replication Studies in Mathematics Education · doi:10.1163/26670127-01010003 · Cited by 18
Nicht replizieren: publizieren!?
Rost, D. H., & Bienefeld, M. A. (2019) · Zeitschrift für Pädagogische Psychologie · doi:10.1024/1010-0652/a000253 · Cited by 14
Falsifiability Is Not OptionalPreprint
LeBel, E. P., Berger, D. M., Campbell, L., & Loving, T. J. (2017) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/dv94b · Cited by 15
Петитта, Л., Probst, T. M., Ghezzi, V., Bettac, E. L., Lavaysse, L. M., & Barbaranelli, C. (2023) · Journal of Occupational and Organizational Psychology · doi:10.1111/joop.12478 · Cited by 14
García-Cerdá, C., Segura, C., & Ferrando, I. (2024) · Implementation and Replication Studies in Mathematics Education · doi:10.1163/26670127-bja10017 · Cited by 12
The Effect of the EU ‐Brand on Citizens’ Trust in Policies: Replicating an Experiment
Eshuis, J., van de Geest, T., Klijn, E., Voets, J., Florek, M., & George, B. (2021) · Public Administration Review · doi:10.1111/puar.13367 · Cited by 15
“This raises questions about validity of earlier experimental research in PA, and calls for replication studies (Pedersen and Stritch 2018).”
Metadata
- Authors
- Schmidt, S.
- Affiliations
- University Medical Center Freiburg (DE)
- Corresponding author
- Schmidt, Stefan
- ORCID
- Schmidt, Stefan
- Venue
- Review of General Psychology
- Publisher
- SAGE Publications
- Published
- 2009-06
- Volume
- 13
- Issue
- 2
- Pages
- 90-100
- ISSN
- 1089-2680 (ISSN-L)
- 1939-1552
- OpenAlex
- W2021458971
- Cited by
- 1,125
- References
- 55
- Citations by year
- 2026: 42 · 2025: 95 · 2024: 88 · 2023: 85 · 2022: 75 · 2021: 138 · 2020: 94 · 2019: 105 · 2018: 93 · 2017: 74 · 2016: 84 · 2015: 52 · 2014: 35 · 2013: 26 · 2012: 23
- FWCI
- 16.48
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Review of General Psychology (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Keywords
- replication, social sciences, fact-checking, publication practices, research methodology, epistemology of science
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.