CoChat

Creative destruction in science

Open access
Tierney, Warren, Hardy, Jay H., Ebersole, Charles R., Leavitt, Keith, Viganola, Domenico, Clemente, Elena Giulia, Gordon, Michael, Dreber, Anna, Johannesson, Magnus, Pfeiffer, Thomas, Uhlmann, Eric Luis · 2020 · Organizational Behavior and Human Decision Processes
Cited by 66

The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.

Abstract

Drawing on the concept of a gale of creative destruction in a capitalistic economy, we argue that initiatives to assess the robustness of findings in the organizational literature should aim to simultaneously test competing ideas operating in the same theoretical space. In other words, replication efforts should seek not just to support or question the original findings, but also to replace them with revised, stronger theories with greater explanatory power. Achieving this will typically require adding new measures, conditions, and subject populations to research designs, in order to carry out conceptual tests of multiple theories in addition to directly replicating the original findings. To illustrate the value of the creative destruction approach for theory pruning in organizational scholarship, we describe recent replication initiatives re-examining culture and work morality, working parents’ reasoning about day care options, and gender discrimination in hiring decisions. It is becoming increasingly clear that many, if not most, published research findings across scientific fields are not readily replicable when the same method is repeated. Although extremely valuable, failed replications risk leaving a theoretical void— reducing confidence the original theoretical prediction is true, but not replacing it with positive evidence in favor of an alternative theory. We introduce the creative destruction approach to replication, which combines theory pruning methods from the field of management with emerging best practices from the open science movement, with the aim of making replications as generative as possible. In effect, we advocate for a Replication 2.0 movement in which the goal shifts from checking on the reliability of past findings to actively engaging in competitive theory testing and theory building. The materials, code, and data for this article are posted publicly on the Open Science Framework, with links provided in the article.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.obhdp.2020.07.002

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 48 references

Research works printed but not linked: 99

  1. Aguinis, H., & Solarino, A. M. (in press). Transparency and replicability in qualitative research: The case of interviews with elite informants. Strategic Management Journal.
  2. Albertini, D. F. (2017). On strong inferences and irreproducibility in reproductive medicine. Journal of Assisted Reproduction and Genetics, 34, 695–696. · doi:10.1007/s10815-017-0950-2
  3. Aguinis, 2009, First decade of organizational research methods: Trends in design, measurement, and data-analysis topics, Organizational Research Methods, 12, 9, 10.1177/1094428108322641 · DOI · Paper page
  4. Agnoli, 2017, Questionable research practices among Italian research psychologists, PLoS One, 12, e0172792, 10.1371/journal.pone.0172792 · DOI · Paper page
  5. Armor, 2006, Accuracy, error, and bias in predictions for real versus hypothetical events, Journal of Personality and Social Psychology, 91, 583, 10.1037/0022-3514.91.4.583 · DOI · Paper page
  6. Bamberger, 2019, On the replicability of abductive research in management and organizations: Internal replication and its alternatives, Academy of Management Discoveries, 5, 103, 10.5465/amd.2019.0121 · DOI
  7. Bamberger, 2008, Beyond contextualization: Using context theories to narrow the micro-macro gap in management research, Academy of Management Journal, 51, 839, 10.5465/amj.2008.34789630 · DOI · Paper page
  8. Baribault, 2018, Metastudies for robust tests of theory, Proceedings of the National Academy of Sciences, 115, 2607, 10.1073/pnas.1708285114 · DOI · Paper page
  9. Baron, 2019, False equivalence: Are liberals and conservatives in the United States equally biased?, Perspectives on Psychological Science, 14, 292, 10.1177/1745691618788876 · DOI · Paper page
  10. Barrick, 2005, Reducing voluntary, avoidable turnover through selection, Journal of Applied Psychology, 90, 159, 10.1037/0021-9010.90.1.159 · DOI · Paper page
  11. Bastardi, 2011, Wishful thinking: Belief, desire, and the motivated evaluation of scientific evidence, Psychological Science, 22, 731, 10.1177/0956797611406447 · DOI
  12. Bedeian, 2010, Management science on the credibility bubble: Cardinal sins and various misdemeanors, Academy of Management Learning & Education, 9, 715
  13. Bergh, 2017, Is there a credibility crisis in strategic management research? Evidence on the reproducibility of study findings, Strategic Organization, 15, 423, 10.1177/1476127017701076 · DOI · Paper page
  14. Bond, 1996, Cross-cultural social and organizational psychology, Annual Review of Psychology, 47, 205, 10.1146/annurev.psych.47.1.205 · DOI · Paper page
  15. Bosco, 2016, HARKing’s threat to organizational research: Evidence from primary and meta-analytic sources, Personnel Psychology, 69, 709, 10.1111/peps.12111 · DOI · Paper page
  16. Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F., Geller, J., Giner-Sorolla, R., … van‘t Veer, A. (2014). The replication recipe: What makes for a convincing replication? Journal of Experimental Social Psychology, 50, 217–224. · DOI · Paper page
  17. Brainerd, 2018, Replication, registration, and scientific creativity, Perspectives on Psychological Science, 13, 428, 10.1177/1745691617739421 · DOI
  18. Byington, 2017, Solutions to the credibility crisis in management science, Academy of Management Learning & Education, 16, 142, 10.5465/amle.2015.0035 · DOI
  19. Byrd, 2019, What we can (and can’t) infer about implicit bias from debiasing experiments, Synthese, 1
  20. Cairo, A.H., Green, J.D., Forsyth, D.R., Behler, A.M.C., & Raldiris, T.L. (in press). Gray (Literature) Matters: Evidence of Selective Hypothesis Reporting in Social Psychological Research. Personality and Social Psychology Bulletin.
  21. Carpenter, 2005, The effects of stakes in distribution experiments, Economics Letters, 86, 393, 10.1016/j.econlet.2004.08.007 · DOI · Paper page
  22. Cashen, 2004, Statistical power and the testing of null hypotheses: A review of contemporary management research and recommendations for future studies, Organizational Research Methods, 7, 151, 10.1177/1094428104263676 · DOI · Paper page
  23. Chang, 2017, A preanalysis plan to replicate sixty economics research papers that worked half of the time, American Economic Review, 107, 60, 10.1257/aer.p20171034 · DOI
  24. Crandall, 2016, On the scientific superiority of conceptual replications for scientific progress, Journal of Experimental Social Psychology, 66, 93, 10.1016/j.jesp.2015.10.002 · DOI · Paper page
  25. Davis, 2006, Mechanisms and the theory of organizations, Journal of Management Inquiry, 15, 114, 10.1177/1056492605280238 · doi:10.1177/1056492605280238
  26. Devine, 1990, Diagnostic and confirmation strategies in trait hypothesis testing, Journal of Personality and Social Psychology, 58, 952, 10.1037/0022-3514.58.6.952 · doi:10.1037/0022-3514.58.6.952 · Paper page
  27. DellaVigna, 2018, Predicting experimental results: Who knows what?, Journal of Political Economy, 126, 2410, 10.1086/699976 · DOI · Paper page
  28. Desai, 2014, The implications of marriage structure for men’s workplace attitudes, beliefs, and behaviors toward women, Administrative Science Quarterly, 59, 330, 10.1177/0001839214528704 · DOI
  29. Ditto, 2019, At least bias is bipartisan: A meta-analytic comparison of partisan bias in liberals and conservatives, Perspectives on Psychological Science, 14, 273, 10.1177/1745691617746796 · DOI · Paper page
  30. Ebersole, C. (2019). Pre-commitment and updating beliefs. Unpublished doctoral dissertation. · DOI
  31. Edwards, 2010, Reconsidering theoretical progress in organizational and management research, Organizational Research Methods, 13, 615, 10.1177/1094428110380468 · DOI · Paper page
  32. Evangelou, 2012, Perceived information gain from randomized trials correlates with publication in high–impact factor journals, Journal of Clinical Epidemiology, 65, 1274, 10.1016/j.jclinepi.2012.06.009 · DOI
  33. Forsell, 2019, Predicting replication outcomes in the Many Labs 2 study, Journal of Economic Psychology, 75, 102, 10.1016/j.joep.2018.10.009 · DOI · Paper page
  34. Funder, 1987, Errors and mistakes: Evaluating the accuracy of social judgment, Psychological Bulletin, 101, 75, 10.1037/0033-2909.101.1.75 · doi:10.1037/0033-2909.101.1.75 · Paper page
  35. Gelman, 2015, The connection between varying treatment effects and the crisis of unreplicable research: A Bayesian perspective, Journal of Management, 41, 632, 10.1177/0149206314525208 · DOI · Paper page
  36. Greenberg, J., & Jonas, E. (2003). Psychological motives and political orientation—The left, the right, and the rigid: Comment on Jost et al. (2003). Psychological Bulletin, 129(3), 376–382. · DOI · Paper page
  37. Goff, 1990, Employer supported child care, work/family conflict, and absenteeism: A field study, Personnel Psychology, 43, 793, 10.1111/j.1744-6570.1990.tb00683.x · DOI · Paper page
  38. Greenwald, 1995, Implicit social cognition: Attitudes, self-esteem, and stereotypes, Psychological Review, 102, 4, 10.1037/0033-295X.102.1.4 · DOI · Paper page
  39. Greenwald, 2002, A unified theory of implicit attitudes, beliefs, self-esteem and self-concept, Psychological Review, 109, 3, 10.1037/0033-295X.109.1.3 · DOI · Paper page
  40. Grossmann, 2011, Social class, culture, and cognition, Social Psychological and Personality Science, 2, 81, 10.1177/1948550610377119 · DOI · Paper page
  41. Hanselman, P., Rozek, C.S., Grigg, J., & Borman, G.D. (in press). New evidence on self-affirmation effects and theorized sources of heterogeneity from large-scale replications. Journal of Educational Psychology.
  42. Hambrick, 2007, The field of management’s devotion to theory: Too much of a good thing?, Academy of Management Journal, 50, 1346, 10.5465/amj.2007.28166119 · DOI · Paper page
  43. Harrington, 2014, Tightness–looseness across the 50 United States, Proceedings of the National Academy of Sciences, 111, 7990, 10.1073/pnas.1317937111 · DOI · Paper page
  44. Harvie, 1998, Gender-based biases in occupational hiring decisions, Journal of Applied Social Psychology, 28, 1698, 10.1111/j.1559-1816.1998.tb01341.x · DOI
  45. He, 2019, Self-insight into emotional and cognitive abilities is not related to higher adjustment, Nature Human Behavior, 3, 867, 10.1038/s41562-019-0644-0 · DOI
  46. Jordan, 2016, Third-party punishment as a costly signal of trustworthiness, Nature, 530, 473, 10.1038/nature16981 · DOI · Paper page
  47. Katz, 1988, Racial ambivalence and American value conflict: Correlational and priming studies of dual cognitive structures, Journal of Personality and Social Psychology, 55, 893, 10.1037/0022-3514.55.6.893 · doi:10.1037/0022-3514.55.6.893 · Paper page
  48. Kepes, 2012, Publication bias in the organizational sciences, Organizational Research Methods, 15, 624, 10.1177/1094428112452760 · DOI · Paper page
  49. Kirkman, 2019, Is it just me, or was that sexist? The role of sexism type and perpetrator race in identifying sexism, The Journal of Social Psychology, 160, 1
  50. Kitayama, 2006, Voluntary settlement and the spirit of independence: Evidence from Japan’s “northern frontier”, Journal of Personality and Social Psychology, 91, 369, 10.1037/0022-3514.91.3.369 · DOI · Paper page
  51. Kluger, 2001, The error of accepting the “‘theoretical’” null hypothesis: The rise, fall, and resurrection of commonsense hypotheses in psychology, Psychological Bulletin, 127, 408, 10.1037/0033-2909.127.3.408 · doi:10.1037/0033-2909.127.3.408 · Paper page
  52. Köhler, T., & Cortina, J. M. (in press). Play it again, Sam! An analysis of constructive replication in the organizational sciences. Journal of Management.
  53. Krueger, 2004, Towards a balanced social psychology: Causes, consequences and cures for the problem-seeking approach to social behavior and cognition, Behavioral and Brain Sciences, 27, 313, 10.1017/S0140525X04000081 · DOI · Paper page
  54. Kunert, 2016, Internal conceptual replications do not increase independent replication success, Psychonomic Bulletin and Review, 23, 1631, 10.3758/s13423-016-1030-9 · DOI
  55. Kunst, 2019, Sexism, rape myths and feminist identification explain gender differences in attitudes toward the #metoo social media campaign in two countries, Media Psychology, 22, 818, 10.1080/15213269.2018.1532300 · DOI · Paper page
  56. Latham, 1988, Resolving scientific disputes by the joint design of crucial experiments by the antagonists – Application to the Erez-Latham dispute regarding participation in goal setting, Journal of Applied Psychology, 73, 753, 10.1037/0021-9010.73.4.753 · DOI · Paper page
  57. Leavitt, 2010, Theory pruning: Strategies for reducing our dense theoretical landscape, Organizational Research Methods, 13, 644, 10.1177/1094428109345156 · DOI · Paper page
  58. List, 2006, The behavioralist meets the market: Measuring social preferences and reputation effects in actual transactions, Journal of Political Economy, 114, 1, 10.1086/498587 · DOI · Paper page
  59. Lord, 1979, Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence, Journal of Personality and Social Psychology, 37, 2098, 10.1037/0022-3514.37.11.2098 · DOI · Paper page
  60. Luttrell, 2017, Replicating and fixing failed replications: The case of need for cognition and argument quality, Journal of Experimental Social Psychology, 69, 178, 10.1016/j.jesp.2016.09.006 · DOI
  61. Lykken, 1968, Statistical significance in psychological research, Psychological Bulletin, 70, 151, 10.1037/h0026141 · DOI · Paper page
  62. Makel, M. C., Hodges, J., Cook, B. G., & Plucker, J. A. (2019). Questionable and open research practices in education research. Unpublished manuscript. Available at: https://edarxiv.org/f7srb/. · DOI
  63. McCullough, 2006, Lessons from the JMCB archive, Journal of Money, Credit and Banking, 38, 1093, 10.1353/mcb.2006.0061 · DOI · Paper page
  64. Meehl, 1978, Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology, Journal of Consulting and Clinical Psychology, 46, 806, 10.1037/0022-006X.46.4.806 · DOI · Paper page
  65. Mitchell, 2001, Building better theory: Time and the specification of when things happen, Academy of Management Review, 26, 530, 10.2307/3560240 · DOI · Paper page
  66. Murphy, 2019, HARKing: How badly can cherry-picking and question trolling produce bias in published results?, Journal of Business and Psychology, 34, 1, 10.1007/s10869-017-9524-7 · DOI
  67. Olian, 1988, The impact of applicant gender compared to qualifications on hiring recommendations: A meta-analysis of experimental studies, Organizational Behavior and Human Decision Processes, 41, 180, 10.1016/0749-5978(88)90025-8 · DOI · Paper page
  68. Parboteeah, 2003, Social institutions and work centrality: Explorations beyond national culture, Organization Science, 14, 137, 10.1287/orsc.14.2.137.14989 · DOI · Paper page
  69. Payne, 2019, Historical roots of implicit bias in slavery, Proceedings of the National Academy of Sciences, 116, 11693, 10.1073/pnas.1818816116 · DOI · Paper page
  70. Perry, 1994, Explaining gender-based selection decisions: A synthesis of contextual and cognitive approaches, Academy of Management Review, 19, 786, 10.2307/258745 · DOI · Paper page
  71. Petty, R. E., & Cacioppo, J. T. (2016). Methodological choices have predictable consequences in replicating studies on motivation to think: Commentary on Ebersole et al. (2016). Journal of Experimental Social Psychology, 67, 86–87. · DOI
  72. Peteraf, 2013, The elephant in the room of dynamic capabilities: Bringing two diverging conversations together, Strategic Management Journal, 34, 1389, 10.1002/smj.2078 · DOI · Paper page
  73. Pitz, 1969, An inertia effect (resistance to change) in the revision of opinion, Canadian Journal of Psychology, 23, 24, 10.1037/h0082790 · DOI
  74. Porter, 1996, Forty years of organization studies: Reflections from a micro perspective, Administrative Science Quarterly, 41, 262, 10.2307/2393718 · DOI
  75. Ramagopalan, 2014, Prevalence of primary outcome changes in clinical trials registered on ClinicalTrials.gov: A cross-sectional study, F1000Research, 3, 77, 10.12688/f1000research.3784.1 · DOI · Paper page
  76. Reynolds, 2014, The role of moral knowledge in everyday immorality: What does it matter if I know what is right?, Organizational Behavior and Human Decision Processes, 123, 124, 10.1016/j.obhdp.2013.10.008 · DOI
  77. Scherbaum, 2009, Estimating statistical power and required sample sizes for organizational research using multilevel modeling, Organizational Research Methods, 12, 347, 10.1177/1094428107308906 · DOI · Paper page
  78. Schlaegel, 2014, Determinants of entrepreneurial intent: A meta-analytic test and integration of competing models, Entrepreneurship Theory and Practice, 38, 291, 10.1111/etap.12087 · DOI · Paper page
  79. Schumpeter, J. A. (1942/1994). Capitalism, socialism and democracy. London: Routledge. pp. 82–83.
  80. Schweinsberg, 2016, The pipeline project: Pre-publication independent replications of a single laboratory’s research pipeline, Journal of Experimental Social Psychology, 66, 55, 10.1016/j.jesp.2015.10.001 · DOI · Paper page
  81. Sherman, 2006, The psychology of self-defense: Self-affirmation theory, Vol. 38, 183
  82. Schwarz, 2014, Does merely going through the same moves make for a “direct” replication? Concepts, contexts, and operationalizations, Social Psychology, 45, 305
  83. Soklaridis, 2018, Men’s fear of mentoring in the #MeToo era – What’s at stake for academic medicine?, New England Journal of Medicine, 379, 2270, 10.1056/NEJMms1805743 · DOI
  84. Stewart, 1996, Fathers' work experiences effect children's behaviors via job-related affect and parenting behaviors, Journal of Organizational Behavior, 17, 221, 10.1002/(SICI)1099-1379(199605)17:3<221::AID-JOB741>3.0.CO;2-G · DOI
  85. Stephens, 2011, When choice does not equal freedom: A sociocultural analysis of agency in working-class American contexts, Social Psychological and Personality Science, 2, 33, 10.1177/1948550610378757 · DOI · Paper page
  86. Strack, 2016, Reflection on the smiling registered replication report, Perspectives on Psychological Science, 11, 929, 10.1177/1745691616674460 · DOI
  87. Tetlock, 1982, Attribution bias: On the inconclusiveness of the cognition-motivation debate, Journal of Experimental Social Psychology, 18, 68, 10.1016/0022-1031(82)90082-8 · DOI · Paper page
  88. Tetlock, 1985, Impression management versus intrapsychic explanations in social psychology: A useful dichotomy?, Psychological Review, 92, 59, 10.1037/0033-295X.92.1.59 · doi:10.1037/0033-295x.92.1.59 · Paper page
  89. Tierney, W., Hardy, J. H., III., Ebersole, C., Viganola, D., Clemente, E., Gordon, M., et al. (2020). A creative destruction approach to replication: Implicit work and sex morality across cultures. Registered Report proposal accepted in principle at the Journal of Experimental Social Psychology. · DOI · Paper page
  90. Trope, 1982, Confirmatory and diagnosing strategies in social information gathering, Journal of Personality and Social Psychology, 43, 22, 10.1037/0022-3514.43.1.22 · doi:10.1037/0022-3514.43.1.22 · Paper page
  91. Tsang, 1999, Replication and theory development in organizational science: A critical realist perspective, Academy of Management Review, 24, 759, 10.2307/259353 · DOI · Paper page
  92. Uhlmann, 2005, Constructed criteria: Redefining merit to justify discrimination, Psychological Science, 16, 474, 10.1111/j.0956-7976.2005.01559.x · DOI · Paper page
  93. Uhlmann, 2012, My culture made me do it: Lay theories of responsibility for automatic prejudice, Social Psychology, 43, 108, 10.1027/1864-9335/a000089 · DOI
  94. Uhlmann, 2011, Implicit Puritanism in American moral cognition, Journal of Experimental Social Psychology, 47, 312, 10.1016/j.jesp.2010.10.013 · DOI
  95. Van de Ven, 2006, Knowledge for theory and practice, Academy of Management Review, 31, 802, 10.5465/amr.2006.22527385 · DOI · Paper page
  96. Vancouver, 2018, Translating informal theories into formal theories: The case of the dynamic computational model of the integrated model of work motivation, Organizational Research Methods, 23, 238, 10.1177/1094428118780308 · DOI
  97. Van den Akker, O., Weston, S. J., Campbell, L., Chopik, W. J., Damian, R. I., Davis-Kean, P., et al., (2019). Preregistration of secondary data analysis: A template and tutorial. Unpublished manuscript. · DOI
  98. Vandello, 1999, Patterns of individualism and collectivism across the United States, Journal of Personality and Social Psychology, 77, 279, 10.1037/0022-3514.77.2.279 · DOI · Paper page
  99. Zwaan, 2017, Making replication mainstream, Behavioral & Brain Sciences., 41, e120, 10.1017/S0140525X17001972 · DOI · Paper page

Cited by

See all 66

Metadata

Authors
Tierney, W., Hardy, J. H., Ebersole, C. R., Leavitt, K., Viganola, D., Clemente, E. G., Gordon, M., Dreber, A., Johannesson, M., Pfeiffer, T., & Uhlmann, E. L.
Venue
Organizational Behavior and Human Decision Processes
Publisher
Elsevier BV
Published
2020-11
Volume
161
Pages
291-309
ISSN
0749-5978 (ISSN-L)
OpenAlex
W3091293017
Cited by
66
References
48
FWCI
7.33
Type
journal article
Language
en
Open access
hybrid
Licence
Funders
Jan Wallanders och Tom Hedelius Stiftelse samt Tore Browaldhs Stiftelse

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Creative destruction in science. https://app.cochat.ai/papers/10.1016/j.obhdp.2020.07.002
The paper
Tierney, W., Hardy, J. H., Ebersole, C. R., Leavitt, K., Viganola, D., Clemente, E. G., Gordon, M., Dreber, A., Johannesson, M., Pfeiffer, T., & Uhlmann, E. L. (2020). Creative destruction in science. Organizational Behavior and Human Decision Processes. https://doi.org/10.1016/j.obhdp.2020.07.002

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.