CoChat

A creative destruction approach to replication: Implicit work and sex morality across cultures

Open access
Tierney, Warren, Hardy, Jay, Ebersole, Charles R., Viganola, Domenico, Clemente, Elena Giulia, Gordon, Michael, Hoogeveen, Suzanne, Haaf, Julia, Dreber, Anna, Johannesson, Magnus, Pfeiffer, Thomas, Huang, Jason L., Vaughn, Leigh Ann, DeMarree, Kenneth, Igou, Eric R., Chapman, Hanah, Gantman, Ana, Vanaman, Matthew, Wylie, Jordan, Storbeck, Justin, Andreychik, Michael R., McPhetres, Jon, Uhlmann, Eric Luis · 2021 · Journal of Experimental Social Psychology
Cited by 51

The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.

Abstract

How can we maximize what is learned from a replication study? In the creative destruction approach to replication, the original hypothesis is compared not only to the null hypothesis, but also to predictions derived from multiple alternative theoretical accounts of the phenomenon. To this end, new populations and measures are included in the design in addition to the original ones, to help determine which theory best accounts for the results across multiple key outcomes and contexts. The present pre-registered empirical project compared the Implicit Puritanism account of intuitive work and sex morality to theories positing regional, religious, and social class differences; explicit rather than implicit cultural differences in values; self-expression vs. survival values as a key cultural fault line; the general moralization of work; and false positive effects. Contradicting Implicit Puritanism's core theoretical claim of a distinct American work morality, a number of targeted findings replicated across multiple comparison cultures, whereas several failed to replicate in all samples and were identified as likely false positives. No support emerged for theories predicting regional variability and specific individual-differences moderators (religious affiliation, religiosity, and education level). Overall, the results provide evidence that work is intuitively moralized across cultures.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.jesp.2020.104060

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 87

  1. Adigun, 1997, Orientations to work: A cross-cultural approach, Journal of Cross-Cultural Psychology, 28, 352, 10.1177/0022022197283009 · doi:10.1177/0022022197283009
  2. Aguinis, 2009, First decade of organizational research methods: Trends in design, measurement, and data-analysis topics, Organizational Research Methods, 11, 9, 10.1177/1094428106292896 · DOI · Paper page
  3. Aquino, 2009, Testing a social-cognitive model of moral behavior: the interactive influence of situations and moral identity centrality, Journal of Personality and Social Psychology, 97, 123, 10.1037/a0015406 · DOI · Paper page
  4. Arkes, 2010, A cross-cultural study of reference point adaptation: Evidence from China, Korea, and the US, Organizational Behavior and Human Decision Processes, 112, 99, 10.1016/j.obhdp.2010.02.002 · DOI
  5. Bargh, 2014, Our unconscious mind, Scientific American, 30, 30
  6. Berman, 2010, Investigator allegiance and the evaluation of psychotherapy outcome research, European Journal of Psychotherapy and Counselling, 12, 11, 10.1080/13642531003637775 · DOI
  7. Billingsley, 2018, Implicit and explicit influences of religious cognition on dictator game transfers, Royal Society Open Science, 5
  8. Bozarth, 1972, Signifying significant significance, American Psychologist, 27, 774, 10.1037/h0038034 · DOI · Paper page
  9. Brainerd, 2018, Replication, registration, and scientific creativity, Perspectives on Psychological Science, 13, 428, 10.1177/1745691617739421 · DOI
  10. Buhrmester, 2011, Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data?, Perspectives on Psychological Science, 6, 3, 10.1177/1745691610393980 · DOI · Paper page
  11. Caruso, 2017, Show me the money: A systematic exploration of manipulations, moderators, and mechanisms of priming effects, Psychological Science, 28, 1148, 10.1177/0956797617706161 · DOI
  12. Chartrand, 2007, Nonconscious relationship reactance: When significant others prime opposing goals, Journal of Experimental Social Psychology, 43, 719, 10.1016/j.jesp.2006.08.003 · DOI
  13. Cohen, 2016, Beyond east vs. west: Social class, region, and religion as forms of culture, Current Opinion in Psychology, 8, 5, 10.1016/j.copsyc.2015.09.006 · DOI
  14. Corney, 2005, A comparative analysis of the desirability of work characteristics: Chile versus the United States, International Journal of Management, 22, 159
  15. Cova, 2018, Estimating the reproducibility of experimental philosophy, Review of Philosophy and Psychology, 1
  16. Cuccolo, 2020, What crowdsourcing can offer to cross-cultural psychological science, Cross-Cultural Research
  17. DellaVigna, 2018, Predicting experimental results: Who knows what?, Journal of Political Economy, 126, 2410, 10.1086/699976 · DOI · Paper page
  18. Dunham, 2006, From American city to Japanese village: The omnipresence of implicit race attitudes, Child Development, 77, 1268, 10.1111/j.1467-8624.2006.00933.x · DOI · Paper page
  19. Dunham, 2008, The development of implicit intergroup cognition, Trends in Cognitive Science, 12, 248, 10.1016/j.tics.2008.04.006 · DOI · Paper page
  20. Dunham, 2016, The development of implicit gender attitudes, Developmental Science, 19, 781, 10.1111/desc.12321 · DOI
  21. Epstein, 1998, Cognitive-experiential self-theory: A dual process personality theory with implications for diagnosis and psychotherapy, Vol. 7, 99
  22. Fabrigar, 2020, A validity-based framework for understanding replication in psychology, Personality and Social Psychology Review, 10.1177/1088868320931366 · DOI
  23. Forsell, 2019, Predicting replication outcomes in the Many Labs 2 study, Journal of Economic Psychology, 75, 102, 10.1016/j.joep.2018.10.009 · DOI · Paper page
  24. Furnham, 1982, The Protestant work ethic and attitudes towards unemployment, Journal of Occupational Psychology, 55, 277, 10.1111/j.2044-8325.1982.tb00101.x · DOI · Paper page
  25. Furnham, 1993, A comparison of Protestant work ethic beliefs in thirteen nations, Journal of Social Psychology, 133, 185, 10.1080/00224545.1993.9712136 · DOI · Paper page
  26. Gawronski, 2006, Associative and propositional processes in evaluation: An integrative review of implicit and explicit attitude change, Psychological Bulletin, 132, 692, 10.1037/0033-2909.132.5.692 · DOI · Paper page
  27. Gelfand, 2002, Culture and egocentric perceptions of fairness in conflict and negotiation, Journal of Applied Psychology, 87, 833, 10.1037/0021-9010.87.5.833 · doi:10.1037/0021-9010.87.5.833 · Paper page
  28. Greenwald, 1995, Implicit social cognition: Attitudes, self- esteem, and stereotypes, Psychological Review, 102, 4, 10.1037/0033-295X.102.1.4 · DOI · Paper page
  29. Greenwald, 2002, A unified theory of implicit attitudes, beliefs, self-esteem and self-concept, Psychological Review, 109, 3, 10.1037/0033-295X.109.1.3 · DOI · Paper page
  30. Greenwald, 2003, Targets of discrimination: Effects of race on responses to weapons holders, Journal of Experimental Social Psychology, 39, 399, 10.1016/S0022-1031(03)00020-9 · DOI · Paper page
  31. Gregg, 2006, Easier done than undone: Asymmetry in the malleability of implicit preferences, Journal of Personality and Social Psychology, 90, 1, 10.1037/0022-3514.90.1.1 · DOI · Paper page
  32. Grossmann, 2011, Social class, culture, and cognition, Social Psychological and Personality Science, 2, 81, 10.1177/1948550610377119 · DOI · Paper page
  33. Haaf, 2017, Developing constraint in Bayesian mixed models, Psychological Methods, 22, 779, 10.1037/met0000156 · DOI
  34. Hambrick, 2007, The field of management’s devotion to theory: Too much of a good thing?, Academy of Management Journal, 50, 1346, 10.5465/amj.2007.28166119 · DOI · Paper page
  35. Harris, 2013, Two failures to replicate high-performance-goal priming effects, PLoS One, 8, 10.1371/journal.pone.0072467 · DOI · Paper page
  36. Henrich, 2010, The weirdest people in the world?, Behavioral & Brain Sciences, 33, 61, 10.1017/S0140525X0999152X · DOI · Paper page
  37. Hubbard, 1994, Replications and extensions in marketing: Rarely published but quite contrary, International Journal of Research in Marketing, 11, 233, 10.1016/0167-8116(94)90003-5 · DOI · Paper page
  38. Ioannidis, 2005, Why most published research findings are false, PLoS Medicine, 2, 124, 10.1371/journal.pmed.0020124 · DOI · Paper page
  39. Jordan, 2016, Third-party punishment as a costly signal of trustworthiness, Nature, 530, 473, 10.1038/nature16981 · DOI · Paper page
  40. Jussim, 1987, The nature of stereotypes: A comparison and integration of three theories, Journal of Personality and Social Psychology, 52, 536, 10.1037/0022-3514.52.3.536 · doi:10.1037/0022-3514.52.3.536 · Paper page
  41. Katz, 1988, Racial ambivalence and American value conflict: Correlational and priming studies of dual cognitive structures, Journal of Personality and Social Psychology, 55, 893, 10.1037/0022-3514.55.6.893 · doi:10.1037/0022-3514.55.6.893 · Paper page
  42. King, 2005, Perceptions of the mutual obligations between employees and employers: A comparative study of new generation IT professionals in China and the United States, International Journal of Human Resource Management, 16, 46, 10.1080/0958519042000295948 · DOI
  43. Kitayama, 2006, Voluntary settlement and the spirit of independence: Evidence from Japan’s “northern frontier”, Journal of Personality and Social Psychology, 91, 369, 10.1037/0022-3514.91.3.369 · DOI · Paper page
  44. Kluger, 2001, The error of accepting the “theoretical” null hypothesis: The rise, fall, and resurrection of commonsense hypotheses in psychology, Psychological Bulletin, 127, 408, 10.1037/0033-2909.127.3.408 · doi:10.1037/0033-2909.127.3.408 · Paper page
  45. Kohn, 1990, Position in the class structure and psychological functioning in the United States, Japan, and Poland, American Journal of Sociology, 95, 964, 10.1086/229382 · DOI · Paper page
  46. Kohn, 2002, Structural location and personality during the transformation of Poland and Ukraine, Social Psychology Quarterly, 65, 364, 10.2307/3090108 · DOI
  47. Leavitt, 2010, Theory pruning: Strategies for reducing our dense theoretical landscape, Organizational Research Methods, 13, 644, 10.1177/1094428109345156 · DOI · Paper page
  48. Leong, 2014, Protestant work ethic, Confucian values, and work-related attitudes in Singapore, Journal of Career Assessment, 22, 304, 10.1177/1069072713493985 · DOI
  49. Manzoli, 2014, Non-publication and delayed publication of randomized trials on vaccines: Survey, British Medical Journal, 348, 10.1136/bmj.g3058 · DOI · Paper page
  50. McCarthy, 2018, Registered replication report: Srull & Wyer (1979), Advances in Methods and Practices in Psychological Science, 1, 321, 10.1177/2515245918777487 · DOI · Paper page
  51. Milfont, 2018, Replication and reproducibility in cross-cultural psychology, Journal of Cross-Cultural Psychology, 49, 735, 10.1177/0022022117744892 · DOI
  52. Mirels, 1971, Protestant ethic as a personality variable, Journal of Consulting and Clinical Psychology, 36, 40, 10.1037/h0030477 · DOI · Paper page
  53. Moon, 2018, Religious targets are trusted because they are viewed as slow life-history strategists, Psychological Science, 29, 947, 10.1177/0956797617753606 · DOI
  54. Moshontz, 2018, The Psychological Science Accelerator: Advancing psychology through a distributed collaborative network, Advances in Methods and Practices in Psychological Science, 1, 501, 10.1177/2515245918797607 · DOI · Paper page
  55. Mynatta, 1977, Confirmation bias in a simulated research environment: An experimental study of scientific inference, Quarterly Journal of Experimental Psychology, 29, 85, 10.1080/00335557743000053 · DOI · Paper page
  56. Nisbett, 1977, Telling more than we can know: Verbal reports on mental processes, Psychological Review, 84, 231, 10.1037/0033-295X.84.3.231 · DOI · Paper page
  57. O’Donnell, 2018, Registered replication report: Dijksterhuis & van Knippenberg (1998), Perspectives on Psychological Science, 13, 268, 10.1177/1745691618755704 · DOI · Paper page
  58. Olsson-Collentine, 2020, Heterogeneity in direct replications in psychology and its association with effect size, Psychological Bulletin, 10.1037/bul0000294 · DOI · Paper page
  59. Oyserman, 2002, Rethinking individualism and collectivism: Evaluation of theoretical assumptions and meta-analyses, Psychological Bulletin, 128, 3, 10.1037/0033-2909.128.1.3 · DOI · Paper page
  60. Pashler, 2012, Priming of social distance? Failure to replicate effects on social and food judgements, PLoS One, 7, 10.1371/journal.pone.0042510 · DOI · Paper page
  61. Pashler, 2013, Can the goal of honesty be primed?, Journal of Experimental Social Psychology, 49, 959, 10.1016/j.jesp.2013.05.011 · DOI
  62. Poortinga, 1989, Equivalence of cross-cultural data: An overview of basic issues, International Journal of Psychology, 24, 737
  63. Rohrer, 2015, Do subtle reminders of money change people’s political views?, Journal of Experimental Psychology: General, 144, e73, 10.1037/xge0000058 · DOI · Paper page
  64. Rouder, 2019, Beyond overall effects: A Bayesian approach to finding constraints in meta-analysis, Psychological Methods, 24, 606, 10.1037/met0000216 · DOI
  65. Sanchez-Burks, 2002, Protestant relational ideology and (in) attention to relational cues in work settings, Journal of Personality and Social Psychology, 83, 919, 10.1037/0022-3514.83.4.919 · doi:10.1037/0022-3514.83.4.919 · Paper page
  66. Sanchez-Burks, 2005, Protestant relational ideology: The cognitive underpinnings and organizational implications of an American anomaly, Research in Organizational Behavior, 26, 265, 10.1016/S0191-3085(04)26007-5 · DOI
  67. Sanchez-Burks, 2007, Culture and workways, Vol. 1, 346
  68. Schein, 1990, Organizational culture, American Psychologist, 45, 109, 10.1037/0003-066X.45.2.109 · doi:10.1037/0003-066x.45.2.109 · Paper page
  69. Schwarz, 2014, Does merely going through the same moves make for a “direct” replication? Concepts, contexts, and operationalizations, Social Psychology, 45, 305
  70. Shariff, 2016, Religious priming: A meta-analysis with a focus on prosociality, Personality and Social Psychology Review, 20, 27, 10.1177/1088868314568811 · DOI · Paper page
  71. Sinclair, 2005, The relationship between parental racial attitudes and children’s automatic prejudice, Journal of Experimental Social Psychology, 3, 283, 10.1016/j.jesp.2004.06.003 · DOI · Paper page
  72. Sinclair, 2005, The social tuning of automatic ethnic attitudes: The role of affiliative motivation, Journal of Personality and Social Psychology, 89, 583, 10.1037/0022-3514.89.4.583 · DOI · Paper page
  73. Snir, 2009, Cross-cultural differences concerning heavy work investment, Cross Cultural Research, 43, 309, 10.1177/1069397109336988 · DOI
  74. Srull, 1979, The role of category accessibility in the interpretation of information about persons: Some determinants and implications, Journal of Personality and Social Psychology, 37, 1660, 10.1037/0022-3514.37.10.1660 · DOI · Paper page
  75. Stephens, 2011, When choice does not equal freedom: A sociocultural analysis of agency in working-class American contexts, Social Psychological and Personality Science, 2, 33, 10.1177/1948550610378757 · DOI · Paper page
  76. Thompson, 1978, Ancestral inference. II. The founders of Tristan da Cunha, Annals of Human Genetics, 42, 239, 10.1111/j.1469-1809.1978.tb00656.x · doi:10.1111/j.1469-1809.1978.tb00656.x
  77. Tierney, 2020, Creative destruction in science, Organizational Behavior and Human Decision Processes, 161, 291, 10.1016/j.obhdp.2020.07.002 · DOI · Paper page
  78. Uhlmann, 2013, Acting professional: An exploration of culturally bounded norms against non-work role referencing, Journal of Organizational Behavior, 34, 866, 10.1002/job.1874 · DOI
  79. Uhlmann, 2011, Implicit puritanism in American moral cognition, Journal of Experimental Social Psychology, 47, 312, 10.1016/j.jesp.2010.10.013 · DOI
  80. Van de Ven, 2006, Knowledge for theory and practice, Academy of Management Review, 31, 802, 10.5465/amr.2006.22527385 · DOI · Paper page
  81. van de Vijver, 2010, Equivalence and bias: A review of concepts, models, and data analytic procedures, 15
  82. van Elk, 2015, Meta-analyses are no substitute for registered replications: A skeptical perspective on religious priming, Frontiers in Psychology, 6, 1365, 10.3389/fpsyg.2015.01365 · DOI · Paper page
  83. Vandello, 1999, Patterns of individualism and collectivism across the United States, Journal of Personality and Social Psychology, 77, 279, 10.1037/0022-3514.77.2.279 · DOI · Paper page
  84. Van't Veer, 2016, Pre-registration in social psychology: A discussion and suggested template, Journal of Experimental Social Psychology, 67, 2, 10.1016/j.jesp.2016.03.004 · DOI · Paper page
  85. Varnum, 2012, Social class differences in N400 indicate differences in spontaneous trait inference, Journal of Experimental Psychology: General, 141, 518, 10.1037/a0026104 · DOI
  86. Weeden, 2013, What predicts religiosity? A multinational analysis of reproductive and cooperative morals, Evolution and Human Behavior, 340, 440, 10.1016/j.evolhumbehav.2013.08.006 · DOI
  87. Weingarten, 2016, On priming action: A meta-analysis of the behavioral effects of incidentally presented stimuli, Psychological Bulletin, 142, 472, 10.1037/bul0000030 · DOI · Paper page

Cited by

See all 51

Metadata

Authors
Tierney, W., Hardy, J., Ebersole, C. R., Viganola, D., Clemente, E. G., Gordon, M., Hoogeveen, S., Haaf, J., Dreber, A., Johannesson, M., Pfeiffer, T., Huang, J. L., Vaughn, L. A., DeMarree, K., Igou, E. R., Chapman, H., Gantman, A., Vanaman, M., Wylie, J., Storbeck, J., Andreychik, M. R., McPhetres, J., & Uhlmann, E. L.
Venue
Journal of Experimental Social Psychology
Publisher
Elsevier BV
Published
2021-03
Volume
93
Pages
104060
ISSN
0022-1031 (ISSN-L)
OpenAlex
W3095810629
Cited by
51
References
45
FWCI
10.05
Type
journal article
Language
en
Open access
hybrid
Licence
Funders
Jan Wallanders och Tom Hedelius Stiftelse samt Tore Browaldhs Stiftelse

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.