Generalizability is not optional: insights from a cross-cultural study of social discounting
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Current scientific reforms focus more on solutions to the problem of reliability (e.g. direct replications) than generalizability. Here, we use a cross-cultural study of social discounting to illustrate the utility of a complementary focus on generalizability across diverse human populations. Social discounting is the tendency to sacrifice more for socially close individuals—a phenomenon replicated across countries and laboratories. Yet, when adapting a typical protocol to low-literacy, resource-scarce settings in Bangladesh and Indonesia, we find no independent effect of social distance on generosity, despite still documenting this effect among US participants. Several reliability and validity checks suggest that methodological issues alone cannot explain this finding. These results illustrate why we must complement replication efforts with investment in strong checks on generalizability. By failing to do so, we risk developing theories of human nature that reliably explain behaviour among only a thin slice of humanity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Inclusion of Other in the Self Scale and the structure of interpersonal closeness.
Aron, A., Aron, E. N., & Smollan, D. (1992) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.63.4.596 · Cited by 5,157
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490
SIMR : an R package for power analysis of generalized linear mixed models by simulation
Green, P., & MacLeod, C. J. (2016) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12504 · Cited by 2,467
Sears, D. O. (1986) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.51.3.515 · Cited by 2,119
“Economic man” in cross-cultural perspective: behavioral experiments in 15 small-scale societies
Henrich, J., Boyd, R., Bowles, S., Camerer, C., Fehr, E., Gintis, H., McElreath, R., Alvard, M., Barr, A., Ensminger, J., et al. (2005) · Behavioral and Brain Sciences · doi:10.1017/s0140525x05000142 · Cited by 1,898
Costly Punishment Across Human Societies
Henrich, J., McElreath, R., Barr, A., Ensminger, J., Barrett, C., Bolyanatz, A., Cardenas, J. C., Gurven, M., Gwako, E., Henrich, N., et al. (2006) · Science · doi:10.1126/science.1127333 · Cited by 1,771 · cited 1×
“Another is to invest in large-scale collaborative projects (e.g. AHRC Culture and the Mind [70]) that investigate the same phenomenon in diverse contexts [71–73].”
Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., et al. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0399-z · Cited by 1,521
Cited by
Why Hypothesis Testers Should Spend Less Time Testing Hypotheses
Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620966795 · Cited by 382
Self-citationCrowdsourcing hypothesis tests: Making transparent how design choices shape research results.
Landy, J. F., Jia, M. (., Ding, I. L., Viganola, D., Tierney, W., Dreber, A., Johannesson, M., Pfeiffer, T., Ebersole, C. R., Gronau, Q. F., et al. (2020) · Psychological Bulletin · doi:10.1037/bul0000220 · Cited by 168
Generalization Bias in Science
Peters, U., Krauss, A., & Braganza, O. (2022) · Cognitive Science · doi:10.1111/cogs.13188 · Cited by 63
“Several researchers have argued that the replication crisis is closely connected to a “generalizability crisis” in the sciences (Redish, Kummerfeld, Morris, & Love, 2018; Tiokhin et al., 2019; Yarkoni, 2020).”
BackgroundMcDanal, R., Rubin, A., Fox, K. R., & Schleider, J. L. (2022) · Behavior Therapy · doi:10.1016/j.beth.2021.10.004 · Cited by 43
Parochial altruism: What it is and why it varies
Pisor, A. C., & Ross, C. T. (2024) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2023.06.005 · Cited by 38
Metadata
- Authors
- Tiokhin, L., Hackman, J., Munira, S., Jesmin, K., & Hruschka, D.
- Venue
- Royal Society Open Science
- Publisher
- The Royal Society
- Published
- 2019-02
- Volume
- 6
- Issue
- 2
- Pages
- 181386
- ISSN
- 2054-5703 (ISSN-L)
- OpenAlex
- W2915497009
- Cited by
- 57
- References
- 22
- FWCI
- 11.34
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generalizability is not optional: insights from a cross-cultural study of social discounting. https://app.cochat.ai/papers/10.1098/rsos.181386
- The paper
- Tiokhin, L., Hackman, J., Munira, S., Jesmin, K., & Hruschka, D. (2019). Generalizability is not optional: insights from a cross-cultural study of social discounting. Royal Society Open Science. https://doi.org/10.1098/rsos.181386
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.