The DOI matches its Crossref record; 31 of the 36 references it cites were found in a registry or an index.
Do Student Samples Provide an Accurate Estimate of the General Public?
Open accessAbstract
Most psychological studies rely on student samples. Students are usually considered as more homogenous than representative samples both within and across countries. However, little is known about the nature of the differences between student and representative samples. This is an important gap, also because knowledge about the degree of difference between student and representative samples may allow to infer from the former to the latter group. Across 59 countries and 12 personality (Big-5) and attitudinal variables we found that differences between students and general public were partly substantial, incoherent, and contradicted previous findings. Two often used cultural variables, embeddedness and intellectual autonomy, failed to explain the differences between both groups across countries. We further found that students vary as much as the general population both between and within countries. In summary, our results indicate that generalizing from students to the general public can be problematic when personal and attitudinal variables are used, as students vary mostly randomly from the general public. Findings are also discussed in terms of the replication crisis within psychology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A further comment, which is true for much psychology literature (e.g., Hanel & Vione, 2016), is that most of this research depends on student samples, most commonly psychology student samples.”
From Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review (2025) · cites this paper as Hanel & Vione, 2016
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Schwartz, S. H. (1992) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60281-6 · Cited by 14,619 · cited 3×
“This was done to correct for individual scale use tendencies and to investigate the relative importance of CVOs [10,16].”
BackgroundThe weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307 · cited 2×
“Nevertheless, other psychometric qualities such as convergent validity or test-retest reliabilities were found to be good [20].”
BackgroundMethodPower failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
WEIRD languages have misled us, too
Majid, A., & Levinson, S. C. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1000018x · Cited by 4,177
Differences Between Tight and Loose Cultures: A 33-Nation Study
Gelfand, M. J., Raver, J. L., Nishii, L., Leslie, L. M., Lun, J., Lim, B. C., Duan, L., Almaliach, A., Ang, S., Arnadottir, J., et al. (2011) · Science · doi:10.1126/science.1197754 · Cited by 3,252 · cited 1×
“We have further added the tightness scores [24] as an independent variables for 20 countries.”
BackgroundAdjusting for multiple testing—when and how?
Bender, R., & Lange, S. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00314-0 · Cited by 3,168 · cited 1×
“, differences between both sample types for country X) should be tested in further confirmatory studies [25].”
BackgroundThe neglected 95%: Why American psychology needs to become less American.
Arnett, J. J. (2008) · American Psychologist · doi:10.1037/0003-066x.63.7.602 · Cited by 2,178 · cited 4×
“10) nor the seven attitudinal variables (all Fs[1, 116]< 1.”
BackgroundResultInfluentialA Theory of Cultural Value Orientations: Explication and Applications
Schwartz, S. (2006) · Comparative Sociology · doi:10.1163/156913306778667357 · Cited by 2,150 · cited 11×
“Both are part of Schwartz’ [10] cultural value orientation model (CVO).”
BackgroundMethodResultInfluentialSears, D. O. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.3.515 · Cited by 2,111 · cited 1×
“, [5,7]), at least whilst we do not know what predicts those differences.”
BackgroundUsing Bayes to get the most out of non-significant results
Dienes, Z. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00781 · Cited by 1,998 · cited 1×
“For this, we computed the Bayes factor (BF) for all 537 student-general public comparison, as the classical frequentist approach does not allow to address the question whether the data supports the null hypothesis [26].”
MethodHARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Factors predicting the subjective well-being of nations.
Diener, E., Diener, E., & Diener, C. I. (1995) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.69.5.851 · Cited by 1,917
Development of personality in early and middle adulthood: Set like plaster or persistent change?
Srivastava, S., John, O. P., Gosling, S. D., & Potter, J. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.84.5.1041 · Cited by 1,290 · cited 3×
“For example, a large cross-sectional study found systematic personality (Big-5) differences between younger and older participants [11].”
BackgroundResultInfluentialPeterson, R. A. (2001) · Journal of Consumer Research · doi:10.1086/323732 · Cited by 1,258 · cited 3×
“However, consistent with previous findings [9], we were neither able to explain the difference between students and the general public nor the variability of students across countries.”
BackgroundResultInvestigating Variation in Replicability: A “Many Labs” Replication Project
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., et al. (2014) · Social Psychology · doi:10.1027/1864-9335/a000178 · Cited by 1,208
de Pietri-Tonelli, A. (2006) · doi:10.1007/978-1-349-13322-2_2 · Cited by 946
Convenience samples of college students and research reproducibility
Peterson, R. A., & Merunka, D. R. (2014) · Journal of Business Research · doi:10.1016/j.jbusres.2013.08.010 · Cited by 567
Students as Experimental Participants: A Defense of the "Narrow Data Base" (WP-09-05)Preprint
Druckman, J., & Kam, C. (2009) · SSRN Electronic Journal · doi:10.2139/ssrn.1498843 · Cited by 509 · cited 1×
“This challenges the claim that students are a moderately accurate estimate for the representative sample [3,28].”
BackgroundWhence Differences in Value Priorities?
Fischer, R. D., & Schwartz, S. H. (2010) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022110381429 · Cited by 455
Contextual sensitivity in scientific reproducibility
Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1521897113 · Cited by 398 · cited 1×
“However, this reproducibility project was criticized for not taking contextual differences into account, as some studies were replicated in another country, which may have partly caused the low reproducibility rate [13].”
BackgroundBardi, A., Buchanan, K. E., Goodwin, R. B., Slabu, L., & Robinson, M. (2013) · Journal of Personality and Social Psychology · doi:10.1037/a0034818 · Cited by 322 · cited 1×
“For example, it was found that psychology students increasingly value benevolence over the course of three years, whereas business students increasingly value achievement and power [29].”
BackgroundDo cultural values predict individuals' moral attitudes? A cross‐cultural multilevel approach
Vauclair, C., & Fischer, R. D. (2011) · European Journal of Social Psychology · doi:10.1002/ejsp.794 · Cited by 170 · cited 1×
“[19]): attitudes towards liberal personal-sexual behaviors (7 items, α = .”
BackgroundCross-Cultural Differences in a Global “Survey of World Views”
Saucier, G., Kenner, J., Iurino, K., Malham, P. B., Chen, Z., Thalmayer, A., Kemmelmeier, M., Tov, W., Boutti, R., Metaferia, H., et al. (2015) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022114551791 · Cited by 128 · cited 1×
“In cross-cultural research for example, students are thought to reduce the variability in the sample due to differences in education levels [2], and were found to give a moderately good estimate of representative or teachers samples [3,4].”
BackgroundThe Connection Between Varying Treatment Effects and the Crisis of Unreplicable Research
Gelman, A. (2015) · Journal of Management · doi:10.1177/0149206314525208 · Cited by 125 · cited 1×
“This is in line with calls for studies to explore whether effects vary across contexts and if so, how [14].”
ResultFactors Predicting the Subjective Well-Being of Nations
Diener, E., Diener, E., & Diener, C. I. (2009) · Social indicators research series · doi:10.1007/978-90-481-2352-0_3 · Cited by 123 · cited 2×
“In cross-cultural research for example, students are thought to reduce the variability in the sample due to differences in education levels [2], and were found to give a moderately good estimate of representative or teachers samples [3,4].”
BackgroundWeird people, yes, but also weird experiments
Baumard, N., & Sperber, D. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x10000038 · Cited by 71 · cited 1×
“, [5,7]), at least whilst we do not know what predicts those differences.”
Sociodemographic Differences Within Countries Produce Variable Cultural Values
Greenfield, P. M. (2013) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022113513402 · Cited by 61 · cited 2×
“For example, Greenfield [8] argued that differences between socioeconomic groups are larger than differences between countries.”
BackgroundResult
Flagged first, then by citations
5 of the 36 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Schwartz SH. The 7 Schwartz cultural value orientation scores for 80 countries [Internet]. Dataset; 2008. https://www.researchgate.net/publication/304715744_The_7_Schwartz_cultural_value_orientation_scores_for_80_countries
- JN Druckman, 2011, Cambridge Handbook of Experimental Political Science, 41
Cited by
Sorted by citations
Lattie, E. G., Adkins, E. C., Winquist, N., Stiles-Shields, C., Wafford, Q. E., & Graham, A. K. (2019) · Journal of Medical Internet Research · doi:10.2196/12869 · Cited by 843
“College students are often used as a convenience sample for psychological research [139].”
Comprehensive meta-analysis of resilience interventions
Liu, J. J. W., Ein, N., Gervasio, J., Battaion, M., Reed, M. J., & Vickers, K. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101919 · Cited by 259
Insecure Attachment and Relationship Satisfaction: A Meta-Analysis of Actor and Partner Associations
Candel, O.-S., & Turliuc, M. N. (2019) · Personality and Individual Differences · doi:10.1016/j.paid.2019.04.037 · Cited by 186
Self-compassion, trauma, and posttraumatic stress disorder: a systematic review
Winders, S., Murphy, O., Looney, K., & O'Reilly, G. (2020) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2429 · Cited by 168
“Overall, a large portion of the studies utilized college or general population samples, which can be problematic when generalizing findings (Hanel & Vione, 2016).”
BackgroundStress management interventions for college students: A systematic review and meta-analysis.
Amanvermez, Y., Rahmadiana, M., Karyotaki, E., de Wit, L. M., Ebert, D. D., Kessler, R. C., & Cuijpers, P. J. (2020) · Clinical Psychology Science and Practice · doi:10.1111/cpsp.12342 · Cited by 194
Metadata
- Authors
- Hanel, P. H. P., & Vione, K. C.
- Affiliations
- Cardiff University (GB)
- Corresponding author
- Hanel, Paul H. P.
- ORCID
- Hanel, Paul H. P.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-12-21
- Volume
- 11
- Issue
- 12
- Pages
- e0168354
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5176168
- OpenAlex
- W2569236205
- Cited by
- 510
- References
- 36
- Citations by year
- 2026: 45 · 2025: 48 · 2024: 71 · 2023: 59 · 2022: 56 · 2021: 80 · 2020: 87 · 2019: 40 · 2018: 18 · 2017: 5
- FWCI
- 23.02
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- ORCA Online Research @Cardiff (Cardiff University) (acceptedVersion)
- Open Access at Essex (University of Essex) (acceptedVersion)
- University of Derby Online Research Archive. (University of Derby) (submittedVersion, cc-by)
- The University of Bath Online Publications Store (The University of Bath) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Pure (University of Bath) (submittedVersion, other-oa)
- Funders
- Economic and Social Research Council (GB) (ES/J500197/1)
- CAPES Foundation
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Social Psychology › Social Representations and Identity
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Keywords
- student samples, cross-cultural comparison, Big Five personality traits, attitudes, embeddedness, intellectual autonomy, sampling bias, replication crisis, cross-national comparison
- MeSH terms
- Attitude, Female, Humans, Male, Personality, Psychological Tests, Students, Knowledge, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
