Do Student Samples Provide an Accurate Estimate of the General Public?
Open accessAbstract
Most psychological studies rely on student samples. Students are usually considered as more homogenous than representative samples both within and across countries. However, little is known about the nature of the differences between student and representative samples. This is an important gap, also because knowledge about the degree of difference between student and representative samples may allow to infer from the former to the latter group. Across 59 countries and 12 personality (Big-5) and attitudinal variables we found that differences between students and general public were partly substantial, incoherent, and contradicted previous findings. Two often used cultural variables, embeddedness and intellectual autonomy, failed to explain the differences between both groups across countries. We further found that students vary as much as the general population both between and within countries. In summary, our results indicate that generalizing from students to the general public can be problematic when personal and attitudinal variables are used, as students vary mostly randomly from the general public. Findings are also discussed in terms of the replication crisis within psychology.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most of this research depends on student samples, most commonly psychology student samples.
“A further comment, which is true for much psychology literature (e.g., Hanel & Vione, 2016), is that most of this research depends on student samples, most commonly psychology student samples.”
From Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review (2025) · cites this paper as Hanel & Vione, 2016
The Introduction says student samples are extremely common in psychological studies, supporting the general point about psychology research. The Discussion (Implications and limitations) mentions psychology students but says the authors had no data on students’ subjects; it does not establish that psychology students are the most common student sample. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Schwartz, S. H. (1992) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60281-6 · Cited by 14,619 · cited 3× · see citing works
“This was done to correct for individual scale use tendencies and to investigate the relative importance of CVOs [10,16].”
BackgroundThe weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641 · see citing works
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · see citing works
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307 · cited 2× · see citing works
“Nevertheless, other psychometric qualities such as convergent validity or test-retest reliabilities were found to be good [20].”
BackgroundMethodPower failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · see citing works
Weird people, yes, but also weird experiments
Henrich, J., Heine, S., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x10000038 · Cited by 6,808 · cited 1× · see citing works
“, [5,7]), at least whilst we do not know what predicts those differences.”
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
WEIRD languages have misled us, too
Majid, A., & Levinson, S. C. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1000018x · Cited by 4,169 · see citing works
Differences Between Tight and Loose Cultures: A 33-Nation Study
Gelfand, M. J., Raver, J. L., Nishii, L., Leslie, L. M., Lun, J., Lim, B. C., Duan, L., Almaliach, A., Ang, S., Arnadottir, J., et al. (2011) · Science · doi:10.1126/science.1197754 · Cited by 3,252 · cited 1× · see citing works
“We have further added the tightness scores [24] as an independent variables for 20 countries.”
BackgroundAdjusting for multiple testing—when and how?
Bender, R., & Lange, S. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00314-0 · Cited by 3,059 · cited 1× · see citing works
“, differences between both sample types for country X) should be tested in further confirmatory studies [25].”
BackgroundA Theory of Cultural Value Orientations: Explication and Applications
Schwartz, S. (2006) · Comparative Sociology · doi:10.1163/156913306778667357 · Cited by 2,150 · cited 11× · see citing works
“Both are part of Schwartz’ [10] cultural value orientation model (CVO).”
BackgroundMethodResultInfluentialSears, D. O. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.3.515 · Cited by 2,138 · cited 1× · see citing works
“, [5,7]), at least whilst we do not know what predicts those differences.”
BackgroundUsing Bayes to get the most out of non-significant results
Dienes, Z. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00781 · Cited by 1,998 · cited 1× · see citing works
“For this, we computed the Bayes factor (BF) for all 537 student-general public comparison, as the classical frequentist approach does not allow to address the question whether the data supports the null hypothesis [26].”
MethodHARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · see citing works
Factors predicting the subjective well-being of nations.
Diener, E., Diener, M. L., & Diener, C. (1995) · Journal of Personality and Social Psychology · doi:10.1007/978-90-481-2352-0_3 · Cited by 1,932 · cited 2× · see citing works
“In cross-cultural research for example, students are thought to reduce the variability in the sample due to differences in education levels [2], and were found to give a moderately good estimate of representative or teachers samples [3,4].”
BackgroundThe neglected 95%: why American psychology needs to become less American.
Arnett, J. (2008) · American Psychologist · doi:10.1037/0003-066x.63.7.602 · Cited by 1,656 · cited 4× · see citing works
“10) nor the seven attitudinal variables (all Fs[1, 116]< 1.”
BackgroundResultInfluentialPeterson, R. A. (2001) · Journal of Consumer Research · doi:10.1086/323732 · Cited by 1,258 · cited 3× · see citing works
“However, consistent with previous findings [9], we were neither able to explain the difference between students and the general public nor the variability of students across countries.”
BackgroundResultInvestigating Variation in Replicability: A “Many Labs” Replication Project
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., et al. (2014) · Social Psychology · doi:10.1027/1864-9335/a000178 · Cited by 1,208 · see citing works
Development of personality in early and middle adulthood: Set like plaster or persistent change?
Srivastava, S., John, O. P., Gosling, S. D., & Potter, J. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.84.5.1041 · Cited by 1,202 · cited 3× · see citing works
“For example, a large cross-sectional study found systematic personality (Big-5) differences between younger and older participants [11].”
BackgroundResultInfluentialde Pietri-Tonelli, A. (2006) · doi:10.1007/978-1-349-13322-2_2 · Cited by 946
Convenience samples of college students and research reproducibility
Peterson, R. A., & Merunka, D. R. (2014) · Journal of Business Research · doi:10.1016/j.jbusres.2013.08.010 · Cited by 567 · see citing works
Students as Experimental Participants: A Defense of the "Narrow Data Base" (WP-09-05)Preprint
Druckman, J., & Kam, C. (2009) · SSRN Electronic Journal · doi:10.2139/ssrn.1498843 · Cited by 509 · cited 1× · see citing works
“This challenges the claim that students are a moderately accurate estimate for the representative sample [3,28].”
BackgroundContextual sensitivity in scientific reproducibility
Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1521897113 · Cited by 398 · cited 1× · see citing works
“However, this reproducibility project was criticized for not taking contextual differences into account, as some studies were replicated in another country, which may have partly caused the low reproducibility rate [13].”
BackgroundBardi, A., Buchanan, K., Goodwin, R., Slabu, L., & Robinson, M. (2014) · Journal of Personality and Social Psychology · doi:10.1037/a0034818 · Cited by 267 · cited 1× · see citing works
“For example, it was found that psychology students increasingly value benevolence over the course of three years, whereas business students increasingly value achievement and power [29].”
BackgroundDo cultural values predict individuals' moral attitudes? A cross-cultural multilevel approach
Vauclair, C., & Fischer, R. (2011) · European Journal of Social Psychology · doi:10.1002/ejsp.794 · Cited by 149 · cited 1× · see citing works
“[19]): attitudes towards liberal personal-sexual behaviors (7 items, α = .”
BackgroundCross-Cultural Differences in a Global “Survey of World Views”
Saucier, G., Kenner, J., Iurino, K., Malham, P. B., Chen, Z., Thalmayer, A., Kemmelmeier, M., Tov, W., Boutti, R., Metaferia, H., et al. (2015) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022114551791 · Cited by 128 · cited 1× · see citing works
“In cross-cultural research for example, students are thought to reduce the variability in the sample due to differences in education levels [2], and were found to give a moderately good estimate of representative or teachers samples [3,4].”
BackgroundThe Connection Between Varying Treatment Effects and the Crisis of Unreplicable Research
Gelman, A. (2015) · Journal of Management · doi:10.1177/0149206314525208 · Cited by 125 · cited 1× · see citing works
“This is in line with calls for studies to explore whether effects vary across contexts and if so, how [14].”
ResultSociodemographic Differences Within Countries Produce Variable Cultural Values
Greenfield, P. (2014) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022113513402 · Cited by 46 · cited 2× · see citing works
“For example, Greenfield [8] argued that differences between socioeconomic groups are larger than differences between countries.”
BackgroundResult
Flagged first, then by citations
7 of the 36 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- [4]Fischer R, Schwartz S. Whence differences in value priorities? Individual, cultural, or artifactual sources. J Cross-Cult Psychol. 2011; 42: 1127– 1144.
- [15]Klein RA, Ratliff KA, Vianello M, Adams RB, Bahník Š, Bernstein MJ, et al Investigating variation in replicability. Soc Psychol. 2014; 45: 142– 152.
- [17]Schwartz SH. The 7 Schwartz cultural value orientation scores for 80 countries [Internet]. Dataset; 2008. https://www.researchgate.net/publication/304715744_The_7_Schwartz_cultural_value_orientation_scores_for_80_countries
- [28]Druckman JN, Kam CD. Students as Experimental Participants: A Defense of the “Narrow Data Base.” In: Druckman JN, Green DP, Kuklinski JH, Lupia A, editors. Cambridge Handbook of Experimental Political Science. New York: CUP; 2011 pp. 41– 57.
Cited by
Sorted by citations
Lattie, E. G., Adkins, E. C., Winquist, N., Stiles-Shields, C., Wafford, Q. E., & Graham, A. K. (2019) · Journal of Medical Internet Research · doi:10.2196/12869 · Cited by 843 · see citing works
“College students are often used as a convenience sample for psychological research [139].”
Comprehensive meta-analysis of resilience interventions.
Liu, J., Ein, N., Gervasio, J., Battaion, M., Reed, M., & Vickers, K. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101919 · Cited by 204 · see citing works
Insecure Attachment and Relationship Satisfaction: A Meta-Analysis of Actor and Partner Associations
Candel, O.-S., & Turliuc, M. N. (2019) · Personality and Individual Differences · doi:10.1016/j.paid.2019.04.037 · Cited by 186 · see citing works
Self-compassion, trauma, and posttraumatic stress disorder: a systematic review
Winders, S., Murphy, O., Looney, K., & O'Reilly, G. (2020) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2429 · Cited by 168 · see citing works
“Overall, a large portion of the studies utilized college or general population samples, which can be problematic when generalizing findings (Hanel & Vione, 2016).”
BackgroundStress management interventions for college students: A systematic review and meta-analysis.
Amanvermez, Y., Rahmadiana, M., Karyotaki, E., de Wit, L. D., Ebert, D., Kessler, R., & Cuijpers, P. (2020) · Clinical Psychology Science and Practice · doi:10.1111/cpsp.12342 · Cited by 163 · see citing works
Restorativeness in Natural and Urban Environments: A Meta-Analysis
Menardo, E., Brondino, M., Hall, R., & Pasini, M. (2019) · Psychological Reports · doi:10.1177/0033294119884063 · Cited by 149 · see citing works
“Moreover, the extensive use of university students has often been criticized (e.g., Hanel & Vione, 2016).”
BackgroundBrowning, M., Saeidi-Rizi, F., McAnirlin, O., Yoon, H., & Pei, Y. (2020) · Environment and Behavior · doi:10.1177/0013916520906481 · Cited by 145 · see citing works
“These samples may have been used because they were accessible and incurred low recruitment costs (Hanel & Vione, 2016).”
BackgroundJenkins, E. L., Ilicic, J., Barklamb, A. M., & McCaffrey, T. (2020) · Journal of Medical Internet Research · doi:10.2196/17296 · Cited by 119 · see citing works
Ali, F., Ciftci, O., Nanu, L., Çobanoğlu, C., & Ryu, K. (2020) · Cornell Hospitality Quarterly · doi:10.1177/1938965520943094 · Cited by 118 · see citing works
“It is noteworthy that many concerns are raised in using student samples for research studies regarding issues of representativeness, generalizability, and comparability of results (Hanel & Vione, 2016).”
BackgroundCross-Cultural Differences and Similarities in Human Value Instantiation
Hanel, P., Maio, G., Soares, A. K. S., Vione, K. C., Coelho, G. L. D. H., Gouveia, V., Patil, A. C., Kamble, S., & Manstead, A. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00849 · Cited by 111 · see citing works
Self-citationHeissel, A., Pietrek, A., Flunger, B., Fydrich, T., Rapp, M. A., Heinzel, S., & Vansteenkiste, M. (2018) · European Journal of Health Psychology · doi:10.1027/2512-8442/a000017 · Cited by 106 · see citing works
Methods for Heart Rate Variability Biofeedback (HRVB): A Systematic Review and Guidelines
Lalanza, J. F., Lorente, S., Bullich, R., García, C., Losilla, J.-M., & Capdevila, L. (2023) · Applied Psychophysiology and Biofeedback · doi:10.1007/s10484-023-09582-6 · Cited by 102 · see citing works
“The use of university student participants is a common research practice, but could hamper the generalization of the results to the general public (Hanel & Vione, 2016).”
BackgroundEnglish, M. C. W., Gignac, G., Visser, T., Whitehouse, A., & Maybery, M. (2019) · Autism Research · doi:10.1002/aur.2198 · Cited by 81 · see citing works
Argyriou, A., Goldsmith, K., & Rimes, K. (2021) · Archives of Sexual Behavior · doi:10.1007/s10508-020-01862-0 · Cited by 78 · see citing works
“For example, the wide use of university samples by the studies, while common in psychology research, has been argued to be problematic as university students tend to have higher socioeconomic status and are largely homogenous (Hanel & Vione, 2016).”
BackgroundMethodPrevalence and Impact of Dysmenorrhea Among University Students in Ireland
Durand, H., Monahan, K., & McGuire, B. (2021) · Pain medicine (Malden, Mass.) · doi:10.1093/pm/pnab122 · Cited by 76 · see citing works
“Furthermore, students may not be representative of young adults in the community [54] as well as adolescent schoolchildren who also experience PD. PD may affect these groups in different ways, for example, in terms of their development or the economic impact of missing work.”
BackgroundMascareño, J., Rietzschel, E. F., & Wisse, B. M. (2019) · Creativity and Innovation Management · doi:10.1111/caim.12341 · Cited by 89 · see citing works
“…was a student sample, which may raise concerns with respect to representativeness and generalizability (for instance because students are a relatively homogeneous group of well educated, young people; see Hanel & Vione, 2016; Peterson & Merunka, 2014; however, see also Rietzschel et al., 2017).”
Martinelli, A., Grüll, J., & Baum, C. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104180 · Cited by 79 · see citing works
Blake, H., Knight, H., Jia, R., Corner, J., Morling, J., Denning, C., Ball, J., Bolton, K., Figueredo, G., Morris, D. E., et al. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18084182 · Cited by 73 · see citing works
“These data relate to the views of students in a higher education setting, which might vary from that of the general public in terms of personal and attitudinal variables [69].”
BackgroundLi, H., Ding, Y., Zhao, B., Xu, Y., & Wei, W. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1058177 · Cited by 71 · see citing works
Problems with the Big Five assessment in the World Values Survey
Ludeke, S. G., & Larsen, E. (2017) · Personality and Individual Differences · doi:10.1016/j.paid.2017.02.042 · Cited by 68 · see citing works
Wild, H. A., Kyröläinen, A.-J., & Kuperman, V. (2022) · PLoS ONE · doi:10.1371/journal.pone.0271191 · Cited by 72 · see citing works
Baldissarri, C., & Andrighetto, L. (2021) · Human Performance · doi:10.1080/08959285.2021.1878182 · Cited by 70 · see citing works
“The use of this kind of sample has been widely criticized (e.g., Hanel & Vione, 2016; Henrich, Heine, & Norenzayan, 2010), so it is important that future research replicate our findings among real workers.”
BackgroundThe effect of brand identity-cognitive style fit and social influence on consumer-based brand equity
Alvarado-Karste, D., & Guzmán, F. (2020) · Journal of Product & Brand Management · doi:10.1108/jpbm-06-2019-2419 · Cited by 65 · see citing works
Butavicius, M., Taib, R., & Han, S. (2022) · Computers & security · doi:10.1016/j.cose.2022.102937 · Cited by 60 · see citing works
Burnay, J., Kepes, S., & Bushman, B. (2021) · Aggressive Behavior · doi:10.1002/ab.21998 · Cited by 57 · see citing works
Fostering ocean empathy through future scenarios
Blythe, J., Baird, J., Bennett, N. J., Dale, G., Nash, K. L., Pickering, G. J., & Wabnitz, C. (2021) · People and Nature · doi:10.1002/pan3.10253 · Cited by 57 · see citing works
Creative destruction in science
Tierney, W., Hardy, J. H., Ebersole, C. R., Leavitt, K., Viganola, D., Clemente, E. G., Gordon, M., Dreber, A., Johannesson, M., Pfeiffer, T., et al. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.07.002 · Cited by 64 · see citing works
Mapping the implications and competencies for Industry 4.0 to hard and soft total quality management
Babatunde, O. (2020) · The TQM Journal · doi:10.1108/tqm-07-2020-0158 · Cited by 57 · see citing works
Quantitative Data From Rating Scales: An Epistemological and Methodological Enquiry
Uher, J. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02599 · Cited by 55 · see citing works
“On widely used Big Five personality scales, differences between student and general public samples varied substantially and randomly across 59 countries, showing that, contrary to common assumptions, student findings cannot be generalized (Hanel and Vione, 2016).”
BackgroundLow-carbon travel mode choices: The role of time perceptions and familiarity
Dällenbach, N. (2020) · Transportation Research Part D Transport and Environment · doi:10.1016/j.trd.2020.102378 · Cited by 53 · see citing works
Understanding the Representation and Representativeness of Age in AI Data Sets
Park, J., Bernstein, M. S., Brewer, R. N., Kamar, E., & Morris, M. (2021) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462590 · Cited by 51 · see citing works
“…is highly reminiscent of many psychology studies that recruited from the undergraduate population, a field which now has growing concerns for the generalizability of research findings towards the larger population outside of a university (Hanel and Vione 2016; Henrich, Heine, and Norenzayan 2010).”
BackgroundResultThe relationship between rumination and NSSI: A systematic review and meta-analysis.
Coleman, S. E., Dunlop, B., Hartley, S., & Taylor, P. (2021) · British Journal of Clinical Psychology · doi:10.1111/bjc.12350 · Cited by 50 · see citing works
Willroth, E. C., Flett, J. A. M., & Mauss, I. (2019) · Journal of Personality · doi:10.1111/jopy.12475 · Cited by 50 · see citing works
“This is important given that the majority of previous studies of the association between depressive symptoms and emotion differentiation in nonclinical populations have used student samples and findings from student samples often do not generalize to the general population (Hanel & Vione, 2016).”
Effect of Acute Cardiovascular Exercise on Cerebral Blood Flow: A Systematic Review
Mulser, L., & Moreau, D. (2022) · medRxiv · doi:10.1101/2022.08.14.22278765 · Cited by 48
Weigold, A., & Weigold, I. K. (2021) · Social science computer review · doi:10.1177/08944393211006847 · Cited by 48 · see citing works
Newson, M., Buhrmester, M. D., Xygalatas, D., & Whitehouse, H. (2020) · Journal for the Cognitive Science of Religion · doi:10.1558/jcsr.38413 · Cited by 48 · see citing works
“Across fifty-nine countries, student samples have been found to be problematic when generalizing to broader populations – both within and between populations (Hanel et al. 2016), an ongoing problem (Sears 1986).”
BackgroundHow university blue and green space affect students’ mental health: A scoping review
Aghabozorgi, K., van der Jagt, A., Bell, S., & Smith, H. C. (2024) · Urban forestry & urban greening · doi:10.1016/j.ufug.2024.128394 · Cited by 52 · see citing works
What is Cyberstalking? A Review of Measurements
Wilson, C., Sheridan, L., & Garratt-Reed, D. (2021) · Journal of Interpersonal Violence · doi:10.1177/0886260520985489 · Cited by 46 · see citing works
“In research conducted by Hanel and Vione (2016), these variables were found to significantly differ between general and university samples.”
BackgroundTaras, V., Steel, P. D. G., & Stackhouse, M. R. D. (2023) · Journal of World Business · doi:10.1016/j.jwb.2022.101386 · Cited by 45 · see citing works
Lazarov, A., Basel, D., Dolan, S. C., Dillon, D. G., Pizzagalli, D. A., & Schneier, F. R. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.04.063 · Cited by 45 · see citing works
“This is important as student participants, especially those scoring extremely low on different measures of psychopathology, may be inherently different from the general population, thereby driving some of the observed group differences (Hanel and Vione, 2016).”
BackgroundNo matter what you do, I still love you: an examination of consumer reaction to brand transgressions
Kennedy, E. B., & Guzmán, F. (2020) · Journal of Product & Brand Management · doi:10.1108/jpbm-07-2019-2450 · Cited by 45 · see citing works
Epistemological and Ethical Aspects of Research in the Social Sciences
Dettweiler, U., Hanfstingl, B., & Schröter, H. (2020) · Frontiers Research Topics · doi:10.3389/978-2-88963-688-4 · Cited by 44
Verčič, A. T., Ćorić, D. S., & Vokić, N. P. (2021) · doi:10.1108/ccij-01-2021-0006 · Cited by 43
Binfet, J., Green, F. L. L., & Draper, Z. A. (2021) · Anthrozoos · doi:10.1080/08927936.2021.1944558 · Cited by 41 · see citing works
“University students are certainly a well-studied population (Hanel & Vione, 2016; Peterson & Merunka, 2014) and complexities in studying this group for the present investigation include the role incentives might play in influencing participation in studies and the role that self-report measures…”
Background“I’ll Worry About It Tomorrow” – Fostering Emotion Regulation Skills to Overcome Procrastination
Schuenemann, L., Scherenberg, V., von Salisch, M., & Eckert, M. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.780675 · Cited by 40 · see citing works
“Therefore, generalizing predictions about other populations should be considered with caution, as generalizations from students to the general public can bear difficulties (Hanel and Vione, 2016).”
BackgroundBurakova, M., & Schmitt, D. (2017) · Psihologijske teme · doi:10.31820/pt.26.1.5 · Cited by 40 · see citing works
“These factors may have affected the range of scores on many of our sexuality and personality measures in ways that limit the generalizability of our findings to general populations (Hanel & Vione, 2017).”
The Development of the Physical Activity and Social Support Scale
Golaszewski, N. M., & Bartholomew, J. (2019) · Journal of Sport & Exercise Psychology (JSEP) · doi:10.1123/jsep.2018-0234 · Cited by 39 · see citing works
“Furthermore, most scale development efforts are overly reliant on samples of young participants that undermine the ability to generalize the findings (Hanel & Vione, 2016).”
BackgroundThe Perception of Family, City, and Country Values Is Often Biased
Hanel, P., Wolfradt, U., Coelho, G. L. D. H., Wolf, L. J., Vilar, R., Monteiro, R. P., Gouveia, V., Crompton, T., & Maio, G. (2018) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022118767574 · Cited by 39 · see citing works
Self-citationCoelho, G. L. H., Vilar, R., Hanel, P. H. P., Monteiro, R. P., Ribeiro, M. G. C., & Gouveia, V. V. (2018) · Personality and Individual Differences · doi:10.1016/j.paid.2018.06.030 · Cited by 38 · see citing works
Self-citationA qualitative study of young peoples’ thoughts and attitudes to follow a more plant-based diet
McInnes, C., Carstairs, S. A., & Cecil, J. E. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1196142 · Cited by 42 · see citing works
King, A., Wong-Padoongpatt, G., Barrita, A. M., Phung, D., & Tong, T. (2020) · Issues in Mental Health Nursing · doi:10.1080/01612840.2020.1803461 · Cited by 38 · see citing works
The Generalizability of Personality Effects in Politics
Vitriol, J., Larsen, E., Ludeke, S. G., & Kandler, C. (2019) · European Journal of Personality · doi:10.1002/per.2222 · Cited by 37 · see citing works
Passive data collection on Reddit: a practical approach
Rocha-Silva, T., Nogueira, C., & Rodrigues, L. (2023) · Research Ethics · doi:10.1177/17470161231210542 · Cited by 36 · see citing works
“This strength is particularly relevant in the field of psychology since psychological research still tends to predominantly resort to college-based samples (Hanel and Vione, 2016).”
BackgroundSchlaegel, C., Gunkel, M., & Taras, V. (2022) · Journal of Organizational Behavior · doi:10.1002/job.2671 · Cited by 43 · see citing works
“Although the generalizability of findings based on student samples has been called into question (e.g., Bello et al., 2009; Shen et al., 2011), the main concerns about student samples only have limited relevance in the context of the present study (Hanel & Vione, 2016).”
BackgroundLi, H. (2021) · Personality and Individual Differences · doi:10.1016/j.paid.2021.110678 · Cited by 36 · see citing works
“We called that a behavior”: The making of institutional data
Whitman, M. (2020) · Big Data & Society · doi:10.1177/2053951720932200 · Cited by 44 · see citing works
Markova, V., Sandal, G. M., & Pallesen, S. (2020) · BMC Health Services Research · doi:10.1186/s12913-020-05478-x · Cited by 39 · see citing works
A review of experimental research on organizational trust
Schilke, O., Powell, A., & Schweitzer, M. (2023) · Journal of Trust Research · doi:10.1080/21515581.2023.2214202 · Cited by 34 · see citing works
“There is an ongoing argument regarding the extent to which data obtained from student participants is generalisable to wider populations (Hanel & Vione, 2016).”
BackgroundBasic Empathy Scale: A Systematic Review and Reliability Generalization Meta-Analysis
Cabedo-Peris, J., Vilar, M. M., Merino‐Soto, C., & Morán, M. O. (2021) · Healthcare · doi:10.3390/healthcare10010029 · Cited by 34 · see citing works
“Although student samples can be efficiently used to help with the measurement design [9], there is sufficient scientific literature which makes robust claims about the poor generalization of student samples [10], applied to several psychosocial constructs, such as empathy.”
BackgroundRange Restriction Affects Factor Analysis: Normality, Estimation, Fit, Loadings, and Reliability
Franco-Martínez, A., Alvarado, J. M., & Sorrel, M. (2021) · Educational and Psychological Measurement · doi:10.1177/00131644221081867 · Cited by 34 · see citing works
“The convenience sampling is broadly used in quantitative studies (Etikan et al., 2016) and, as its name indicates, accounts for the researcher’s accessibility in recruitment and administration (Hanel & Vione, 2016).”
BackgroundNudelman, G., & Otto, K. (2019) · British Journal of Psychology · doi:10.1111/bjop.12438 · Cited by 34 · see citing works
Anggraini, D., Glasauer, S., & Wunderlich, K. (2018) · Scientific Reports · doi:10.1038/s41598-018-28241-z · Cited by 34 · see citing works
Negative online reviews, brand equity and emotional contagion
Ahmad, F., & Guzmán, F. (2021) · European Journal of Marketing · doi:10.1108/ejm-10-2019-0820 · Cited by 33 · see citing works
Westgarth, M. M. P., Hogan, C., Neumann, D. L., & Shum, D. (2021) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsab017 · Cited by 40 · see citing works
Genç, E., Schlüter, C., Fraenz, C., Arning, L., Metzen, D., Nguyen, H., Voelkle, M., Streit, F., Güntürkün, O., Kumsta, R., et al. (2020) · Molecular Neurobiology · doi:10.1007/s12035-021-02398-7 · Cited by 33 · see citing works
Food Reward and Food Choice. An Inquiry Through The Liking and Wanting Model
Recio-Román, A., Recio-Menéndez, M., & Román-González, M. V. (2020) · Nutrients · doi:10.3390/nu12030639 · Cited by 33 · see citing works
“Generalizing from student’s samples to the general public can be problematic when personal and attitudinal variables are used [115].”
BackgroundEvaluating Reddit as a Crowdsourcing Platform for Psychology Research Projects
Luong, R., & Lomanowska, A. M. (2021) · Teaching of psychology · doi:10.1177/00986283211020739 · Cited by 32 · see citing works
“Unlike traditionally used university student samples that suffer from problems with generalizability and representativeness (Hanel & Vione, 2016), online recruitment offers greater demographic diversity and data quality (Buhrmester et al., 2011; Peer et al., 2017; Rouse, 2015; Sciutto, 2015).”
Stimulating aged brains with transcranial direct current stimulation: Opportunities and challenges.
Habich, A., Fehér, K., Antonenko, D., Boraxbekk, C., Flöel, A., Nissen, C., Siebner, H., Thielscher, A., & Klöppel, S. (2020) · Psychiatry research. Neuroimaging · doi:10.1016/j.pscychresns.2020.111179 · Cited by 32 · see citing works
“…al., Psychiatry Research: Neuroimaging, https://doi.org/10.1016/j.pscychresns.2020.111179 The generalizability and above all the transferability of findings between distinct populations (e.g. age groups) has been called into question with regard to many psychology studies (Hanel and Vione, 2016).”
BackgroundResultInfluentialLynch, L., Moorhead, A., Long, M., & Hawthorne-Steele, I. (2020) · Youth & society · doi:10.1177/0044118x20902786 · Cited by 32 · see citing works
“Academic journal publishing guidelines often recommend the exclusion of research that includes university students due to issues concerning generalizability and representativeness (Hanel & Vione, 2016).”
BackgroundExamining Cyberstalking Perpetration and Victimization: A Scoping Review
Wilson, C., Sheridan, L., & Garratt-Reed, D. (2022) · Trauma, Violence, & Abuse · doi:10.1177/15248380221082937 · Cited by 31 · see citing works
“Because many studies have focussed on university/college populations who may share culture and attitudes (Hanel & Vione, 2016), fi ndings cannot be assumed to re fl ect actual rates of these behaviours within entire populations.”
BackgroundThe Relationship Between Narcissism and Empathy : A Meta-Analytic Review
Simard, P., Simard, V., Laverdière, O., & Descôteaux, J. (2022) · Journal of Research in Personality · doi:10.1016/j.jrp.2022.104329 · Cited by 30 · see citing works
Ahmad, F., & Guzmán, F. (2021) · Journal of Consumer Marketing · doi:10.1108/jcm-09-2020-4119 · Cited by 30 · see citing works
Ethical guidelines for social justice in psychology.
Hailes, H. P., Ceccolini, C. J., Gutowski, E. R., & Liang, B. (2020) · Professional Psychology Research and Practice · doi:10.1037/pro0000291 · Cited by 46 · see citing works
“For instance, as researchers, psychologists often find that the most affordable and readily accessible research participants are college students, who are disproportionately White, middle class, and highly educated (Hanel & Vione, 2016; Henrich, Heine, & Norenzayan, 2010).”
Childhood adversity and mental health outcomes among university students: A longitudinal study
Bhattarai, A., King, N., Adhikari, K., Dimitropoulos, G., Devoe, D., Byun, J., Li, M., Rivera, D., Cunningham, S., Bulloch, A. G. M., et al. (2023) · The Canadian Journal of Psychiatry · doi:10.1177/07067437221111368 · Cited by 29 · see citing works
“Finally, the generalization of findings to other university student populations requires caution, especially when personal and attitudinal variables are analyzed.(60) Table 4.”
BackgroundRethinking fast and slow processing in autism.
Taylor, E., Farmer, G., Livingston, L. A., Callan, M. J., & Shah, P. (2022) · Journal of Psychopathology and Clinical Science · doi:10.1037/abn0000734 · Cited by 30 · see citing works
“Further, the samples were homogeneous, consisting largely of current or prospective university students and staff, which are unrepresentative of autistic or non-autistic populations (see also, Hanel & Vione, 2016).”
BackgroundFischer, C. (2021) · Review of Public Personnel Administration · doi:10.1177/0734371x20986839 · Cited by 29 · see citing works
“They conducted lab experiments with students, which are criticized for not being valid externally in several disciplines (Bello et al., 2009; Hanel & Vione, 2016; Henry, 2008; Peterson, 2001).”
BackgroundWang, N., Ren, F., & Zhou, X.-L. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01355 · Cited by 29 · see citing works
“We chose to recruit college students for two reasons: students were evidenced as heterogeneous as the community people (Hanel and Vione, 2016) and the facility of collecting test–retest data.”
BackgroundThe impact of customer engagement on customer happiness
de Azambuja, G. P., Peña, G. A. R., & Vargas, E. G. T. (2023) · Journal of Consumer Behaviour · doi:10.1002/cb.2147 · Cited by 28 · see citing works
Ong, W. Y., & Sündermann, O. (2022) · JMIR mhealth and uhealth · doi:10.2196/41800 · Cited by 32 · see citing works
“First, student participants may limit the generalizability of findings, as university students and the public may differ in factors such as level of education [91].”
BackgroundPrevalence and correlates of individuals with sexual interest in children: A systematic review.
Savoie, V., Quayle, E., & Flynn, E. (2021) · International Journal of Child Abuse & Neglect · doi:10.1016/j.chiabu.2021.105005 · Cited by 27 · see citing works
Brand equity, online reviews, and message trust: the moderating role of persuasion knowledge
Ahmad, F., & Guzmán, F. (2020) · Journal of Product & Brand Management · doi:10.1108/jpbm-09-2019-2564 · Cited by 27 · see citing works
The Minds of God(s) and Humans: Differences in Mind Perception in Fiji and North America
Willard, A., & McNamara, R. (2019) · Cognitive Sciences · doi:10.1111/cogs.12703 · Cited by 27 · see citing works
“Psychological subject pools have been claimed as problematic and often do not generalize to the population at large (Hanel & Vione, 2016), yet they are still used at a high level in psychological research.”
BackgroundUher, J. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1009893 · Cited by 30 · see citing works
“…questionnaires’ reliability and validity not only outside of Western, educated, industrialised, rich and democratic (WEIRD) populations but also their adequacy and predictive utility for studying individuals within Western countries (Hanel and Vione, 2016; Laajaj et al., 2019; Condon et al., 2021).”
BackgroundLekkas, D., Price, G. D., & Jacobson, N. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.107029 · Cited by 26 · see citing works
“Further, student self-reports on personal and attitudinal variables do not generalize well to the public (Hanel & Vione, 2016), which may be extended to the self-reported fatigue and boredom metrics considered in the present study.”
BackgroundAlbayay, J., Villarroel-Gruner, P., Bascour–Sandoval, C., Parma, V., & Galvéz-García, G. (2019) · Brain and Cognition · doi:10.1016/j.bandc.2019.103618 · Cited by 32 · see citing works
Why, when, and from whom: considerations for collecting and reporting race and ethnicity data in HCI
Chen, Y. T., Smith, A. D. R., Reinecke, K., & To, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581122 · Cited by 22 · see citing works
“dustrialized, Rich, and Democratic (WEIRD) samples [7, 45, 73, 90] and among college students [41, 46, 90].”
BackgroundThe effect of seeing scientists as intellectually humble on trust in scientists and their research
Koetke, J., Schumann, K., Bowes, S. M., & Vaupotič, N. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02060-x · Cited by 30 · see citing works
“While this does make them more diverse than typical university pools 70 , it also limits their generalizability (for example, all participants chose to opt in to academic research websites and therefore may have more positive attitudes towards science).”
Informal learning research in hospitality and tourism: A systematic literature review
Kodom-Wiredu, J. K., Coetzer, A., Redmond, J., & Sharafizad, J. (2022) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2022.05.011 · Cited by 24 · see citing works
Zinn, S., Landrock, U., & Gnambs, T. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01480-7 · Cited by 24 · see citing works
“I Just Go with It”: Negotiating Sexual Desire Discrepancies for Women in Partnered Relationships
Fahs, B., Swank, E., & Shambe, A. (2019) · Sex Roles · doi:10.1007/s11199-019-01098-w · Cited by 23 · see citing works
Howard, M. C., & O’Sullivan, R. (2024) · The Journal of Marketing Theory and Practice · doi:10.1080/10696679.2024.2385377 · Cited by 18 · see citing works
“Many methodologists have commented on the frequent inadequacy of research samples, as convenient sources may not always generalize to intended target populations (Hanel et al., 2016; Stevens, 2011).”
Uher, J. (2023) · Social & Personality Psychology Compass · doi:10.1111/spc3.12740 · Cited by 22 · see citing works
Miranda, A. R., Scotta, A. V., Méndez, A., Serra, S., & Soria, E. A. (2020) · Journal of preventive medicine and public health = Yebang Uihakhoe chi · doi:10.3961/jpmph.20.229 · Cited by 22 · see citing works
“However, further studies are needed to assess the reliability and validity of these instruments in other Latin American Spanish-speaking populations, since psychological research performed in students is not appropriate for generalizations to the general public [10].”
BackgroundArousal enhances herding tendencies when decision making
Coker, B. (2020) · Journal of Consumer Behaviour · doi:10.1002/cb.1811 · Cited by 22 · see citing works
Communication blackouts in power outages: Findings from scenario exercises in Germany and France
Mahdavian, F., Platt, S., Wiens, M., Klein, M., & Schultmann, F. (2020) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2020.101628 · Cited by 21 · see citing works
Castro, N., Curley, T., & Hertzog, C. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01454-9 · Cited by 21
“…variety of research that is reliant on category norms and continuing concern regarding the replicability of psychological research (Arnett, 2016; Hanel & Vione, 2016; Henrich, Heine, & Norenzayan, 2010), it is critical that future norms continue to make attempts Table 8 Stability of categories…”
Is intellectual humility related to more accuracy and less overconfidence?
Bowes, S. M., Ringwood, A., & Tasimi, A. (2023) · The Journal of Positive Psychology · doi:10.1080/17439760.2023.2208100 · Cited by 30 · see citing works
“…differences between community and college samples has been a challenging endeavor for decades (see Peterson, 2001); for instance, even cultural worldviews have been found not to explain differences between community and college samples in studies from dozens of countries (e.g. Hanel & Vione, 2016).”
BackgroundFerrar, J., Griffith, G. J., Skirrow, C., Cashdollar, N. M., Taptiklis, N., Dobson, J., Cree, F., Cormack, F. K., Barnett, J. H., & Munafo, M. R. (2021) · Journal of Medical Internet Research · doi:10.2196/26004 · Cited by 23 · see citing works
Culture and Sampling Issues With “Green” Attitude Research
Larson, R. B., & Kinsey, J. (2019) · Social Marketing Quarterly · doi:10.1177/1524500419838989 · Cited by 20 · see citing works
Hanel, P., Zarzeczna, N., & Haddock, G. (2018) · Social Psychology and Personality Science · doi:10.1177/1948550618803348 · Cited by 20
“This replicates Hanel and Vione (2016) who investigated whether student samples and the general public are more heterogeneous in more autonomous and less embedded countries (cf. Schwartz, 2006).”
BackgroundSelf-citation
- Authors
- Hanel, P. H. P., & Vione, K. C.
- Affiliations
- Cardiff University (GB)
- Corresponding author
- Hanel, Paul H. P.
- ORCID
- Hanel, Paul H. P.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-12-21
- Volume
- 11
- Issue
- 12
- Pages
- e0168354
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC5176168
- OpenAlex
- W2569236205
- Cited by
- 510
- References
- 36
- Citations by year
- 2026: 45 · 2025: 48 · 2024: 71 · 2023: 59 · 2022: 56 · 2021: 80 · 2020: 87 · 2019: 40 · 2018: 18 · 2017: 5 · 2013: 1
- FWCI
- 23.02
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- ORCA Online Research @Cardiff (Cardiff University) (acceptedVersion)
- Open Access at Essex (University of Essex) (acceptedVersion)
- University of Derby Online Research Archive. (University of Derby) (submittedVersion, cc-by)
- The University of Bath Online Publications Store (The University of Bath) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Pure (University of Bath) (submittedVersion, other-oa)
- Funders
- Economic and Social Research Council (GB) (ES/J500197/1)
- CAPES Foundation
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Social Psychology › Social Representations and Identity
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Keywords
- student samples, cross-cultural comparison, Big Five personality traits, attitudes, embeddedness, intellectual autonomy, sampling bias, replication crisis, cross-national comparison
- MeSH terms
- Attitude, Female, Humans, Male, Personality, Psychological Tests, Students, Knowledge, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
