Validity problems comparing values across cultures and possible solutions.
Abstract
The authors argue that commonly used ranking and rating methods of value surveys may have low validity in cross-cultural value comparisons because participants' reports about values can be affected by factors such as cultural differences in the meaning of particular value terms as well as the possibility that some value judg-ments are based on social comparison or deprivation rather than on any "direct reading " of personal preferences. Four different value survey methods—ranking, rating, attitude scale, and behavioral scenario methods—were compared. It was found that ranking and rating methods of assessing differences between Chinese and Americans had low convergence with each other and with the criterion of cultural experts ' independent judgments. Attitude questions had slight and nonsig-nificant convergence with the expert judgment criterion. A scenario method of value assessment, however, yielded reasonable criterion validity. Cross-cultural value comparison has long been the main tool for understanding similarities and differ-ences among people with different cultural and ethnic
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Human evaluations can have high variance and instability, which could be due to cultural and individual differences.
“even human evaluations can have high variance and instability, which could be due to cultural and individual differences [155].”
From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [155]
Cites
Culture and the self: Implications for cognition, emotion, and motivation.
Markus, H. R., & Kitayama, S. (1991) · Psychological Review · doi:10.1037/0033-295x.98.2.224 · Cited by 21,995 · see citing works
A theory of social comparison processes
Festinger, L. (1954) · Human Relations · doi:10.1177/001872675400700202 · Cited by 20,926 · see citing works
Culture's Consequences: International Differences in Work-Related Values.
Sorge, A. M., & Hofstede, G. (1983) · Administrative Science Quarterly · doi:10.2307/2393017 · Cited by 15,186 · see citing works
Schwartz, S. H. (1992) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60281-6 · Cited by 14,619 · see citing works
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195 · see citing works
(1973) · DigitalGeorgetown (Georgetown University Library) · Cited by 10,144 · see citing works
Essentials of Psychological Testing
Rosner, B., & Cronbach, L. J. (1960) · The American Journal of Psychology · doi:10.2307/1419921 · Cited by 4,522 · see citing works
Culture shift in advanced industrial society
(1990) · Choice Reviews Online · doi:10.5860/choice.28-1180 · Cited by 3,800 · see citing works
Toward a universal psychological structure of human values.
Schwartz, S. H., & Bilsky, W. (1987) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.53.3.550 · Cited by 3,617 · see citing works
International Encyclopedia of the Social Sciences.
Duncan, O. D., & Sills, D. L. (1968) · American Sociological Review · doi:10.2307/2092889 · Cited by 3,486 · see citing works
Norm theory: Comparing reality to its alternatives
Kahneman, D., & Miller, D. T. (1986) · Psychological Review · doi:10.1037/0033-295x.93.2.136 · Cited by 3,158 · see citing works
Individualism and collectivism: Cross-cultural perspectives on self-ingroup relationships.
Triandis, H. C., Bontempo, R. N., Villareal, M. J., Asai, M., & Irizarry, N. L. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.2.323 · Cited by 2,974 · see citing works
Educational measurement, 3rd ed.
Linn, R. L. (1989) · Cited by 2,858 · see citing works
Toward a General Theory of Action
(1951) · Harvard University Press eBooks · doi:10.4159/harvard.9780674863507 · Cited by 2,567 · see citing works
Schwartz, S. H., & Bilsky, W. (1990) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.58.5.878 · Cited by 2,338 · see citing works
The person and the situation: perspectives of social psychology
(1992) · Choice Reviews Online · doi:10.5860/choice.29-3604 · Cited by 2,205 · see citing works
A structural equation test of the value-attitude-behavior hierarchy.
Homer, P., & Kahle, L. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.4.638 · Cited by 2,041 · see citing works
A model of egoistical relative deprivation.
Crosby, F. J. (1976) · Psychological Review · doi:10.1037/0033-295x.83.2.85 · Cited by 1,538 · see citing works
(1983) · Cambridge University Press eBooks · doi:10.1017/cbo9780511620836 · Cited by 1,531 · see citing works
Multimethod probes of individualism and collectivism.
Triandis, H. C., McCusker, C., & Hui, C.-C. H. (1990) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.59.5.1006 · Cited by 1,515 · see citing works
Direct Experience And Attitude-Behavior Consistency
Fázio, R. H., & Zanna, M. P. (1981) · Advances in experimental social psychology · doi:10.1016/s0065-2601(08)60372-x · Cited by 1,215 · see citing works
The Psychology of the Chinese people
Bond, M. H. (1986) · Oxford University Press eBooks · Cited by 890 · see citing works
Chen, C., Lee, S.-Y., & Stevenson, H. W. (1995) · Psychological Science · doi:10.1111/j.1467-9280.1995.tb00327.x · Cited by 877 · see citing works
A value pluralism model of ideological reasoning.
Tetlock, P. E. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.50.4.819 · Cited by 708 · see citing works
Contexts of Achievement: A Study of American, Chinese, and Japanese Children
Stevenson, H. W., Lee, S.-Y., Chen, C., Stigler, J. W., Hsu, C., Kitamura, S., & Hatano, G. (1990) · Monographs of the Society for Research in Child Development · doi:10.2307/1166090 · Cited by 578 · see citing works
Bond, M. H. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.55.6.1009 · Cited by 538 · see citing works
Self-Protection and the Culture of Honor: Explaining Southern Violence
Cohen, D., & Nisbett, R. E. (1994) · Personality and Social Psychology Bulletin · doi:10.1177/0146167294205012 · Cited by 532 · see citing works
Self-citationMorris, C. W. (1956) · University of Chicago Press eBooks · doi:10.1037/10819-000 · Cited by 460 · see citing works
Mathematics Achievement of Chinese, Japanese, and American Children: Ten Years Later
Stevenson, H. W., Chen, C., & Lee, S.-Y. (1993) · Science · doi:10.1126/science.8418494 · Cited by 432 · see citing works
Structure of human values: Testing the adequacy of the Rokeach Value Survey.
Braithwaite, V. A., & Law, H. G. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.49.1.250 · Cited by 420 · see citing works
Stability and change in American value priorities, 1968–1981.
Rokeach, M., & Ball‐Rokeach, S. J. (1989) · American Psychologist · doi:10.1037/0003-066x.44.5.775 · Cited by 415 · see citing works
Violence and U.S. regional culture.
Nisbett, R. E. (1993) · American Psychologist · doi:10.1037/0003-066x.48.4.441 · Cited by 345 · see citing works
Self-citationChinese personality and its change.
Yang, K. (1986) · Cited by 341 · see citing works
Culture and the Problems of Chinese Management
Lockett, M. (1988) · Organization Studies · doi:10.1177/017084068800900402 · Cited by 330 · see citing works
Americans and Chinese : passage to differences
Hsü, F. L. K. (1981) · Cited by 313 · see citing works
Āllport, G. W., Vernon, P. E., & Lindzey, G. E. (1960) · Medical Entomology and Zoology · Cited by 309 · see citing works
Bloom, A. H. (1981) · Medical Entomology and Zoology · Cited by 280 · see citing works
Filial piety, authoritarian moralism, and cognitive conservatism in Chinese societies.
Ho, D. Y. F. (1994) · PubMed · Cited by 271 · see citing works
Personal values and consumer psychology
Pitts, R. E., & Woodside, A. G. (1984) · Medical Entomology and Zoology · Cited by 255 · see citing works
Beginnings: The Social and Affective Development of Black Children.
Scott‐Jones, D., Spencer, M. B., Brookins, G. K., & Allen, W. R. (1987) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2069963 · Cited by 222 · see citing works
Tesser, A., & Shaffer, D. R. (1990) · Annual Review of Psychology · doi:10.1146/annurev.ps.41.020190.002403 · Cited by 208 · see citing works
Ralston, D. A., Gustafson, D. J., Elsass, P. M., Cheung, F. M., & Terpstra, R. H. (1992) · Journal of Applied Psychology · doi:10.1037/0021-9010.77.5.664 · Cited by 199 · see citing works
Goodnow, J. J. (1998) · American Psychological Association eBooks · doi:10.1037/10294-004 · Cited by 155 · see citing works
Time Use and Mathematics Achievement among American, Chinese, and Japanese High School Students
Fuligni, A. J., & Stevenson, H. W. (1995) · Child Development · doi:10.1111/j.1467-8624.1995.tb00908.x · Cited by 150 · see citing works
Intergenerational comparisons of child-rearing attitudes and practices in Hong Kong.
Ho, D. Y., & Kang, T. K. (1984) · Developmental Psychology · doi:10.1037/0012-1649.20.6.1004 · Cited by 147 · see citing works
The Idea of Social Structure: Papers in Honor of Robert K. Merton.
Caplovitz, D., & Coser, L. A. (1977) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2065764 · Cited by 144 · see citing works
Traditional patterns of socialization in Chinese society.
Ho, D. Y. F. (1981) · Cited by 144 · see citing works
Childrearing by Black Parents: A Description and Comparison to Anglo and Chicano Parents
Bartz, K. W., & LeVine, E. S. (1978) · Journal of Marriage and the Family · doi:10.2307/351192 · Cited by 134 · see citing works
A comparison of ranking and rating procedures for value system measurement
Rankin, W. L., & Grube, J. W. (1980) · European Journal of Social Psychology · doi:10.1002/ejsp.2420100303 · Cited by 133 · see citing works
Munro, D. (1981) · doi:10.3998/mpub.19095 · Cited by 113 · see citing works
The Chinese Mind: Essentials of Chinese Philosophy and Culture
Wilhelm, H., & Moore, C. A. (1968) · Journal of the American Oriental Society · doi:10.2307/596920 · Cited by 112 · see citing works
Value self-confrontation as a method to aid in weight loss.
Schwartz, S. H., & Inbar-Saban, N. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.3.396 · Cited by 87 · see citing works
VALUE SYSTEMS ACROSS CULTURES: AUSTRALIA AND CHINA*
Feather, N. T. (1986) · International Journal of Psychology · doi:10.1080/00207598608247616 · Cited by 85 · see citing works
Responses to change: Society, culture, and personality
Vos, D., & George, A. R. (1976) · Medical Entomology and Zoology · Cited by 76 · see citing works
The measurement of values: Effects of different assessment procedures
Feather, N. T. (1973) · Australian Journal of Psychology · doi:10.1080/00049537308255849 · Cited by 75 · see citing works
No place to run, no place to hide: Comparative status and future prospects of black boys.
Hare, B. R., & Castenell, L. A. (1985) · Cited by 66 · see citing works
A Cross-Cultural Study of Authoritarianism
Meade, R. D., & Whittaker, J. O. (1967) · The Journal of Social Psychology · doi:10.1080/00224545.1967.9922293 · Cited by 63 · see citing works
Singh, P. N., Huang, S. C., & Thompson, G. G. (1962) · The Journal of Social Psychology · doi:10.1080/00224545.1962.9710910 · Cited by 60 · see citing works
Authoritarianism and Attitude toward Filial Piety in Chinese Teachers
Yau-Fai, H., & Yü, L. L. (1974) · The Journal of Social Psychology · doi:10.1080/00224545.1974.9923111 · Cited by 58 · see citing works
THE VALUE ORIENTATIONS OF CHINESE UNIVERSITY STUDENTS IN HONG KONG
Lau, S. (1988) · International Journal of Psychology · doi:10.1080/00207598808247787 · Cited by 54 · see citing works
IS CHINESE CONFLICT-HANDLING BEHAVIOR INFLUENCED BY CHINESE VALUES?
Chiu, R. K., & Kosinski, F. A. (1994) · Social Behavior and Personality An International Journal · doi:10.2224/sbp.1994.22.1.81 · Cited by 51 · see citing works
A Cross-Cultural and Cross-Language Comparison of Dogmatism Scores
Earle, M. J. (1969) · The Journal of Social Psychology · doi:10.1080/00224545.1969.9922382 · Cited by 43 · see citing works
Value and Sex‐Role Orientation of Chinese Adolescents
Lau, S., & Wong, K. Y. A. (1992) · International Journal of Psychology · doi:10.1080/00207599208246862 · Cited by 42 · see citing works
Choosing between the ranking and rating procedures for the comparison of values across cultures
Ng, S. H. (1982) · European Journal of Social Psychology · doi:10.1002/ejsp.2420120204 · Cited by 38 · see citing works
Leadership Studies of Chinese and Chinese-Americans
Meade, R. D. (1970) · Journal of Cross-Cultural Psychology · doi:10.1177/135910457000100404 · Cited by 29 · see citing works
Similarity of values systems within the same nation: Evidence from australia and papua new guinea
Feather, N. T. (1980) · Australian Journal of Psychology · doi:10.1080/00049538008254669 · Cited by 26 · see citing works
Value Change in Chinese Society
Wilson, R. W., Wilson, A. A., & Greenblatt, S. L. (1979) · Medical Entomology and Zoology · Cited by 25 · see citing works
The meeting of East and West : an inquiry concerning world understanding
Northrop, F. S. C. (1952) · Virtual Defense Library (Ministerio de Defensa) · Cited by 20 · see citing works
A note on the validity of the rimland diagnostic checklist
Prior, M. R., & Bence, R. (1975) · Journal of Clinical Psychology · doi:10.1002/1097-4679(197507)31:3 · Cited by 19 · see citing works
Authoritarianism and Evaluation of Appropriateness of Role Behavior
Yang, K. (1970) · The Journal of Social Psychology · doi:10.1080/00224545.1970.9712537 · Cited by 16 · see citing works
Young, L. W. L. (1983) · Cambridge University Press eBooks · doi:10.1017/cbo9780511620836.006 · Cited by 15 · see citing works
The Values of Chinese Students: At Home and Abroad
Katz, B., Juni, S., Shope, C. B., et al. (1993) · International Journal of Psychology · doi:10.1080/00207599308246961 · Cited by 14 · see citing works
Using the cross-cultural method to study the development of values
Keats, D. M. (1986) · Australian Journal of Psychology · doi:10.1080/00049538608259016 · Cited by 13 · see citing works
Iwata, O. (1991) · PSYCHOLOGIA · doi:10.2117/psysoc.1991.149 · Cited by 4 · see citing works
Lee, K. Y. (1978) · Medical Entomology and Zoology · Cited by 3 · see citing works
China at the Crossroads: Reform after Tiananmen
Goldstein, S. M. (2017) · The SHAFR Guide Online · doi:10.1163/2468-1733_shafr_sim240090069 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641 · see citing works
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 2,842 · see citing works
“Note that even human evaluations can have high variance and instability, which could be due to cultural and individual differences (Peng et al., 1997).”
BackgroundIs there a universal need for positive self-regard?
Heine, S. J., Lehman, D. R., Markus, H. R., & Kitayama, S. (1999) · Psychological Review · doi:10.1037/0033-295x.106.4.766 · Cited by 2,247 · see citing works
Large-Scale Psychological Differences Within China Explained by Rice Versus Wheat Agriculture
Talhelm, T., Zhang, X., Oishi, S., Shimin, C., Duan, D., Lan, X., & Kitayama, S. (2014) · Science · doi:10.1126/science.1246850 · Cited by 1,374 · see citing works
Brewer, M. B., & Chen, Y.-R. (2007) · Psychological Review · doi:10.1037/0033-295x.114.1.133 · Cited by 1,200 · see citing works
Deviance or uniqueness, harmony or conformity: A cultural analysis
Kim, H., & Markus, H. R. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.4.785 · Cited by 1,216 · see citing works
Chen, F. F. (2008) · Journal of Personality and Social Psychology · doi:10.1037/a0013193 · Cited by 962 · see citing works
Psychological universals: what are they and how can we know?
Norenzayan, A., & Heine, S. J. (2005) · Psychological Bulletin · doi:10.1037/0033-2909.131.5.763 · Cited by 820 · see citing works
Cohen, A. B. (2009) · American Psychologist · doi:10.1037/a0015308 · Cited by 589 · see citing works
Self as Cultural Product: An Examination of East Asian and North American Selves
Heine, S. J. (2001) · Journal of Personality · doi:10.1111/1467-6494.696168 · Cited by 531 · see citing works
Bai, Y., Maruskin, L. A., Chen, S., Gordon, A. M., Stellar, J. E., McNeil, G. D., Peng, K., & Keltner, D. (2017) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000087 · Cited by 476 · see citing works
Self-citationKitayama, S. (2002) · Psychological Bulletin · doi:10.1037/0033-2909.128.1.89 · Cited by 465 · see citing works
Cross-national variation in the size of sex differences in values: Effects of gender equality.
Schwartz, S. H., & Rubel-Lifschitz, T. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0015546 · Cited by 395 · see citing works
Zou, X., Tam, K.-P., Morris, M. W., Lee, S.-L., Lau, I. Y.-M., & Chiu, C.-Y. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0016399 · Cited by 369 · see citing works
Fostering and Measuring Skills: Interventions That Improve Character and Cognition
Heckman, J., & Kautz, T. (2013) · doi:10.3386/w19656 · Cited by 311 · see citing works
Self-assertive interdependence in Arab culture
San Martin, A., Sinaceur, M., Madi, A., Tompson, S., Maddux, W. W., & Kitayama, S. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0435-z · Cited by 124 · see citing works
Culture and Unmerited Authorship Credit: Who Wants It and Why?
Ren, X., Su, H., Lu, K., Dong, X., Ouyang, Z., & Talhelm, T. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.02017 · Cited by 16 · see citing works
- Authors
- Peng, K., Nisbett, R. E., & Wong, N. Y. C.
- Affiliations
- University of Michigan (US)
- University of Hawaiʻi at Mānoa (US)
- Venue
- Psychological Methods
- Published
- 1997-12
- Volume
- 2
- Issue
- 4
- Pages
- 329-344
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2152594618
- Cited by
- 408
- References
- 76
- Citations by year
- 2026: 6 · 2025: 7 · 2024: 12 · 2023: 8 · 2022: 12 · 2021: 16 · 2020: 15 · 2019: 14 · 2018: 23 · 2017: 19 · 2016: 20 · 2015: 24 · 2014: 19 · 2013: 21 · 2012: 9
- FWCI
- 7.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Cultural Differences and Values
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Keywords
- validity, ranking algorithms, rating methodology, attitude scales, social comparison, deprivation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review