What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research
Abstract
It is a common practice to export instruments developed in one culture to another. Little is known about the consequences of making inappropriate comparisons in cross-cultural research. Several studies were conducted to fill in this gap. Study 1 examined the impact of lacking factor loading invariance on regression slope comparisons. When factor loadings of a predictor are higher in the reference group (e.g., United States), for which the scale was developed, than in the focal group (e.g., China), into which the scale was imported, the predictive relationship (e.g., self-esteem predicting life satisfaction) is artificially stronger in the reference group but weaker in the focal group, creating a bogus interaction effect of predictor by group (e.g., self-esteem by culture); the opposite pattern is found when the reference group has higher loadings in an outcome variable. Studies 2 and 3 examined the impact of lacking loading and intercept (i.e., point of origin) invariance on factor means, respectively. When the reference group has higher loadings or intercepts, the mean is overestimated in that group but underestimated in the focal group, resulting in a pseudo group difference.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Contrasts in how people interpret and respond to questionnaire items have been shown to threaten the validity of findings from cross-cultural studies.
“Highlighting the significance of these differences, contrasts in how people interpret and respond to questionnaire items have been shown to threaten the validity of findings from cross-cultural studies (Chen, 2008).”
From Couples becoming parents: Trajectories for psychological distress and buffering effects of social support (2020) · cites this paper as Chen, 2008
Cites
Tabachnick, B. G., & Fidell, L. S. (1983) · Cited by 79,888 · see citing works
The satisfaction with life scale
Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4901_13 · Cited by 31,768 · see citing works
Society and the Adolescent Self-Image
Rosenberg, M. (1965) · doi:10.1515/9781400876136 · Cited by 30,126 · see citing works
Culture and the self: Implications for cognition, emotion, and motivation.
Markus, H. R., & Kitayama, S. (1991) · Psychological Review · doi:10.1037/0033-295x.98.2.224 · Cited by 21,995 · see citing works
Structural Equations with Latent Variables.
Clogg, C. C., & Bollen, K. A. (1991) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2072165 · Cited by 17,471 · see citing works
The Handbook of Social Psychology
Gravatt, A. E., Lindzey, G. E., & Aronson, F. (1969) · The Family Coordinator · doi:10.2307/582242 · Cited by 12,518 · see citing works
Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance
Chen, F. F. (2007) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510701301834 · Cited by 12,217 · see citing works
Self-citationLISREL 7 : user's reference guide
Jöreskog, K. G., & Sörbom, D. (1989) · Medical Entomology and Zoology · Cited by 6,783 · see citing works
Rethinking individualism and collectivism: Evaluation of theoretical assumptions and meta-analyses.
Oyserman, D., Coon, H. M., & Kemmelmeier, M. (2002) · Psychological Bulletin · doi:10.1037/0033-2909.128.1.3 · Cited by 5,758 · see citing works
Assessing Measurement Invariance in Cross‐National Consumer Research
Steenkamp, J. E. M., & Baumgartner, H. (1998) · Journal of Consumer Research · doi:10.1086/209528 · Cited by 4,760 · see citing works
Measurement Invariance, Factor Analysis and Factorial Invariance
Meredith, W. (1993) · Psychometrika · doi:10.1007/bf02294825 · Cited by 4,536 · see citing works
(2000) · Cited by 4,106 · see citing works
Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.3.456 · Cited by 3,928 · see citing works
Is there a universal need for positive self-regard?
Heine, S. J., Lehman, D. R., Markus, H. R., & Kitayama, S. (1999) · Psychological Review · doi:10.1037/0033-295x.106.4.766 · Cited by 2,247 · see citing works
Simultaneous Factor Analysis in Several Populations
Jöreskog, K. G. (1971) · Psychometrika · doi:10.1007/bf02291366 · Cited by 2,051 · cited 1× · see citing works
“However, it is the development of measurement invariance tests (Jöreskog, 1971; Meredith, 1993; Millsap & Everson, 1993; Sörbom, 1978; Widaman & Reise, I would like to express appreciation to Donna Coffman, Larry Cohen, Samuel Gaertner, Kimberly Juliano, Shanhong Luo, Beth Morling, Kristopher…”
MethodLittle, T. D. (1997) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3201_3 · Cited by 1,397 · see citing works
Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993) · Psychological Bulletin · doi:10.1037/0033-2909.114.3.552 · Cited by 1,225 · see citing works
Brewer, M. B., & Chen, Y.-R. (2007) · Psychological Review · doi:10.1037/0033-295x.114.1.133 · Cited by 1,200 · see citing works
Widaman, K. F., & Reise, S. P. (1997) · The science of prevention: Methodological advances from alcohol and substance abuse research. · doi:10.1037/10222-009 · Cited by 1,146 · see citing works
Measurement in Cross-Cultural Psychology
Hui, C.-C. H., & Triandis, H. C. (1985) · Journal of Cross-Cultural Psychology · doi:10.1177/0022002185016002001 · Cited by 1,092 · see citing works
Heine, S. J., Lehman, D. R., Peng, K., & Greenholtz, J. (2002) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.82.6.903 · Cited by 1,080 · see citing works
TEACHER’S CORNER: Testing Measurement Invariance of Second-Order Factor Models
Chen, F. F., Sousa, K. H., & West, S. G. (2005) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1203_7 · Cited by 1,032 · see citing works
Self-citationThe cultural matrix of social psychology
Fiske, A. P., Kitayama, S., Markus, H. R., & Nisbett, R. E. (1998) · Medical Entomology and Zoology · Cited by 990 · see citing works
Chen, C., Lee, S.-Y., & Stevenson, H. W. (1995) · Psychological Science · doi:10.1111/j.1467-9280.1995.tb00327.x · Cited by 877 · see citing works
On Cross-Cultural Comparability
Berry, J. W. (1969) · International Journal of Psychology · doi:10.1080/00207596908247261 · Cited by 863 · see citing works
Differential Item Performance and the Mantel–Haenszel Procedure
(2013) · Routledge eBooks · doi:10.4324/9780203056905-19 · Cited by 715 · see citing works
Cheung, G. W., & Rensvold, R. B. (2000) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022100031002003 · Cited by 577 · see citing works
Lehman, D. R., Chiu, C., & Schaller, M. (2004) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.141927 · Cited by 532 · see citing works
Methodology Review: Statistical Approaches for Assessing Measurement Bias
Millsap, R. E., & Everson, H. T. (1993) · Applied Psychological Measurement · doi:10.1177/014662169301700401 · Cited by 492 · see citing works
The Issue Of Measurement Invariance Revisited
Byrne, B. M., & Watkins, D. (2003) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022102250225 · Cited by 488 · see citing works
The science of prevention: Methodological advances from alcohol and substance abuse research.
Bryant, K. J. (1997) · American Psychological Association eBooks · doi:10.1037/10222-000 · Cited by 477 · see citing works
Cross-Cultural Comparisons and the Presumption of Equivalent Measurement and Theoretical Structure
Byrne, B. M., & Campbell, T. L. (1999) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022199030005001 · Cited by 472 · see citing works
Evaluating the Impact of Partial Factorial Invariance on Selection in Two Populations
Millsap, R. E., & Kwok, O.-M. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.1.93 · Cited by 414 · see citing works
Methodological Issues in Psychological Research on Culture
van de Vijver, F. J. R., & Leung, K. Y. (2000) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022100031001004 · Cited by 411 · see citing works
Validity problems comparing values across cultures and possible solutions.
Peng, K., Nisbett, R. E., & Wong, N. Y. C. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.4.329 · Cited by 408 · see citing works
Moorman, R. H., & Podsakoff, P. M. (1992) · Journal of Occupational and Organizational Psychology · doi:10.1111/j.2044-8325.1992.tb00490.x · Cited by 377 · see citing works
Riordan, C. M., & Vandenberg, R. J. (1994) · Journal of Management · doi:10.1016/0149-2063(94)90007-8 · Cited by 373 · see citing works
EQUIVALENCE OF CROSS‐CULTURAL DATA: AN OVERVIEW OF BASIC ISSUES
Poortinga, Y. H. (1989) · International Journal of Psychology · doi:10.1080/00207598908247842 · Cited by 307 · see citing works
Campbell, D. P., & Boruch, R. F. (1975) · Elsevier eBooks · doi:10.1016/b978-0-12-088850-4.50008-2 · Cited by 207 · see citing works
Variations in collectivism and individualism by ingroup and culture: Confirmatory factor analysis.
Rhee, E., Uleman, J. S., & Lee, H. K. (1996) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.71.5.1037 · Cited by 185 · see citing works
Individuation: Conceptual analysis and assessment.
Maslach, C., Stapp, J., & Santee, R. T. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.49.3.729 · Cited by 179 · see citing works
Evaluation and experiment : some critical issues in assessing social programs
Bennett, C. A., & Lumsdaine, A. A. (1975) · Cited by 138 · see citing works
Chen, F., & West, S. G. (2007) · Journal of Research in Personality · doi:10.1016/j.jrp.2007.05.006 · Cited by 132 · see citing works
Self-citationSmith, L. L., & Reise, S. P. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.75.5.1350 · Cited by 118 · see citing works
The Chinese Mind: Essentials of Chinese Philosophy and Culture
Wilhelm, H., & Moore, C. A. (1968) · Journal of the American Oriental Society · doi:10.2307/596920 · Cited by 112 · see citing works
An Alternative to the Methodology for Analysis of Covariance
Sörbom, D. (1978) · Psychometrika · doi:10.1007/bf02293647 · Cited by 99 · see citing works
The Mediating Role of Parental Expectations in Culture and Well‐Being
Oishi, S., & Sullivan, H. W. (2005) · Journal of Personality · doi:10.1111/j.1467-6494.2005.00349.x · Cited by 91 · see citing works
McArdle, J. J., & Cattell, R. B. (1994) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2901_3 · Cited by 81 · see citing works
Cross-cultural comparisons using non-invariant measurement items
Cheung, G. W., & Rensvold, R. B. (1998) · Applied Behavioral Science Review · doi:10.1016/s1068-8595(99)80006-3 · Cited by 78 · see citing works
Horn, J. L., McArdle, J. J., & Mason, R. (1983) · Cited by 66 · see citing works
Four Unresolved Problems in Studies of Factorial Invariance.
Millsap, R. E. (2005) · Cited by 50 · see citing works
The Construct of Individuation: More Complex in Collectivist than in Individualist Cultures
Kwan, V. S. Y., Bond, M. H., Boucher, H. C., Maslach, C., & Gan, Y. (2002) · Personality and Social Psychology Bulletin · doi:10.1177/0146167202286002 · Cited by 26 · see citing works
Filial Piety and Chinese Society
Yu-wei, H. (1959) · Philosophy East and West · doi:10.2307/1397210 · Cited by 23 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance
Chen, F. F. (2007) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510701301834 · Cited by 12,217 · see citing works
Self-citationPutnick, D. L., & Bornstein, M. H. (2016) · Developmental Review · doi:10.1016/j.dr.2016.06.004 · Cited by 4,369 · see citing works
“To manage partial noninvariance, Chen (2008) suggested comparing the results of interest (e.g., mean differences across groups, regression coefficients between latent variables, etc.) using a partially invariant model (imposing constraints on invariant items only) to those using a fully invariant…”
BackgroundMethodInfluentialAnxiety and working memory capacity: A meta-analysis and narrative review.
Moran, T. P. (2016) · Psychological Bulletin · doi:10.1037/bul0000051 · Cited by 776 · see citing works
Sass, D. A. (2011) · Journal of Psychoeducational Assessment · doi:10.1177/0734282911406661 · Cited by 531 · see citing works
Swami, V., & Barron, D. (2019) · Body Image · doi:10.1016/j.bodyim.2018.08.014 · Cited by 454 · see citing works
The Very Efficient Assessment of Need for Cognition: Developing a Six-Item Version
Lins de Holanda Coelho, G., H. P. Hanel, P., & J. Wolf, L. (2020) · Assessment · doi:10.1177/1073191118793208 · Cited by 358 · see citing works
Sischka, P. E., Costa, A. P., Steffgen, G., & Schmidt, A. F. (2020) · Journal of Affective Disorders Reports · doi:10.1016/j.jadr.2020.100020 · Cited by 245 · see citing works
“Manifest sum (or mean) scoring assumes that the measure under tudy is invariant across groups ( Chen, 2008 ), however, it is important hat cross-country studies take possible DTF effects of the WHO-5 into ccount (for a gentle introduction into measurement invariance testing ee e.g., Tay et al.,…”
BackgroundMethodInfluentialKarcher, N. R., Barch, D. M., Avenevoli, S., Savill, M., Huber, R. S., Simon, T. J., Leckliter, I. N., Sher, K. J., & Loewy, R. L. (2018) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.1334 · Cited by 198 · see citing works
Bibi, A., Lin, M., Zhang, X. C., & Margraf, J. (2020) · International Journal of Psychology · doi:10.1002/ijop.12671 · Cited by 176 · see citing works
Data Analysis in Social Psychology: Recent and Recurring Issues
Judd, C. M., & Kenny, D. A. (2010) · Handbook of Social Psychology · doi:10.1002/9780470561119.socpsy001004 · Cited by 151 · see citing works
Zanon, C., Bardagi, M. P., Layous, K., & Hutz, C. S. (2014) · Social Indicators Research · doi:10.1007/s11205-013-0478-5 · Cited by 140 · see citing works
Van Dierendonck, D., Sousa, M., Gunnarsdóttir, S., Bobbio, A., Hakanen, J., Pircher Verdorfer, A., Cihan Duyan, E., & Rodriguez-Carvajal, R. (2017) · Administrative Sciences · doi:10.3390/admsci7020008 · Cited by 132 · see citing works
de Beer, L. T., Schaufeli, W. B., De Witte, H., Hakanen, J. J., Shimazu, A., Glaser, J., Seubert, C., Bosak, J., Sinval, J., & Rudnev, M. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17155604 · Cited by 130 · see citing works
Schroeders, U., Kubera, F., & Gnambs, T. (2022) · Assessment · doi:10.1177/10731911211033894 · Cited by 76 · see citing works
Hughes, C., T. Devine, R., Foley, S., D. Ribner, A., Mesman, J., & Blair, C. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.01.133 · Cited by 67 · see citing works
Wang, Y., Benner, A. D., & Kim, S. Y. (2015) · Psychological Assessment · doi:10.1037/pas0000136 · Cited by 51 · see citing works
Krammer, G., Sommer, M., & Arendasy, M. E. (2017) · Journal of Personality Assessment · doi:10.1080/00223891.2017.1285781 · Cited by 25 · see citing works
“This implies that mean score differences do not only reflect mean individual differences, but also measurement bias (cf. Chen, 2008; Cheung & Rensvold, 2002; Li & Zumbo, 2009; Millsap, 2011).”
BackgroundInfluentialPauls, F., Petermann, F., & Lepach, A. C. (2013) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803395.2013.820257 · Cited by 13 · see citing works
Warren, M. T., Wray-Lake, L., & Syvertsen, A. K. (2017) · The Journal of Positive Psychology · doi:10.1080/17439760.2017.1350741 · Cited by 13 · see citing works
- Authors
- Chen, F. F.
- Venue
- Journal of Personality and Social Psychology
- Published
- 2008-11
- Volume
- 95
- Issue
- 5
- Pages
- 1005-1018
- ISSN
- 0022-3514 (ISSN-L)
- 1939-1315
- OpenAlex
- W2157438378
- Cited by
- 962
- References
- 53
- Citations by year
- 2026: 62 · 2025: 67 · 2024: 74 · 2023: 85 · 2022: 87 · 2021: 76 · 2020: 76 · 2019: 61 · 2018: 58 · 2017: 64 · 2016: 55 · 2015: 45 · 2014: 51 · 2013: 36 · 2012: 23
- FWCI
- 21.05
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Personality and Social Psychology (publishedVersion)
- PubMed (publishedVersion)
- The HKU Scholars Hub (University of Hong Kong) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Social Psychology › Cultural Differences and Values
- Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
- Keywords
- cross-cultural research, scale adaptation, self-esteem, life satisfaction
- MeSH terms
- China, Cross-Cultural Comparison, Humans, Personality Inventory, Psychometrics, Quality of Life, Reference Values, Research, Self Concept, United States, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
