Measurement Errors in the Behavioral Sciences: The Case of Personality/Attitude Research

Buckley, M. Ronald; Cote, Joseph A.; Comstock, S. Mark

doi:10.1177/0013164490503001

Cited by 147Psychometric Methodologies and TestingPersonality Traits and PsychologyReliability and Agreement in Measurement

Measurement Errors in the Behavioral Sciences: The Case of Personality/Attitude Research

Buckley, M. Ronald, Cote, Joseph A., Comstock, S. Mark·1990·Educational and Psychological Measurement

DOI 10.1177/0013164490503001

Open this paper in CoChat — free account

Abstract

It has been established that construct measures often contain measurement error and method effects. However, the extent of these measurement problems and their effects on personality constructs has not been fully examined. In this study, an analysis of published data sets affords estimates of the amount of trait, method, and error variance of measures used in the social/behavioral sciences. It was found that traits accounted for less than 50 percent of the variance in construct measures among the entire group of data sets included in the study. Measures of attitudes and personality variables contain considerably less trait variance. Implications for theory testing are discussed.

Integrity signals

Registry

Crossref · 10.1177/0013164490503001

Cites

79 of 79 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Common method variance is a known limitation in surveys using self-reported measures.
“Common method variance is a known limitation in surveys using selfreported measures and refers to the amount of spurious covariance shared among variables because of the common method used in collecting data (Buckley et al., 1990).”
From How the interplay between consumer motivations and values influences organic food identity and behavior (2018) · cites this paper as Buckley et al., 1990

Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116 · see citing works
Social Science Quarterly
Southwestern Social Science Association (2010) · Social Science Quarterly · doi:10.1111/(issn)1540-6237 · Cited by 13,619 · see citing works
Book Reviews: Kerlinger, Fred N. Foundations of Behavioral Research. (2nd ed.) New York: Holt, Rinehart and Winston, 1973. 741 + xx pp. $13.95
Stockton, R. A., & Kerlinger, F. N. (1974) · American Educational Research Journal · doi:10.2307/1162206 · Cited by 7,356 · see citing works
Reliability and Validity Assessment
Carmines, E., & Zeller, R. (1979) · doi:10.4135/9781412985642 · Cited by 5,884 · see citing works
Reliability and Validity Assessment
Kurian, G. T. (2011) · CQ Press eBooks · doi:10.4135/9781608712434.n1341 · Cited by 5,045 · see citing works
Foundations of Behavioral Research
Buttram, J. B. (1967) · Journal of Music Therapy · doi:10.1093/jmt/4.4.144-a · Cited by 2,977 · see citing works
Attitudes versus Actions: The Relationship of Verbal and Overt Behavioral Responses to Attitude Objects
Wicker, A. W. (1969) · Journal of Social Issues · doi:10.1111/j.1540-4560.1969.tb00619.x · Cited by 2,152 · see citing works
Representing and Testing Organizational Theories: A Holistic Construal
Bagozzi, R. P., & Phillips, L. W. (1982) · Administrative Science Quarterly · doi:10.2307/2392322 · Cited by 2,064 · see citing works
The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis
Anderson, J. C., & Gerbing, D. (1984) · Psychometrika · doi:10.1007/bf02294170 · Cited by 1,775 · see citing works
Construct Validity: A Review of Basic Issues and Marketing Practices
Peter, J. P. (1981) · Journal of Marketing Research · doi:10.2307/3150948 · Cited by 1,189 · see citing works
Journal of Applied Social Psychology
Milfont, T. L., Duckitt, J. H., & Wagner, C. (2010) · Journal of Applied Social Psychology · doi:10.1111/(issn)1559-1816 · Cited by 1,094 · see citing works
Hierarchically Nested Covariance Structure Models for Multitrait-Multimethod Data
Widaman, K. F. (1985) · Applied Psychological Measurement · doi:10.1177/014662168500900101 · Cited by 1,037 · see citing works
Measurement Theory for the Behavioral Sciences.
Elliott, G. C., Ghiselli, E. Ε., Campbell, J. P., & Zedeck, S. (1982) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2067145 · Cited by 1,011 · see citing works
Estimating Trait, Method, and Error Variance: Generalizing across 70 Construct Validation Studies
Cote, J. A., & Buckley, M. R. (1987) · Journal of Marketing Research · doi:10.2307/3151642 · Cited by 750 · see citing works
Self-citation
The relationship between the affective, behavioral, and cognitive components of attitude
Ostrom, T. M. (1969) · Journal of Experimental Social Psychology · doi:10.1016/0022-1031(69)90003-1 · Cited by 625 · see citing works
Nonconvergence, Improper Solutions, and Starting Values in Lisrel Maximum Likelihood Estimation
Boomsma, A. (1985) · Psychometrika · doi:10.1007/bf02294248 · Cited by 621 · see citing works
Personality and consumer behavior: A review
Kassarjian, H. H. (1971) · Journal of Marketing Research · doi:10.1177/002224377100800401 · Cited by 601 · see citing works
Personality and Consumer Behavior: A Review
Kassarjian, H. H. (1971) · Journal of Marketing Research · doi:10.2307/3150229 · Cited by 467 · see citing works
Offending estimates in covariance structure analysis: Comments on the causes of and solutions to Heywood cases.
Dillon, W. R., Kumar, A. S., & Mulani, N. (1987) · Psychological Bulletin · doi:10.1037/0033-2909.101.1.126 · Cited by 334 · see citing works
Toward the measurement of organizational practices: Scale development and validation.
House, R. J., & Rizzo, J. (1972) · Journal of Applied Psychology · doi:10.1037/h0033444 · Cited by 328 · see citing works
The multitrait-multirater approach to measuring managerial job performance.
Lawler, E. E. (1967) · Journal of Applied Psychology · doi:10.1037/h0025095 · Cited by 304 · see citing works
The Fishbein Extended Model and Consumer Behavior
Ryan, M. J., & Bonfield, E. H. (1975) · Journal of Consumer Research · doi:10.1086/208623 · Cited by 285 · see citing works
Relationships among Research Design Choices and Psychometric Properties of Rating Scales: A Meta-Analysis
Peter, J. P., & Churchill, G. A. (1986) · Journal of Marketing Research · doi:10.2307/3151771 · Cited by 253 · see citing works
Validation of feeling, belief, and intention to act as three components of attitude and their contribution to prediction of contraceptive behavior.
Kothandapani, V. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0031448 · Cited by 229 · see citing works
A four-dimensional interpretation of risk taking1
Jackson, D. N., Hourany, L., & Vidmar, N. J. (1972) · Journal of Personality · doi:10.1111/j.1467-6494.1972.tb00075.x · Cited by 214 · see citing works
On Various Causes of Improper Solutions in Maximum Likelihood Factor Analysis
van Driel, O. P. (1978) · Psychometrika · doi:10.1007/bf02293865 · Cited by 214 · see citing works
The development and evaluation of behaviorally based rating scales.
Campbell, J. P., Dunnette, M. D., Arvey, R. D., & Hellervik, L. W. (1973) · Journal of Applied Psychology · doi:10.1037/h0034185 · Cited by 208 · see citing works
Validation of an S-O-R Model for Situation, Enduring, and Response Components of Involvement
Arora, R. (1982) · Journal of Marketing Research · doi:10.1177/002224378201900411 · Cited by 207 · see citing works
Measurement of guilt in females by self-report inventories.
Mosher, D. L. (1968) · Journal of Consulting and Clinical Psychology · doi:10.1037/h0026589 · Cited by 184 · see citing works
Measuring Causal Attributions for Success and Failure
Elig, T. W., & Frieze, I. H. (1980) · Springer eBooks · doi:10.1007/978-1-4757-8997-3_13 · Cited by 173 · see citing works
Multitrait-Multimethod Analyses of the Self-description Questionnaire: Student-Teacher Agreement on Multidimensional Ratings of Student Self-concept
Marsh, H. W., Smith, I. D., & Barnes, J. (1983) · American Educational Research Journal · doi:10.3102/00028312020003333 · Cited by 167 · see citing works
Measuring causal attibutions for success and failure.
Elig, T. W., & Frieze, I. H. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.4.621 · Cited by 165 · see citing works
A Path Analysis of Causes and Consequences of Salespeople's Perceptions of Role Clarity
Teas, R. K., Wacker, J. G., & Hughes, R. E. (1979) · Journal of Marketing Research · doi:10.1177/002224377901600308 · Cited by 148 · see citing works
Validity of students' evaluations of college teaching: A multitrait–multimethod analysis.
Marsh, H. W. (1982) · Journal of Educational Psychology · doi:10.1037/0022-0663.74.2.264 · Cited by 145 · see citing works
Methods Factors In Multitrait-Multimethod Matrices : Multiplicative Rather Than Additive?
Campbell, D. T., & O'Connell, E. J. (1967) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr0204_1 · Cited by 139 · see citing works
Measurement Error and Theory Testing in Consumer Research: An Illustration of the Importance of Construct Validation
Cote, J. A., & Buckley, M. R. (1988) · Journal of Consumer Research · doi:10.1086/209137 · Cited by 134 · see citing works
Self-citation
Construct Interpretations of Three Self-Concept Inventories
Marx, R. W., & Winne, P. H. (1978) · American Educational Research Journal · doi:10.3102/00028312015001099 · Cited by 116 · see citing works
Response format of the Job Descriptive Index: Assessment of reliability and validity by the multitrait–multimethod matrix.
Johnson, S. M., Smith, P. C., & Tucker, S. M. (1982) · Journal of Applied Psychology · doi:10.1037/0021-9010.67.4.500 · Cited by 109 · see citing works
Examination of personality-ability relationships in the elderly: The role of the contextual (interface) assessment mode
Lachman, M. E., Baltes, P. B., Nesselroade, J. R., & Willis, S. L. (1982) · Journal of Research in Personality · doi:10.1016/0092-6566(82)90007-1 · Cited by 108 · see citing works
Construct validity for measures of childhood depression: Application of multitrait–multimethod methodology.
Saylor, C. F., Finch, A. J., Jr., Furey, W., et al. (1984) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.52.6.977 · Cited by 96 · see citing works
Is achievement a unitary construct?
Jackson, D. N., Ahmed, S. A., & Heapy, N. (1976) · Journal of Research in Personality · doi:10.1016/0092-6566(76)90079-9 · Cited by 90 · see citing works
Standardized observations: An approach to measuring the nature of jobs.
Jenkins, G. D., Nadler, D. A., Lawler, E. E., & Cammann, C. (1975) · Journal of Applied Psychology · doi:10.1037/h0076541 · Cited by 89 · see citing works
A Review And Critique Of Analyses Of Multitrait-Multimethod Matrices
Schmitt, N., Coyle, B. W., & Saari, B. B. (1977) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr1204_3 · Cited by 85 · see citing works
Validation of an S-O-R Model for Situation, Enduring, and Response Components of Involvement
Arora, R. (1982) · Journal of Marketing Research · doi:10.2307/3151723 · Cited by 74 · see citing works
A Path Analysis of Causes and Consequences of Salespeople's Perceptions of Role Clarity
Teas, R. K., Wacker, J. G., & Hughes, R. E. (1979) · Journal of Marketing Research · doi:10.2307/3150710 · Cited by 74 · see citing works
Sex Effects on Managerial Hiring Decisions.
Rose, G. L., & Andiappan, P. (1978) · Academy of Management Journal · doi:10.2307/255666 · Cited by 70 · see citing works
Convergent and discriminant validity for areas and methods of rating job satisfaction.
Locke, E. A., Smith, P. C., Kendall, L. M., et al. (1964) · Journal of Applied Psychology · doi:10.1037/h0043202 · Cited by 68 · see citing works
The development and validation of a test of emotional styles.
Allen, J. G., & Hamsher, J. H. (1974) · Journal of Consulting and Clinical Psychology · doi:10.1037/h0037046 · Cited by 61 · see citing works
Nursing performance as measured by behavioral expectation scales: A multitrait-multirater analysis
Zedeck, S., & Baker, H. T. (1972) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(72)90029-3 · Cited by 58 · see citing works
An Experimental Test of the Harmful Effects of Premium-Oriented Commercials on Children
Shimp, T. A., Dyer, R. F., & Divita, S. F. (1976) · Journal of Consumer Research · doi:10.1086/208645 · Cited by 55 · see citing works
Measurement of Salespeople's Job Satisfaction: Convergent and Discriminant Validity of Corresponding INDSALES and Job Descriptive Index Scales
Futrell, C. M. (1979) · Journal of Marketing Research · doi:10.1177/002224377901600418 · Cited by 54 · see citing works
Assessment of the Multitrait-Multimethod Matrix Validity of Likert Scales Via Confirmatory Factor Analysis
Flamer, S. (1983) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr1803_3 · Cited by 53 · see citing works
Maximum-likelihood estimation in common factor analysis: A cautionary note.
Jackson, D. N., & Chan, D. W. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.2.502 · Cited by 48 · see citing works
A Multitrait-Multimethod Matrix Analysis of Sense of Humor
Koppel, M. A., & SECHREST, L. B. (1970) · Educational and Psychological Measurement · doi:10.1177/001316447003000107 · Cited by 45 · see citing works
A multitrait-multi method analysis of scales developed by retranslation
Dickinson, T. L., & Tice, T. E. (1973) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(73)90063-9 · Cited by 41 · see citing works
Convergent and discriminant validities between the Cornell Job Descriptive Index and a measure of goal attainment.
Evans, M. (1969) · Journal of Applied Psychology · doi:10.1037/h0027098 · Cited by 37 · see citing works
A Multitrait Multimethod Measurement Approach to the Traits (or States) of Anxiety, Depression and Hostility
Zuckerman, M., Persky, H., Eckman, K. M., & Hopkins, T. R. (1967) · Journal of Projective Techniques and Personality Assessment · doi:10.1080/0091651x.1967.10120357 · Cited by 35 · see citing works
Spousal Conflict Arousal: Scale Development
Seymour, D. T., & Lessne, G. J. (1984) · Journal of Consumer Research · doi:10.1086/209016 · Cited by 34 · see citing works
Evaluating Reading Diagnostic Tests: An Application of Confirmatory Factor Analysis To Multitrait-Multimethod Data
Marsh, H. W., & Butler, S. R. (1984) · Applied Psychological Measurement · doi:10.1177/014662168400800308 · Cited by 30 · see citing works
A multitrait-multimethod analysis of variance of teachers' ratings of aggression, hyperactivity, and inattention
Roberts, M. A., Milich, R. S., Loney, J., et al. (1981) · Research on Child and Adolescent Psychopathology · doi:10.1007/bf00916841 · Cited by 25 · see citing works
The Construct Validity of three Dimensions of Leadership Research
Mitchell, T. R. (1970) · The Journal of Social Psychology · doi:10.1080/00224545.1970.9712523 · Cited by 24 · see citing works
Student Evaluations of Courses and Faculty Based on a Perceived Learning Criterion: Scale Construction, Validation, and Comparison of Results
Freedman, R. D., & Stumpf, S. A. (1978) · Applied Psychological Measurement · doi:10.1177/014662167800200203 · Cited by 22 · see citing works
The redundancy of work alienation and job satisfaction: Some evidence of convergent and discriminant validity
Lefkowitz, J. M., & Brigando, L. (1980) · Journal of Vocational Behavior · doi:10.1016/0001-8791(80)90043-3 · Cited by 21 · see citing works
A Multitrait-Multimethod Validation of Measures of Student Attitudes Toward School, Toward Learning, and Toward Technology in Sixth Grade Children
Roshal, S. M., Frieze, I. H., & Wood, J. T. (1971) · Educational and Psychological Measurement · doi:10.1177/001316447103100429 · Cited by 19 · see citing works
Measurement of Salespeople's Job Satisfaction: Convergent and Discriminant Validity of Corresponding INDSALES and Job Descriptive Index Scales
Futrell, C. M. (1979) · Journal of Marketing Research · doi:10.2307/3150836 · Cited by 19 · see citing works
A comparison of stress measures in an experimental analogue of public speaking.
Karst, T. O., & Most, R. B. (1973) · Journal of Consulting and Clinical Psychology · doi:10.1037/h0035258 · Cited by 14 · see citing works
An investigation of the construct validity of three recently developed personality instruments: An application of confirmatory multimethod factor analysis
Watkins, D., & Hattie, J. (1981) · Australian Journal of Psychology · doi:10.1080/00049538108254697 · Cited by 12 · see citing works
Relationships between Various Criteria of Leadership in Small Groups
Schneider, B. (1970) · The Journal of Social Psychology · doi:10.1080/00224545.1970.9919956 · Cited by 12 · see citing works
Convergent and Discriminant Validity of the Adjective Check List and Edwards Personal Preference Schedule
Bouchard, T. J. (1968) · Educational and Psychological Measurement · doi:10.1177/001316446802800417 · Cited by 8 · see citing works
Auxiliary Theory and Multitrait-Multimethod Validation: A Review of Two Approaches
Avison, W. R. (1978) · Applied Psychological Measurement · doi:10.1177/014662167800200318 · Cited by 8 · see citing works
A Three Dimensional Test for White's Effectance Motive
Lamont, D. J. (1983) · Journal of Personality Assessment · doi:10.1207/s15327752jpa4701_11 · Cited by 7 · see citing works
Comparative Validation of Attitude Measures By The Multitrait-Multimethod Matrix
Hicks, J. M. (1967) · Educational and Psychological Measurement · doi:10.1177/001316446702700446 · Cited by 6 · see citing works
QUESTIONNAIRE VALIDATION BY CONFIRMATORY FACTOR ANALYSIS: AN IMPROVEMENT OVER MULTITRAIT‐MULTIMETHOD MATRICES*
Elbert, N. F. (1979) · Decision Sciences · doi:10.1111/j.1540-5915.1979.tb00052.x · Cited by 6 · see citing works
Convergent and Discriminant Validation of the Trait of Visual Figure-Ground Perception
Ritter, D. R., & Ysseldyke, J. E. (1976) · Journal of Learning Disabilities · doi:10.1177/002221947600900511 · Cited by 5 · see citing works
Convergent and Discriminant Validity of the Barclay Classroom Climate Inventory
Tapp, G. S., & Barclay, J. R. (1974) · Educational and Psychological Measurement · doi:10.1177/001316447403400234 · Cited by 4 · see citing works
A COMPARATIVE STUDY OF ASSESSING CHILDREN'S SCHOOL‐RELATED ATTITUDES
Khan, S. B. (1978) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1978.tb00058.x · Cited by 4 · see citing works
Convergent and Discriminant Validity of the Iowa Social Competency Scale for Preschool Children
Wirth, S., & Pease, D. (1983) · Educational and Psychological Measurement · doi:10.1177/001316448304300141 · Cited by 2 · see citing works
Convergent and Discriminant Validation of Chartered Life Underwriter (CLU) Examinations by the Multitrait-Multimethod Matrix
Bajtelsmit, J. W. (1979) · Educational and Psychological Measurement · doi:10.1177/001316447903900422 · Cited by 2 · see citing works
Item-Analysis Procedures for Enhancing Validity of Existing Personality Scales
Haley, G. A. (1970) · Psychological Reports · doi:10.1177/003329417002700301 · Cited by 1 · see citing works

Flagged first, then by citations

Sorted by citations

Sources of method bias in social science research and recommendations on how to control it
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100452 · Cited by 15,744 · see citing works
Common Method Bias in Marketing: Causes, Mechanisms, and Procedural Remedies
MacKenzie, S. B., & Podsakoff, P. M. (2012) · Journal of Retailing · doi:10.1016/j.jretai.2012.08.001 · Cited by 3,051 · see citing works
A meta-analysis of shared leadership and team effectiveness
Wang, D., Waldman, D. A., & Zhang, Z. (2014) · Journal of Applied Psychology · doi:10.1037/a0034531 · Cited by 731 · see citing works
How the interplay between consumer motivations and values influences organic food identity and behavior
Hansen, T., Sørensen, M. I., & Eriksen, M.-L. R. (2018) · Food Policy · doi:10.1016/j.foodpol.2017.11.003 · Cited by 311 · see citing works
Do Environmental Concern and Perceived Risk Contribute to Consumers’ Intention toward Buying Remanufactured Products? An Empirical Study from China
Zhang, W., & Luo, B. (2021) · Clean Technologies and Environmental Policy · doi:10.1007/s10098-021-02061-8 · Cited by 54 · see citing works
Moderating effect of cognitive conflict on the relationship between value consciousness and online shopping cart abandonment
Mishra, S., Malhotra, G., & Tiwari, S. R. (2021) · The International Review of Retail, Distribution and Consumer Research · doi:10.1080/09593969.2021.2002386 · Cited by 26 · see citing works

Authors: Buckley, M. R., Cote, J. A., & Comstock, S. M.
Affiliations: University of Oklahoma (US)
Washington State University (US)
ORCID: Buckley, M. Ronald
Venue: Educational and Psychological Measurement
Publisher: SAGE Publications
Published: 1990-09
Volume: 50
Issue: 3
Pages: 447-474
ISSN: 0013-1644 (ISSN-L)
1552-3888
DOI: 10.1177/0013164490503001
OpenAlex: W2143445213
Cited by: 147
References: 79
Citations by year: 2026: 3 · 2025: 2 · 2024: 3 · 2023: 5 · 2022: 13 · 2021: 11 · 2020: 10 · 2019: 5 · 2018: 5 · 2017: 5 · 2016: 3 · 2015: 7 · 2014: 6 · 2013: 9 · 2012: 14
FWCI: 1.27
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Educational and Psychological Measurement (publishedVersion)
Topics: Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Psychology › Clinical Psychology › Personality Traits and Psychology
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Keywords: measurement error, personality traits, attitudes, trait variation, method variance, error variance, construct validity, theory testing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Measurement Errors in the Behavioral Sciences: The Case of Personality/Attitude Research

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks