Measurement error in psychological research: Lessons from 26 research scenarios.

Schmidt, Frank L.; Hunter, John E.

doi:10.1037/1082-989x.1.2.199

Cited by 568Psychometric Methodologies and TestingReliability and Agreement in MeasurementMeta-analysis and systematic reviews

Measurement error in psychological research: Lessons from 26 research scenarios.

Schmidt, Frank L., Hunter, John E.·1996·Psychological Methods

DOI 10.1037/1082-989x.1.2.199

Open this paper in CoChat — free account

Abstract

technical principles to concrete research problems that appear in a wide variety of forms and with a myriad of obscuring features, details, and idiosyncracies. This article is based on a different approach: examination of a series of concrete research scenarios that we have encountered in our work as researchers, advisors to researchers, or reviewers. This article examines 26 real-world case studies and explicates them on the basis of the principles of reliability theory found in Cronbach (1947, 1951) and Cronbach et al. (1972). Although we cite and rely on these sources, other sources (e.g., Thorndike, 1951) would yield identical resolutions; only the terminology would be sometimes (slightly) different. In classical measurement theory, the fundamental general formula for the observed correlation between any two measures, x and y, is yields the dissattenuation formula:

Integrity signals

Registry

Crossref · 10.1037/1082-989x.1.2.199

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The Hunter and Schmidt method aims to investigate relations on the level of constructs rather than measured values and allows correction for statistical artifacts including internal consistencies.
“This method aims to investigate relations on the level of constructs rather than measured values; therefore, it allows correction for the effects of several statistical artifacts, inter alia, concerning measuring accuracy (or more specifically for internal consistencies [78]).”
From Conceptualizations of cyberchondria and relations to the anxiety spectrum: Systematic review and meta-analysis (2021) · cites this paper as [78]

Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · see citing works
Handbook of Industrial and Organizational Psychology.
Hammer, T. H., Anderson, J. F., Gurdon, M. A., Jick, T. D., Petit, A., Stern, R. N., Schwartz, H., Vardi, Y., & Dunnette, M. D. (1977) · Administrative Science Quarterly · doi:10.2307/2391755 · Cited by 11,947 · see citing works
Reviews: Lord, Frederic M., and Novick, Melvin R. (with contributions by Allan Birnbaum). Statistical Theories of Mental Test Scores. Reading, Mass.: Addison-Wesley, 1968. xii + 568 pp. $14.95.
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012 · see citing works
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542 · see citing works
Self-citation
The American Voter. By A. Campbell, P. Converse, W. Miller, and D. Stokes. (New York: John Wiley & Sons, Inc. 1960. Pp. viii, 573. $8.50.)
Eulau, H. H. F. (1960) · American Political Science Review · doi:10.2307/1952653 · Cited by 4,749 · see citing works
Educational measurement, 3rd ed.
Linn, R. L. (1989) · Cited by 2,858 · see citing works
The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis.
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · see citing works
The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles
Merrifield, P. R., Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1974) · American Educational Research Journal · doi:10.2307/1162145 · Cited by 1,819 · see citing works
Personnel Selection in Organizations
Murphy, K. R., Schmitt, N., & Borman, W. C. (1993) · Academy of Management Review · doi:10.2307/258598 · Cited by 1,639 · see citing works
Modeling the performance prediction problem in industrial and organizational psychology.
Campbell, J. P. (1990) · Cited by 1,467 · see citing works
Meta-Analysis: Cumulating Research Findings across Studies
Feldman, J. M., Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1984) · Academy of Management Review · doi:10.2307/258247 · Cited by 1,199 · see citing works
Self-citation
Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales.
Smith, P. C., & Kendall, L. M. (1963) · Journal of Applied Psychology · doi:10.1037/h0047060 · Cited by 838 · see citing works
The validity of employment interviews: A comprehensive review and meta-analysis.
McDaniel, M. A., Whetzel, D. L., Schmidt, F., & Maurer, S. D. (1994) · Journal of Applied Psychology · doi:10.1037/0021-9010.79.4.599 · Cited by 751 · see citing works
Self-citation
What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology.
Schmidt, F. L. (1992) · American Psychologist · doi:10.1037/0003-066x.47.10.1173 · Cited by 564 · cited 5× · see citing works
“These advances have led to the demise of the older belief that population relations among variables are unique to settings, organizations, or subgroups and have therefore encouraged attempts to develop general theories and explanations (Schmidt, 1992).”
BackgroundMethodInfluentialSelf-citation
Test “Reliability”: Its Meaning and Determination
Cronbach, L. J. (1947) · Psychometrika · doi:10.1007/bf02289289 · Cited by 460 · see citing works
Validity generalization results for tests used to predict job proficiency and training success in clerical occupations.
Pearlman, K., Schmidt, F. L., & Hunter, J. E. (1980) · Journal of Applied Psychology · doi:10.1037/0021-9010.65.4.373 · Cited by 343 · see citing works
Self-citation
Interrater reliability of job performance ratings: Growth to asymptote level with increasing opportunity to observe.
Rothstein, H. R. (1990) · Journal of Applied Psychology · doi:10.1037/0021-9010.75.3.322 · Cited by 220 · see citing works
Development of a Causal Model of Processes Determining Job Performance
Schmidt, F. L., & Hunter, J. E. (1992) · Current Directions in Psychological Science · doi:10.1111/1467-8721.ep10768758 · Cited by 208 · see citing works
Self-citation
Halo in a multidimensional forced-choice performance evaluation scale.
King, L. M., Hunter, J. E., & Schmidt, F. L. (1980) · Journal of Applied Psychology · doi:10.1037/0021-9010.65.5.507 · Cited by 143 · see citing works
Self-citation
The case for confidence intervals in controlled clinical trials
Borenstein, M. (1994) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)90036-1 · Cited by 84 · see citing works
Validity generalization results for two job groups in the petroleum industry.
Schmidt, F. L., Hunter, J. E., & Caplan, J. R. (1981) · Journal of Applied Psychology · doi:10.1037/0021-9010.66.3.261 · Cited by 80 · see citing works
Self-citation
Satisfying a Simplex Structure is Simpler than it should be
Rogosa, D., & Willett, J. B. (1985) · Journal of Educational Statistics · doi:10.3102/10769986010002099 · Cited by 68 · see citing works
Within-task intercorrelations of skilled performance: Implications for predicting individual differences? A comment on Henry & Hulin, 1987.
Ackerman, P. L. (1989) · Journal of Applied Psychology · doi:10.1037/0021-9010.74.2.360 · Cited by 60 · see citing works
A WITHIN SETTING EMPIRICAL TEST OF THE SITUATIONAL SPECIFICITY HYPOTHESIS IN PERSONNEL SELECTION
Schmidt, F. L., & Hunter, J. E. (1984) · Personnel Psychology · doi:10.1111/j.1744-6570.1984.tb01453.x · Cited by 49 · see citing works
Self-citation

Flagged first, then by citations

Sorted by citations

A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research
Vandenberg, R. J., & Lance, C. E. (2000) · Organizational Research Methods · doi:10.1177/109442810031002 · Cited by 7,997 · see citing works
The job satisfaction–job performance relationship: A qualitative and quantitative review.
Judge, T. A., Thoresen, C. J., Bono, J. E., & Patton, G. K. (2001) · Psychological Bulletin · doi:10.1037/0033-2909.127.3.376 · Cited by 4,760 · see citing works
Business-unit-level relationship between employee satisfaction, employee engagement, and business outcomes: a meta-analysis.
Harter, J. K., Schmidt, F. L., & Hayes, T. L. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.2.268 · Cited by 4,623 · see citing works
Self-citation
Personality and leadership: A qualitative and quantitative review
Judge, T. A., Bono, J. E., Ilies, R., & Gerhardt, M. W. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.4.765 · Cited by 3,190 · see citing works
A meta-analysis of the five-factor model of personality and academic performance.
Poropat, A. E. (2009) · Psychological Bulletin · doi:10.1037/a0014996 · Cited by 2,729 · see citing works
The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and the Interpretation of Research Results
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461 · see citing works
Uses and abuses of coefficient alpha
Schmitt, N. (1996) · Psychological Assessment · doi:10.1037/1040-3590.8.4.350 · Cited by 2,588 · see citing works
Five-factor model of personality and job satisfaction: A meta-analysis.
Judge, T. A., Heller, D., & Mount, M. K. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.3.530 · Cited by 2,164 · see citing works
The validity of the multi-informant approach to assessing child and adolescent mental health.
Reyes, A. D. L., Augenstein, T. M., Wang, M., Thomas, S. A., Drabick, D. A. G., Burgers, D. E., & Rabinowitz, J. A. (2015) · Psychological Bulletin · doi:10.1037/a0038498 · Cited by 1,706 · see citing works
The five factor model of personality and job performance in the European Community.
Salgado, J. F. (1997) · Journal of Applied Psychology · doi:10.1037/0021-9010.82.1.30 · Cited by 1,582 · see citing works
Moral Virtues, Fairness Heuristics, Social Entities, and Other Denizens of Organizational Justice
Cropanzano, R., Byrne, Z. S., Bobocel, D. R., & Rupp, D. E. (2001) · Journal of Vocational Behavior · doi:10.1006/jvbe.2001.1791 · Cited by 1,453 · see citing works
The relationship of age to ten dimensions of job performance.
Ng, T. W. H., & Feldman, D. C. (2008) · Journal of Applied Psychology · doi:10.1037/0021-9010.93.2.392 · Cited by 1,224 · see citing works
Predicting and Explaining Intentions and Behavior: How Well Are We Doing?
Sutton, S. R. (1998) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1998.tb01679.x · Cited by 1,032 · see citing works
Role of social desirability in personality testing for personnel selection: The red herring.
Öneş, D. S., Viswesvaran, C., & Reiss, A. D. (1996) · Journal of Applied Psychology · doi:10.1037/0021-9010.81.6.660 · Cited by 924 · see citing works
The Affective Underpinnings of Job Perceptions and Attitudes: A Meta-Analytic Review and Integration.
Thoresen, C. J., Kaplan, S. A., Barsky, A. P., Warren, C. R., & de Chermont, K. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.6.914 · Cited by 919 · see citing works
Individual differences in rational thought.
Stanovich, K. E., & West, R. F. (1998) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.127.2.161 · Cited by 875 · see citing works
Well-Being for Public Policy
Diener, E., Lucas, R. E., Schimmack, U., & Helliwell, J. F. (2009) · doi:10.1093/acprof:oso/9780195334074.001.0001 · Cited by 868 · see citing works
Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers.
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.115 · Cited by 1,031 · see citing works
Self-citation
A meta-analytic review of occupational commitment: Relations with person- and work-related variables.
Lee, K., Carswell, J. J., & Allen, N. J. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.5.799 · Cited by 809 · see citing works
Comparative analysis of the reliability of job performance ratings.
Viswesvaran, C., Öneş, D. S., & Schmidt, F. L. (1996) · Journal of Applied Psychology · doi:10.1037/0021-9010.81.5.557 · Cited by 774 · see citing works
Self-citation
The meta-analysis of clinical judgment project: fifty-six years of accumulated research on clinical versus statistical prediction
Ægisdóttir, S., White, M. J., Spengler, P. M., Maugherman, A. S., Anderson, L. A., Cook, R. S., Nichols, C. N., Lampropoulos, G. K., Walker, B. S., Cohen, G., et al. (2006) · The Counseling Psychologist · doi:10.1177/0011000005285875 · Cited by 737 · see citing works
Climate for Creativity: A Quantitative Review
Hunter, S. T., Bedell, K. E., & Mumford, M. D. (2007) · Creativity Research Journal · doi:10.1080/10400410709336883 · Cited by 709 · see citing works
MEASUREMENT ERROR IN RESEARCH ON HUMAN RESOURCES and FIRM PERFORMANCE: HOW MUCH ERROR IS THERE AND HOW DOES IT INFLUENCE EFFECT SIZE ESTIMATES?
Gerhart, B. A., Wright, P. M., MAHAN, G. C. M., & Snell, S. A. (2000) · Personnel Psychology · doi:10.1111/j.1744-6570.2000.tb02418.x · Cited by 679 · see citing works
Deliberate practice and performance in music, games, sports, education, and professions: a meta-analysis.
Macnamara, B. N., Hambrick, D. Z., & Oswald, F. L. (2014) · Psychological Science · doi:10.1177/0956797614535810 · Cited by 656 · see citing works
Handbook of Industrial, Work & Organizational Psychology
Anderson, N. R., Öneş, D. S., Sinangil, H. K., & Viswesvaran, C. (2001) · Sage eBooks · Cited by 632 · see citing works
Facets of visual aesthetics
Moshagen, M., & Thielsch, M. T. (2010) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2010.05.006 · Cited by 611 · see citing works
DOES PERFORMANCE IMPROVE FOLLOWING MULTISOURCE FEEDBACK? A THEORETICAL MODEL, META‐ANALYSIS, AND REVIEW OF EMPIRICAL FINDINGS
SMITHER, J. W., LONDON, M., & REILLY, R. R. (2005) · Personnel Psychology · doi:10.1111/j.1744-6570.2005.514_1.x · Cited by 601 · see citing works
Thriving at work: A meta‐analysis
Kleine, A., Rudolph, C. W., & Zacher, H. (2019) · Journal of Organizational Behavior · doi:10.1002/job.2375 · Cited by 606 · see citing works
Influence tactics and work outcomes: a meta analysis
Higgins, C. A., Judge, T. A., & Ferris, G. R. (2003) · Journal of Organizational Behavior · doi:10.1002/job.181 · Cited by 573 · see citing works
An Overview of Coefficient Alpha and a Reliability Matrix for Estimating Adequacy of Internal Consistency Coefficients with Psychological Research Measures
Ponterotto, J. G., & Ruckdeschel, D. E. (2007) · Perceptual and Motor Skills · doi:10.2466/pms.105.3.997-1014 · Cited by 555 · see citing works
The role of affectivity in job satisfaction: a meta-analysis
Connolly, J. J., & Viswesvaran, C. (2000) · Personality and Individual Differences · doi:10.1016/s0191-8869(99)00192-0 · Cited by 548 · see citing works
The causal relation between job attitudes and performance: A meta-analysis of panel studies.
Riketta, M. (2008) · Journal of Applied Psychology · doi:10.1037/0021-9010.93.2.472 · Cited by 547 · see citing works
Methodological Issues in Cross‐Sectional and Panel Estimates of the Human Resource‐Firm Performance Link
Huselid, M. A., & Becker, B. E. (1996) · Industrial Relations A Journal of Economy and Society · doi:10.1111/j.1468-232x.1996.tb00413.x · Cited by 533 · see citing works
Objective Light-Intensity Physical Activity Associations With Rated Health in Older Adults
Buman, M. P., Hekler, E. B., Haskell, W. L., Pruitt, L., Conway, T. L., Cain, K. L., Sallis, J. F., Saelens, B. E., Frank, L. D., & King, A. C. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq249 · Cited by 528 · see citing works
Turnover rates and organizational performance: A meta-analysis.
Park, T., & Shaw, J. D. (2012) · Journal of Applied Psychology · doi:10.1037/a0030723 · Cited by 511 · see citing works
Reliability from α to ω: A tutorial.
Revelle, W., & Condon, D. M. (2019) · Psychological Assessment · doi:10.1037/pas0000754 · Cited by 514 · see citing works
A meta-analytic review of behavior modeling training.
Taylor, P. J., Russ-Eft, D. F., & Chan, D. W. L. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.4.692 · Cited by 517 · see citing works
Effects of task performance and contextual performance on systemic rewards.
Scotter, J. R. V., Motowidlo, S. J., & Cross, T. C. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.4.526 · Cited by 495 · see citing works
Coefficient Alpha: A Reliability Coefficient for the 21st Century?
Yang, Y., & Green, S. B. (2011) · Journal of Psychoeducational Assessment · doi:10.1177/0734282911406668 · Cited by 480 · see citing works
HOW BROADLY DOES EDUCATION CONTRIBUTE TO JOB PERFORMANCE?
Ng, T. W. H., & Feldman, D. C. (2009) · Personnel Psychology · doi:10.1111/j.1744-6570.2008.01130.x · Cited by 474 · see citing works
Striking a Balance in Boundary-Spanning Positions: An Investigation of Some Unconventional Influences of Role Stressors and Job Characteristics on Job Outcomes of Salespeople
Singh, J. (1998) · Journal of Marketing · doi:10.1177/002224299806200305 · Cited by 459 · see citing works
An occupational information system for the 21st century: The development of O*NET.
Peterson, N. G., Mumford, M. D., Borman, W., Jeanneret, P., & Fleishman, E. (1999) · American Psychological Association eBooks · doi:10.1037/10313-000 · Cited by 430 · see citing works
Psychological Science Needs a Standard Practice of Reporting the Reliability of Cognitive-Behavioral Measurements
Parsons, S. D., Kruijt, A., & Fox, E. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919879695 · Cited by 424 · see citing works
Is There a General Factor in Ratings of Job Performance? A Meta-Analytic Framework for Disentangling Substantive and Error Influences.
Viswesvaran, C., Schmidt, F. L., & Öneş, D. S. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.1.108 · Cited by 417 · see citing works
Self-citation
Tackling the Problem of Construct Proliferation: A Guide to Assessing the Discriminant Validity of Conceptually Related Constructs
Shaffer, J. A., DeGeest, D., & Li, A. (2016) · Organizational Research Methods · doi:10.1177/1094428115598239 · Cited by 393 · see citing works
Detecting and measuring rater effects using many-facet Rasch measurement: Part II.
Myford, C. M., & Wolfe, E. W. (2003) · PubMed · Cited by 384 · see citing works
Research methods in sport and exercise psychology: quantitative and qualitative issues
Biddle, S., Markland, D. A., Gilbourne, D., Chatzisarantis, N. L. D., & Sparkes, A. C. (2001) · Journal of Sports Sciences · doi:10.1080/026404101317015438 · Cited by 379 · see citing works
Religion and the Forgiving Personality
McCullough, M. E., & Worthington, E. L. (1999) · Journal of Personality · doi:10.1111/1467-6494.00085 · Cited by 379 · see citing works
Transgression-Related Motivational Dispositions: Personality Substrates of Forgiveness and their Links to the Big Five
McCullough, M. E., & Hoyt, W. T. (2002) · Personality and Social Psychology Bulletin · doi:10.1177/014616702237583 · Cited by 367 · see citing works
Coefficient alpha and related internal consistency reliability coefficients.
Osburn, H. G. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.3.343 · Cited by 356 · see citing works
A meta-analytic study of general mental ability validity for different occupations in the European community.
Salgado, J., Anderson, N., Moscoso, S., Bertua, C., Fruyt, F. D., & Rolland, J. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.6.1068 · Cited by 328 · see citing works
Self-report multicultural counseling competence scales: Their relation to social desirability attitudes and multicultural case conceptualization ability.
Constantine, M. G., & Ladany, N. (2000) · Journal of Counseling Psychology · doi:10.1037/0022-0167.47.2.155 · Cited by 324 · see citing works
Rater bias in psychological research: When is it a problem and what can we do about it?
Hoyt, W. T. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.1.64 · Cited by 323 · see citing works
The Convergent Validity between Self and Observer Ratings of Personality: A meta‐analytic review
Connolly, J. J., Kavanagh, E. J., & Viswesvaran, C. (2007) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2007.00371.x · Cited by 320 · see citing works
Stability and change of personality traits, self-esteem, and well-being: Introducing the meta-analytic stability and change model of retest correlations.
Anusic, I., & Schimmack, U. (2015) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000066 · Cited by 321 · see citing works
Implications of direct and indirect range restriction for meta-analysis methods and findings.
Hunter, J. E., Schmidt, F. L., & Le, H. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.3.594 · Cited by 315 · see citing works
Self-citation
Personnel selection: looking toward the future--remembering the past.
Hough, L., & Oswald, F. (2000) · Annual Review of Psychology · doi:10.1146/annurev.psych.51.1.631 · Cited by 358 · see citing works
Principles for the Validation and Use of Personnel Selection Procedures
Anonymous (2018) · Industrial and Organizational Psychology · doi:10.1017/iop.2018.195 · Cited by 308 · see citing works
Religion and depression: a review of the literature
McCullough, M. E., & Larson, D. B. (1999) · Twin Research · doi:10.1375/136905299320565997 · Cited by 297 · see citing works
Human Resources and Business Performance: Findings, Unanswered Questions, and an Alternative Approach
Gerhart, B. (2005) · management revue · doi:10.5771/0935-9915-2005-2-174 · Cited by 295 · see citing works
The need for work: Jahoda's latent functions of employment in a representative sample of the German population
Paul, K. I., & Batinic, B. (2009) · Journal of Organizational Behavior · doi:10.1002/job.622 · Cited by 278 · see citing works
The problem of empirical redundancy of constructs in organizational research: An empirical investigation
Le, H., Schmidt, F. L., Harter, J. K., & Lauver, K. J. (2010) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2010.02.003 · Cited by 272 · see citing works
Self-citation
Beyond alpha: An empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual-differences constructs.
Schmidt, F. L., Le, H., & Ilies, R. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.2.206 · Cited by 362 · see citing works
Self-citation
Understanding attitudes toward affirmative action programs in employment: Summary and meta-analysis of 35 years of research.
Harrison, D. A., Kravitz, D. A., Mayer, D. M., Leslie, L. M., & Lev-Arey, D. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.5.1013 · Cited by 358 · see citing works
Theory testing and measurement error
Schmidt, F. L., & Hunter, J. E. (1999) · Intelligence · doi:10.1016/s0160-2896(99)00024-0 · Cited by 254 · see citing works
Self-citation
We need to talk about reliability: making better use of test-retest studies for study design and interpretation
Matheson, G. J. (2019) · PeerJ · doi:10.7717/peerj.6918 · Cited by 246 · see citing works
What Doesn’t Get Measured Does Exist: Improving the Accuracy of Computer-Aided Text Analysis
McKenny, A. F., Aguinis, H., Short, J. C., & Anglin, A. H. (2016) · Journal of Management · doi:10.1177/0149206316657594 · Cited by 257 · see citing works
INTERNATIONAL VALIDITY GENERALIZATION OF GMA AND COGNITIVE ABILITIES: A EUROPEAN COMMUNITY META‐ANALYSIS
Salgado, J. F., Anderson, N. R., Moscoso, S., Bertua, C., & Fruyt, F. D. (2003) · Personnel Psychology · doi:10.1111/j.1744-6570.2003.tb00751.x · Cited by 238 · see citing works
Test of a service profit chain model in the retail banking sector
Gelade, G. A., & Young, S. (2005) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317904x22926 · Cited by 231 · see citing works
Using regression to measure holistic face processing reveals a strong link with face recognition ability
Degutis, J. M., Wilmer, J. B., Mercado, R. J., & Cohan, S. (2012) · Cognition · doi:10.1016/j.cognition.2012.09.004 · Cited by 229 · see citing works
The jingle and jangle of emotion assessment: Imprecise measurement, casual scale usage, and conceptual fuzziness in emotion research.
Weidman, A. C., Steckler, C. M., & Tracy, J. L. (2016) · Emotion · doi:10.1037/emo0000226 · Cited by 219 · see citing works
Multi-criteria supplier segmentation using a fuzzy preference relations based AHP
Rezaei, J., & Ortt, R. (2012) · European Journal of Operational Research · doi:10.1016/j.ejor.2012.09.037 · Cited by 217 · see citing works
Strategic objectives for improving understanding of informant discrepancies in developmental psychopathology research
Reyes, A. D. L. (2013) · Development and Psychopathology · doi:10.1017/s0954579413000096 · Cited by 216 · see citing works
Measuring Belief in Conspiracy Theories: Validation of a French and English Single-Item Scale
Lantian, A., Muller, D., Nurra, C., & Douglas, K. M. (2016) · International Review of Social Psychology · doi:10.5334/irsp.8 · Cited by 187 · see citing works
The Five-Factor Model, forced-choice personality inventories and performance: A comprehensive meta-analysis of academic and occupational validity studies
Salgado, J. F., & Táuriz, G. (2014) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2012.716198 · Cited by 205 · see citing works
The Dispositional Sources of Job Satisfaction: A Comparative Test
Judge, T. A., Heller, D. A., & Klinger, R. L. (2008) · Applied Psychology · doi:10.1111/j.1464-0597.2007.00318.x · Cited by 201 · see citing works
Time is on my side: Time, general mental ability, human capital, and extrinsic career success.
Judge, T. A., Klinger, R. L., & Simon, L. (2010) · Journal of Applied Psychology · doi:10.1037/a0017594 · Cited by 200 · see citing works
A Monte Carlo Simulation of Observable Versus Latent Variable Structural Equation Modeling Techniques
Stephenson, M. T., & Holbert, R. L. (2003) · Communication Research · doi:10.1177/0093650203030003004 · Cited by 197 · see citing works
Some better practices for measuring racial and ethnic identity constructs.
Helms, J. E. (2007) · Journal of Counseling Psychology · doi:10.1037/0022-0167.54.3.235 · Cited by 197 · see citing works
Explaining the Weak Relationship Between Job Performance and Ratings of Job Performance
Murphy, K. R. (2008) · Industrial and Organizational Psychology · doi:10.1111/j.1754-9434.2008.00030.x · Cited by 195 · see citing works
Ancient Pygmalion joins contemporary management: A meta-analysis of the result.
McNatt, D. B. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.2.314 · Cited by 192 · see citing works
Psychometric and Rationalization Accounts of the Religion‐Forgiveness Discrepancy
Tsang, J.-A. C., McCullough, M. E., & Hoyt, W. T. (2005) · Journal of Social Issues · doi:10.1111/j.1540-4560.2005.00432.x · Cited by 191 · see citing works
EXPLORING RELATIONS BETWEEN TYPICAL AND MAXIMUM PERFORMANCE RATINGS AND THE FIVE FACTOR MODEL OF PERSONALITY
Ployhart, R. E., Lim, B., & Chan, K.-Y. (2001) · Personnel Psychology · doi:10.1111/j.1744-6570.2001.tb00233.x · Cited by 189 · see citing works
Measurement Error in “Big Five Factors” Personality Assessment: Reliability Generalization across Studies and Measures
Viswesvaran, C., & Öneş, D. S. (2000) · Educational and Psychological Measurement · doi:10.1177/00131640021970475 · Cited by 187 · see citing works
To prosper, organizational psychology should … overcome methodological barriers to progress
Edwards, J. R. (2008) · Journal of Organizational Behavior · doi:10.1002/job.529 · Cited by 186 · see citing works
INTERRATER CORRELATIONS DO NOT ESTIMATE THE RELIABILITY OF JOB PERFORMANCE RATINGS
Murphy, K. R., & DeShon, R. P. (2000) · Personnel Psychology · doi:10.1111/j.1744-6570.2000.tb02421.x · Cited by 183 · see citing works
Developmental change in the acuity of approximate number and area representations.
Odic, D., Libertus, M. E., Feigenson, L., & Halberda, J. (2012) · Developmental Psychology · doi:10.1037/a0029472 · Cited by 182 · see citing works
What is being assessed and why it matters: The impact of transient error on trait research.
Chmielewski, M. S., & Watson, D. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0015618 · Cited by 175 · see citing works
The antecedents and effects of national corruption: A meta-analysis
Judge, W. Q., McNatt, D. B., & Xu, W. (2010) · Journal of World Business · doi:10.1016/j.jwb.2010.05.021 · Cited by 174 · see citing works
Supervisee multicultural case conceptualization ability and self-reported multicultural competence as functions of supervisee racial identity and supervisor focus.
Ladany, N., Inman, A. G., Constantine, M. G., & Hofheinz, E. W. (1997) · Journal of Counseling Psychology · doi:10.1037//0022-0167.44.3.284 · Cited by 173 · see citing works
Alpha's standard error (ASE): an accurate and precise confidence interval estimate.
Duhachek, A., & Iacobucci, D. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.5.792 · Cited by 176 · see citing works
Intuitive sense of number correlates with math scores on college-entrance examination
Libertus, M. E., Odic, D., & Halberda, J. (2012) · Acta Psychologica · doi:10.1016/j.actpsy.2012.09.009 · Cited by 168 · see citing works
Detecting and Correcting the Lies That Data Tell
Schmidt, F. (2010) · Perspectives on Psychological Science · doi:10.1177/1745691610369339 · Cited by 163 · see citing works
Self-citation
The Impact of Job Complexity and Performance Measurement on the Temporal Consistency, Stability, and Test-Retest Reliability of Employee Job Performance Ratings.
Sturman, M. C., Cheramie, R., & Cashen, L. H. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.2.269 · Cited by 164 · see citing works
Cognitive mediators and sex-related differences in mathematics
Delgado, A. R., & Prieto, G. (2003) · Intelligence · doi:10.1016/s0160-2896(03)00061-8 · Cited by 161 · see citing works
The correction for attenuation due to measurement error: clarifying concepts and creating confidence sets
Charles, E. P. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.2.206 · Cited by 154 · see citing works
Personality Traits in Adolescence as Predictors of Religiousness in Early Adulthood: Findings from the Terman Longitudinal Study
McCullough, M. E., Tsang, J.-A. C., & Brion, S. L. (2003) · Personality and Social Psychology Bulletin · doi:10.1177/0146167203253210 · Cited by 157 · see citing works
Assessment of Individual Job Performance: A Review of the past Century and a Look Ahead
Viswesvaran, C. (2012) · SAGE Publications Ltd eBooks · doi:10.4135/9781848608320.n7 · Cited by 155 · see citing works
A meta‐analysis of the interrelationships between employee lateness, absenteeism, and turnover: Implications for models of withdrawal behavior
Berry, C. M., Lelchook, A. M., & Clark, M. A. (2011) · Journal of Organizational Behavior · doi:10.1002/job.778 · Cited by 154 · see citing works
Narcissists as “Victims”: The Role of Narcissism in the Perception of Transgressions
McCullough, M. E., Emmons, R. A., Kilpatrick, S. D., & Mooney, C. N. (2003) · Personality and Social Psychology Bulletin · doi:10.1177/0146167203029007007 · Cited by 154 · see citing works

Authors: Schmidt, F. L., & Hunter, J. E.
Affiliations: University of Iowa (US)
ORCID: Hunter, John E.
Venue: Psychological Methods
Publisher: American Psychological Association (APA)
Published: 1996-06
Volume: 1
Issue: 2
Pages: 199-223
ISSN: 1082-989X (ISSN-L)
1939-1463
DOI: 10.1037/1082-989x.1.2.199
OpenAlex: W2051130243
Cited by: 567
References: 24
Citations by year: 2026: 19 · 2025: 15 · 2024: 28 · 2023: 16 · 2022: 16 · 2021: 25 · 2020: 13 · 2019: 21 · 2018: 24 · 2017: 17 · 2016: 22 · 2015: 21 · 2014: 18 · 2013: 21 · 2012: 29
FWCI: 26.81
Type: journal article
Language: en
Open access: closed
Locations: Psychological Methods (publishedVersion)
Topics: Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Keywords: measurement error, reliability theory, classical test theory, research scenarios, case studies

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Measurement error in psychological research: Lessons from 26 research scenarios.

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks