On the Relationship Between Maximal Reliability and Maximal Validity of Linear Composites
Open accessAbstract
A linear combination of a set of measures is often sought as an overall score summarizing subject performance. The weights in this composite can be selected to maximize its reliability or to maximize its validity, and the optimal choice of weights is in general not the same for these two optimality criteria. We explore several relationships between the resulting reliability and validity estimates in different situations. Only in the case of congeneric tests are maximal reliability and maximal validity attained with the same weights, and a precise relationship between these two maximality concepts can be derived. A widely and readily applicable procedure for point and interval estimation of maximal validity is also outlined. Several inequalities are established for the case when the measures are not congeneric.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The estimation of internal consistency reliability of unequally weighted composite scores is more complicated and less well-established.
“The estimation of internal consistency reliability of unequally weighted composite scores is more complicated and less well-established [67].”
From The Role of Conspiracist Ideation and Worldviews in Predicting Rejection of Science (2013) · cites this paper as [67]
Cites
Structural Equations with Latent Variables
Bollen, K. A. (1989) · Wiley series in probability and statistics · doi:10.1002/9781118619179 · Cited by 17,512 · see citing works
Structural Equations with Latent Variables.
Clogg, C. C., & Bollen, K. A. (1991) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2072165 · Cited by 17,471 · see citing works
Linear Statistical Inference and its Applications
Rao, C. R. (1973) · Wiley series in probability and statistics · doi:10.1002/9780470316436 · Cited by 10,474 · see citing works
EQS : structural equations program manual
Bentler, P. M. (1989) · Medical Entomology and Zoology · Cited by 9,669 · see citing works
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012 · see citing works
Linear Statistical Inference and Its Applications.
Robertson, C., & Rao, C. R. (1975) · Biometrics · doi:10.2307/2529568 · Cited by 3,611 · see citing works
Matrix Differential Calculus with Applications in Statistics and Econometrics.
Magnus, J. R., & Neudecker, H. (1988) · Biometrics · doi:10.2307/2531754 · Cited by 2,734 · see citing works
Statistical analysis of sets of congeneric tests
Jöreskog, K. G. (1971) · Psychometrika · doi:10.1007/bf02291393 · Cited by 1,481 · see citing works
STATISTICAL ANALYSIS OF SETS OF CONGENERIC TESTS*
Jöreskog, K. G. (1969) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1969.tb00777.x · Cited by 1,205 · see citing works
Statistical theories of mental test scores
Lord, F. M., Novick, M. R., & Birnbaum, A. (1974) · Cited by 1,174 · see citing works
Time-Limit Tests: Estimating their Reliability and Degree of Speeding
Cronbach, L. J., & Warrington, W. G. (1951) · Psychometrika · doi:10.1007/bf02289113 · Cited by 675 · see citing works
British Journal of Mathematical and Statistical Psychology
British Psychological Society (2010) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/(issn)2044-8317 · Cited by 389 · see citing works
The Orthogonal Approximation of an Oblique Structure in Factor Analysis
Green, B. F. (1952) · Psychometrika · doi:10.1007/bf02288918 · Cited by 234 · see citing works
THE ORTHOGONAL APPROXIMATION OF AN OBLIQUE STRUCTURE IN FACTOR ANALYSIS
Green, B. F. (1951) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1951.tb00218.x · Cited by 205 · see citing works
Reliability of measurement in psychology: From Spearman-Brown to maximal reliability.
Li, H., Rosenthal, R., & Rubin, D. B. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.98 · Cited by 132 · see citing works
The Statistical Approach to Social Measurement
Bartholomew, D. J. (1996) · Cited by 123 · see citing works
Yuan, K., & Bentler, P. M. (2002) · Psychometrika · doi:10.1007/bf02294845 · Cited by 88 · see citing works
Estimation of maximal reliability: A note on a covariance structure modelling approach
Raykov, T. (2004) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711004849295 · Cited by 79 · see citing works
Self-citationA Unifying Expression for the Maximal Reliability of a Linear Composite
Li, H. (1997) · Psychometrika · doi:10.1007/bf02295278 · Cited by 63 · see citing works
A Note on the Calculation of Weights for Maximum Battery Reliability
Green, B. F. (1950) · Psychometrika · doi:10.1007/bf02289178 · Cited by 41 · see citing works
Informed Test Component Weighting
Rudner, L. M. (2001) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.2001.tb00054.x · Cited by 36 · see citing works
The Resolution of Some Paradoxes Related to Reliability and Validity
Li, H. (2003) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986028002089 · Cited by 25 · see citing works
Maximally Reliable Composites for Unidimensional Measures
Conger, A. J. (1980) · Educational and Psychological Measurement · doi:10.1177/001316448004000213 · Cited by 18 · see citing works
Raykov, T., & Penev, S. I. (2006) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr4101_2 · Cited by 13 · see citing works
Self-citationElectronic structure of testosterone: A semiempirical and ab initio assessment
Kubli‐Garfias, C. (1997) · International Journal of Quantum Chemistry · doi:10.1002/(sici)1097-461x(1997)62:3 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Role of Conspiracist Ideation and Worldviews in Predicting Rejection of Science
Lewandowsky, S., Gignac, G. E., & Oberauer, K. (2013) · PLoS ONE · doi:10.1371/journal.pone.0075637 · Cited by 693 · see citing works
- Authors
- Penev, S., & Raykov, T.
- Venue
- Multivariate Behavioral Research
- Publisher
- Informa UK Limited
- Published
- 2006-06-01
- Volume
- 41
- Issue
- 2
- Pages
- 105-126
- ISSN
- 0027-3171 (ISSN-L)
- OpenAlex
- W2028358230
- Cited by
- 26
- References
- 25
- Citations by year
- 2026: 1 · 2020: 1 · 2019: 1 · 2018: 3 · 2017: 1 · 2016: 3 · 2015: 2 · 2014: 4 · 2013: 3 · 2012: 2
- FWCI
- 1.45
- Type
- journal article
- Language
- en
- Open access
- green
- handle.unsw.edu.au
- Licence
- cc-by-nc-nd
- Locations
- Multivariate Behavioral Research (publishedVersion)
- PubMed (publishedVersion)
- UNSWorks (UNSW Sydney) (submittedVersion, cc-by-nc-nd)
- UNSWorks (University of New South Wales, Sydney, Australia) (submittedVersion, cc-by-nc-nd)
- Funders
- University at Buffalo
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Education › Educational Assessment and Pedagogy
- Keywords
- inequality (mathematics), psychometrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
