Stamp collecting versus science: Validation as hypothesis testing.

Abstract

There have been continuing arguments with respect to the unitarian versus trinitarian view of the validation process. An emerging position is that the threefold distinction among validation models presented by Cronbach and Meehl is not as useful as it once was. This article joins in the unitarian argument and suggests that the validation process be considered nothing more and nothing less than traditional hypothesis testing. The issue of the role of constructs in psychological measurement is considered, as is the general status of construct-validation strategies in employment testing. In addition, the unique knowledges, skills, and abilities that psychologists bring to the discussion of validity and inference are discussed. During the past 15 years, there have been many occasions to regret the simple taxonomy of validity strategies suggested by the American Psychological Association in their Technical for Psychological Tests and Diagnostic Techniques (1954). Cronbach and Meehl were members of the APA committee that developed those recommendations. They further elaborated on the notion of construct validity in a subsequent publication in 1955 (Cronbaeh & Meehl, 1955). The Technical Recommendations proposed a four-category taxonomy of validity The taxonomy consisted of predictive, concurrent, content, and construct validity. The predictive and concurrent models are usually combined into a category known as criterion-related validation strategies. Thus, it is common to hear reference made to three of validity. This view of validity has been aptly labeled the trinitarian view (Guion, 1980). At the time that the Technical Recommendations were published, the taxonomy was quite valuable. It provided order where before there was chaos. It added the preliminary structure necessary for progress. For some time, this taxonomy proved a useful tool for validity discussions. The taxonomy represented a point of departure. Prior to that point, validity was considered a correlation between a predictor and a criterion. It was a literalist view directed toward the technology of prediction.Such a positivist view was (and remains) only minimally helpful in developing theories of behavior or a basic understanding of what was being measured by the tests in question. The introduction of the Technical Recommendations and the concepts of content-oriented and construct validation represented an attempt to correct the past excesses of dust-bowl empiricism and focus attention on understanding rather than on the simpler and less meaningful search for predictability. The value of that reorientation has been substantially blunted in the past 15 years as a result of a reification of the validity taxonomy that was introduced in 1954. This result is unfortunate because the emphasis on the meaning of test scores rather than the predictive value of those scores is as important today as it was 30 years ago. As a result of the efforts of practitioners (lawyers, judges, expert witnesses, and personnel administrators) to make the taxonomy into something it was never intended to be, the point of departure suggested by Cronbach and Meehl (1955) has become a point of arrival. The three types of validity that were originally proposed as a method for broadening the appreciation of the inferential limits of test scores have become an orthodoxy. Because forensic activity is usually deductive in nature, there has been the tendency in Title VII litigation to try to identify the validity model most appropriate for a given situation. Once the validity model has been identified, the case revolves around whether the requirements of the model have been met. In principle, this is no different from the way in which any court case is approached: Determine which principle covers the issue under scrutiny, and then determine if that particular principle has been violated. This approach can be quite effective if the are clear and distinct from one another and if all the parameters of the are accepted by all the parties involved. In psychology, an example of the value of this approach has been the development of the DSM-III for the diagnosis of psychological disorders (American Psychiatric Association, 1980). Although it is not without its detractors, the DSM-III is a useful way of determining from a set of observable symptoms what might be responsible for those symptoms and how those symptoms might be best treated. The parameters of the diagnostic model are clearly defined and understood (the five axes of the DSM-III), and there is a clear listing of the symptoms that warrant one diagnosis as opposed to another. Many would like to use the Uniform Guidelines on Employee Selection Procedures (1978) in a similar manner. Unfortunately, there is substantial disagreement about the meaning and the independence of the involved in determining the legality of certain classes of employment decisions. The are, for the most part, validity models. However, if these models are accept

Cites (16)

1. Convergent and discriminant validation by the multitrait-multimethod matrix.
   Campbell, Donald T., Fiske, Donald W. · 1959 · Psychological Bulletin
2. Construct validity in psychological tests.
   Cronbach, Lee J., Meehl, Paul E. · 1955 · Psychological Bulletin
3. Diagnostic and Statistical Manual of Mental Disorders, 3rd ed
   COOPER, ARNOLD M., MICHELS, ROBERT · 1981 · American Journal of Psychiatry
4. Objective Tests as Instruments of Psychological Theory
   Loevinger, Jane · 1957 · Psychological Reports
5. On a distinction between hypothetical constructs and intervening variables.
   MacCorquodale, Kenneth, Meehl, Paul E. · 1948 · Psychological Review
6. Test validity and the ethics of assessment.
   Messick, Samuel · 1980 · American Psychologist
7. Construct validity: Construct representation versus nomothetic span.
   Whitely, Susan E. · 1983 · Psychological Bulletin
8. The Nature of Physical Reality
   Margenau, Henry, Lindsay, R. B. · 1951 · American Journal of Physics
9. Personnel Selection and Classification Systems
   Dunnette, M D, Borman, W C · 1979 · Annual Review of Psychology
10. On Trinitarian doctrines of validity.
    Guion, Robert M. · 1980 · Professional Psychology

Show all 16 references

Cited by (14)

1. Emotional intelligence: An integrative meta-analysis and cascading model
   Joseph, Dana L., Newman, Daniel A. · 2010 · Journal of Applied Psychology
2. Construct Validity: Advances in Theory and Methodology
   Strauss, Milton E., Smith, Gregory T. · 2009 · Annual Review of Clinical Psychology
3. A Review of Key Likert Scale Development Advances: 1995–2019
   Jebb, Andrew T., Ng, Vincent, Tay, Louis · 2021 · Frontiers in Psychology
4. The reliability and validity of discrete and continuous measures of psychopathology: A quantitative review.
   Markon, Kristian E., Chmielewski, Michael, Miller, Christopher J. · 2011 · Psychological Bulletin
5. Introduction to health measurement scales
   Keszei, András P., Novak, Márta, Streiner, David L. · 2010 · Journal of Psychosomatic Research

See all 263