Cited by 940Psychometric Methodologies and TestingPersonality Traits and PsychologyPsychological Testing and Assessment

The DOI matches its Crossref record; 98 of the 100 references it cites were found in a registry or an index.

Behavioral Development and Construct Validity: The Principle of Aggregation

Rushton, J. Philippe, Brainerd, Charles J., Pressley, Michael·1983·Psychological Bulletin

Open this paper in CoChat — free account

Abstract

Many important variables in behavioral development are presumed to be unrelated because of repeated failures to obtain substantial correlations. In this article, we explore the possibility that such null findings have often been due to failures to aggregate. The principle of aggregation states that the sum of a set of multiple measurements is a more stable and representative estimator than any single measurement. This greater representation occurs because there is inevitably some error associated with measurement. By combining numerous exemplars, such errors of measurement are averaged out, leaving a clearer view of underlying relationships. We illustrate the usefulness of this principle in 12 major areas of developmental research in which the issue of negligible correlations figures prominently: (a) the validity of judges' ratings, (b) the cross-situational consistency of moral character and personality, (c) the longitudinal stability of personality, (d) the coherence of stages of cognitive development, (e) metacognition, (f) the attitude—behavior relationship, (g) the personality—behavior relationship, (h) the role-taking/altruism relationship, (i) the moral-judgment/altruism relationship, (j) the legitimacy of the construct of attachment, (k) the existence of sex differences, and (1) the assessment of emotionality in animals. In a final section, we also discuss the implications of the principle of aggregation for conducting experimental research.

Integrity signals

Registry

Crossref · 10.1037/0033-2909.94.1.18

Cites

98 of 98 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Social Learning Theory
Kytle, J., & Bandura, A. (1978) · Contemporary Sociology · doi:10.2307/2065952 · Cited by 25,422
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
Reviews: Lord, Frederic M., and Novick, Melvin R. (with contributions by Allan Birnbaum). Statistical Theories of Mental Test Scores. Reading, Mass.: Addison-Wesley, 1968. xii + 568 pp. $14.95.
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012
The psychology of sex differences
Maccoby, E. E. (1975) · Behavior Therapy · doi:10.1016/s0005-7894(75)80031-1 · Cited by 5,714
Principles of behavior modification
Bandura, A. (1969) · Medical Entomology and Zoology · Cited by 4,631
Précis of The neuropsychology of anxiety: An enquiry into the functions of the septo-hippocampal system
Gray, J. A. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00013066 · Cited by 4,477
The Moral Judgment Of The Child
Piaget, J. (2013) · Routledge eBooks · doi:10.4324/9781315009681 · Cited by 4,409
Personality and Assessment
Madison, P., & Mischel, W. (1968) · The American Journal of Psychology · doi:10.2307/1421067 · Cited by 4,395
Aggression: A Social Learning Analysis
Bandura, A. (1973) · Stanford Law Review · doi:10.2307/1227918 · Cited by 4,302
Toward a cognitive social learning reconceptualization of personality
Mischel, W. (1973) · Psychological Review · doi:10.1037/h0035002 · Cited by 3,181
The psychology of fear and stress
(1988) · Choice Reviews Online · doi:10.5860/choice.26-2399 · Cited by 3,133
The two disciplines of scientific psychology.
Cronbach, L. J. (1957) · American Psychologist · doi:10.1037/h0043943 · Cited by 2,590
The developmental psychology of Jean Piaget.
Flavell, J. H. (1963) · D Van Nostrand eBooks · doi:10.1037/11449-000 · Cited by 2,389
Emotion, a psychoevolutionary synthesis
Plutchik, R. (1980) · The American Journal of Psychology · doi:10.2307/1422394 · Cited by 2,316
Theory of Mental Tests
Gulliksen, H. O. (1950) · John Wiley & Sons, Inc. eBooks · doi:10.1037/13240-000 · Cited by 2,184
Meta-analysis of psychotherapy outcome studies.
Smith, M. L., & Glass, G. V. (1977) · American Psychologist · doi:10.1037/0003-066x.32.9.752 · Cited by 2,154
Progress of Veterinary Science.
Jackson, D. N. (1882) · PubMed Central · Cited by 1,971
Bridging moral cognition and moral action: A critical review of the literature.
Blasi, A. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.1.1 · Cited by 1,500
The stability of behavior: I. On predicting most of the people much of the time.
Epstein, S. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.7.1097 · Cited by 1,468
The Q-sort method in personality assessment and psychiatric research.
Block, J. (1961) · Charles C Thomas Publisher eBooks · doi:10.1037/13141-000 · Cited by 1,405
Attitudes towards objects as predictors of single and multiple behavioral criteria.
Fishbein, M., & Ajzen, I. (1974) · Psychological Review · doi:10.1037/h0035872 · Cited by 1,404
The structure of human personality
Eysenck, H. J. (1953) · Medical Entomology and Zoology · Cited by 1,322
CORRELATION CALCULATED FROM FAULTY DATA
Spearman, C. (1910) · British Journal of Psychology 1904-1920 · doi:10.1111/j.2044-8295.1910.tb00206.x · Cited by 1,215
Continuity of Adaptation in the Second Year: The Relationship between Quality of Attachment and Later Competence
Matas, L., Arend, R. A., & Sroufe, L. A. (1978) · Child Development · doi:10.2307/1128221 · Cited by 1,164
A Model for Personality
Eysenck, H. J. (1981) · Springer eBooks · doi:10.1007/978-3-642-67783-0 · Cited by 1,026
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022 · cited 1×
“The nongeneralizability of experimental findings to similar situations and, sometimes, their nonreplicability have been sources of concern (Greenwald, 1975).”
Background
Toward a general psychobiological theory of emotions
Panksepp, J. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00012759 · Cited by 999
Perspectives on the Development of Memory and Cognition
Kail, R. V., & Hagen, J. W. (1977) · Medical Entomology and Zoology · Cited by 927
Combining results of independent studies.
Rosenthal, R. (1978) · Psychological Bulletin · doi:10.1037/0033-2909.85.1.185 · Cited by 910
Lives Through Time, 1932-1958
Block, J. (1987) · Harvard Dataverse · doi:10.7910/dvn/8wg8xd · Cited by 862
Infant Irritability, Mother Responsiveness, and Social Support Influences on the Security of Infant-Mother Attachment
Crockenberg, S. B. (1981) · Child Development · doi:10.1111/j.1467-8624.1981.tb03124.x · Cited by 793
Gender effects in decoding nonverbal cues.
Hall, J. A. (1978) · Psychological Bulletin · doi:10.1037/0033-2909.85.4.845 · Cited by 766
Sex differences in empathy and related behaviors.
Hoffman, M. L. (1977) · Psychological Bulletin · doi:10.1037/0033-2909.84.4.712 · Cited by 755
The stability of behavior: II. Implications for psychological research.
Epstein, S. J. (1980) · American Psychologist · doi:10.1037/0003-066x.35.9.790 · Cited by 744
Attachment, Positive Affect, and Competence in the Peer Group: Two Studies in Construct Validation
Waters, E., Wippman, J., & Sroufe, L. A. (1979) · Child Development · doi:10.1111/j.1467-8624.1979.tb02431.x · Cited by 709
Studies in the Nature of Character.
Borgeson, F. C., Hartshorne, H., May, M. A., Maller, J. B., & Shuttleworth, F. K. (1930) · Journal of Educational Sociology · doi:10.2307/2960812 · Cited by 693
Socialization and Society.
Moore, J. E., & Clausen, J. A. (1969) · American Sociological Review · doi:10.2307/2092350 · Cited by 624
Beyond déjà vu in the search for cross-situational consistency.
Mischel, W., & Peake, P. K. (1982) · Psychological Review · doi:10.1037/0033-295x.89.6.730 · Cited by 617
Continuity of Individual Adaptation from Infancy to Kindergarten: A Predictive Study of Ego-Resiliency and Curiosity in Preschoolers
Arend, R., Gove, F. L., & Sroufe, L. A. (1979) · Child Development · doi:10.1111/j.1467-8624.1979.tb02454.x · Cited by 591
Perspective-taking and altruism.
Underwood, B., & Moore, B. S. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.91.1.143 · Cited by 544
Attachment and Early Maltreatment
Egeland, B. R., & Sroufe, L. A. (1981) · Child Development · doi:10.2307/1129213 · Cited by 544
An Interview Study of Children's Knowledge about Memory
Kreutzer, M. A., Leonard, C., Flavell, J. H., & Hagen, J. W. (1975) · Monographs of the Society for Research in Child Development · doi:10.2307/1165955 · Cited by 514
The stage question in cognitive-developmental theory
Brainerd, C. J. (1978) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00073842 · Cited by 512
Self-citation
An exercise in mega-silliness.
Eysenck, H. J. (1978) · American Psychologist · doi:10.1037/0003-066x.33.5.517.a · Cited by 466
Statistically combining independent studies: A meta-analysis of sex differences in conformity research.
Cooper, H. M. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.1.131 · Cited by 418
Constancy and Change in Human Development.
Schooler, C., Brim, O. G., & Kagan, J. (1982) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2067080 · Cited by 402
Personality: Structure And Assessment
Wiggins, J. S., & Pincus, A. L. (1992) · Annual Review of Psychology · doi:10.1146/annurev.ps.43.020192.002353 · Cited by 393
The Reliability and Stability of Individual Differences in Infant-Mother Attachment
Waters, E. (1978) · Child Development · doi:10.2307/1128714 · Cited by 387
Piaget's theory of intelligence
Brainerd, C. J. (1978) · Cited by 384
Self-citation
Stimulation in Early Infancy.
Jones, N. B., & Ambrose, A. (1970) · Man · doi:10.2307/2799173 · Cited by 383
Personality traits and the eye of the beholder: Crossing some traditional philosophical boundaries in the search for consistency in all of the people.
Kenrick, D. T., & Stringfield, D. O. (1980) · Psychological Review · doi:10.1037/0033-295x.87.1.88 · Cited by 328
Studies in Deceit.
H., A. B., Hartshorne, H., & May, M. A. (1929) · Journal Of The Royal Statistical Society · doi:10.2307/2341443 · Cited by 307
How and How Much Can Intelligence Be Increased
Detterman, D. K., & Sternberg, R. J. (1982) · Medical Entomology and Zoology · Cited by 307
Annals of Theoretical Psychology
Mos, L. P. (1986) · Springer eBooks · doi:10.1007/978-1-4615-6453-9 · Cited by 258
Altruism, Socialization and Society.
Neal, M. A., & Rushton, J. P. (1982) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2068841 · Cited by 228
Self-citation
Validity comparison of behavior counts and ratings by knowledgeable informants.
Moskowitz, D. S., & Schwarz, J. C. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.42.3.518 · Cited by 192
Generality of honesty reconsidered.
Burton, R. V. (1963) · Psychological Review · doi:10.1037/h0047594 · Cited by 168
Studies in the nature of character, Vol. 3: Studies in the organization of character.
Hartshorne, H., May, M. A., & Shuttleworth, F. K. (1930) · MacMillan Co eBooks · doi:10.1037/13357-000 · Cited by 157
Further explorations in personality
Rabin, A. I. (1981) · Wiley eBooks · Cited by 144
The Relationship between Out-of-Home Care and the Quality of Infant-Mother Attachment in an Economically Disadvantaged Population
Vaughn, B. E., Gove, F. L., & Egeland, B. R. (1980) · Child Development · doi:10.2307/1129562 · Cited by 142
Clinical and statistical prediction
Wiggins, J. S. (1981) · Clinical Psychology Review · doi:10.1016/0272-7358(81)90015-5 · Cited by 134
Present and Past in Middle Life
Brown, S. R., Eichorn, D. H., Clausen, J. A., Haan, N., Honzik, M. P., & Mussen, P. H. (1983) · Political Psychology · doi:10.2307/3791073 · Cited by 134
Individual Differences in Infant-Mother Attachment Relationships at Age One: Antecedents in Neonatal Behavior in an Urban, Economically Disadvantaged Sample
Waters, E., Vaughn, B. E., & Egeland, B. R. (1980) · Child Development · doi:10.2307/1129608 · Cited by 129
Group Judgments in the Field of Lifted Weights.
Gordon, K. (1924) · Journal of Experimental Psychology · doi:10.1037/h0065015 · Cited by 126
Critical periods in the organization of systems
Scott, J. G., Stewart, J. M., & Ghett, V. J. D. (1974) · Developmental Psychobiology · doi:10.1002/dev.420070602 · Cited by 120
Studies in the Organization of Character
Kirkpatrick, E. A., Hartshorne, H., May, M. A., & Shuttleworth, F. K. (1932) · The American Journal of Psychology · doi:10.2307/1414998 · Cited by 117
Personality, research creativity, and teaching effectiveness in university professors
Rushton, J. P., Murray, H. G., & Paunonen, S. V. (1983) · Scientometrics · doi:10.1007/bf02072856 · Cited by 115
Self-citation
Studies in the nature of character, [part] 1: Studies in deceit: book 1, General methods and results; book 2, Statistical methods and results.
Hartshorne, H., May, M. A., & Inquiry, C. E. (1930) · MacMillan Co eBooks · doi:10.1037/13386-000 · Cited by 114
Predicting social behavior from personality traits
Jaccard, J. J. (1974) · Journal of Research in Personality · doi:10.1016/0092-6566(74)90057-9 · Cited by 110
Preterm Infant Interaction with the Caregiver in the First Year of Life and Competence at Age Two
Cohen, S. E., & Beckwith, L. (1979) · Child Development · doi:10.1111/j.1467-8624.1979.tb02426.x · Cited by 109
Traits, tests, and personality research.
Hogan, R. T., DeSoto, C. B., & Solano, C. H. (1977) · American Psychologist · doi:10.1037//0003-066x.32.4.255 · Cited by 104
The construct validity of egocentrism.
Ford, M. E. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.6.1169 · Cited by 104
Attachment to Substitute Caregivers as a Function of Center Quality and Caregiver Involvement
Anderson, C. W., Nagle, R. J., Roberts, W. A., & Smith, J. W. (1981) · Child Development · doi:10.1111/j.1467-8624.1981.tb03016.x · Cited by 100
The Development of Quantitative Thinking: A Systematic Replication of Piaget's Studies
Elkind, D. (1961) · The Journal of Genetic Psychology · doi:10.1080/00221325.1961.10534349 · Cited by 97
Steepness of approach and avoidance gradient in humans as a function of experience: Theory and experiment.
Epstein, S. J., & Fenz, W. D. (1965) · Journal of Experimental Psychology · doi:10.1037/h0021990 · Cited by 95
Social Role-Taking: A Review of the Constructs, Measures, and Measurement Properties
Enright, R. D., & Lapsley, D. K. (1980) · Review of Educational Research · doi:10.3102/00346543050004647 · Cited by 91
Searching for metamemory-memory connections: A developmental study.
Cavanaugh, J. C., & Borkowski, J. G. (1980) · Developmental Psychology · doi:10.1037//0012-1649.16.5.441 · Cited by 88
The study of human infant attachment: A procedural critique.
Masters, J. C., & Wellman, H. M. (1974) · Psychological Bulletin · doi:10.1037/h0036184 · Cited by 87
In defense of measurement.
Green, B. F. (1978) · American Psychologist · doi:10.1037/0003-066x.33.7.664 · Cited by 78
Personality: Nomothetic or idiographic? A response to Kenrick and Stringfield.
Rushton, J. P., Jackson, D. N., & Paunonen, S. V. (1981) · Psychological Review · doi:10.1037//0033-295x.88.6.582 · Cited by 77
Self-citation
Prescription for reduction of aggression.
Eron, L. D. (1980) · American Psychologist · doi:10.1037//0003-066x.35.3.244 · Cited by 75
Cognitive development and concept learning: An interpretative review.
Brainerd, C. J. (1977) · Psychological Bulletin · doi:10.1037//0033-2909.84.5.919 · Cited by 75
Self-citation
Traité de logique
Piaget, J. (1951) · Cited by 66
Behavioral and Metamnemonic Indicators of Strategic Behaviors under Remember Instructions in First Grade
Salatas, H., & Flavell, J. H. (1976) · Child Development · doi:10.1111/j.1467-8624.1976.tb03395.x · Cited by 64
Training strategic study time apportionment in educable retarded children
Brown, A. L., & Campione, J. C. (1977) · Intelligence · doi:10.1016/0160-2896(77)90029-0 · Cited by 58
Role Taking in Childhood: Some Methodological Considerations
Rubin, K. H. (1978) · Child Development · doi:10.1111/j.1467-8624.1978.tb02332.x · Cited by 57
Children's understanding of number concepts: Characteristics of an individual and of a group test.
Dodwell, P. C. (1961) · Canadian Journal of Psychology/Revue Canadienne de Psychologie · doi:10.1037/h0083202 · Cited by 57
Response Criteria in Concept Development Research
Brainerd, C. J. (1977) · Child Development · doi:10.1111/j.1467-8624.1977.tb01172.x · Cited by 54
Self-citation
The validity of judgments as a function of the number of judges.
Eysenck, H. J. (1939) · Journal of Experimental Psychology · doi:10.1037/h0058754 · Cited by 45
General and Specific Factors in Character
Maller, J. B. (1934) · The Journal of Social Psychology · doi:10.1080/00224545.1934.9921587 · Cited by 39
On the reliability and validity of children’s metamemory
Kurtz‐Costes, B., Reid, M. K., Borkowski, J. G., & Cavanaugh, J. C. (1982) · Bulletin of the Psychonomic Society · doi:10.3758/bf03330211 · Cited by 36
A longitudinal analysis of logical reasoning relationships: Conservation and transitive inference.
Hooper, F. H., Toniolo, T. A., & Sipple, T. S. (1978) · Developmental Psychology · doi:10.1037//0012-1649.14.6.674 · Cited by 36
Relations among Memory, Memory Appraisal, and Memory Strategies
Kelly, M. L., Scholnick, E. K., Travers, S. H., & Johnson, J. W. (1976) · Child Development · doi:10.1111/j.1467-8624.1976.tb02228.x · Cited by 29
Interdependence of Inhelder and Piaget's model of logical thinking.
Berzonsky, M. D. (1971) · Developmental Psychology · doi:10.1037/h0030961 · Cited by 27
Classes, relations et nombres : essai sur les groupements de la logistique et sur la reversibilité de la pensée
Piaget, J. (1942) · J. Vrin eBooks · Cited by 25
Person by situation interactions in academic achievement1
Rushton, J. P., & Endler, N. S. (1977) · Journal of Personality · doi:10.1111/j.1467-6494.1977.tb00153.x · Cited by 11
Self-citation
TRAITS, BEHAVIORAL SYSTEMS, AND RELATIONSHIPS: THREE MODELS OF INFANT-ADULT ATTACHMENT
Waters, E. (1981) · Cited by 10
Does introspection have a role in brain-behavior research?
Vanderwolf, C. H., & Goodale, M. (1982) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00013030 · Cited by 1

Sorted by citations

Using Self-Report Assessment Methods to Explore Facets of Mindfulness
Baer, R. A., Smith, G. T., Hopkins, J., Krietemeyer, J., & Toney, L. (2006) · Assessment · doi:10.1177/1073191105283504 · Cited by 7,511
To parcel or not to parcel: Exploring the question, weighing the merits
Little, T. D., Cunningham, W. A., Shahar, G., & Widaman, K. F. (2002) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0902_1 · Cited by 6,552
Validation of the five-factor model of personality across instruments and observers.
McCrae, R. R., & Costa, P. T. (1987) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.52.1.81 · Cited by 6,404
The processing-speed theory of adult age differences in cognition
Salthouse, T. A. (1996) · Psychological Review · doi:10.1037/0033-295x.103.3.403 · Cited by 5,902
A TYPOLOGY OF DEVIANT WORKPLACE BEHAVIORS: A MULTIDIMENSIONAL SCALING STUDY.
Robinson, S. L., & BENNETT, R. J. (1995) · Academy of Management Journal · doi:10.2307/256693 · Cited by 3,402

See all 940

Authors: Rushton, J. P., Brainerd, C. J., & Pressley, M.
Affiliations: Western University (CA)
ORCID: Brainerd, Charles J.
Venue: Psychological Bulletin
Publisher: American Psychological Association (APA)
Published: 1983-07
Volume: 94
Issue: 1
Pages: 18-38
ISSN: 0033-2909 (ISSN-L)
1939-1455
DOI: 10.1037/0033-2909.94.1.18
OpenAlex: W2152174164
Cited by: 940
References: 100
Citations by year: 2026: 10 · 2025: 17 · 2024: 16 · 2023: 15 · 2022: 22 · 2021: 28 · 2020: 17 · 2019: 23 · 2018: 29 · 2017: 37 · 2016: 26 · 2015: 18 · 2014: 33 · 2013: 20 · 2012: 38
FWCI: 32.85
Type: journal article
Language: en
Open access: closed
Locations: Psychological Bulletin (publishedVersion)
Topics: Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Psychology › Clinical Psychology › Personality Traits and Psychology
Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
Keywords: construct validity, behavioral development, measurement error, personality stability, moral judgment, metacognition, attitude-behavior relationship, altruism, attachment, sex differences, cross-situational consistency, longitudinal stability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Behavioral Development and Construct Validity: The Principle of Aggregation

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks