Semantic Response Consistency and Protocol Validity in Structured Personality Assessment: The Case of the NEO-PI-R
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
In this study we tested the hypothesis that groups of NEO Personality Inventory-Revised (NEO-PI-R; Costa & McCrae, 1992a) protocols identified as potentially invalid by an inconsistency scale (INC; Schinka, Kinder, & Kremer, 1997) would show reduced reliability and validity according to a series of psychometric tests. Data were obtained from 2 undergraduate student samples, a self-report group (n = 132) who provided NEO-PI-R self-ratings on 2 occasions separated by a 7- to 14-day interval and an informant group (n = 109) who provided ratings of well-known friends or relatives on 2 occasions separated by a 6 month interval. INC scores from the Time 1 protocols were used to divide these samples into low, moderate, and elevated inconsistency groups. In both samples, these 3 groups showed equivalent levels of reliability and validity as measured by: contingency coefficients for the 20 INC item responses across occasions; test-retest intraclass correlations of NEO-PI-R domain scores; convergent correlations with Goldberg's (1992) Bipolar Adjective Scale scores; and discriminant correlations between the 5 NEO-PI-R domain scores. The similarity of results across self-report and informant assessment contexts provides additional evidence that semantic consistency approaches to assessing protocol validity may overestimate the prevalence of random or careless response behavior in standard administration conditions. Several theories are discussed that accommodate the existence of valid inconsistency in structured personality assessment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.6.1173 · Cited by 91,532
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Cicchetti, D. V. (1994) · Psychological Assessment · doi:10.1037/1040-3590.6.4.284 · Cited by 9,349
The development of markers for the Big-Five factor structure
Goldberg, L. R. (1992) · Psychological Assessment · doi:10.1037/1040-3590.4.1.26 · Cited by 5,530
Normal personality assessment in clinical practice: the NEO personality inventory
Costa, P. T., & McCrae, R. R. (1992) · Psychological Assessment · doi:10.1037/1040-3590.4.1.5 · Cited by 3,414
Cited by
Identifying careless responses in survey data.
Meade, A. W., & Craig, S. B. (2012) · Psychological Methods · doi:10.1037/a0028085 · Cited by 3,865
The international personality item pool and the future of public-domain personality measures
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. G. (2006) · Journal of Research in Personality · doi:10.1016/j.jrp.2005.08.007 · Cited by 3,185
Detecting and Deterring Insufficient Effort Responding to Surveys
Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012) · Journal of Business and Psychology · doi:10.1007/s10869-011-9231-8 · Cited by 1,295
Kosinski, M., Matz, S. C., Gosling, S. D., Popov, V., & Stillwell, D. (2015) · American Psychologist · doi:10.1037/a0039210 · Cited by 925
Internal consistency, retest reliability, and their implications for personality scale validity
McCrae, R. R., Kurtz, J. E., Yamagata, S., & Terracciano, A. (2011) · Personality and Social Psychology Review · doi:10.1177/1088868310366253 · Cited by 874
“The only study that conducted a short-term test–retest study in an American sample appears to be Kurtz and Parrish (2001), and that article did not report reliabilities for the individual facet scales.”
InfluentialSelf-citation
Metadata
- Authors
- Kurtz, J. E., & Parrish, C. L.
- Venue
- Journal of Personality Assessment
- Publisher
- Informa UK Limited
- Published
- 2001-04
- Volume
- 76
- Issue
- 2
- Pages
- 315-332
- ISSN
- 0022-3891 (ISSN-L)
- 1532-7752
- OpenAlex
- W1966265031
- Cited by
- 83
- References
- 5
- FWCI
- 0.59
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Semantic Response Consistency and Protocol Validity in Structured Personality Assessment: The Case of the NEO-PI-R. https://app.cochat.ai/papers/10.1207/s15327752jpa7602_12
- The paper
- Kurtz, J. E., & Parrish, C. L. (2001). Semantic Response Consistency and Protocol Validity in Structured Personality Assessment: The Case of the NEO-PI-R. Journal of Personality Assessment. https://doi.org/10.1207/s15327752jpa7602_12
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.