Outliers in Questionnaire Data: Can They Be Detected and Should They Be Removed?
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Outliers in questionnaire data are unusual observations, which may bias statistical results, and outlier statistics may be used to detect such outliers. The authors investigated the effect outliers have on the specificity and the sensitivity of each of six different outlier statistics. The Mahalanobis distance and the item-pair based outlier statistics were found to have the best combination of specificity and sensitivity. Next, it was investigated how outliers influenced the bias in the percentile rank score, Cronbach’s alpha, and the validity coefficient. Outliers due to random responding and faking produced considerable bias, and outliers due to extreme responding produced little bias. Finally, the influence of removing discordant observations on bias was studied. Removing observations due to random responding identified by means of the Mahalanobis distance, the local outlier factor, and the item-pair based outlier statistic reduced bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LOF: identifying density-based local outliers
Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335388 · Cited by 5,207
Robust Statistics: The Approach Based on Influence Functions
Ruppert, D. (1987) · Technometrics · doi:10.2307/1269782 · Cited by 3,880
Hawkins, D. M. (1980) · doi:10.1007/978-94-015-3994-4 · Cited by 2,982
Variance in faking across noncognitive measures.
McFarland, L. A., & Ryan, A. M. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.5.812 · Cited by 410 · cited 2×
“They are extreme response style (e.g., Bachman & O’Malley, 1984), random response style (e.g., Emons, 2008), and faking (e.g., McFarland & Ryan, 2000; Zickar & Drasgow, 1996).”
BackgroundIndividual response spread in self-report scales: personality correlations and consequences
Austin, E. J., Deary, I. J., Gibson, G. J., McGregor, M. J., & Dent, J. B. (1998) · Personality and Individual Differences · doi:10.1016/s0191-8869(97)00175-x · Cited by 138
Cited by
Exploratory Factor Analysis: A Guide to Best Practice
Watkins, M. W. (2018) · Journal of Black Psychology · doi:10.1177/0095798418771807 · Cited by 2,218
Kliem, S., Mößle, T., Rehbein, F., Hellmann, D. F., Zenger, M., & Brähler, E. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.11.003 · Cited by 263
A tutorial on how to do a Mokken scale analysis on your test and questionnaire data
Sijtsma, K., & van der Ark, L. A. (2017) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/bmsp.12078 · Cited by 258
Self-citationDeserno, M. K., Bathelt, J., Groenman, A. P., & Geurts, H. M. (2023) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-022-01986-9 · Cited by 22
Social Networks Addiction (SNA-6) – Short: Validity of Measurement in Mexican Youths
Salas-Blas, E., Merino-Soto, C., Pérez-Amezcua, B., & Toledano-Toledano, F. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.774847 · Cited by 20
Metadata
- Authors
- Zijlstra, W. P., van der Ark, L. A., & Sijtsma, K.
- Venue
- Journal of Educational and Behavioral Statistics
- Published
- 2011-04
- Volume
- 36
- Issue
- 2
- Pages
- 186-212
- ISSN
- 1076-9986 (ISSN-L)
- 1935-1054
- OpenAlex
- W2161805882
- Cited by
- 87
- References
- 5
- FWCI
- 3.92
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Outliers in Questionnaire Data: Can They Be Detected and Should They Be Removed? https://app.cochat.ai/papers/10.3102/1076998610366263
- The paper
- Zijlstra, W. P., van der Ark, L. A., & Sijtsma, K. (2011). Outliers in Questionnaire Data: Can They Be Detected and Should They Be Removed? Journal of Educational and Behavioral Statistics. https://doi.org/10.3102/1076998610366263
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.