Identifying mislabelled samples: Machine learning models exceed human performance
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Background: It is difficult for clinical laboratories to identify samples that are labelled with the details of an incorrect patient. Many laboratories screen for these errors with delta checks, with final decision-making based on manual review of results by laboratory staff. Machine learning models have been shown to outperform delta checks for identifying these errors. However, a comparison of machine learning models to human-level performance has not yet been made. Methods: Deidentified data for current and previous (within seven days) electrolytes, urea and creatinine results was used in the computer simulation of mislabelled samples. Eight different machine learning models were developed on 127,256 sets of results using different algorithms: artificial neural network, extreme gradient boosting, support vector machine, random forest, logistic regression, k-nearest neighbours and two decision trees (one complex and one simple). A separate test data-set ( n = 14,140) was used to evaluate the performance of these models as well as laboratory staff volunteers, who manually reviewed a random subset of this data ( n = 500). Results: The best performing machine learning model was the artificial neural network (92.1% accuracy), with the simple decision tree demonstrating the poorest accuracy (86.5%). The accuracy of laboratory staff for identifying mislabelled samples was 77.8%. Conclusions: The results of this preliminary investigation suggest that even relatively simple machine learning models can exceed human performance for identifying mislabelled samples. Machine learning techniques should be considered for implementation in clinical laboratories to assist with error identification.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In their study, Farrell et al. conducted research to detect mislabeled samples [24].”
From Machine learning-based clinical decision support using laboratory data (2024) · cites this paper as [24]
In the citing paper
“Notably, the ANN model yielded impressive results, achieving an accuracy of 92.1 % and an area under the curve (AUC) of 0.977.”
From Machine learning-based clinical decision support using laboratory data (2024) · cites this paper as [24]
In the citing paper
“These findings unequivocally demonstrate the superiority of ML algorithms over human capabilities in accurately identifying incorrectly labeled samples [24].”
From Machine learning-based clinical decision support using laboratory data (2024) · cites this paper as [24]
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Bonini, P., Plebani, M., Ceriotti, F., & Rubboli, F. (2002) · Clinical Chemistry · doi:10.1093/clinchem/48.5.691 · Cited by 850 · cited 1×
“Finally, the simulated error rate was significantly higher than sample mislabelling in practice.(6) This was necessary to assess human-level performance, as only 500 sample pairs were reviewed by volunteers.”
BackgroundDelta Checks in the clinical laboratory
Randell, E. W., & Yenice, S. (2019) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/10408363.2018.1540536 · Cited by 65 · cited 1×
“Current laboratory practice is to use delta checks to screen for these errors and have results failing delta checks manually reviewed by laboratory staff.(1) Preliminary investigations have demonstrated that machine learning (ML) models are superior to delta checks in identifying mislabelled samples.”
BackgroundUsing Machine Learning-Based Multianalyte Delta Checks to Detect Wrong Blood in Tube Errors
Rosenbaum, M. W., & Baron, J. M. (2018) · American Journal of Clinical Pathology · doi:10.1093/ajcp/aqy085 · Cited by 58 · cited 1×
“Preliminary investigations have demonstrated that machine learning (ML) models are superior to delta checks in identifying mislabelled samples.(2,3) The relative performance of different ML algorithms has not been established, however.”
BackgroundDevelopment and characterization of neural network-based multianalyte delta checks
Jackson, C. R., & Cervinski, M. A. (2020) · Journal of Laboratory and Precision Medicine · doi:10.21037/jlpm.2020.02.03 · Cited by 9 · cited 1×
“Preliminary investigations have demonstrated that machine learning (ML) models are superior to delta checks in identifying mislabelled samples.(2,3) The relative performance of different ML algorithms has not been established, however.”
Background
Cited by
Applications of machine learning in routine laboratory medicine: Current state and future directions
Rabbani, N., Kim, G. Y. E., Suarez, C. J., & Chen, J. H. (2022) · Clinical Biochemistry · doi:10.1016/j.clinbiochem.2022.02.011 · Cited by 118
Clinlabomics: leveraging clinical laboratory data by data mining strategies
Wen, X., Leng, P., Wang, J., Yang, G., Zu, R., Jia, X., Zhang, K., Mengesha, B. A., Huang, J. H., Wang, D., et al. (2022) · BMC Bioinformatics · doi:10.1186/s12859-022-04926-1 · Cited by 81
Integration of artificial intelligence in clinical laboratory medicine: Advancements and challenges
Xie, H., Jia, Y. H., & Liu, S. (2024) · Interdisciplinary medicine · doi:10.1002/inmd.20230056 · Cited by 80
Machine learning-based clinical decision support using laboratory data
Çubukçu, H. C., Topcu, D. İ., & Yenice, S. (2024) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2023-1037 · Cited by 66
“In their study, Farrell et al. conducted research to detect mislabeled samples [24].”
BackgroundArtificial intelligence in the clinical laboratory
Hou, H., Zhang, R., & Li, J. (2024) · Clinica Chimica Acta · doi:10.1016/j.cca.2024.119724 · Cited by 64
Metadata
- Authors
- Farrell, C.-J.
- Affiliations
- Nepean Hospital (AU)
- Corresponding author
- Farrell, Christopher-John
- Venue
- Annals of Clinical Biochemistry: International Journal of Laboratory Medicine
- Publisher
- SAGE Publications
- Published
- 2021-07-16
- Volume
- 58
- Issue
- 6
- Pages
- 650-652
- ISSN
- 0004-5632 (ISSN-L)
- 1758-1001
- OpenAlex
- W3176752500
- Cited by
- 35
- References
- 5
- Citations by year
- 2026: 4 · 2025: 10 · 2024: 7 · 2023: 5 · 2022: 8 · 2021: 1
- FWCI
- 3.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Annals of Clinical Biochemistry International Journal of Laboratory Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Medical Laboratory Technology › Clinical Laboratory Practices and Quality Control
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- patient identification errors, machine learning, artificial neural networks, human performance comparison, delta check, electrolytes, urea and creatinine, clinical laboratory automation
- MeSH terms
- Machine Learning, Laboratories, Clinical, Algorithms, Computer Simulation, Humans, Logistic Models, Neural Networks, Computer, Support Vector Machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Identifying mislabelled samples: Machine learning models exceed human performance. https://app.cochat.ai/papers/10.1177/00045632211032991
- The paper
- Farrell, C.-J. (2021). Identifying mislabelled samples: Machine learning models exceed human performance. Annals of Clinical Biochemistry: International Journal of Laboratory Medicine. https://doi.org/10.1177/00045632211032991
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.