CoChat

Identifying mislabelled samples: Machine learning models exceed human performance

Farrell, Christopher-John · 2021 · Annals of Clinical Biochemistry: International Journal of Laboratory Medicine

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Abstract

Background: It is difficult for clinical laboratories to identify samples that are labelled with the details of an incorrect patient. Many laboratories screen for these errors with delta checks, with final decision-making based on manual review of results by laboratory staff. Machine learning models have been shown to outperform delta checks for identifying these errors. However, a comparison of machine learning models to human-level performance has not yet been made. Methods: Deidentified data for current and previous (within seven days) electrolytes, urea and creatinine results was used in the computer simulation of mislabelled samples. Eight different machine learning models were developed on 127,256 sets of results using different algorithms: artificial neural network, extreme gradient boosting, support vector machine, random forest, logistic regression, k-nearest neighbours and two decision trees (one complex and one simple). A separate test data-set ( n = 14,140) was used to evaluate the performance of these models as well as laboratory staff volunteers, who manually reviewed a random subset of this data ( n = 500). Results: The best performing machine learning model was the artificial neural network (92.1% accuracy), with the simple decision tree demonstrating the poorest accuracy (86.5%). The accuracy of laboratory staff for identifying mislabelled samples was 77.8%. Conclusions: The results of this preliminary investigation suggest that even relatively simple machine learning models can exceed human performance for identifying mislabelled samples. Machine learning techniques should be considered for implementation in clinical laboratories to assist with error identification.

Overview

Integrity signals

Registry

Crossref · 10.1177/00045632211032991

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 5 references

Cited by

See all 39

Metadata

Authors
Farrell, C.-J.
Affiliations
Nepean Hospital (AU)
Corresponding author
Farrell, Christopher-John
Venue
Annals of Clinical Biochemistry: International Journal of Laboratory Medicine
Published
2021-07-16
Volume
58
Issue
6
Pages
650-652
ISSN
  • 0004-5632 (ISSN-L)
  • 1758-1001
OpenAlex
W3176752500
Cited by
35
References
5
Citations by year
2026: 4 · 2025: 10 · 2024: 7 · 2023: 5 · 2022: 8 · 2021: 1
FWCI
3.29
Type
journal article
Language
en
Open access
closed
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
  • Annals of Clinical Biochemistry International Journal of Laboratory Medicine (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Health Professions › Medical Laboratory Technology › Clinical Laboratory Practices and Quality Control
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
patient identification errors, machine learning, artificial neural networks, human performance comparison, delta check, electrolytes, urea and creatinine, clinical laboratory automation
MeSH terms
Machine Learning, Laboratories, Clinical, Algorithms, Computer Simulation, Humans, Logistic Models, Neural Networks, Computer, Support Vector Machine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Identifying mislabelled samples: Machine learning models exceed human performance. https://app.cochat.ai/papers/10.1177/00045632211032991
The paper
Farrell, C.-J. (2021). Identifying mislabelled samples: Machine learning models exceed human performance. Annals of Clinical Biochemistry: International Journal of Laboratory Medicine. https://doi.org/10.1177/00045632211032991

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.