Performance evaluation of internal quality control rules, EWMA, CUSUM, and the novel machine learning model
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Objectives: The present study set out to build a machine learning model to incorporate conventional quality control (QC) rules, exponentially weighted moving average (EWMA), and cumulative sum (CUSUM) with random forest (RF) algorithm to achieve better performance and to evaluate the performances the models using computer simulation to aid laboratory professionals in QC procedure planning. Methods: Conventional QC rules, EWMA, CUSUM, and RF models were implemented on the simulation data using an in-house algorithm. The models’ performances were evaluated on 170,000 simulated QC results using outcome metrics, including the probability of error detection (Ped), probability of false rejection (Pfr), average run length (ARL), and power graph. Results: The highest Pfr (0.0404) belonged to the 1–2s rule. The 1–3s rule could not detect errors with a 0.9 Ped up to 4 SD of systematic error. The random forest model had the highest Ped for systematic errors lower than 1 SD. However, ARLs of the model require the combined utility of the RF model with conventional QC rules having lower ARLs or more than one QC measurement is required. Conclusions: The RF model presented in this study showed acceptable Ped for most degrees of systematic error. The outcome metrics established in this study will help laboratory professionals planning internal QC.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The study identified CUSUM and EWMA as the most important features in terms of predictive capability.”
From Machine learning-based clinical decision support using laboratory data (2024) · cites this paper as [40]
In the citing paper
“The RF model achieved a low false rejection probability of 0.0048 and demonstrated the highest error detection rate for errors less than one standard deviation.”
From Machine learning-based clinical decision support using laboratory data (2024) · cites this paper as [40]
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · cited 1×
“Therandomforestclassi fi ermodelused200decisiontrees,entropyfor information gain, and other default parameters based on the scikit-learn 0.24.1 package [15, 16] (the source code can be downloaded from: https://github.com/hikmetc/IQCAI).”
BackgroundInfluentialScikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1×
“1 package [15, 16] (the source code can be downloaded”
BackgroundHarris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., et al. (2020) · Nature · doi:10.1038/s41586-020-2649-2 · Cited by 23,826 · cited 1×
“The gaussian distributed 32,000 in-control internal QC results were generated using Python 3.7.6 [12]. with Numpy package 1.19.1 [13].”
MethodInfluentialHarris, C., Millman, K., van der Walt, S., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N., et al. (2020) · RePEc: Research Papers in Economics · Cited by 17,844
PAGE, E. S. (1954) · Biometrika · doi:10.1093/biomet/41.1-2.100 · Cited by 5,739 · cited 1×
“Page developed the CUSUM chart in 1954 by plotting deviations of the measured values from the target value [6].”
MethodExponentially Weighted Moving Average Control Schemes: Properties and Enhancements
Lucas, J. M., & Saccucci, M. S. (1990) · Technometrics · doi:10.1080/00401706.1990.10484583 · Cited by 1,934 · cited 1×
“Robert fi rst presented the EWMA chart in 1959, created by calculating the weighted average of current and previous data [3].”
MethodEstimating classification error rate: Repeated cross-validation, repeated hold-out and bootstrap
Kim, J.-H. (2009) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2009.04.009 · Cited by 792
Internal quality control: planning and implementation strategies
Westgard, J. O. (2003) · Annals of Clinical Biochemistry: International Journal of Laboratory Medicine · doi:10.1258/000456303770367199 · Cited by 172 · cited 4×
“The QC rules corresponding to the Ped of ≥ 0.9 were regarded as acceptable for an intended degree of systematic error [2].”
BackgroundMethodInfluentialThe Inertial Properties of Quality Control Charts
Woodall, W. H., & Mahmoud, M. A. (2005) · Technometrics · doi:10.1198/004017005000000256 · Cited by 154 · cited 1×
“Furthermore, the combined utility of EWMA and Shewhart chart (conventional QC rules) was recommended previously [19] due to the inertia effect that leads to a lag in error detection, particularly when a lower weighting factor is used [7].”
BackgroundWestgard, J. O., Groth, T., Aronsson, T., Falk, H., & de Verdier, C. H. (1977) · Clinical Chemistry · doi:10.1093/clinchem/23.10.1857 · Cited by 152 · cited 2×
“While these metrics are important for implementing proper QC strategy according to CLSI C24ed4, only a few old-dated studies [9, 10] investigated mostly conventional rules, and one study addressed the comparison of EWMA with conventional rules [11].”
BackgroundMethodInfluential
Research works printed but not linked: 1
- Montgomery, DC. Introduction to statistical quality control, 7th ed. Danvers: John Wiley & Sons; 2013.
Cited by
Machine learning-based clinical decision support using laboratory data
Çubukçu, H. C., Topcu, D. İ., & Yenice, S. (2024) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2023-1037 · Cited by 66
“However, the model lacked performance evaluation with multi rules and real-world implementation [40].”
BackgroundMethodSelf-citationÇubukçu, H. C., Boursier, G., Linko, S., Andreu, F. A. B., Brguljan, P. M., Trajkovska, K. T., Brugnoni, D., Milinković, N., Padoan, A., & Thelen, M. (2025) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2025-0482 · Cited by 40
Self-citationCadamuro, J. (2022) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2022-0620 · Cited by 31
Advances in clinical chemistry patient-based real-time quality control (PBRTQC)
Cervinski, M. A., Bietenbeck, A., Katayev, A., Loh, T. P., van Rossum, H. H., & Badrick, T. C. (2023) · Advances in clinical chemistry · doi:10.1016/bs.acc.2023.08.003 · Cited by 18
Çubukçu, H. C. (2024) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2024-0156 · Cited by 16
Self-citation
Metadata
- Authors
- Çubukçu, H. C.
- Affiliations
- Ankara University (TR)
- Corresponding author
- Çubukçu, Hikmet Can
- ORCID
- Çubukçu, Hikmet Can
- Venue
- Turkish Journal of Biochemistry
- Publisher
- Walter de Gruyter GmbH
- Published
- 2021-12-01
- Volume
- 46
- Issue
- 6
- Pages
- 661-670
- ISSN
- 0250-4685 (ISSN-L)
- 1303-829X
- OpenAlex
- W4200198924
- Cited by
- 13
- References
- 19
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 2 · 2023: 2 · 2022: 1
- FWCI
- 0.79
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Turkish Journal of Biochemistry (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Health Professions › Medical Laboratory Technology › Clinical Laboratory Practices and Quality Control
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Advanced Statistical Process Monitoring
- Physical Sciences › Engineering › Control and Systems Engineering › Fault Detection and Control Systems
- Keywords
- internal quality control, random forest, EWMA control chart, CUSUM control chart, average run length, systematic error detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Performance evaluation of internal quality control rules, EWMA, CUSUM, and the novel machine learning model. https://app.cochat.ai/papers/10.1515/tjb-2021-0199
- The paper
- Çubukçu, H. C. (2021). Performance evaluation of internal quality control rules, EWMA, CUSUM, and the novel machine learning model. Turkish Journal of Biochemistry. https://doi.org/10.1515/tjb-2021-0199
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.