CoChat

Case-Based Data Quality Management for IoT Logs: A Case Study Focusing on Detection of Data Quality Issues

Open access

The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index; 20 of the 37 sources its claims cite were checked for support.

Abstract

This paper presents an end-to-end case-based reasoning framework that detects and remedies data quality issues in IoT logs in near real time, even with only a handful of annotated fault instances.

Overview

CoChat Summary

TL;DR

The paper presents an end-to-end case-based reasoning framework for detecting and remedying data quality issues in Industrial IoT time-series logs, using expert-encoded vocabulary, case base, similarity measures, and adaptation rules. In a 500-case smart-factory case study, the approach detected almost no true DQI positives, with overall recall 0.38% and F1 0.75%, while also showing severe memory and runtime limitations.

Key findings

  • The case study used a reduced 500-case base with 25 cases of each of three DQI types—missing sensor values, missing sensors, and time shifts—and 10-fold cross-validation. Overall accuracy was 64.21%, but average recall was 0.38% and average F1-score was 0.75%.
  • For missing sensor values, precision was 62.50%, recall 1.15%, F1-score 2.25%, specificity 50.00%, and false positive rate 50.00%; 41 queries were aborted, and average calculation time per query was 21 min 46 s.
  • For missing sensors, no true positives were detected: precision, recall, and F1-score were all 0.00, while accuracy was 95.00% and specificity 1.000; average calculation time per query was 12 s.
  • For time shifts, no true positives were detected: precision, recall, and F1-score were all 0.00, while accuracy was 95.83% and specificity 0.9973; 116 queries were aborted, retrieval took over 10 days, average calculation time per query was 29 min 21 s, and memory use reached almost 400 GB.

Methods

  • Design: a Temporal Case-Based Reasoning framework populating the four CBR knowledge containers, with a DataStream-format vocabulary, a fault-annotated case base, tailored similarity measures, and lightweight adaptation rules. Similarity measures included a weighted Dynamic Time Warping variant for missing sensor values, hierarchical list mapping for missing sensors, and increased timestamp weighting for time shifts.
  • Data and sample: error-free sensor logs from an extended Fischertechnik Smart Factory in the IoT Lab Trier were used; a domain expert generated DQI-injected event logs for the three DQI types. The case study used a reduced, randomly chosen data set of 500 cases, containing 25 cases of each DQI type, implemented in the ProCAKE framework.
  • Analysis: 10-fold cross-validation was performed for retrieval and classification using majority voting over the five most similar cases with a similarity threshold of 0.9. Confusion matrices and performance measures—precision, recall, F1-score, accuracy, specificity, and error rates—were computed; the further CBR phases were omitted in the case study.

Limitations

  • Stated: the authors report only limited DQI identification, poor recall and F1-scores, zero true positives for missing sensors and time shifts, small numbers of faulty cases, heterogeneous DQI manifestations, unsuitable similarity measures, fragile expert-only similarity weights, non-scalable naïve retrieval, and failure of a one-size-fits-all similarity measure across DQI types.
  • Stated: the approach showed severe technical limitations, including memory consumption exceeding 400 GB RAM, throttling and aborted queries, long runtimes, and unsuitability for edge devices or real-world real-time use.
  • Evident from the text: the case study evaluates only retrieval and classification, not the full detection-handling-solving pipeline; the paper also reports inconsistent DQI prevalence figures (25 cases per DQI type versus 5% of 500 cases labeled with a DQI), and the missing-sensor-value confusion matrix implies very few actual negatives, complicating interpretation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/iot6040063

Cites

78 of 78 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Application domains of data quality in cyber-physical systems and IoT environments (based on Goknil et al. [7]).
Figure 1 Application domains of data quality in cyber-physical systems and IoT environments (based on Goknil et al. [7]).From the PDF
Figure 2 Illustration of the CBR cycle with the corresponding knowledge containers (based on Aamodt and Plaza [21]).
Figure 2 Illustration of the CBR cycle with the corresponding knowledge containers (based on Aamodt and Plaza [21]).From the PDF
Figure 3 The data quality management cycle according to Bertrand et al. [11].
Figure 3 The data quality management cycle according to Bertrand et al. [11].From the PDF
Figure 4 Procedure of using CBR for detection of DQIs, with references to DQMC: (1)-(4).
Figure 4 Procedure of using CBR for detection of DQIs, with references to DQMC: (1)-(4).From the PDF
Figure 5 The Vocabulary for Representing an Event in the Data Stream Format As a Case (Adapted from and Extended Based On Schultheis et al. [49]).
Figure 5 The Vocabulary for Representing an Event in the Data Stream Format As a Case (Adapted from and Extended Based On Schultheis et al. [49]).From the PDF
Figure 6 The undesired mapping in the DQI of a missing sensor value (based on Schultheis et al. [49]).
Figure 6 The undesired mapping in the DQI of a missing sensor value (based on Schultheis et al. [49]).From the PDF
Figure 7 The desired mapping in the DQI of a missing sensor value.
Figure 7 The desired mapping in the DQI of a missing sensor value.From the PDF
Figure 8 The proposed hierarchical similarity measures for the DQI type of missing sensors.
Figure 8 The proposed hierarchical similarity measures for the DQI type of missing sensors.From the PDF
Figure 9 The undesired mapping in the DQI of a time shift.
Figure 9 The undesired mapping in the DQI of a time shift.From the PDF
Figure 10 The desired mapping in the DQI of a time shift.
Figure 10 The desired mapping in the DQI of a time shift.From the PDF

Verification

Claims the paper makes

14 Valid · 5 Not valid · 13 Not checked · 108 Own results and interpretations

49 more claims

Create a free account to see all 140 claims and to verify the 4 references still open.

See all claims — free account
  • It may state two separate claims as one. 53
  • Probably not a substantial claim. 48
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 78 references

Research works printed but not linked: 17

  1. 19. Mitchell, T.M. Machine Learning ; McGraw-Hill: New York, NY, USA, 1997.
  2. 22. Bergmann, R. Experience Management: Foundations, Development Methodology, and Internet-Based Applications ; LNCS; Springer: Berlin/Heidelberg, Germany, 2003; Volume 2432. [CrossRef]
  3. 25. Richter, M.M. Knowledge Containers. In Readings in Case-Based Reasoning ; Morgan Kaufmann Publishers: Cambridge, MA, USA, 2003.
  4. 38. Verhulst, R. Evaluating Quality of Event Data within Event Logs: An Extensible Framework. Master's Thesis, Eindhoven University of Technology, Eindhoven, The Netherlands, 2016.
  5. 47. Malburg, L.; Schultheis, A.; Bergmann, R. Modeling and Using Complex IoT Time Series Data in Case-Based Reasoning: From Application Scenarios to Implementations. In ICCBR Workshops ; Springer: Cham, Switzerland, 2023; Volume 3438, pp. 81-96.
  6. 50. Stahl, A. Learning of Knowledge-Intensive Similarity Measures in Case-Based Reasoning. Ph.D. Thesis, University of Kaiserslautern, der Pfalz, Germany, 2004.
  7. 58. English, L.P. Improving Data Warehouse and Business Information Quality: Methods for Reducing Costs and Increasing Profits ; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 1999.
  8. 65. Ehrlinger, L.; Wöß, W. Automated Data Quality Monitoring. In Proceedings of the 22nd ICIQ, Little Rock, AR, USA, 6-7 October 2017.
  9. 69. IEEE Std 1849-2016 ; IEEE Standard for eXtensible Event Stream (XES) for Achieving Interoperability in Event Logs and Event Streams. IEEE: New York, NY, USA, 2016. [CrossRef]
  10. 70. IEEE Std 1849-2023 (Revision of IEEE Std 1849-2016) ; IEEE Standard for eXtensible Event Stream (XES) for Achieving Interoperability in Event Logs and Event Streams. IEEE: New York, NY, USA, 2023. [CrossRef]
  11. 73. Levenshtein, V.I. Binary Codes Capable of Correcting Deletions, Insertions and Reversals. In Soviet Physics-Doklady ; Soviet Union: Moscow, Russia, 1966; Volume 10, pp. 707-710.
  12. 74. Bergmann, R.; Grumbach, L.; Malburg, L.; Zeyen, C. ProCAKE: A Process-Oriented Case-Based Reasoning Framework. In Proceedings of the 27th ICCBR Workshop, Otzenhausen, Germany, 8-12 September 2019; Volume 2567, pp. 156-161.
  13. 75. Mahalanobis, P.C. On the Generalized Distance in Statistics. Proc. Natl. Inst. Sci. India 1936 , 2 , 49-55, reprinted in Sankhy¯ a: Indian J. Stat. 2018 , 80-A (Suppl. 1), S1-S7.
  14. 82. Hoffmann, M. Hybrid AI for Process Management: Improving Similarity Assessment in Process-Oriented Case-Based Reasoning via Deep Learning. Ph.D. Thesis, Trier University, Trier, Germany, 2025.
  15. 84. Augusti, F.; Albertini, D.; Esmer, K.; Sannino, R.; Bernardini, A. IMAD-DS: A Dataset for Industrial Multi-Sensor Anomaly Detection Under Domain Shift Conditions. In Proceedings of the Detection and Classification of Acoustic Scenes and Events 2024, Tokyo, Japan, 23-25 October 2024. [CrossRef]
  16. 91. Smyth, B.; Keane, M.T. Remembering To Forget: A Competence-Preserving Case Deletion Policy for Case-Based Reasoning Systems. In Proceedings of the 14th IJCAI, Montreal, QC, Canada, 20-25 August 1995; Morgan Kaufmann: Cambridge, MA, USA, 1995; pp. 377-383.
  17. 92. Buduma, N. Fundamentals of Deep Learning: Designing Next-Generation Machine Intelligence Algorithms , 1st ed.; O'Reilly: Sebastopol, CA, USA, 2017. Disclaimer/Publisher's Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Cited by

Metadata

Authors
Schultheis, A., Bertrand, Y., Grüger, J., Malburg, L., Bergmann, R., & Serral Asensio, E.
Affiliations
  • German Research Centre for Artificial Intelligence (DE)
  • Ghent University (BE)
  • Research Centre for Information Systems Engineering (LIRIS), KU Leuven, Warmoesberg 26, 1000 Brussels, Belgium
Corresponding author
Schultheis, Alexander
Venue
IoT
Publisher
MDPI AG
Published
2025-10-23
Volume
6
Issue
4
Pages
63
ISSN
2624-831X (ISSN-L)
OpenAlex
W4415491316
Cited by
2
References
63
Citations by year
2026: 2
FWCI
0.55
Type
journal article
Language
en
Open access
Locations
  • IoT (publishedVersion, cc-by)
  • Ghent University Academic Bibliography (Ghent University) (submittedVersion, cc-by)
  • Lirias (KU Leuven) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Federal Ministry for Economic Affairs and Energy (01MD22002C)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Fault Detection and Control Systems
  • Physical Sciences › Computer Science › Information Systems › Data Quality and Management
  • Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Digital Transformation in Industry
Keywords
case-based reasoning, data quality issues, Industrial Internet of Things, sensor data, data quality management, missing data imputation, data alignment, missing data, similarity measures, dynamic time warping, expert systems, smart manufacturing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.