Requirements for Data Quality Metrics
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Data quality and especially the assessment of data quality have been intensively discussed in research and practice alike. To support an economically oriented management of data quality and decision making under uncertainty, it is essential to assess the data quality level by means of well-founded metrics. However, if not adequately defined, these metrics can lead to wrong decisions and economic losses. Therefore, based on a decision-oriented framework, we present a set of five requirements for data quality metrics. These requirements are relevant for a metric that aims to support an economically oriented management of data quality and decision making under uncertainty. We further demonstrate the applicability and efficacy of these requirements by evaluating five data quality metrics for different data quality dimensions. Moreover, we discuss practical implications when applying the presented requirements.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mining association rules between sets of items in large databases
Agrawal, R., Imieliński, T., & Swami, A. (1993) · Proceedings of the 1993 ACM SIGMOD international conference on Management of data · doi:10.1145/170035.170072 · Cited by 14,907
Bell, E., Bryman, A., & Harley, B. (2022) · Oxford University Press eBooks · doi:10.1093/hebz/9780198869443.001.0001 · Cited by 10,965
Analyzing the Past to Prepare for the Future: Writing a Literature Review
Webster, J., & Watson, R. T. (2002) · MIS Quarterly · doi:10.2307/4132319 · Cited by 7,449
Rational choice and the structure of the environment.
Simon, H. A. (1956) · Psychological Review · doi:10.1037/h0042769 · Cited by 5,919
On the Theory of Scales of Measurement
Stevens, S. S. (1946) · Science · doi:10.1126/science.103.2684.677 · Cited by 4,761
Pipino, L. L., Lee, Y. W., & Wang, R. Y. (2002) · Communications of the ACM · doi:10.1145/505248.506010 · Cited by 1,724 · cited 2×
“To avoid such problems, both researchers and practitioners set out to propose requirements for data quality metrics [e.g., Pipino et al. 2002; Even and Shankaranarayanan 2007; Heinrich et al. 2007; Mosley et al. 2009; Loshin 2010; Hüner 2011].”
BackgroundAIMQ: a methodology for information quality assessment
Lee, Y. W., Strong, D. M., Kahn, B. K., & Wang, R. Y. (2002) · Information & Management · doi:10.1016/s0378-7206(02)00043-5 · Cited by 1,611
A product perspective on total data quality management
Wang, R. Y. (1998) · Communications of the ACM · doi:10.1145/269012.269022 · Cited by 911
Khatri, V., & Brown, C. V. (2010) · Communications of the ACM · doi:10.1145/1629175.1629210 · Cited by 816
Analyzing the past to prepare for the future
(2002) · MIS Quarterly · doi:10.5555/2017160.2017162 · Cited by 179
The Effects and Interactions of Data Quality and Problem Complexity on Classification
Blake, R., & Mangiameli, P. (2011) · Journal of Data and Information Quality · doi:10.1145/1891879.1891881 · Cited by 132
Evaluating a Model for Cost-Effective Data Quality Management in a Real-World CRM Setting
Even, A., Shankaranarayanan, G., & Berger, P. D. (2010) · Decision Support Systems · doi:10.1016/j.dss.2010.07.011 · Cited by 56
An Accuracy Metric: Percentages, Randomness, and Probabilities
Fisher, C. W., Lauria, E. J. M., & Matheus, C. C. (2009) · Journal of Data and Information Quality · doi:10.1145/1659225.1659229 · Cited by 40
Cited by
Data Governance Model to Enhance Data Quality in Financial Institutions
Karkošková, S. (2023) · Information Systems Management · doi:10.1080/10580530.2022.2042628 · Cited by 78
“Development of modern technologies, financial innovations (Hasan et al., 2020) and evolving approaches to data processing (Polak et al., 2020; Wahyudi et al., 2018) reveal that there still exist open research gaps and challenges in data quality (Hussain & Prieto, 2016).”
BackgroundResultAI system engineering key challenges and lessons learned
Fischer, L., Ehrlinger, L., Geist, V., Ramler, R., Sobiezky, F., Zellinger, W., Brunner, D., Kumar, M., & Moser, B. (2020) · Machine Learning and Knowledge Extraction · doi:10.3390/make3010004 · Cited by 71
“Such high-dimensional effects can cause instabilities as illustrated, for example, by the emergence of so-called adversarial examples, see for example, References [22,23].”
InfluentialA Framework for Current and New Data Quality Dimensions: An Overview
Miller, R., Whelan, H., Chrubasik, M., Whittaker, D., Duncan, P., & Gregório, J. (2024) · Data · doi:10.3390/data9120151 · Cited by 33
A Survey of Data Quality Measurement and Monitoring Tools
Ehrlinger, L., & Wöß, W. (2022) · Frontiers in Big Data · doi:10.3389/fdata.2022.850611 · Cited by 25
Understanding Data Quality in a Data-Driven Industry Context: Insights from the Fundamentals
Fu, Q., Nicholson, G. L., & Easton, J. M. (2024) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2024.100729 · Cited by 17
Metadata
- Authors
- Heinrich, B., Hristova, D., Klier, M., Schiller, A., & Szubartowicz, M.
- Venue
- Journal of Data and Information Quality
- Published
- 2017-06-30
- Volume
- 9
- Issue
- 2
- Pages
- 1-32
- ISSN
- 1936-1955 (ISSN-L)
- 1936-1963
- OpenAlex
- W2785061420
- Cited by
- 104
- References
- 13
- FWCI
- 5.56
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.