The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Holistic data cleaning: Putting violations into context
Abstract
Data cleaning is an important problem and data quality rules are the most promising way to face it with a declarative approach. Previous work has focused on specific formalisms, such as functional dependencies (FDs), conditional functional dependencies (CFDs), and matching dependencies (MDs), and those have always been studied in isolation. Moreover, such techniques are usually applied in a pipeline or interleaved. In this work we tackle the problem in a novel, unified framework. First, we let users specify quality rules using denial constraints with ad-hoc predicates. This language subsumes existing formalisms and can express rules involving numerical values, with predicates such as “greater than” and “less than”. More importantly, we exploit the interaction of the heterogeneous constraints by encoding them in a conflict hypergraph. Such holistic view of the conflicts is the starting point for a novel definition of repair context which allows us to compute automatically repairs of better quality w.r.t. previous approaches in the literature. Experimental results on real datasets show that the holistic approach outperforms previous algorithms in terms of quality and efficiency of the repair.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mining association rules between sets of items in large databases
Agrawal, R., Imieliński, T., & Swami, A. (1993) · Proceedings of the 1993 ACM SIGMOD international conference on Management of data · doi:10.1145/170035.170072 · Cited by 14,907
Mining association rules between sets of items in large databases
Agrawal, R., Imieliński, T., & Swami, A. (1993) · ACM SIGMOD Record · doi:10.1145/170036.170072 · Cited by 4,551
Duplicate Record Detection: A Survey
Elmagarmid, A. K., Ipeirotis, P. G., & Verykios, V. S. (2007) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.250581 · Cited by 1,267
(2006) · Data-Centric Systems and Applications · doi:10.1007/3-540-33173-5 · Cited by 659
Conditional functional dependencies for capturing data inconsistencies
Fan, W., Geerts, F., Jia, X., & Kementsietsidis, A. (2008) · ACM Transactions on Database Systems · doi:10.1145/1366102.1366103 · Cited by 460
A cost-based model and effective heuristic for repairing constraints by value modification
Bohannon, P., Fan, W., Flaster, M., & Rastogi, R. (2005) · Proceedings of the 2005 ACM SIGMOD international conference on Management of data · doi:10.1145/1066157.1066175 · Cited by 424
Improving data quality: consistency and accuracy
Cong, G., Fan, W., Geerts, F., et al. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 313
Discovering data quality rules
Chiang, F., & Miller, R. J. (2008) · Proceedings of the VLDB Endowment · doi:10.14778/1453856.1453980 · Cited by 265
Reasoning about record matching rules
Fan, W., Jia, X., Li, J., & Ma, S. (2009) · Proceedings of the VLDB Endowment · doi:10.14778/1687627.1687674 · Cited by 226
Yakout, M., Elmagarmid, A. K., Neville, J., Ouzzani, M., & Ilyas, I. F. (2011) · Proceedings of the VLDB Endowment · doi:10.14778/1952376.1952378 · Cited by 216
Self-citationOn approximating optimum repairs for functional dependency violations
Kolahi, S., & Lakshmanan, L. V. S. (2009) · Proceedings of the 12th International Conference on Database Theory · doi:10.1145/1514894.1514901 · Cited by 206
Towards certain fixes with editing rules and master data
Fan, W., Li, J., Ma, S., Tang, N., & Yu, W. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920867 · Cited by 202
On generating near-optimal tableaux for conditional functional dependencies
Golab, L., Karloff, H., Korn, F., Srivastava, D., & Yu, B. (2008) · Proceedings of the VLDB Endowment · doi:10.14778/1453856.1453900 · Cited by 200
Towards certain fixes with editing rules and master data
Fan, W., Li, J., Ma, S., Tang, N., & Yu, W. (2011) · The VLDB Journal · doi:10.1007/s00778-011-0253-7 · Cited by 128
Sampling the repairs of functional dependency violations under hard constraints
Beskales, G., Ilyas, I. F., & Golab, L. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920870 · Cited by 127
Self-citationInteraction between record matching and data repairing
Fan, W., Li, J., Ma, S., Tang, N., & Yu, W. (2011) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/1989323.1989373 · Cited by 116
Bertossi, L., Bravo, L., Enrico Franconi, B., & Lopatenko, A. (2008) · Information Systems · doi:10.1016/j.is.2008.01.005 · Cited by 99
Complexity and Approximation of Fixing Numerical Attributes in Databases Under Integrity Constraints
Bertossi, L., Bravo, L., Enrico Franconi, B., & Lopatenko, A. (2005) · Lecture notes in computer science · doi:10.1007/11601524_17 · Cited by 37
Cited by
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781
Data Cleaning: Overview and Emerging Challenges
Chu, X., Ilyas, I. F., Krishnan, S., & Wang, J. (2016) · Proceedings of the 2016 International Conference on Management of Data · doi:10.1145/2882903.2912574 · Cited by 488
Self-citationHoloClean: holistic data repairs with probabilistic inference
Rekatsinas, T., Chu, X., Ilyas, I. F., & Ré, C. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3137628.3137631 · Cited by 480
Self-citationDai, H., Jiang, B., Hu, X., Lin, X., Wei, X., & Pecht, M. (2021) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.110480 · Cited by 405
KATARA: A Data Cleaning System Powered by Knowledge Bases and Crowdsourcing
Chu, X., Morcos, J., Ilyas, I. F., Ouzzani, M., Papotti, P., Tang, N., & Ye, Y. (2015) · Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2723372.2749431 · Cited by 279
Self-citation
Metadata
- Authors
- Chu, X., Ilyas, I. F., & Papotti, P.
- Venue
- 2013 IEEE 29th International Conference on Data Engineering (ICDE)
- Publisher
- IEEE
- Published
- 2013-04
- Pages
- 458-469
- ISSN
- 1084-4627 (ISSN-L)
- OpenAlex
- W2044469685
- Cited by
- 326
- References
- 18
- FWCI
- 31.30
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.