The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Discovering Conditional Functional Dependencies
Open accessAbstract
This paper investigates the discovery of conditional functional dependencies (CFDs). CFDs are a recent extension of functional dependencies (FDs) by supporting patterns of semantically related constants, and can be used as rules for cleaning relational data. However, finding quality CFDs is an expensive process that involves intensive manual effort. To effectively identify data cleaning rules, we develop techniques for discovering CFDs from relations. Already hard for traditional FDs, the discovery problem is more difficult for CFDs. Indeed, mining patterns in CFDs introduces new challenges. We provide three methods for CFD discovery. The first, referred to as CFDMiner, is based on techniques for mining closed item sets, and is used to discover constant CFDs, namely, CFDs with constant patterns only. Constant CFDs are particularly important for object identification, which is essential to data cleaning and data integration. The other two algorithms are developed for discovering general CFDs. One algorithm, referred to as CTANE, is a levelwise algorithm that extends TANE, a well-known algorithm for mining FDs. The other, referred to as FastCFD, is based on the depth-first approach used in FastFD, a method for discovering FDs. It leverages closed-item-set mining to reduce the search space. As verified by our experimental study, CFDMiner can be multiple orders of magnitude faster than CTANE and FastCFD for constant CFD discovery. CTANE works well when a given relation is large, but it does not scale well with the arity of the relation. FastCFD is far more efficient than CTANE when the arity of the relation is large; better still, leveraging optimization based on closed-item-set mining, FastCFD also scales well with the size of the relation. These algorithms provide a set of cleaning-rule discovery tools for users to choose for different applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Spencer, J. (1991) · Medical Entomology and Zoology · Cited by 4,310
Abiteboul, S., Hull, R. B., & Vianu, V. (1994) · Cited by 3,461
Fast discovery of association rules
Agrawal, R., Mannila, H., Srikant, R., Toivonen, H. T. T., & Verkamo, A. I. (1996) · Knowledge Discovery and Data Mining · Cited by 2,376
(1995) · Choice Reviews Online · doi:10.5860/choice.33-0359 · Cited by 1,754
(2000) · Cited by 1,467
Discovery of Frequent Episodes in Event Sequences
Mannila, H., Toivonen, H., & Inkeri Verkamo, A. (1997) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009748302351 · Cited by 1,463
Alon, N., & Spencer, J. H. (2000) · Wiley eBooks · doi:10.1002/0471722154 · Cited by 949
Levelwise Search and Borders of Theories in Knowledge Discovery
Mannila, H., & Toivonen, H. T. T. (1997) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009796218281 · Cited by 889
Consistent query answers in inconsistent databases
Arenas, M., Bertossi, L., & Chomicki, J. (1999) · Proceedings of the eighteenth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems · doi:10.1145/303976.303983 · Cited by 868
(2006) · Data-Centric Systems and Applications · doi:10.1007/3-540-33173-5 · Cited by 659
Tane: An Efficient Algorithm for Discovering Functional and Approximate Dependencies
Huhtala, Y. (1999) · The Computer Journal · doi:10.1093/comjnl/42.2.100 · Cited by 598
Conditional functional dependencies for capturing data inconsistencies
Fan, W., Geerts, F., Jia, X., & Kementsietsidis, A. (2008) · ACM Transactions on Database Systems · doi:10.1145/1366102.1366103 · Cited by 460
Self-citationMinimal-change integrity maintenance using tuple deletions
Chomicki, J., & Marcinkowski, J. (2005) · Information and Computation · doi:10.1016/j.ic.2004.04.007 · Cited by 347
CORDS: automatic discovery of correlations and soft functional dependencies
Ilyas, I. F., Markl, V., Haas, P., Brown, P., & Aboulnaga, A. (2004) · Proceedings of the 2004 ACM SIGMOD international conference on Management of data · doi:10.1145/1007568.1007641 · Cited by 315
Improving data quality: consistency and accuracy
Cong, G., Fan, W., Geerts, F., et al. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 313
Self-citationDiscovering data quality rules
Chiang, F., & Miller, R. J. (2008) · Proceedings of the VLDB Endowment · doi:10.14778/1453856.1453980 · Cited by 265
Database repairing using updates
Wijsen, J. (2005) · ACM Transactions on Database Systems · doi:10.1145/1093382.1093385 · Cited by 208
On generating near-optimal tableaux for conditional functional dependencies
Golab, L., Karloff, H., Korn, F., Srivastava, D., & Yu, B. (2008) · Proceedings of the VLDB Endowment · doi:10.14778/1453856.1453900 · Cited by 200
Database dependency discovery: a machine learning approach
Flach, P. A., & Savnik, I. (1999) · doi:10.5555/1216155.1216159 · Cited by 171
Wyss, C., Giannella, C., & Robertson, E. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44801-2_11 · Cited by 170
Mining statistically important equivalence classes and delta-discriminative emerging patterns
Li, J., Liu, G., & Wong, L. (2007) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1281192.1281240 · Cited by 117
Self-citationExtending dependencies with conditions
Bravo, L., Fan, W., & Ma, S. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 107
Self-citationMannila, H., & Räihä, K. (1987) · Very Large Data Bases · Cited by 80
Li, H., Li, J., Wong, L., Feng, M., & Tan, Y. (2005) · ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS) · doi:10.1145/1065167.1065215 · Cited by 65
Self-citationEstimating the confidence of conditional functional dependencies
Cormode, G., Golab, L., Korn, F. R., McGregor, A., Srivastava, D., & Zhang, X. (2009) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/1559845.1559895 · Cited by 48
Mining Association Rules of Simple Conjunctive Queries
Goethals, B., Page, W. L., & Mannila, H. (2008) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972788.9 · Cited by 32
Cited by
Profiling relational data: a survey
Abedjan, Z., Golab, L., & Naumann, F. (2015) · The VLDB Journal · doi:10.1007/s00778-015-0389-y · Cited by 283
Data quality: The other face of Big Data
Saha, B., & Srivastava, D. (2014) · 2014 IEEE 30th International Conference on Data Engineering · doi:10.1109/icde.2014.6816764 · Cited by 262
Discovering denial constraints
Chu, X., Ilyas, I. F., & Papotti, P. (2013) · Proceedings of the VLDB Endowment · doi:10.14778/2536258.2536262 · Cited by 262
IoT data cleaning techniques: A survey
Ding, X., Wang, H., Li, G., Li, H., Li, Y., & Liu, Y. (2022) · Intelligent and Converged Networks · doi:10.23919/icn.2022.0026 · Cited by 35
Conflict resolution with data currency and consistency
Fan, W., Geerts, F., Tang, N., & Yu, W. (2014) · Journal of Data and Information Quality · doi:10.1145/2631923 · Cited by 30
Self-citation
Metadata
- Authors
- Fan, W., Geerts, F., Li, J., & Xiong, M.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2011-05
- Volume
- 23
- Issue
- 5
- Pages
- 683-698
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2113607096
- Cited by
- 253
- References
- 26
- FWCI
- 16.13
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.