The DOI matches its Crossref record.
Supporting fine-grained data lineage in a database visualization environment
Abstract
The lineage of a datum records its processing history. Because such information can be used to trace the source of anomalies and errors in processed data sets, it is valuable to users for a variety of applications, including the investigation of anomalies and debugging. Traditional data lineage approaches rely on metadata. However, metadata does not scale well to fine-grained lineage, especially in large data sets. For example, it is not feasible to store all of the information that is necessary to trace from a specific floating-point value in a processed data set to a particular satellite image pixel in a source data set. In this paper, we propose a novel method to support fine-grained data lineage. Rather than relying on metadata, our approach lazily computes the lineage using a limited amount of information about the processing operators and the base data. We introduce the notions of weak inversion and verification. While our system does not perfectly invert the data, it uses weak inversion and verification to provide a number of guarantees about the lineage it generates. We propose a design for the implementation of weak inversion and verification in an object-relational database management system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Why and Where: A Characterization of Data Provenance
Buneman, P., Khanna, S., & Wang-Chiew, T. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44503-x_20 · Cited by 1,334
A survey of data provenance in e-science
Simmhan, Y. L., Plale, B., & Gannon, D. (2005) · ACM SIGMOD Record · doi:10.1145/1084805.1084812 · Cited by 1,153
Guidelines for using multiple views in information visualization
Wang Baldonado, M. Q., Woodruff, A., & Kuchinsky, A. (2000) · Proceedings of the working conference on Advanced visual interfaces · doi:10.1145/345513.345271 · Cited by 747
Self-citationChimera: a virtual data system for representing, querying, and automating data derivation
Foster, I., Vockler, J., Wilde, M., & Zhao, Y. (2002) · Proceedings 14th International Conference on Scientific and Statistical Database Management · doi:10.1109/ssdm.2002.1029704 · Cited by 605
Provenance in Databases: Why, How, and Where
Cheney, J., Chiticariu, L., & Tan, W.-C. (2009) · Foundations and Trends in Databases · doi:10.1561/1900000006 · Cited by 547
Tracing the lineage of view data in a warehousing environment
Cui, Y., Widom, J., & Wiener, J. L. (2000) · ACM Transactions on Database Systems · doi:10.1145/357775.357777 · Cited by 434
Lineage retrieval for scientific data processing: a survey
Bose, R., & Frew, J. (2005) · ACM Computing Surveys · doi:10.1145/1057977.1057978 · Cited by 409
Lineage tracing for general data warehouse transformations
Cui, Y., & Widom, J. (2003) · The VLDB Journal The International Journal on Very Large Data Bases · doi:10.1007/s00778-002-0083-8 · Cited by 345
A survey on provenance: What for? What form? What from?
Herschel, M., Diestelkämper, R., & Ben Lahmar, H. (2017) · The VLDB Journal · doi:10.1007/s00778-017-0486-1 · Cited by 300
Provenance management in curated databases
Buneman, P., Chapman, A., & Cheney, J. (2006) · Proceedings of the 2006 ACM SIGMOD international conference on Management of data · doi:10.1145/1142473.1142534 · Cited by 297
The Foundations for Provenance on the Web
Moreau, L. (2010) · Foundations and Trends® in Web Science · doi:10.1561/1800000010 · Cited by 168
A systematic review of provenance systems
Pérez, B., Rubio, J., & Sáenz-Adán, C. (2018) · Knowledge and Information Systems · doi:10.1007/s10115-018-1164-3 · Cited by 66
Towards Enabling Trusted Artificial Intelligence via Blockchain
Sarpatwar, K., Vaculin, R., Min, H., Su, G., Heath, T., Ganapavarapu, G., & Dillenberger, D. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-17277-0_8 · Cited by 50
Metadata
- Authors
- Woodruff, A., & Stonebraker, M.
- Venue
- Proceedings 13th International Conference on Data Engineering
- Publisher
- IEEE Comput. Soc. Press
- Published
- 2002-11-22
- Pages
- 91-102
- ISSN
- 1084-4627 (ISSN-L)
- OpenAlex
- W2171588311
- Cited by
- 197
- References
- 0
- FWCI
- 5.38
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.