CoChat
Sign Up
Cited by 14Data Quality and ManagementAdvanced Database Systems and QueriesScientific Computing and Data Management

Auto-Validate by-History: Auto-Program Data Quality Constraints to Validate Recurring Data Pipelines

Open access
Tu, Dezhan, He, Yeye, Cui, Weiwei, Ge, Song, Zhang, Haidong, Han, Shi, Zhang, Dongmei, Chaudhuri, Surajit·2023·Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
DOI 10.1145/3580305.3599776
Open this paper in CoChat — free accountOpen PDF

Abstract

Data pipelines are widely employed in modern enterprises to power a variety of Machine-Learning (ML) and Business-Intelligence (BI) applications. Crucially, these pipelines are recurring (e.g., daily or hourly) in production settings to keep data updated so that ML models can be re-trained regularly, and BI dashboards refreshed frequently. However, data quality (DQ) issues can often creep into recurring pipelines because of upstream schema and data drift over time. As modern enterprises operate thousands of recurring pipelines, today data engineers have to spend substantial efforts to manually monitor and resolve DQ issues, as part of their DataOps and MLOps practices.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3580305.3599776

Cites

47 of 47 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • On-demand and continuous monitoring of model performance and safety in production is a critical practice associated with MLOps.

    “On-demand and continuous monitoring of model performance and safety in a production environment is yet another critical practice associated with MLOps (e.g., [24])”

    From A Multivocal Review of MLOps Practices, Challenges and Open Issues (2026) · cites this paper as [24]

    The Introduction describes monitoring data-quality issues in recurring production pipelines as part of MLOps and explains that those issues can harm downstream model predictions. It does not describe on-demand monitoring or direct monitoring of model performance and safety. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 19

  1. n. d.]. Full version of the paper: Auto-Validate-by-History. https://arxiv.org/abs/2306.02421. · arXiv
  2. Eric Breck, Neoklis Polyzotis, Sudip Roy, Steven Whang, and Martin Zinkevich. 2019. Data Validation for Machine Learn- ing. In Conference on Systems and Machine Learning (SysML). https://www. sysml. cc/doc/2019/167. pdf.
  3. Fei Chiang and Renée J Miller. 2008. Discovering data quality rules. VLDB 1, 1 (2008).
  4. Xu Chu, Ihab F Ilyas, and Paolo Papotti. 2013. Discovering denial constraints. VLDB 6, 13 (2013).
  5. Helena Galhardas, Daniela Florescu, Dennis Shasha, Eric Si- mon, and Cristian Saita. 2001. Declarative data cleaning: Lan- guage, model, and algorithms. Ph. D. Dissertation. INRIA.
  6. Alireza Heidari, Joshua McGrath, Ihab F Ilyas, and Theodoros Rekatsinas. 2019. Holodetect: Few-shot learning for error detection. In Proceedings of the 2019 International Conference on Management of Data. 829-846.
  7. Victoria Hodge and Jim Austin. 2004. A survey of outlier detection methodologies. Artificial intelligence review (2004).
  8. Zhipeng Huang and Yeye He. 2018. Auto-Detect: Data-Driven Error Detection in Tables. In SIGMOD.
  9. Philipp Langer and Felix Naumann. 2016. Efficient order de- pendency detection. The VLDB Journal 25, 2 (2016), 223-241.
  10. Moisés F Lima, Bruno B Zarpelao, Lucas DH Sampaio, Joel JPC Rodrigues, Taufik Abrao, and Mario Lemes Proença. 2010. Anomaly detection using baseline and k-means clustering. In SoftCOM 2010, 18th International Conference on Software, Telecommunications and Computer Networks. IEEE, 305-309.
  11. Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. 2008. Isola- tion forest. In 2008 eighth ieee international conference on data mining. IEEE, 413-422.
  12. Mohammad Mahdavi, Ziawasch Abedjan, Raul Castro Fernan- dez, Samuel Madden, Mourad Ouzzani, Michael Stonebraker, and Nan Tang. 2019. Raha: A configuration-free error detec- tion system. In Proceedings of the 2019 International Conference on Management of Data. 865-882.
  13. Vijayshankar Raman and Joseph M Hellerstein. 2001. Potter's wheel: An interactive data cleaning system. In VLDB, Vol. 1.
  14. Sebastian Schelter, Stefan Grafberger, Philipp Schmidt, Tammo Rukat, Mario Kiessling, Andrey Taptunov, Felix Biessmann, and Dustin Lange. 2018. Deequ-data quality validation for ma- chine learning pipelines. In Machine Learning Systems Work- shop at the Conference on Neural Information Processing Systems (NeurIPS).
  15. Sebastian Schmidl, Phillip Wenig, and Thorsten Papenbrock. 2022. Anomaly detection in time series: a comprehensive evaluation. Proceedings of the VLDB Endowment 15, 9 (2022), 1779-1797.
  16. Bernhard Schölkopf, Robert C Williamson, Alex Smola, John Shawe-Taylor, and John Platt. 1999. Support vector method for novelty detection. Advances in neural information processing systems 12 (1999).
  17. Jaroslaw Szlichta, Parke Godfrey, Lukasz Golab, Mehdi Kargar, and Divesh Srivastava. 2016. Effective and complete discovery of order dependencies via set-based axiomatization. arXiv preprint arXiv:1608.06169 (2016). · arXiv
  18. Jian Tang, Zhixiang Chen, Ada Wai-Chee Fu, and David W Cheung. [n. d.]. Enhancing effectiveness of outlier detections for low density patterns. In PAKDD 2002. Springer.
  19. Pei Wang and Yeye He. 2019. Uni-Detect: A Unified Approach to Automated Error Detection in Tables. In Proceedings of the 2019 International Conference on Management of Data. 811- 828.

Cited by

Sorted by citations

Related

Authors
Tu, D., He, Y., Cui, W., Ge, S., Zhang, H., Han, S., Zhang, D., & Chaudhuri, S.
Affiliations
  • University of California, Los Angeles (US)
  • Microsoft (United States) (US)
  • Microsoft Research Asia (China) (CN)
ORCID
Venue
Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
Publisher
ACM
Published
2023-08-04
Pages
4991-5003
DOI
10.1145/3580305.3599776
OpenAlex
W4385568252
Cited by
14
References
47
Citations by year
2026: 6 · 2025: 6 · 2024: 2
FWCI
7.69
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Data Quality and Management
  • Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Database Systems and Queries
  • Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
Keywords
data quality, data validation, data drift, schema drift, data quality constraints, DataOps, MLOps, business intelligence dashboard

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review