CoChat

Data collection and quality challenges for deep learning

Whang, Steven Euijong, Lee, Jae-Gil · 2020 · Proceedings of the VLDB Endowment
Cited by 153

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Abstract

Software 2.0 refers to the fundamental shift in software engineering where using machine learning becomes the new norm in software with the availability of big data and computing infrastructure. As a result, many software engineering practices need to be rethought from scratch where data becomes a first-class citizen, on par with code. It is well known that 80--90% of the time for machine learning development is spent on data preparation. Also, even the best machine learning algorithms cannot perform well without good data or at least handling biased and dirty data during model training. In this tutorial, we focus on data collection and quality challenges that frequently occur in deep learning applications. Compared to traditional machine learning, there is less need for feature engineering, but more need for significant amounts of data. We thus go through state-of-the-art data collection techniques for machine learning. Then, we cover data validation and cleaning techniques for improving data quality. Even if the data is still problematic, hope is not lost, and we cover fair and robust training techniques for handling data bias and errors. We believe that the data management community is well poised to lead the research in these directions. The presenters have extensive experience in developing machine learning platforms and publishing papers in top-tier database, data mining, and machine learning venues.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.14778/3415478.3415562

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Research works printed but not linked: 17

  1. E. Breck, M. Zinkevich, N. Polyzotis, S. Whang, and S. Roy. Data validation for machine learning. In MLSys, 2019.
  2. 10.1038/s41598-018-24271-9 · DOI · Paper page
  3. L. Dong and T. Rekatsinas. Data integration and machine learning: A natural synergy. PVLDB, 11(12):2094--2097, 2018.
  4. 10.1145/3310205 · DOI · Paper page
  5. 10.1145/3035918.3054775 · DOI
  6. P. Li, X. Rao, J. Blase, Y. Zhang, X. Chu, and C. Zhang. CleanML: A benchmark for joint data cleaning and machine learning. CoRR, abs/1904.09483, 2019. · arXiv
  7. 10.1109/ICDE.2018.00062 · DOI · Paper page
  8. F. Nargesian, E. Zhu, R. J. Miller, K. Q. Pu, and P. C. Arocena. Data lake management: Challenges and opportunities. PVLDB, 12(12):1986--1989, 2019.
  9. 10.1145/3366423.3380233 · doi:10.1145/3366423.3380233
  10. 10.1145/3035918.3054782 · DOI · Paper page
  11. Y. Roh, G. Heo, and S. E. Whang. A survey on data collection for machine learning: a big data - AI integration perspective. IEEE TKDE, 2019.
  12. Y. Roh, K. Lee, S. E. Whang, and C. Suh. FR-Train: A mutual information-based approach to fair and robust training. In ICML, 2020.
  13. H. Song, M. Kim, and J. Lee. SELFIE: Refurbishing unclean samples for robust deep learning. In ICML, pages 5907--5915, 2019.
  14. M. Stonebraker and E. K. Rezig. Machine learning and big data: What is important? IEEE Data Eng. Bull., 2019.
  15. 10.14778/2831360.2831371 · DOI · Paper page
  16. 10.1145/3294052.3322192 · DOI
  17. 10.1145/3035918.3064019 · DOI · Paper page

Cited by

See all 153

Metadata

Authors
Whang, S. E., & Lee, J.-G.
Venue
Proceedings of the VLDB Endowment
Published
2020-08
Volume
13
Issue
12
Pages
3429-3432
ISSN
2150-8097 (ISSN-L)
OpenAlex
W3085364681
Cited by
153
References
2
FWCI
8.60
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Data collection and quality challenges for deep learning. https://app.cochat.ai/papers/10.14778/3415478.3415562
The paper
Whang, S. E., & Lee, J.-G. (2020). Data collection and quality challenges for deep learning. Proceedings of the VLDB Endowment. https://doi.org/10.14778/3415478.3415562

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.