Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 334

The DOI matches its Crossref record; 180 of the 180 references it cites were found in a registry or an index.

Data-centric Artificial Intelligence: A Survey

Open access
Zha, Daochen, Bhat, Zaid Pervaiz, Lai, Kwei-Herng, Yang, Fan, Jiang, Zhimeng, Zhong, Shaochen, Hu, Xia·2025·ACM Computing Surveys
DOI 10.1145/3711118
Open this paper in CoChat — free account

Abstract

Artificial Intelligence (AI) is making a profound impact in almost every domain. A vital enabler of its great success is the availability of abundant and high-quality data for building machine learning models. Recently, the role of data in AI has been significantly magnified, giving rise to the emerging concept of data-centric AI . The attention of researchers and practitioners has gradually shifted from advancing model design to enhancing the quality and quantity of the data. In this survey, we discuss the necessity of data-centric AI, followed by a holistic view of three general data-centric goals (training data development, inference data development, and data maintenance) and the representative methods. We also organize the existing literature from automation and collaboration perspectives, discuss the challenges, and tabulate the benchmarks for various tasks. We believe this is the first comprehensive survey that provides a global view of a spectrum of tasks across various stages of the data lifecycle. We hope it can help the readers efficiently grasp a broad picture of this field, and equip them with the techniques and further research ideas to systematically engineer data for building AI systems. A companion list of data-centric AI resources will be regularly updated on https://github.com/daochenzha/data-centric-AI .

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3711118

Cites

180 of 180 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 180 references

Research works printed but not linked: 101

  1. Mohamed Abdelaal Christian Hammacher and Harald Schoening. 2023. REIN: A comprehensive benchmark framework for data cleaning methods in ML pipelines. Retrieved from https://arXiv:2302.04702
  2. Peshawa Jamal Muhammad Ali, Rezhna H. Faraj, Erbil Koya, Peshawa J. Muhammad Ali, and Rezhna H. Faraj. 2014. Data normalization and standardization: A technical report. Mach. Learn. Tech. Rep. 1, 1 (2014), 1–6.
  3. 10.1109/4236.612229 · doi:10.1109/4236.612229
  4. Patricia C. Arocena, Boris Glavic, Giansalvatore Mecca, Renée J. Miller, Paolo Papotti, and Donatello Santoro. 2016. Benchmarking data curation systems. IEEE Data Eng. Bull. 39, 2 (2016), 47–62.
  5. B. Azhagusundari, Antony Selvadoss Thanamani et al. 2013. Feature selection based on information gain. Int. J. Innov.e Technol. Explor. Eng. 2, 2 (2013), 18–21.
  6. Matias Barenstein. 2019. Propublica’s compas data revisited. Retrieved from https://arXiv:1906.04711
  7. 10.1109/HICSS.1995.375612 · doi:10.1109/hicss.1995.375612
  8. Eyal Betzalel Coby Penso Aviv Navon and Ethan Fetaya. 2022. A study on the evaluation of generative models. Retrieved from https://arXiv:2206.10935
  9. Anant Bhardwaj, Souvik Bhattacherjee, Amit Chavan, Amol Deshpande, Aaron J. Elmore, Samuel Madden, and Aditya G. Parameswaran. 2015. Datahub: Collaborative data science & dataset version management at scale. In Proceedings of the CIDR.
  10. Pierre Blanchart. 2021. An exact counterfactual-example-based approach to tree-ensemble models interpretability. Retrieved from https://arXiv:2105.14820
  11. Benedikt Boecking, Willie Neiswanger, Eric Xing, and Artur Dubrawski. 2021. Interactive weak supervision: Learning useful heuristics for data labeling. In Proceedings of the ICLR.
  12. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the FAccT.
  13. 10.1007/978-1-4614-7485-2_17 · doi:10.1007/978-1-4614-7485-2_17
  14. Surajit Chaudhuri and Vivek R. Narasayya. 1997. An efficient, cost-driven index selection tool for Microsoft SQL server. In Proceedings of the VLDB.
  15. 10.1109/ICDE.2011.5767949 · doi:10.1109/icde.2011.5767949
  16. 10.5555/1622407.1622416 · doi:10.5555/1622407.1622416
  17. Paul F. Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Proceedings of the NeurIPS.
  18. 10.5555/1622737.1622744 · doi:10.5555/1622737.1622744
  19. Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V. Le. 2019. Autoaugment: Learning augmentation policies from data. In Proceedings of the CVPR.
  20. Ofer Dekel and Ohad Shamir. 2009. Vox Populi: Collecting high-quality labels from a crowd. In Proceedings of the COLT.
  21. Meghana Deodhar Xiao Ma Yixin Cai Alex Koes Alex Beutel and Jilin Chen. 2022. A Human-ML collaboration framework for improving video content reviews. Retrieved from https://arXiv:2210.09500
  22. Luc Desnoyers. 2011. Toward a taxonomy of visuals in science communication. Techn. Commun. 58, 2 (2011), 119–134.
  23. Amit Dhurandhar Tejaswini Pedapati Avinash Balakrishnan Pin-Yu Chen Karthikeyan Shanmugam and Ruchir Puri. 2019. Model agnostic contrastive explanations for structured data. Retrieved from https://arXiv:1906.00117
  24. Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. 2021. Retiring adult: New datasets for fair machine learning. In Proceedings of the NeurIPS.
  25. Sirui Ding Ruixiang Tang Daochen Zha Na Zou Kai Zhang Xiaoqian Jiang and Xia Hu. 2023. Fairly predicting graft failure in liver transplant for organ assigning. Retrieved from https://arXiv:2302.09400
  26. Iddo Drori Yamuna Krishnamurthy Remi Rampin Raoni de Paula Lourenco Jorge Piazentin Ono Kyunghyun Cho Claudio Silva and Juliana Freire. 2021. AlphaD3M: Machine learning pipeline synthesis. Retrieved from https://arXiv:2111.02508
  27. Abolfazl Farahani, Sahar Voghoei, Khaled Rasheed, and Hamid R. Arabnia. 2021. A brief review of domain adaptation. In Proceedings from ICDATA and IKE. 877–894.
  28. Raul Castro Fernandez, Ziawasch Abedjan, Famien Koko, Gina Yuan, Samuel Madden, and Michael Stonebraker. 2018. Aurum: A data discovery system. In Proceedings of the ICDE.
  29. Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum, and Frank Hutter. 2015. Efficient and robust automated machine learning. In Proceedings of the NeurIPS.
  30. 10.1145/3448016.3457334 · doi:10.1145/3448016.3457334
  31. Edwin Gamboa, Alejandro Libreros, Matthias Hirth, and Dan Dubiner. 2022. Human-AI collaboration for improving the identification of cars for autonomous driving. In Proceedings of the CIKM Workshop.
  32. Amirata Ghorbani, Michael Kim, and James Zou. 2020. A distributional framework for data valuation. In Proceedings of the ICML.
  33. Amirata Ghorbani and James Zou. 2019. Data shapley: Equitable valuation of data for machine learning. In Proceedings of the ICML.
  34. Pieter Gijsbers Marcos L. P. Bueno Stefan Coors Erin LeDell Sébastien Poirier Janek Thomas Bernd Bischl and Joaquin Vanschoren. 2022. Amlb: An automl benchmark. Retrieved from https://arXiv:2207.12560
  35. Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, and Bernhard Schölkopf. 2009. Covariate shift by kernel mean matching. Dataset Shift Mach. Learn. 3, 4 (2009), 5.
  36. Georges G. Grinstein Patrick Hoffman Ronald M. Pickett and Sharon J. Laskowski. 2002. Benchmark Development for the Evaluation of Visualization for Data Mining. Morgan Kaufmann Publishers Inc. San Francisco CA USA 129–176.
  37. Keren Gu Brandon Yang Jiquan Ngiam Quoc Le and Jonathon Shlens. 2019. Using videos to evaluate image model robustness. Retrieved from https://arXiv:1904.10076
  38. Xiaotian Han, Zhimeng Jiang, Ninghao Liu, and Xia Hu. 2022. G-Mixup: Graph data augmentation for graph classification. In Proceedings of the ICML.
  39. Haibo He, Yang Bai, Edwardo A. Garcia, and Shutao Li. 2008. ADASYN: Adaptive synthetic sampling approach for imbalanced learning. In Proceedings of the WCCI.
  40. Herodotos Herodotou, Harold Lim, Gang Luo, Nedyalko Borisov, Liang Dong, Fatma Bilgen Cetin, and Shivnath Babu. 2011. Starfish: A self-tuning system for big data analytics. In Proceedings of the CIDR.
  41. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In Proceedings of the NeurIPS.
  42. Sarah Hooper, Michael Wornow, Ying Hang Seah, Peter Kellman, Hui Xue, Frederic Sala, Curtis Langlotz, and Christopher Re. 2021. Cut out the annotator, keep the cutout: Better segmentation with weak supervision. In Proceedings of the ICLR.
  43. Johannes Jakubik Michael Vössing Niklas Kühl Jannis Walk and Gerhard Satzger. 2022. Data-centric artificial intelligence. Retrieved from https://arXiv:2212.11854
  44. Mohammad Hossein Jarrahi Ali Memariani and Shion Guha. 2022. The principles of data-centric AI (DCAI). Retrieved from https://arXiv:2211.14611
  45. Minqi Jiang Chaochuan Hou Ao Zheng Xiyang Hu Songqiao Han Hailiang Huang Xiangnan He Philip S. Yu and Yue Zhao. 2023. Weakly supervised anomaly detection: A survey. Retrieved from https://arXiv:2302.04549
  46. Zhimeng Jiang, Kaixiong Zhou, Zirui Liu, Li Li, Rui Chen, Soo-Hyun Choi, and Xia Hu. 2022. An information fusion approach to learning with instance-dependent label noise. In Proceedings of the ICLR.
  47. Shankar Kanthara, Rixie Tiffany Ko Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. 2022. Chart-to-text: A large-scale benchmark for chart summarization. In Proceedings of the ACL.
  48. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the NAACL.
  49. Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. 2021. Variational diffusion models. In Proceedings of the NeurIPS.
  50. Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao et al. 2021. Wilds: A benchmark of in-the-wild distribution shifts. In Proceedings of the ICML.
  51. Sanjay Krishnan and Eugene Wu. 2019. Alphaclean: Automatic generation of data cleaning pipelines. Retrieved from https://arXiv:1904.11827
  52. 10.1609/aaai.v35i18.18012 · doi:10.1609/aaai.v35i18.18012
  53. Kwei-Herng Lai, Daochen Zha, Junjie Xu, Yue Zhao, Guanchu Wang, and Xia Hu. 2021. Revisiting time series outlier detection: Definitions and benchmarks. In Proceedings of the NeurIPS.
  54. Kamakshi Lakshminarayan, Steven A. Harp, Robert P. Goldman, Tariq Samad et al. 1996. Imputation of missing data using machine learning techniques. In Proceedings of the KDD.
  55. Peng Li Xi Rao Jennifer Blase Yue Zhang Xu Chu and Ce Zhang. 2019. Cleanml: A benchmark for joint data cleaning and machine learning [experiments and analysis]. Retrieved from https://arXiv:1904.09483
  56. Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. 2018. Detecting and correcting for label shift with black box predictors. In Proceedings of the ICML.
  57. Zhining Liu, Pengfei Wei, Jing Jiang, Wei Cao, Jiang Bian, and Yi Chang. 2020. MESA: Boost ensemble imbalanced learning with meta-sampler. In Proceedings of the NeurIPS.
  58. 10.14778/3421424.3421425 · doi:10.14778/3421424.3421425
  59. Diego Martinex Daochen Zha Qiaoyu Tan and Xia Hu. 2023. Towards personalized preprocessing pipeline search. Retrieved from https://arXiv:2302.14329
  60. 10.1145/3318464.3380597 · doi:10.1145/3318464.3380597
  61. Mitar Milutinovic, Brandon Schoenfeld, Diego Martinez-Garcia, Saswati Ray, Sujen Shah, and David Yan. 2020. On evaluation of automl systems. In Proceedings of the ICML Workshop.
  62. Volodymyr Mnih Koray Kavukcuoglu David Silver Alex Graves Ioannis Antonoglou Daan Wierstra and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. Retrieved from https://arXiv:1312.5602
  63. Andrew Ng. 2021. Data-centric AI resource hub. Snorkel AI. Retrieved February 8, 2023 from https://snorkel.ai/
  64. Andrew Ng. 2023. Landing AI. Landing AI. Retrieved February 8, 2023 from https://landing.ai/
  65. Erkin Otles, Jeeheh Oh, Benjamin Li, Michelle Bochinski, Hyeon Joo, Justin Ortwine, Erica Shenoy, Laraine Washer, Vincent B. Young, Krishna Rao et al. 2021. Mind the performance gap: Examining dataset shift during prospective validation. In Proceedings of the MLHC.
  66. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray et al. 2022. Training language models to follow instructions with human feedback. In Proceedings of the NeurIPS.
  67. 10.1007/978-3-319-92639-1_60 · doi:10.1007/978-3-319-92639-1_60
  68. 10.1007/978-3-319-18818-8_2 · doi:10.1007/978-3-319-18818-8_2
  69. 10.14778/2733004.2733009 · doi:10.14778/2733004.2733009
  70. Neoklis Polyzotis and Matei Zaharia. 2021. What can data-centric AI learn from data and ML engineering? Retrieved from https://arXiv:2112.06439
  71. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever et al. 2018. Improving language understanding by generative pre-training. OpenAI (2018).
  72. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever et al. 2019. Language models are unsupervised multitask learners. OpenAI (2019).
  73. Alexander Ratner. 2023. Scale AI. Snorkel AI. Retrieved February 8, 2023 from https://snorkel.ai/
  74. 10.1109/ICDEW49219.2020.00035 · doi:10.1109/icdew49219.2020.00035
  75. Timo Schick and Hinrich Schütze. 2020. Few-shot text generation with pattern-exploiting training. Retrieved from https://arXiv:2012.11926
  76. Nabeel Seedat Fergus Imrie and Mihaela van der Schaar. 2022. DC-Check: A data-centric AI checklist to guide the development of reliable machine learning systems. Retrieved from https://arXiv:2211.05764
  77. Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. 2018. Poison frogs! Targeted clean-label poisoning attacks on neural networks. In Proceedings of the NeurIPS.
  78. Zheyan Shen Jiashuo Liu Yue He Xingxuan Zhang Renzhe Xu Han Yu and Peng Cui. 2021. Towards out-of-distribution generalization: A survey. Retrieved from https://arXiv:2108.13624
  79. Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. 2020. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. In Proceedings of the NeurIPS.
  80. Michael Stonebraker, Daniel Bruckner, Ihab F. Ilyas, George Beskales, Mitch Cherniack, Stanley B. Zdonik, Alexander Pagan, and Shan Xu. 2013. Data curation at scale: The data tamer system. In Proceedings of the CIDR.
  81. Oliver Sutton. 2012. Introduction to k nearest neighbour classification and condensed nearest neighbour data reduction. University lectures, University of Leicester 1 (2012).
  82. Wei Tang and Matthew Lease. 2011. Semi-supervised consensus labeling for crowdsourcing. In Proceedings of the SIGIR Workshop.
  83. Yuchao Tao Ryan McKenna Michael Hay Ashwin Machanavajjhala and Gerome Miklau. 2021. Benchmarking differentially private synthetic data generation algorithms. Retrieved from https://arXiv:2112.09238
  84. Saravanan Thirumuruganathan, Nan Tang, Mourad Ouzzani, and AnHai Doan. 2020. Data curation with deep learning. In Proceedings of the EDBT.
  85. 10.1109/ICDE.2000.839397 · doi:10.1109/icde.2000.839397
  86. Alexandr Wang. 2023. Scale AI. Scale AI. Retrieved February 8, 2023 from https://scale.com/
  87. Yidong Wang, Hao Chen, Yue Fan, Sun Wang, Ran Tao, Wenxin Hou, Renjie Wang, Linyi Yang, Zhi Zhou, Lan-Zhe Guo et al. 2022. Usb: A unified semi-supervised learning benchmark for classification. In Proceedings of the NeurIPS.
  88. Patrick Henry Winston. 1984. Artificial Intelligence. Addison-Wesley Longman Publishing.
  89. Jin Yong Yoo John X. Morris Eli Lifland and Yanjun Qi. 2020. Searching for a search method: Benchmarking search algorithms for generating nlp adversarial examples. Retrieved from https://arXiv:2009.06368
  90. Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. Bartscore: Evaluating generated text as text generation. In Proceedings of the NeurIPS.
  91. 10.1190/1.1444352 · doi:10.1190/1.1444352
  92. Daochen Zha Zaid Pervaiz Bhat Kwei-Herng Lai Fan Yang and Xia Hu. 2023. Data-centric AI: Perspectives and challenges. Retrieved from https://arXiv:2301.04819
  93. 10.1007/s10115-018-1280-0 · doi:10.1007/s10115-018-1280-0
  94. 10.24963/ijcai.2022/862 · doi:10.24963/ijcai.2022/862
  95. Daochen Zha, Jingru Xie, Wenye Ma, Sheng Zhang, Xiangru Lian, Xia Hu, and Ji Liu. 2021. Douzero: Mastering doudizhu with self-play deep reinforcement learning. In Proceedings of the ICML.
  96. Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. 2018. mixup: Beyond empirical risk minimization. In Proceedings of the ICLR.
  97. Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. 2019. Self-attention generative adversarial networks. In Proceedings of the IICML.
  98. Jieyu Zhang Cheng-Yu Hsieh Yue Yu Chao Zhang and Alexander Ratner. 2022. A survey on programmatic weak supervision. Retrieved from https://arXiv:2202.05433
  99. 10.1109/TFUZZ.2019.2959995 · doi:10.1109/tfuzz.2019.2959995
  100. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Proceedings of the NeurIPS.
  101. Barret Zoph, Golnaz Ghiasi, Tsung-Yi Lin, Yin Cui, Hanxiao Liu, Ekin Dogus Cubuk, and Quoc Le. 2020. Rethinking pre-training and self-training. In Proceedings of the NeurIPS.

Cited by

See all 334

Metadata

Authors
Zha, D., Bhat, Z. P., Lai, K.-H., Yang, F., Jiang, Z., Zhong, S., & Hu, X.
Venue
ACM Computing Surveys
Publisher
Association for Computing Machinery (ACM)
Published
2025-01-24
Volume
57
Issue
5
Pages
1-42
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
DOI
10.1145/3711118
OpenAlex
W4406085118
Cited by
334
References
180
FWCI
110.69
Type
journal article
Language
en
Open access
green
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Funders
NSF (IIS-2224843, ITE-2429680)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Data-centric Artificial Intelligence: A Survey. https://app.cochat.ai/papers/10.1145/3711118
The paper
Zha, D., Bhat, Z. P., Lai, K.-H., Yang, F., Jiang, Z., Zhong, S., & Hu, X. (2025). Data-centric Artificial Intelligence: A Survey. ACM Computing Surveys. https://doi.org/10.1145/3711118

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.