Cited by 408Big Data Technologies and ApplicationsBig Data and Business IntelligenceInformation Retrieval and Data Mining

Data Science: A Comprehensive Overview

Open access
Cao, Longbing·2018·ACM Computing Surveys

Abstract

The 21st century has ushered in the age of big data and data economy, in which data DNA , which carries important knowledge, insights, and potential, has become an intrinsic constituent of all data-based organisms. An appropriate understanding of data DNA and its organisms relies on the new field of data science and its keystone, analytics . Although it is widely debated whether big data is only hype and buzz, and data science is still in a very early phase, significant challenges and opportunities are emerging or have been inspired by the research, innovation, business, profession, and education of data science. This article provides a comprehensive survey and tutorial of the fundamental aspects of data science: the evolution from data analysis to data science, the data science concepts, a big picture of the era of data science, the major challenges and directions in data innovation, the nature of data analytics, new industrialization and service opportunities in the data economy, the profession and competency of data education, and the future of data science. This article is the first in the field to draw a comprehensive big picture, in addition to offering rich observations, lessons, and thinking about data science and analytics.

Integrity signals

Registry

Crossref · 10.1145/3076253

Cites

100 of 100 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

2 of the 102 references couldn’t be matched to a known paper.

Research works printed but not linked: 50

  1. Mark, 2012, Beyer and Douglas Laney
  2. Cao Longbing, Understand Data Science (to be published)
  3. Cao Longbing, Open Complex Intelligent Systems
  4. Clay Kelly, 2013, CES 2013: The Year of The Quantified Self?
  5. European Commission. 2014. Commission urges governments to embrace potential of big data. Retrieved from europa.eu/rapid/press-release_IP-14-769_en.htm. European Commission. 2014. Commission urges governments to embrace potential of big data. Retrieved from europa.eu/rapid/press-release_IP-14-769_en.htm.
  6. DABS. 2016. Data Analytics Book Series. Retrieved from http://www.springer.com/series/15063. DABS. 2016. Data Analytics Book Series. Retrieved from http://www.springer.com/series/15063.
  7. Thomas H. Davenport and D. J. Patil. 2012. Data scientist: The sexiest job of the 21st century. Harvard Business Review (2012) 70--76. Thomas H. Davenport and D. J. Patil. 2012. Data scientist: The sexiest job of the 21st century. Harvard Business Review (2012) 70--76.
  8. DSAA., 2014, IEEE/ACM/ASA International Conference on Data Science and Advanced Analytics.
  9. DSE. 2015. Data Science and Engineering. Retrieved from http://link.springer.com/journal/41019. DSE. 2015. Data Science and Engineering. Retrieved from http://link.springer.com/journal/41019.
  10. DSJ. 2014. Data Science Journal.Retrieved from datascience.codata.org. DSJ. 2014. Data Science Journal.Retrieved from datascience.codata.org.
  11. EPJDS. 2012. EPJ Data Science. Retrieved from http://epjdatascience.springeropen.com/. EPJDS. 2012. EPJ Data Science. Retrieved from http://epjdatascience.springeropen.com/.
  12. 2016, Google Trends. (2016). https://www.google.com.au/trends/explore#q=datalyticsz=Etc Retrieved on 14 Novermber
  13. 10.5555/2823938 · DOI
  14. IEEEBD. 2014. IEEE Big Data Initiative. (2014). http://bigdata.ieee.org/. IEEEBD. 2014. IEEE Big Data Initiative. (2014). http://bigdata.ieee.org/.
  15. IJDS. 2016. International Journal of Data Science. (2016). http://www.inderscience.com/jhome.php?jcode=ijds. IJDS. 2016. International Journal of Data Science. (2016). http://www.inderscience.com/jhome.php?jcode=ijds.
  16. IJRDS. 2017. International Journal of Research on Data Science. Retrieved from http://www.sciencepublishinggroup.com/journal/index?journalid=310. IJRDS. 2017. International Journal of Research on Data Science. Retrieved from http://www.sciencepublishinggroup.com/journal/index?journalid=310.
  17. JDS. 2002. Journal of Data Science. Retrieved from http://www.jds-online.com/. JDS. 2002. Journal of Data Science. Retrieved from http://www.jds-online.com/.
  18. JDSA. 2015. International Journal of Data Science and Analytics (JDSA). Retrieved from http://www.springer.com/41060. JDSA. 2015. International Journal of Data Science and Analytics (JDSA). Retrieved from http://www.springer.com/41060.
  19. JFDS. 2016. The Journal of Finance and Data Science. Retrieved from http://www.keaipublishing.com/en/journals/the-journal-of-finance-and-data-science/. JFDS. 2016. The Journal of Finance and Data Science. Retrieved from http://www.keaipublishing.com/en/journals/the-journal-of-finance-and-data-science/.
  20. 1989, IJCAI-89 Workshop on Knowledge Discovery in Databases.
  21. Kelly K, 2012, Quantified Self Conference.
  22. Laney Douglas, 3D Data Management: Controlling Data Volume, Velocity and Variety
  23. Loukides Mike, The Evolution of Data Products. O’Reilly
  24. 2011, Big Data: The Next Frontier for Innovation, Competition, and Productivity
  25. Havart Arthur John, 1968, Proceedings of IFIP Congress
  26. Murray-Rust Peter, 2007, NSF/JISC 2007 Digital Repositories Workshop. http://www.sis.pitt.edu/repwkshop/papers/murray.pdf.
  27. Naur Peter, 1968, Proceedings of IFIP Congress
  28. NCSU., Institute for Advanced Analytics
  29. NCSU., Master of Science in Analytics
  30. Nelson Michael L., 2009, Data-driven science: A new paradigm, EDUCAUSE Review, 44, 6
  31. NIST., 2015, NIST Text Retrieval Conference Data.
  32. NSB., Long-Lived Digital Data Collections: Enabling Research and Education in the 21st Century.
  33. Qian Xuesen, 1991, Revisiting issues on open complex giant systems, International Journal of Pattern Recognition and Artificial Intelligence, 4, 5
  34. Qian Xuesen, 1993, A new discipline of science—The study of open complex giant system and its methodology. Chinese Journal of Systems Engineering 8 Electronics. 4, 2
  35. 2016, https://rapidminer.com/
  36. Renae Samantha, 2011, Data analytics: Crunching the future. Bloomberg Businessweek
  37. Solutions Review. 2016. Data Integration and Application Integration Solutions Directory. Retrieved from http://solutionsreview.com/data-integration/data-integration-solutions-directory/. Solutions Review. 2016. Data Integration and Application Integration Solutions Directory. Retrieved from http://solutionsreview.com/data-integration/data-integration-solutions-directory/.
  38. SAS., 2013, Big Data Analytics, An Assessment of Demand for Labour and Skills, 2012
  39. SIAM. 2016. SIAM career center. (2016). http://jobs.siam.org/home/. SIAM. 2016. SIAM career center. (2016). http://jobs.siam.org/home/.
  40. SSDS. 2015. Springer Series in the Data Sciences. Retrieved from http://www.springer.com/series/13852. SSDS. 2015. Springer Series in the Data Sciences. Retrieved from http://www.springer.com/series/13852.
  41. Stanford Data Science Initiatives
  42. 10.5555/39590.39607 · DOI
  43. TFDSAA. 2013. IEEE Task Force on Data Science and Advanced Analytics. Retrieved from http://dsaatf.dsaa.co/. TFDSAA. 2013. IEEE Task Force on Data Science and Advanced Analytics. Retrieved from http://dsaatf.dsaa.co/.
  44. TOBD. 2015. IEEE Transactions on Big Data. Retrieved from https://www.computer.org/web/tbd. TOBD. 2015. IEEE Transactions on Big Data. Retrieved from https://www.computer.org/web/tbd.
  45. Michigan Institute For Data Science
  46. UTS., Master of Analytics (Research) and Doctor of Philosophy Thesis: Analytics
  47. UTSAAI., Advanced Analytics Institute
  48. van Dyk David, 2015, Duncan Temple Lang, and Hadley Wickham
  49. Jeff Wu., Statistics &equals
  50. Bin Yu., 2014, IMS presidential address: Let us own data science. IMS Bulletin Online

Cited by

Sorted by citations

Authors
Cao, L.
Affiliations
University of Technology Sydney (AU)
Corresponding author
Cao, Longbing
Venue
ACM Computing Surveys
Published
2017-06-29
Volume
50
Issue
3
Pages
1-42
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
OpenAlex
W3104305579
Cited by
407
References
102
Citations by year
2026: 38 · 2025: 58 · 2024: 61 · 2023: 61 · 2022: 55 · 2021: 52 · 2020: 33 · 2019: 25 · 2018: 21 · 2017: 3
FWCI
36.49
Type
journal article
Language
en
Open access
Locations
ACM Computing Surveys (publishedVersion, cc-by-nc-sa)
Funders
Australian Research Council Discovery Grant (DP130102691)
Topics
  • Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
  • Social Sciences › Business, Management and Accounting › Management Information Systems › Big Data and Business Intelligence
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Data Mining
Keywords
data science, big data, data analytics, data economy, data science education, data science challenges

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review