Mining big data: current status, and forecast to the future
Abstract
Big Data is a new term used to identify datasets that we can not manage with current methodologies or data mining software tools due to their large size and complexity. Big Data mining is the capability of extracting useful information from these large datasets or streams of data. New mining techniques are necessary due to the volume, variability, and velocity, of such data. The Big Data challenge is becoming one of the most exciting opportunities for the years to come. We present in this issue, a broad overview of the topic, its current status, controversy, and a forecast to the future. We introduce four articles, written by influential scientists in the field, covering the most interesting and state-of-the-art topics on Big Data mining.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A great deal of existing definitions tend to converge on three main attributes of big data: huge volume, wide variety, and velocity of collection and analysis.
“A great deal of the existing definitions tend to converge on three main attributes of big data: the huge volume of data, the wide variety of data types, and the velocity at which the data can be collected and analysed.These are identified as the most agreed upon Vs (e.g., [49,86]).”
From On the sustainability of smart and smarter cities in the era of big data: an interdisciplinary and transdisciplinary literature review (2019) · cites this paper as [49]
Controversies over the benefits of big data analytics and its application involve limited access and related digital divides and other ethical concerns about accessibility.
“In this regard, controversies over the benefits of big data analytics and its application involve limited access and related digital divides and other ethical concerns about accessibility [49].”
From On the sustainability of smart and smarter cities in the era of big data: an interdisciplinary and transdisciplinary literature review (2019) · cites this paper as [49]
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479 · see citing works
Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data
Zikopoulos, P., & Eaton, C. (2011) · Cited by 1,482 · see citing works
The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East
Gantz, J., & Reinsel, D. (2012) · Cited by 1,146 · see citing works
Bifet, A., Holmes, G., Kirkby, R., et al. (2010) · Research Commons (University of Waikato) · Cited by 1,043 · see citing works
Self-citationBig Data: Principles and best practices of scalable realtime data systems
Marz, N., & Warren, J. (2015) · Cited by 1,019 · see citing works
S4: Distributed Stream Computing Platform
Neumeyer, L., Robbins, B. W., Nair, A., & Kesari, A. (2010) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2010.172 · Cited by 900 · see citing works
Efron, B. (2010) · Cambridge University Press eBooks · doi:10.1017/cbo9780511761362 · Cited by 820 · see citing works
Predictive Data Mining: A Practical Guide
Weiss, S. M., & Indurkhya, N. (1997) · Cited by 577 · see citing works
Knowledge Discovery from Data Streams
da Gama, J. M. P. (2010) · Chapman & Hall/CRC eBooks · doi:10.1201/ebk1439826119 · Cited by 572 · see citing works
Knowledge Discovery in Data Streams
Dang, X. H., & Ong, K. (2009) · CRC Press eBooks · doi:10.1081/e-elis3-120043643 · Cited by 563 · see citing works
Managing and Mining Sensor Data
(2013) · doi:10.1007/978-1-4614-6309-2 · Cited by 294 · cited 1× · see citing works
“There are many applications of Big Data, for example the following [17; 2]: • Business: costumer personalization, churn detection • Technology: reducing process time from hours to seconds • Health: mining DNA of each person, to discover, monitor and improve health aspects of every one • Smart…”
MethodChapman & Hall/CRC Data Mining and Knowledge Discovery Series
Hristidis, V. (2010) · Chapman & Hall/CRC data mining and knowledge discovery series · doi:10.1201/chdaminodis · Cited by 201 · see citing works
Anti-fragile: How to Live in a World We Don’t Understand
Keating, C. (2013) · Quantitative Finance · doi:10.1080/14697688.2013.830860 · Cited by 150 · see citing works
On the Origin(s) and Development of the Term 'Big Data'Preprint
Diebold, F. X. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2152421 · Cited by 148 · see citing works
The Conference on Uncertainty in Artificial Intelligence (UAI 2017)
Belle, V. (2017) · Cited by 134 · see citing works
Leinweber, D. (2007) · The Journal of Investing · doi:10.3905/joi.2007.681820 · Cited by 121 · see citing works
Paredaens, J., & Tenenbaum, L. A. (1994) · Springer eBooks · doi:10.1007/978-3-7091-2704-9 · Cited by 111 · see citing works
Mapreduce is Good Enough?If All You Have is a Hammer, Throw Away Everything That's Not a Nail!
Lin, J. (2012) · Big Data · doi:10.1089/big.2012.1501 · Cited by 81 · see citing works
Antifragile : how to live in a world we don't understand
Taleb, N. N. (2012) · Allen Lane eBooks · Cited by 77 · see citing works
Gopalkrishnan, V., Steier, D. M., Lewis, H. E., et al. (2012) · doi:10.1145/2351316.2351318 · Cited by 71 · see citing works
Stupid Data Miner Tricks: Overfitting the S&P 500
Leinweber, D. (2007) · The Journal of Investing · Cited by 67 · see citing works
Smolan, R. (2012) · Flinders Academic Commons (Flinders University) · Cited by 62 · see citing works
Pegasus: Mining billion-scale graphs in the cloud
Kang, U., Chau, D. H., & Faloutsos, C. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6289127 · Cited by 37 · see citing works
Unexpected challenges in large scale machine learning
Parker, C. (2012) · doi:10.1145/2351316.2351317 · Cited by 36 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Smart sustainable cities of the future: An extensive interdisciplinary literature review
Bibri, S. E., & Krogstie, J. (2017) · Sustainable Cities and Society · doi:10.1016/j.scs.2017.02.016 · Cited by 1,599 · see citing works
Agrawal, A., & Choudhary, A. (2016) · APL Materials · doi:10.1063/1.4946894 · Cited by 1,385 · see citing works
The role of big data in smart city
Hashem, I. A. T., Chang, V., Anuar, N. B., Adewole, K., Yaqoob, I., Gani, A., Ahmed, E., & Chiroma, H. (2016) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2016.05.002 · Cited by 1,103 · see citing works
Machine learning with big data: challenges and approaches.
L'Heureux, A., Grolinger, K., Elyamany, H. F., & Capretz, M. A. M. (2017) · IEEE Access · doi:10.1109/access.2017.2696365 · Cited by 1,023 · see citing works
Applications of big data to smart cities
Al Nuaimi, E., Al Neyadi, H., Mohamed, N., & Al-Jaroodi, J. (2015) · Journal of Internet Services and Applications · doi:10.1186/s13174-015-0041-5 · Cited by 960 · see citing works
Tsai, C.-W., Lai, C.-F., Chao, H.-C., & Vasilakos, A. V. (2015) · Journal of Big Data · doi:10.1186/s40537-015-0030-3 · Cited by 818 · see citing works
Bibri, S. E. (2018) · Sustainable Cities and Society · doi:10.1016/j.scs.2017.12.034 · Cited by 799 · see citing works
The ethics of big data: current and foreseeable issues in biomedical contexts
Mittelstadt, B. D., & Floridi, L. (2016) · Science and Engineering Ethics · doi:10.1007/s11948-015-9652-2 · Cited by 714 · see citing works
Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare
Cirillo, D., Catuara-Solarz, S., Morey, C., Guney, E., Subirats, L., Mellino, S., Gigante, A., Valencia, A., Rementeria, M. J., Chadha, A. S., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0288-5 · Cited by 553 · see citing works
Short, C. E., DeSmet, A., Woods, C., Williams, S. L., Maher, C., Middelweerd, A., Müller, A. M., Wark, P. A., Vandelanotte, C., Poppe, L., et al. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9397 · Cited by 400 · see citing works
O’Dwyer, E., Pan, I., Acha, S., & Shah, N. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.01.024 · Cited by 410 · see citing works
Bibri, S. E. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0182-7 · Cited by 278 · see citing works
Towards cloud based big data analytics for smart future cities
Khan, Z., Anjum, A., Soomro, K., & Tahir, M. A. (2015) · Journal of Cloud Computing · doi:10.1186/s13677-015-0026-8 · Cited by 229 · see citing works
Bibri, S. E., & Krogstie, J. (2017) · Sustainable Cities and Society · doi:10.1016/j.scs.2017.04.012 · Cited by 167 · see citing works
How Machine Learning is Changing e-Government
Alexopoulos, C., Lachana, Z., Androutsopoulou, A., Diamantopoulou, V., Charalabidis, Y., & Loutsaris, M. A. (2019) · Proceedings of the 12th International Conference on Theory and Practice of Electronic Governance · doi:10.1145/3326365.3326412 · Cited by 88 · see citing works
Quadir, B., Chen, N.-S., & Isaias, P. (2022) · Interactive Learning Environments · doi:10.1080/10494820.2020.1712427 · Cited by 46 · see citing works
Data philanthropy and the design of the infraethics for information societies
Taddeo, M. (2016) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2016.0113 · Cited by 35 · see citing works
- Authors
- Fan, W., & Bifet, A.
- Affiliations
- Huawei Noah's Ark Lab, Hong Kong Science Park, Shatin, Hong Kong
- Yahoo! Research Barcelona, Barcelona, Catalonia, Spain
- Venue
- ACM SIGKDD Explorations Newsletter
- Published
- 2013-04-30
- Volume
- 14
- Issue
- 2
- Pages
- 1-5
- ISSN
- 1931-0145 (ISSN-L)
- 1931-0153
- OpenAlex
- W2164364358
- Cited by
- 677
- References
- 25
- Citations by year
- 2026: 6 · 2025: 12 · 2024: 11 · 2023: 19 · 2022: 27 · 2021: 57 · 2020: 71 · 2019: 75 · 2018: 94 · 2017: 94 · 2016: 93 · 2015: 76 · 2014: 40 · 2013: 2
- FWCI
- 154.33
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM SIGKDD Explorations Newsletter (publishedVersion)
- Funders
- University of Auckland
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- big data mining, large-scale data, data streams, data volume, data variability, data velocity, data mining techniques
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
