Toward Scalable Systems for Big Data Analytics: A Technology Tutorial
Open accessAbstract
Recent technological advancements have led to a deluge of data from distinctive domains (e.g., health care and scientific sensors, user-generated data, Internet and financial companies, and supply chain systems) over the past two decades. The term big data was coined to capture the meaning of this emerging trend. In addition to its sheer volume, big data also exhibits other unique characteristics as compared with traditional data. For instance, big data is commonly unstructured and require more real-time analysis. This development calls for new system architectures for data acquisition, transmission, storage, and large-scale data processing mechanisms. In this paper, we present a literature survey and system tutorial for big data analytics platforms, aiming to provide an overall picture for nonexpert readers and instill a do-it-yourself spirit for advanced audiences to customize their own big-data solutions. First, we present the definition of big data and discuss big data challenges. Next, we present a systematic framework to decompose big data systems into four sequential modules, namely data generation, data acquisition, data storage, and data analytics. These four modules form a big data value chain. Following that, we present a detailed survey of numerous approaches and mechanisms from research and industry communities. In addition, we present the prevalent Hadoop framework for addressing big data challenges. Finally, we outline several evaluation benchmarks and potential research directions for big data systems.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Traditional batch-based monitoring approaches often fall short in addressing the dynamic nature of data pipelines and the need for real-time insights.
“Traditional batch-based monitoring approaches often fall short in addressing the dynamic nature of data pipelines and the need for real-time insights [3].”
From Real-Time Monitoring of Data Pipelines: Exploring and Experimentally Proving that the Continuous Monitoring in Data Pipelines Reduces Cost and Elevates Quality (2024) · cites this paper as [3]
The “Future Research” section (page not provided) says mature batch-processing paradigms struggle with growing data volumes and substantial real-time requirements. The “Streaming vs. Batch” section describes streaming as suited to rapidly arriving data, but the paper does not specifically assess batch-based monitoring of data pipelines. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Data mining: concepts and techniques
Sharma, R., Shah, K., Shirahatti, Y., Patel, S., Sharma, R., Shah, K., Patel, S., Sharma, R., & Han, J. (2012) · Choice Reviews Online · doi:10.5860/choice.49-3305 · Cited by 28,655 · see citing works
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476 · see citing works
The anatomy of a large-scale hypertextual Web search engine
Brin, S., & Page, L. (1998) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(98)00110-x · Cited by 16,004 · see citing works
The NIST Definition of Cloud Computing
Mell, P., & Grance, T. (2011) · doi:10.6028/nist.sp.800-145 · Cited by 15,338 · see citing works
An index to quantify an individual’s scientific research output
Hirsch, J. E. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507655102 · Cited by 11,669 · see citing works
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816 · see citing works
Opinion mining and sentiment analysis
Pang, B., & Lee, L. (2008) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000011 · Cited by 6,372 · see citing works
Top 10 algorithms in data mining
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., et al. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0114-2 · Cited by 5,667 · see citing works
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516 · see citing works
(2011) · Cited by 5,203 · see citing works
Benchmarking cloud serving systems with YCSB
Cooper, B. F., Silberstein, A., Tam, E., Ramakrishnan, R., & Sears, R. (2010) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/1807128.1807152 · Cited by 3,754 · see citing works
Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J. H., McCauley, M., Franklin, M. J. M., Shenker, S. J., & Stoica, I. G. (2012) · Cited by 3,622 · see citing works
Malewicz, G. G., Austern, M. H., Bik, A. J. C., Dehnert, J. C., Horn, I., Leiser, N., & Czajkowski, G. (2010) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1807167.1807184 · Cited by 3,518 · see citing works
Chang, F. W., Dean, J., Ghemawat, S., Hsieh, W. C.-Y., Wallach, D. A., Burrows, M., Chandra, T., Fikes, A., & Gruber, R. E. (2008) · ACM Transactions on Computer Systems · doi:10.1145/1365815.1365816 · Cited by 3,461 · see citing works
(2007) · doi:10.1145/1294261.1294281 · Cited by 3,447 · see citing works
OpenMP: an industry standard API for shared-memory programming
Dagum, L., & Menon, R. G. (1998) · IEEE Computational Science and Engineering · doi:10.1109/99.660313 · Cited by 3,394 · see citing works
Lenzerini, M. (2002) · ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS) · doi:10.1145/543613.543644 · Cited by 2,520 · see citing works
Isard, M., Budiu, M., Yu, Y., Birrell, A. D., & Fetterly, D. (2007) · European Conference on Computer Systems · doi:10.1145/1272996.1273005 · Cited by 2,433 · see citing works
OFDM for Optical Communications
Armstrong, J. (2009) · Journal of Lightwave Technology · doi:10.1109/jlt.2008.2010061 · Cited by 2,121 · see citing works
Greenberg, A. G., Hamilton, J. R., Jain, N., Kandula, S., Kim, C., Lahiri, P., Maltz, D. A., Patel, P., & Sengupta, S. (2009) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/1592568.1592576 · Cited by 2,114 · see citing works
Distributed Systems: Principles and Paradigms
Tanenbaum, A. S., & van Steen, M. R. (2001) · Cited by 2,084 · see citing works
Karger, D., Lehman, E., Leighton, T., Panigrahy, R., Levine, M., & Lewin, D. (1997) · Proceedings of the twenty-ninth annual ACM symposium on Theory of computing - STOC '97 · doi:10.1145/258533.258660 · Cited by 1,981 · see citing works
Bleiholder, J., & Naumann, F. (2009) · ACM Computing Surveys · doi:10.1145/1456650.1456651 · Cited by 1,874 · see citing works
Korth, H. F., & Silberschatz, A. (1980) · Cited by 1,784 · see citing works
Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services
Gilbert, S. L., & Lynch, N. A. (2002) · ACM SIGACT News · doi:10.1145/564585.564601 · Cited by 1,741 · see citing works
The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines
Barroso, L. A., & Hölzle, U. (2009) · Synthesis lectures on computer architecture · doi:10.2200/s00193ed1v01y200905cac006 · Cited by 1,736 · see citing works
Keulen, D. S. (2014) · Cited by 1,663 · see citing works
Low, Y., Bickson, D., Gonzalez, J. E., Guestrin, C., Kyrola, A., & Hellerstein, J. M. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2212351.2212354 · Cited by 1,656 · see citing works
An Introduction to Multivariate Statistical Analysis
(2004) · Technometrics · doi:10.1198/tech.2004.s754 · Cited by 1,652 · see citing works
Thusoo, A., Sarma, J. S., Jain, N., Shao, Z., Chakka, P., Anthony, S., Liu, H., Wyckoff, P., & Murthy, R. (2009) · Proceedings of the VLDB Endowment · doi:10.14778/1687553.1687609 · Cited by 1,559 · see citing works
Focused crawling: a new approach to topic-specific Web resource discovery
Chakrabarti, S., van den Berg, M., & Dom, B. (1999) · Computer Networks · doi:10.1016/s1389-1286(99)00052-3 · Cited by 1,487 · see citing works
Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data
Zikopoulos, P., & Eaton, C. (2011) · Cited by 1,482 · see citing works
The Fourth Paradigm: Data-Intensive Scientific Discovery
Buchan, I. E., Winn, J. M., & Bishop, C. (2009) · Cited by 1,428 · see citing works
Scale and performance in a distributed file system
Howard, J. H., Kazar, M. L., Menees, S. G., Nichols, D. A., Satyanarayanan, M., Sidebotham, R. N., & West, M. J. (1988) · ACM Transactions on Computer Systems · doi:10.1145/35037.35059 · Cited by 1,402 · see citing works
Zaharia, M., Borthakur, D., Sarma, J. S., Elmeleegy, K., Shenker, S. J., & Stoica, I. G. (2010) · European Conference on Computer Systems · doi:10.1145/1755913.1755940 · Cited by 1,394 · see citing works
Scalable SQL and NoSQL data stores
Cattell, R. G. G. (2011) · ACM SIGMOD Record · doi:10.1145/1978915.1978919 · Cited by 1,380 · see citing works
Six degrees: the science of a connected age
Barabási, A.-L., Watts, D. J., & Review, B. (2003) · Choice Reviews Online · doi:10.5860/choice.40-6452 · Cited by 1,373 · see citing works
Ghemawat, S., Gobioff, H., & Leung, S.-T. A. (2003) · doi:10.1145/945449.945450 · Cited by 1,347 · see citing works
DeWitt, D. J., & Gray, J. (1992) · Communications of the ACM · doi:10.1145/129888.129894 · Cited by 1,291 · see citing works
Named Entity Recognition in Tweets: An Experimental Study
Ritter, A., Clark, S., & Etzioni, O. (2011) · Cited by 1,192 · see citing works
Data mining and knowledge discovery handbook
(2011) · Choice Reviews Online · doi:10.5860/choice.48-5729 · Cited by 1,174 · see citing works
DeCandia, G., Hastorun, D., Jampani, M., et al. (2007) · ACM SIGOPS Operating Systems Review · doi:10.1145/1323293.1294281 · Cited by 1,150 · see citing works
The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East
Gantz, J., & Reinsel, D. (2012) · Cited by 1,146 · see citing works
A comparison of approaches to large-scale data analysis
Pavlo, A., Paulson, E., Rasin, A., et al. (2009) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/1559845.1559865 · Cited by 1,082 · see citing works
Challenges and opportunities with big data
Labrinidis, A., & Jagadish, H. V. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2367502.2367572 · Cited by 1,070 · see citing works
Noy, N. F. (2004) · ACM SIGMOD Record · doi:10.1145/1041410.1041421 · Cited by 1,028 · see citing works
Learning multiple layers of representation
Hinton, G. E. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.09.004 · Cited by 1,016 · see citing works
Empirical comparison of algorithms for network community detection
Leskovec, J., Lang, K. J., & Mahoney, M. W. (2010) · International World Wide Web Conference (WWW) · doi:10.1145/1772690.1772755 · Cited by 965 · see citing works
Cooper, B. F., Ramakrishnan, R., Srivastava, U., et al. (2008) · Proceedings of the VLDB Endowment · doi:10.14778/1454159.1454167 · Cited by 962 · see citing works
Social influence analysis in large-scale networks
Tang, J., Sun, J., Wang, C., & Yang, Z. (2009) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1557019.1557108 · Cited by 952 · see citing works
Farrington, N., Porter, G., Radhakrishnan, S., Bazzaz, H. H., Subramanya, V., Fainman, Y., Papen, G. C., & Vahdat, A. M. (2010) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/1851182.1851223 · Cited by 918 · see citing works
S4: Distributed Stream Computing Platform
Neumeyer, L., Robbins, B. W., Nair, A., & Kesari, A. (2010) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2010.172 · Cited by 900 · see citing works
Gaber, M. M., Zaslavsky, A., & Krishnaswamy, S. P. (2005) · ACM SIGMOD Record · doi:10.1145/1083784.1083789 · Cited by 890 · see citing works
The Chubby lock service for loosely-coupled distributed systems
Burrows, M. (2006) · Operating Systems Design and Implementation · doi:10.5555/1298455.1298487 · Cited by 887 · see citing works
Graph clustering based on structural/attribute similarities
Zhou, Y., Cheng, H., & Yu, J. X. (2009) · Proceedings of the VLDB Endowment · doi:10.14778/1687627.1687709 · Cited by 865 · see citing works
High-performance complex event processing over streams
Wu, E., Diao, Y., & Rizvi, S. (2006) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/1142473.1142520 · Cited by 838 · see citing works
Spanner: Google's globally-distributed database
Corbett, J. C., Dean, J. B., Epstein, M., Fikes, A., Frost, C., Furman, J. J., Ghemawat, S., Gubarev, A., Heiser, C., Hochschild, P., et al. (2012) · doi:10.5555/2387880.2387905 · Cited by 827 · see citing works
Towards robust distributed systems (abstract)
Brewer, E. A. (2000) · doi:10.1145/343477.343502 · Cited by 810 · see citing works
State-of-the-art in privacy preserving data mining
Verykios, V. S., Bertino, E., Fovino, I. N., Provenza, L. P., Saygın, Y., & Theodoridis, Y. (2004) · ACM SIGMOD Record · doi:10.1145/974121.974131 · Cited by 810 · see citing works
Tolle, G., Polastre, J., Szewczyk, R., Culler, D., Turner, N., Tu, K. P., Burgess, S. S. O., Dawson, T. E., Buonadonna, P., Gay, D. E., et al. (2005) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/1098918.1098925 · Cited by 792 · see citing works
Aggarwal, C. C. (2011) · Springer eBooks · doi:10.1007/978-1-4419-8462-3 · Cited by 785 · see citing works
Ekanayake, J., Li, H., Zhang, B., Gunarathne, T., Bae, S.-H., Qiu, J., & Fox, G. (2010) · ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC) · doi:10.1145/1851476.1851593 · Cited by 777 · see citing works
An overview of business intelligence technology
Chaudhuri, S., Dayal, U., & Narasayya, V. R. (2011) · Communications of the ACM · doi:10.1145/1978542.1978562 · Cited by 763 · see citing works
Condie, T., Conway, N., Alvaro, P., Hellerstein, J. M., Elmeleegy, K., & Sears, R. (2010) · doi:10.5555/1855711.1855732 · Cited by 751 · see citing works
Chaiken, R., Jenkins, B., Larson, P.-Å., Ramsey, B., Shakib, D., Weaver, S., & Zhou, J. (2008) · Proceedings of the VLDB Endowment · doi:10.14778/1454159.1454166 · Cited by 727 · see citing works
Melnik, S., Gubarev, A., Long, J. J., Romer, G., Shivakumar, S., Tolton, M., & Vassilakis, T. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920886 · Cited by 713 · see citing works
Finding a team of experts in social networks
Lappas, T., Liu, K., & Terzi, E. (2009) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1557019.1557074 · Cited by 710 · see citing works
Yu, Y., Isard, M., Fetterly, D., Budiu, M., Erlingsson, Ú., Gunda, P. K., & Currey, J. (2008) · doi:10.5555/1855741.1855742 · Cited by 703 · see citing works
Interactive deduplication using active learning
Sarawagi, S., & Bhamidipaty, A. (2002) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/775047.775087 · Cited by 698 · see citing works
Bu, Y., Howe, B., Bałazińska, M., & Ernst, M. D. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920881 · Cited by 694 · see citing works
Megastore: Providing Scalable, Highly Available Storage for Interactive Services
Baker, J. D., Bond, C. T., Corbett, J. C., et al. (2011) · Cited by 652 · see citing works
A survey of Cyber-Physical Systems
Shi, J., Wan, J., Yan, H., & Suo, H. (2011) · International Conference on Wireless Communications and Signal Processing (WCSP) · doi:10.1109/wcsp.2011.6096958 · Cited by 623 · see citing works
Interpreting the Data: Parallel Analysis with Sawzall
Pike, R., Dorward, S., Griesemer, R., & Quinlan, S. (2005) · Scientific Programming · doi:10.1155/2005/962135 · Cited by 616 · see citing works
A Survey on Visual Content-Based Video Indexing and Retrieval
Hu, W., Xie, N., Li, L., Zeng, X., & Maybank, S. J. (2011) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2011.2109710 · Cited by 585 · see citing works
Parallel data processing with MapReduce
Lee, K.-H., Lee, Y.-J., Choi, H., et al. (2012) · ACM SIGMOD Record · doi:10.1145/2094114.2094118 · Cited by 562 · see citing works
Alizadeh, M., Greenberg, A. G., Maltz, D. A., et al. (2010) · ACM SIGCOMM Computer Communication Review · doi:10.1145/1851275.1851192 · Cited by 550 · see citing works
Guo, C., Lu, G., Li, D., Wu, H., Zhang, X., Shi, Y., Tian, C., Zhang, Y., & Lu, S. (2009) · ACM SIGCOMM Computer Communication Review · doi:10.1145/1594977.1592577 · Cited by 535 · see citing works
Exploiting place features in link prediction on location-based social networks
Scellato, S., Noulas, A., & Mascolo, C. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020575 · Cited by 522 · see citing works
van der Aalst, W. M. P. (2012) · ACM Transactions on Management Information Systems · doi:10.1145/2229156.2229157 · Cited by 491 · see citing works
Interactive analytical processing in big data systems
Chen, Y., Alspaugh, S., & Katz, R. H. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2367502.2367519 · Cited by 471 · see citing works
Novelty and redundancy detection in adaptive filtering
Zhang, Y., Callan, J., & Minka, T. P. (2002) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/564376.564393 · Cited by 441 · see citing works
Finding a needle in Haystack: facebook's photo storage
Beaver, D., Kumar, S., Li, H. C., Sobel, J., & Vajgel, P. (2010) · doi:10.5555/1924943.1924947 · Cited by 433 · see citing works
Jain, A. K., Bolle, R. M., & Pankanti, S. (1996) · Springer eBooks · doi:10.1007/b117227 · Cited by 432 · see citing works
The Case for Evaluating MapReduce Performance Using Workload Suites
Chen, Y., Ganapathi, A., Griffith, R., & Katz, R. H. (2011) · Proceedings - International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascots.2011.12 · Cited by 430 · see citing works
The Mobile Data Challenge: Big Data for Mobile Computing Research
Laurila, J. K., Gática-Pérez, D., Aad, I., et al. (2012) · TUbilio (Technical University of Darmstadt) · Cited by 420 · see citing works
A comparison of join algorithms for log processing in MaPreduce
Blanas, S., Patel, J. M., Ercegovac, V., Rao, J., Shekita, E. J., & Tian, Y. (2010) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1807167.1807273 · Cited by 415 · see citing works
Large-scale incremental processing using distributed transactions and notifications
Peng, D. N., & Dabek, F. (2010) · doi:10.5555/1924943.1924961 · Cited by 405 · see citing works
Building a high-level dataflow system on top of Map-Reduce
Gates, A. F., Natkovich, O., Chopra, S., Kamath, P., Narayanamurthy, S., Olston, C., Reed, B. C., Srinivasan, S., & Srivastava, U. (2009) · Proceedings of the VLDB Endowment · doi:10.14778/1687553.1687568 · Cited by 405 · see citing works
Jiang, D., Ooi, B. C., Shi, L., et al. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920903 · Cited by 396 · see citing works
Temporal Link Prediction Using Matrix and Tensor Factorizations
Dunlavy, D., Kolda, T. G., & Acar, E. (2011) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1921632.1921636 · Cited by 375 · see citing works
Health Monitoring of Civil Infrastructures Using Wireless Sensor Networks
Kim, S., Pakzad, S. N., Culler, D., Demmel, J. W., Fenves, G. L., Glaser, S. D., & Turon, M. (2007) · ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2007.4379685 · Cited by 373 · see citing works
Dittrich, J., Quiané-Ruiz, J.-A., Jindal, A., et al. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920908 · Cited by 369 · see citing works
Interactions with big data analytics
Fisher, D., DeLine, R., Czerwinski, M. P., & Drucker, S. M. (2012) · interactions · doi:10.1145/2168931.2168943 · Cited by 363 · see citing works
Web mining in soft computing framework: relevance, state of the art and future directions
Pal, S. K., Talwar, V. G., & Mitra, P. (2002) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2002.1031947 · Cited by 348 · see citing works
Ghazal, A., Rabl, T., Hu, M., Raab, F., Poess, M., Crolotte, A., & Jacobsen, H. (2013) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2463676.2463712 · Cited by 343 · see citing works
On the energy (in)efficiency of Hadoop clusters
Leverich, J., & Kozyrakis, C. (2010) · ACM SIGOPS Operating Systems Review · doi:10.1145/1740390.1740405 · Cited by 343 · see citing works
de Campos, L. M., Fernández-Luna, J. M., Huete, J. F., et al. (2010) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2010.04.001 · Cited by 325 · see citing works
Cho, J. J., & García-Molina, H. (2002) · International World Wide Web Conference (WWW) · doi:10.1145/511446.511464 · Cited by 312 · see citing works
Monitoring heritage buildings with wireless sensor networks: The Torre Aquila deployment
Ceriotti, M., Mottola, L., Picco, G. P., Murphy, A. L., Gunǎ, Ş., Corrà, M., Pozzi, M., Zonta, D., & Zanon, P. (2009) · doi:10.5555/1602165.1602191 · Cited by 304 · see citing works
Discriminant Locally Linear Embedding With High-Order Tensor Data
Li, X., Lin, S. C.-F., Yan, S., & Xu, D. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2007.911536 · Cited by 303 · see citing works
Self-citationBhatotia, P., Wieder, A., Rodrigues, R., Acar, U. A., & Pasquini, R. (2011) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2038916.2038923 · Cited by 298 · see citing works
Chakrabarti, S. (2000) · ACM SIGKDD Explorations Newsletter · doi:10.1145/846183.846187 · Cited by 293 · see citing works
Towards automatic optimization of MapReduce programs
Babu, S. (2010) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/1807128.1807150 · Cited by 284 · see citing works
SensorScope: Out-of-the-Box Environmental Monitoring
Barrenetxea, G., Ingelrest, F., Schaefer, G., Vetterli, M., Couach, O., & Parlange, M. B. (2008) · ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2008.28 · Cited by 273 · see citing works
Networked Wireless Sensor Data Collection: Issues, Challenges, and Approaches
Wang, F., & Liu, J. (2010) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2011.060710.00066 · Cited by 272 · see citing works
Li, X., Pang, Y., & Yuan, Y. (2010) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2009.2035629 · Cited by 272 · see citing works
Self-citationTranslated Learning: Transfer Learning across Different Feature Spaces
Dai, W., Chen, Y., Xue, G.-R., Yang, Q., & Yu, Y. H. (2008) · Cited by 271 · see citing works
Nelson, L. D. (1997) · Critical Care Medicine · doi:10.1097/00003246-199708000-00004 · Cited by 259 · see citing works
W3QS: A Query System for the World-Wide Web
Konopnicki, D., & Shmueli, O. (1995) · Cited by 258 · see citing works
Data integration for the relational web
Cafarella, M., Halevy, A. Y., & Khoussainova, N. (2009) · Proceedings of the VLDB Endowment · doi:10.14778/1687627.1687750 · Cited by 249 · see citing works
Deadline-aware datacenter tcp (D2TCP)
Vamanan, B., Hasan, J., & Vijaykumar, T. N. (2012) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2377677.2377709 · Cited by 235 · see citing works
Selavo, L., Wood, A., Cao, Q., Sookoor, T. I., Liu, H., Srinivasan, A., Wu, Y., Kang, W., Stankovic, J. A., Young, D. R., et al. (2007) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/1322263.1322274 · Cited by 234 · see citing works
Scheduling Hadoop Jobs to Meet Deadlines
Kc, K., & Anyanwu, K. (2010) · IEEE International Conference on Cloud Computing Technology and Science · doi:10.1109/cloudcom.2010.97 · Cited by 234 · see citing works
Solving big data challenges for enterprise application performance management
Rabl, T., Gómez-Villamor, S., Sadoghi, M., et al. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2367502.2367512 · Cited by 234 · see citing works
Community detection in large-scale social networks
Du, N., Wu, B., Pei, X., Wang, B., & Xu, L. (2007) · doi:10.1145/1348549.1348552 · Cited by 232 · see citing works
Lakshman, A., & Malik, P. (2009) · ACM Symposium on Principles of Distributed Computing (PODC) · doi:10.1145/1582716.1582722 · Cited by 231 · see citing works
CIEL: a universal execution engine for distributed data-flow computing
Murray, D. G., Schwarzkopf, M., Smowton, C., Smith, S. M., Madhavapeddy, A., & Hand, S. (2011) · doi:10.5555/1972457.1972470 · Cited by 230 · see citing works
Wang, G., Andersen, D. G., Kaminsky, M., Papagiannaki, K., Ng, T. S. E., Kozuch, M. A., & Ryan, M. P. (2010) · ACM SIGCOMM Computer Communication Review · doi:10.1145/1851275.1851222 · Cited by 230 · see citing works
Singla, A., Singh, A. K., Ramachandran, K., Xu, L., & Zhang, Y. (2010) · ACM Workshop on Hot Topics in Networks (HotNets) · doi:10.1145/1868447.1868455 · Cited by 229 · see citing works
Extending SSD lifetimes with disk-based write caches
Soundararajan, G., Prabhakaran, V., Balakrishnan, M., et al. (2010) · doi:10.5555/1855511.1855519 · Cited by 227 · see citing works
Ghani, N., Dixit, S., & Wang, T. (2000) · IEEE Communications Magazine · doi:10.1109/35.825644 · Cited by 225 · see citing works
Ye, X., Yin, Y., Yoo, S. J. B., Mejia, P., Proietti, R., & Akella, V. (2010) · ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS) · doi:10.1145/1872007.1872037 · Cited by 223 · see citing works
Kohler, E., Handley, M. J., & Floyd, S. (2006) · ACM SIGCOMM Computer Communication Review · doi:10.1145/1151659.1159918 · Cited by 223 · see citing works
Data-Intensive Text Processing with MapReduce
Lin, J., & Dyer, C. (2010) · Synthesis lectures on human language technologies · doi:10.2200/s00274ed1v01y201006hlt007 · Cited by 221 · see citing works
Energy management for MapReduce clusters
Lang, W., & Patel, J. M. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920862 · Cited by 219 · see citing works
Wang, M., Ni, B., Hua, X., & Chua, T. (2012) · ACM Computing Surveys · doi:10.1145/2333112.2333120 · Cited by 219 · see citing works
Self-citationFiber optic communication technologies: What's needed for datacenter network operations
Lam, C. F., Liu, H., Koley, B., Zhao, X., Kamalov, V. F., & Gill, V. (2010) · IEEE Communications Magazine · doi:10.1109/mcom.2010.5496876 · Cited by 216 · see citing works
Consistency rationing in the cloud
Kraska, T., Hentschel, M., Alonso, G., et al. (2009) · Proceedings of the VLDB Endowment · doi:10.14778/1687627.1687657 · Cited by 206 · see citing works
Li, Y., Chen, W., Wang, Y., & Zhang, Z.-L. (2013) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2433396.2433478 · Cited by 202 · see citing works
An exploration of improving collaborative recommender systems via user-item subgroups
Xu, B., Bu, J., Chen, C., et al. (2012) · International World Wide Web Conference (WWW) · doi:10.1145/2187836.2187840 · Cited by 197 · see citing works
Enhancing text clustering by leveraging Wikipedia semantics
Hu, J., Fang, L., Cao, Y. H., Zeng, H.-J., Li, H., Yang, Q., & Chen, Z. (2008) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1390334.1390367 · Cited by 196 · see citing works
Dynamic Proportional Share Scheduling in Hadoop
Sandholm, T. E., & Lai, K. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-16505-4_7 · Cited by 187 · see citing works
Patil, S., Polte, M., Ren, K., et al. (2011) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2038916.2038925 · Cited by 180 · see citing works
Automatic optimization for MapReduce programs
Jahani, E., Cafarella, M., & Ré, C. (2011) · Proceedings of the VLDB Endowment · doi:10.14778/1978665.1978670 · Cited by 177 · see citing works
Data Cleansing: Beyond Integrity Analysis 1
Maletic, J. I., & Marcus, A. (2000) · Cited by 176 · see citing works
Evolution of social-attribute networks
Gong, N. Z., Xu, W., Huang, L., Mittal, P., Stefanov, E., Sekar, V., & Song, D. X. (2012) · ACM Internet Measurement Conference (IMC) · doi:10.1145/2398776.2398792 · Cited by 172 · see citing works
Castillo, C. (2005) · ACM SIGIR Forum · doi:10.1145/1067268.1067287 · Cited by 170 · see citing works
New Directions In Question Answering
Maybury, M. T. (2006) · Text, speech and language technology · doi:10.1007/978-1-4020-4746-6_18 · Cited by 161 · see citing works
Nambiar, R. O., & Poess, M. (2010) · Cited by 157 · see citing works
Kim, Y., Schmid, T., Charbiwala, Z. M., Friedman, J., & Srivastava, M. B. (2008) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/1460412.1460443 · Cited by 156 · see citing works
Cost effective storage using extent based dynamic tiering
Guerra, J., Pucha, H., Glider, J., et al. (2011) · doi:10.5555/1960475.1960495 · Cited by 154 · see citing works
A platform for scalable one-pass analytics using MapReduce
Li, B., Mazur, E., Diao, Y., et al. (2011) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/1989323.1989426 · Cited by 151 · see citing works
OFDM for Flexible High-Speed Optical Networks
Shieh, W. (2011) · Journal of Lightwave Technology · doi:10.1109/jlt.2011.2132115 · Cited by 149 · see citing works
Taming the Big Data Tidal Wave
(2012) · Wiley eBooks · doi:10.1002/9781119204275 · Cited by 143 · see citing works
Lessons and Challenges from Mining Retail E-Commerce Data
Kohavi, R., Mason, L., Parekh, R. G., & Zheng, Z. (2004) · Machine Learning · doi:10.1023/b:mach.0000035473.11134.83 · Cited by 136 · see citing works
Borkar, V., Carey, M. J., & Li, C. (2012) · International Conference on Extending Database Technology (EDBT) · doi:10.1145/2247596.2247598 · Cited by 132 · see citing works
Individual and group behavior-based customer profile model for personalized product recommendation
Park, Y., & Chang, K. (2007) · Expert Systems with Applications · doi:10.1016/j.eswa.2007.12.034 · Cited by 124 · see citing works
Olston, C., Chiou, G., Chitnis, L., et al. (2011) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/1989323.1989439 · Cited by 118 · see citing works
Co-evolution of social and affiliation networks
Zheleva, E., Sharara, H., & Getoor, L. C. (2009) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1557019.1557128 · Cited by 116 · see citing works
Multimedia data mining: state of the art and challenges
Bhatt, C., & Kankanhalli, M. (2010) · Multimedia Tools and Applications · doi:10.1007/s11042-010-0645-5 · Cited by 115 · see citing works
Linear prediction models with graph regularization for web-page categorization
Zhang, T., Popescul, A., & Dom, B. (2006) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1150402.1150510 · Cited by 111 · see citing works
Survey on Improved Scheduling in Hadoop MapReduce in Cloud EnvironmentsPreprint
Rao, B. T., & Reddy, L. S. S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1207.0780 · Cited by 109 · see citing works
All-pairs: An abstraction for data-intensive cloud computing
Moretti, C. M., Bulosan, J., Thain, D., & Flynn, P. J. (2008) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps.2008.4536311 · Cited by 104 · see citing works
Dynamic optical mesh networks: Drivers, challenges and solutions for the future
Jinno, M., TAKARA, H., & Kozicki, B. (2009) · European Conference on Optical Communication · Cited by 97 · see citing works
Salton, G. (1970) · Science · doi:10.1126/science.168.3929.335 · Cited by 97 · see citing works
Building LinkedIn's Real-time Activity Data Pipeline.
Goodhope, K., Koshy, J., Kreps, J., Narkhede, N., Park, R., Rao, J., & Ye, V. Y. (2012) · Cited by 96 · see citing works
Jiang, Y., Zeng, X., Ye, G., Bhattacharya, S., Ellis, D. P. W., Shah, M. A., & Chang, S. (2010) · Journal of International Crisis and Risk Communication Research · Cited by 92 · see citing works
Symbiotic routing in future data centers
Abu-Libdeh, H., Costa, P., Rowstron, A. I. T., O’Shea, G., & Donnelly, A. (2010) · ACM SIGCOMM Computer Communication Review · doi:10.1145/1851275.1851191 · Cited by 90 · see citing works
Leveraging spatio-temporal redundancy for RFID data cleansing
Chen, H., Ku, W., Wang, H., & Sun, M.-T. (2010) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1807167.1807176 · Cited by 87 · see citing works
Rethinking cost and performance of database systems
Florescu, D., & Kossmann, D. (2009) · ACM SIGMOD Record · doi:10.1145/1558334.1558339 · Cited by 87 · see citing works
Big data platforms: What's next?
Borkar, V., Carey, M. J., & Li, C. (2012) · XRDS Crossroads The ACM Magazine for Students · doi:10.1145/2331042.2331057 · Cited by 86 · see citing works
Evolution of a location-based online social network
Allamanis, M., Scellato, S., & Cecilia Mascolo, C. (2012) · ACM Internet Measurement Conference (IMC) · doi:10.1145/2398776.2398793 · Cited by 79 · see citing works
Knowledge adaptation for ad hoc multimedia event detection with few exemplars
Ma, Z., Yang, Y., Cai, Y., Sebe, N., & Hauptmann, A. G. (2012) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2393347.2393414 · Cited by 78 · see citing works
The History of Storage Systems
Goda, K., & Kitsuregawa, M. (2012) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2189787 · Cited by 78 · see citing works
Deterministic Column-Based Matrix Decomposition
Li, X., & Pang, Y. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.64 · Cited by 76 · see citing works
Self-citationEnergy-Efficient Design of a Scalable Optical Multiplane Interconnection Architecture
Liboiron-Ladouceur, O., Cerutti, I., Raponi, P. G., Andriolli, N., & Castoldi, P. (2010) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2010.2049733 · Cited by 72 · see citing works
GFS: Evolution on Fast-forward
McKusick, M. K., & Quinlan, S. (2009) · Queue · doi:10.1145/1594204.1594206 · Cited by 72 · see citing works
Meijer, E. (2011) · Communications of the ACM · doi:10.1145/2001269.2001285 · Cited by 71 · see citing works
Ad-hoc data processing in the cloud
Logothetis, D., & Yocum, K. (2008) · Proceedings of the VLDB Endowment · doi:10.14778/1454159.1454204 · Cited by 70 · see citing works
Data Visualization and Infographics
Dunleavy, D. (2015) · Visual Communication Quarterly · doi:10.1080/15551393.2015.1029070 · Cited by 69 · see citing works
Cloud computing paradigms for pleasingly parallel biomedical applications
Gunarathne, T., Wu, T., Qiu, J., et al. (2010) · doi:10.1145/1851476.1851544 · Cited by 68 · see citing works
Abouzied, A., Bajda-Pawlikowski, K., Huang, J., et al. (2010) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/1807167.1807294 · Cited by 63 · see citing works
Manchester, J. S., Anderson, J. M., Doshi, B. T., & Dravida, S. (1998) · IEEE Communications Magazine · doi:10.1109/35.668282 · Cited by 61 · see citing works
Data pre-processing on web server logs for generalized association rules mining algorithm
Wahab, M. H. A., Mohd, M. N. H., Hanafi, H. F., & Mohsin, M. F. M. (2008) · Universiti Utara Malaysia Institutional Repository (Universiti Utara Malaysia) · Cited by 61 · see citing works
Beyond audio and video retrieval
Ding, D., Metze, F., Rawat, S., Schulam, P., Burger, S., Younessian, E., Bao, L., Christel, M. G., & Hauptmann, A. G. (2012) · International Conference on Multimedia Retrieval · doi:10.1145/2324796.2324799 · Cited by 55 · see citing works
Data Mining on DNA Sequences of Hepatitis B Virus
Leung, K., Lee, K.-H., Wang, J., Ng, E. Y. T., Chan, H. L., Tsui, S. K., Mok, T. S. K., Tse, P. C., & Sung, J. J. �. (2009) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2009.6 · Cited by 55 · see citing works
Probabilistic Event Extraction from RFID Data
Khoussainova, N., Bałazińska, M., & Suciu, D. M. (2008) · Proceedings - International Conference on Data Engineering · doi:10.1109/icde.2008.4497596 · Cited by 54 · see citing works
Streaming data integration: Challenges and opportunities
Tatbul, N. (2010) · IEEE International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew.2010.5452751 · Cited by 54 · see citing works
Karanth, A., & Louri, A. (2010) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2010.2051419 · Cited by 51 · see citing works
Evolution of an online social aggregation network
Garg, S., Gupta, T., Carlsson, N. F., & Mahanti, A. (2009) · ACM Internet Measurement Conference (IMC) · doi:10.1145/1644893.1644931 · Cited by 48 · see citing works
Shute, J., Oancea, M., Ellner, S., et al. (2012) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/2213836.2213954 · Cited by 47 · see citing works
On-line anomaly detection of deployed software
Baah, G. K., Gray, A., & Harrold, M. J. (2006) · doi:10.1145/1188895.1188911 · Cited by 46 · see citing works
Data-Intensive Scalable Computing for Scientific Applications
Bryant, R. E. (2011) · Computing in Science & Engineering · doi:10.1109/mcse.2011.73 · Cited by 45 · see citing works
Gossip-Based Information Spreading in Mobile Networks
Zhang, H., Zhang, Z., & Dai, H. (2013) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2013.100113.130619 · Cited by 42 · see citing works
On Using a Warehouse to Analyze Web Logs
Joshi, K. P., Joshi, A., & Yesha, Y. (2003) · Distributed and Parallel Databases · doi:10.1023/a:1021515408295 · Cited by 42 · see citing works
Kamath, U., Compton, J., Islamaj, R., Jong, K. A. D., & Shehu, A. (2012) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2012.53 · Cited by 38 · see citing works
Video surveillance using JPEG 2000
Dufaux, F., & Ebrahimi, T. (2004) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.564828 · Cited by 37 · see citing works
IncMR: Incremental Data Processing Based on MapReduce
Yan, C., Yang, X. S., Yu, Z., et al. (2012) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud.2012.67 · Cited by 37 · see citing works
A first look at wired sensor networks for video surveillance systems
Chandramohan, V., & Christensen, K. (2003) · Conference on Local Computer Networks · doi:10.1109/lcn.2002.1181851 · Cited by 36 · see citing works
Mining Massive RFID, Trajectory, and Traffic Data Sets
Han, J., Lee, J.-G., González, H., & Li, X. (2008) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1401890.1551566 · Cited by 35 · see citing works
Self-citation(2003) · Springer eBooks · doi:10.1007/978-1-4615-1177-9 · Cited by 35 · see citing works
Crawling rich internet applications: the state of the art
Choudhary, S., Dinçtürk, M. E., Mirtaheri, S. M., Moosavi, A., von Bochmann, G., Jourdan, G.-V., & Onut, I.-V. (2015) · Cited by 33 · see citing works
A Survey of Transport Protocols other than Standard TCP
Goutelle, M., He, E., Hegde, S., et al. (2005) · Cited by 33 · see citing works
Applying statistical machine learning to multicore voltage & frequency scaling
Moeng, M., & Melhem, R. (2010) · ACM International Conference on Computing Frontiers (CF) · doi:10.1145/1787275.1787336 · Cited by 33 · see citing works
A model-based approach for RFID data stream cleansing
Zhao, Z. L., & Ng, W. (2012) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2396761.2396871 · Cited by 31 · see citing works
Multimodal analysis of body sensor network data streams for real-time healthcare
Garg, M. K., Kim, D., Turaga, D. S., & Prabhakaran, B. (2010) · Multimedia Information Retrieval · doi:10.1145/1743384.1743467 · Cited by 31 · see citing works
Automatic text summarization and small-world networks
Balinsky, H. Y., Balinsky, A. A., & Simske, S. J. (2011) · ACM Symposium on Document Engineering (DocEng) · doi:10.1145/2034691.2034731 · Cited by 28 · see citing works
Biological data cleaning: a case study
Herbert, K. G., & Wang, J. T. L. (2007) · International Journal of Information Quality · doi:10.1504/ijiq.2007.013376 · Cited by 27 · see citing works
Li, Y., Hu, X., Lin, H., et al. (2010) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2010.99 · Cited by 27 · see citing works
Tsai, T., & Lin, C.-Y. (2011) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2011.2180705 · Cited by 26 · see citing works
Flagged first, then by citations
Research works printed but not linked: 41
- J. Gantz and D. Reinsel, ''Extracting value from chaos,'' in Proc. IDC iView, 2011, pp. 1-12.
- O. R. Team, Big Data Now: Current Perspectives from O'Reilly Radar. Sebastopol, CA, USA: O'Reilly Media, 2011.
- S. Marche, ''Is Facebook making us lonely,'' Atlantic, vol. 309, no. 4, pp. 60-69, 2012.
- K. Goodhope et al., ''Building linkedins real-time activity data pipeline,'' Data Eng., vol. 35, no. 2, pp. 33-45, 2012.
- M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, ''Dryad: Distributed data-parallel programs from sequential building blocks,'' in Proc. 2nd ACM SIGOPS/EuroSys Eur. Conf. Comput. Syst., Jun. 2007, pp. 59-72.
- G. Malewicz et al., ''Pregel: A system for large-scale graph processing,'' in Proc. ACM SIGMOD Int. Conf. Manag. Data, Jun. 2010, pp. 135-146.
- S. Melnik et al., ''Dremel: Interactive analysis of web-scale datasets,'' Proc. VLDB Endowment, vol. 3, nos. 1-2, pp. 330-339, 2010.
- J. Gantz and D. Reinsel, ''The digital universe decade-are you ready,'' in Proc. White Paper, IDC, 2010.
- L. Selavo et al., ''Luster: Wireless sensor network for environmental research,'' in Proc. 5th Int. Conf. Embedded Netw. Sensor Syst., Nov. 2007, pp. 103-116.
- Y. Kim, T. Schmid, Z. M. Charbiwala, J. Friedman, and M. B. Srivastava, ''Nawms: Nonintrusive autonomous water monitoring system,'' in Proc. 6th ACM Conf. Embedded Netw. Sensor Syst., Nov. 2008, pp. 309-322.
- A. Greenberg et al., ''VL2: A scalable and flexible data center network,'' in Proc. ACM SIGCOMM Conf. Data Commun., 2009, pp. 51-62.
- C. Guo et al., ''BCube: A high performance, server-centric network archi- tecture for modular data centers,'' SIGCOMM Comput. Commun. Rev., vol. 39, no. 4, pp. 63-74, 2009.
- N. Farrington et al., ''Helios: A hybrid electrical/optical switch architec- ture for modular data centers,'' in Proc. ACM SIGCOMM Conf., 2010, pp. 339-350.
- G. Wang et al., ''c-through: Part-time optics in data centers,'' SIGCOMM Comput. Commun. Rev., vol. 41, no. 4, pp. 327-338, 2010.
- X. Ye, Y. Yin, S. B. Yoo, P. Mejia, R. Proietti, and V. Akella, ''DOS- A scalable optical switch for datacenters,'' in Proc. 6th ACM/IEEE Symp. Archit. Netw. Commun. Syst., Oct. 2010, pp. 1-12.
- A. Singla, A. Singh, K. Ramachandran, L. Xu, and Y. Zhang, ''Proteus: A topology malleable data center network,'' in Proc. 9th ACM SIGCOMM Workshop Hot Topics Netw., 2010, pp. 801-806.
- A. Silberschatz, H. F. Korth, and S. Sudarshan, Database System Con- cepts, vol. 4. New York, NY, USA: McGraw-Hill, 1997.
- D. Salomon, Data Compression. New York, NY, USA: Springer-Verlag, 2004.
- J. Bleiholder and F. Naumann, ''Data fusion,'' ACM Comput. Surv., vol. 41, no. 1, pp. 1-41, 2009.
- R. Chaiken et al., ''Scope: Easy and efficient parallel process- ing of massive data sets,'' Proc. VLDB Endowment, vol. 1, no. 2, pp. 1265-1276, 2008.
- F. Chang et al., ''Bigtable: A distributed storage system for structured data,'' ACM Trans. Comput. Syst., vol. 26, no. 2, pp. 4:1-4:26, Jun. 2008.
- A. Lakshman and P. Malik, ''Cassandra: Structured storage system on a p2p network,'' in Proc. 28th ACM Symp. Principles Distrib. Comput., 2009, p. 5. [149(2013). HBase [Online]. Available: http://hbase.apache.org/ [150] (2013). Hypertable [Online]. Available: http://hypertable.org/
- B. F. Cooper et al., ''PNUTS: Yahoo!'s hosted data serving platform,'' in Proc. VLDB Endowment, vol. 1, no. 2, pp. 1277-1288, 2008.
- J. Shute et al., ''F1: The fault-tolerant distributed RDBMS support- ing Google's ad business,'' in Proc. Int. Conf. Manag. Data, 2012, pp. 777-778.
- K. Keeton, C. B. Morrey, III, C. A. Soules, and A. Veitch, ''Lazybase: Freshness vs. performance in information management,'' SIGOPS Oper. Syst. Rev., vol. 44, no. 1, pp. 15-19, Jan. 2010.
- D. W. Walker and J. J. Dongarra, ''MPI: A standard message passing interface,'' Supercomputer, vol. 12, pp. 56-68, 1996.
- A. Thusoo et al., ''Hive: A warehousing solution over a Map-Reduce framework,'' Proc. VLDB Endowment, vol. 2, no. 2, pp. 1626-1629, 2009.
- Y. Bu, B. Howe, M. Balazinska, and M. D. Ernst, ''HaLoop: Efficient iterative data processing on large clusters,'' Proc. VLDB Endowment, vol. 3, nos. 1-2, pp. 285-296, 2010.
- J. Ekanayake et al., ''Twister: A runtime for iterative mapreduce,'' in Proc. 19th ACM Int. Symp. High Perform. Distrib. Comput., 2010, pp. 810-818.
- P. Bhatotia, A. Wieder, R. Rodrigues, U. A. Acar, and R. Pasquin, ''Incoop: Mapreduce for incremental computations,'' in Proc. 2nd ACM Symp. Cloud Comput., 2011, pp. 1-14.
- C. Olston et al., ''Nova: Continuous Pig/hadoop workflows,'' in Proc. Int. Conf. Manag. Data, 2011, pp. 1081-1090.
- G. Salton, ''Automatic text processing,'' Science, vol. 168, no. 3929, pp. 335-343, 1970.
- S. Shridhar, M. Lakhanpuria, A. Charak, A. Gupta, and S. Shridhar, ''Snair: A framework for personalised recommendations based on social network analysis,'' in Proc. 5th Int. Workshop Location-Based Soc. Netw., 2012, pp. 55-61.
- S. Maniu and B. Cautis, ''Taagle: Efficient, personalized search in collaborative tagging networks,'' in Proc. Assoc. Comput. Mach. (ACM) SIGMOD Int. Conf. Manag. Data, 2012, pp. 661-664.
- H. Hu, Y. Wen, H. Luan, T.-S. Chua, and X. Li, ''Towards multi-screen social tv with geo-aware social sense,'' IEEE Multimedia, to be published.
- Y. Bu, B. Howe, M. Balazinska, and M. D. Ernst, ''Haloop: Efficient iterative data processing on large clusters,'' Proc. VLDB Endowment, vol. 3, nos. 1-2, pp. 285-296, 2010.
- J. Ekanayake et al., ''Twister: A runtime for iterative mapreduce,'' in Proc. 19th Assoc. Comput. Mach. (ACM) Int. Symp. High Perform. Distrib. Comput., 2010, pp. 810-818.
- J. Dittrich, J.-A. Quiané-Ruiz, A. Jindal, Y. Kargin, V. Setty, and J. Schad, ''Hadoop++: Making a yellow elephant run like a cheetah (without it even noticing),'' Proc. VLDB Endowment, vol. 3, nos. 1-2, pp. 515-529, 2010.
- M. Yong, N. Garegrat, and S. Mohan, ''Towards a resource aware scheduler in Hadoop,'' in Proc. Int. Conf. Web Services (ICWS), 2009, pp. 102-109.
- S. Patil et al., ''Ycsb++: Benchmarking and performance debugging advanced features in scalable table stores,'' in Proc. 2nd Assoc. Comput. Mach. (ACM) Symp. Cloud Comput., 2011, p. 9.
- A. Ghazal et al., ''Bigbench: Towards an industry standard benchmark for big data analytics,'' Proc. Assoc. Comput. Mach. (ACM) SIGMOD Int. Conf. Manag. Data, 2013, pp. 1197-1208.
Cited by
Sorted by citations
Big Data Analytics in Operations Management
Choi, T., Wallace, S. W., & Wang, Y. (2018) · Production and Operations Management · doi:10.1111/poms.12838 · Cited by 1,027 · see citing works
Tsai, C.-W., Lai, C.-F., Chao, H.-C., & Vasilakos, A. V. (2015) · Journal of Big Data · doi:10.1186/s40537-015-0030-3 · Cited by 818 · see citing works
An integrated big data analytics-enabled transformation model: application to health care
Wang, Y., Kung, L., Wang, W. Y. C., & Cegielski, C. G. (2018) · Information & Management · doi:10.1016/j.im.2017.04.001 · Cited by 508 · see citing works
A survey towards an integration of big data analytics to big insights for value-creation
Saggi, M. K., & Jain, S. (2018) · Information Processing & Management · doi:10.1016/j.ipm.2018.01.010 · Cited by 494 · see citing works
Tourism analytics with massive user-generated content: a case study of Barcelona
Marine-Roig, E., & Anton Clavé, S. (2015) · Journal of Destination Marketing & Management · doi:10.1016/j.jdmm.2015.06.004 · Cited by 356 · see citing works
Big data caching for networking: moving from cloud to edge
Zeydan, E., Bastug, E., Bennis, M., Kader, M. A., Karatepe, I. A., Er, A. S., & Debbah, M. (2016) · IEEE Communications Magazine · doi:10.1109/mcom.2016.7565185 · Cited by 310 · see citing works
Live Data Analytics With Collaborative Edge and Cloud Processing in Wireless IoT Networks
Sharma, S. K., & Wang, X. (2017) · IEEE Access · doi:10.1109/access.2017.2682640 · Cited by 241 · see citing works
Big data in operations and supply chain management: Current trends and future perspectives
Lamba, K., & Singh, S. P. (2017) · Production Planning & Control · doi:10.1080/09537287.2017.1336787 · Cited by 193 · see citing works
Putting the data before the algorithm in big data addressing personalized healthcare
Cahan, E. M., Hernandez-Boussard, T., Thadaney-Israni, S., & Rubin, D. L. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0157-2 · Cited by 149 · see citing works
Big Data Quality Framework: A Holistic Approach to Continuous Quality Management
Taleb, I., Serhani, M. A., Bouhaddioui, C., & Dssouli, R. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00468-0 · Cited by 149 · see citing works
In Search of Big Medical Data Integration Solutions - A Comprehensive Survey
Dhayne, H., Haque, R., Kilany, R., & Taher, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2927491 · Cited by 84 · see citing works
Data-driven event triggering for IoT applications
Kolios, P., Panayiotou, C., Ellinas, G., & Polycarpou, M. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2564428 · Cited by 60 · see citing works
Moretto, A., Ronchi, S., & Patrucco, A. S. (2017) · International Journal of RF Technologies: Research and Applications · doi:10.3233/rft-171670 · Cited by 57 · see citing works
Narayanan, S., S, M., & Zephan, P. (2024) · ICST Transactions on Scalable Information Systems · doi:10.4108/eetsis.5065 · Cited by 5 · see citing works
“Traditional batch-based monitoring approaches often fall short in addressing the dynamic nature of data pipelines and the need for real-time insights [3].”
- Authors
- Hu, H., Wen, Y., Chua, T.-S., & Li, X.
- Affiliations
- National University of Singapore (SG)
- Nanyang Technological University (SG)
- Chinese Academy of Sciences (CN)
- Venue
- IEEE Access
- Published
- 2014
- Volume
- 2
- Pages
- 652-687
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2157954477
- Cited by
- 1,099
- References
- 221
- Citations by year
- 2026: 17 · 2025: 48 · 2024: 47 · 2023: 50 · 2022: 69 · 2021: 100 · 2020: 133 · 2019: 123 · 2018: 143 · 2017: 130 · 2016: 132 · 2015: 97 · 2014: 9 · 2012: 1
- FWCI
- 229.10
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Institutional Repository of Xi'an Institute of Optics and Fine Mechanics, Chinese Academy of Sciences (Xian Institute of Optics and Precision Mechanics) (submittedVersion, cc-by-nc-sa)
- Funders
- Energy Market Authority of Singapore (NRF2012EWT-EIRP002-013)
- Singapore National Research Foundation through the International Research Center at Singapore Funding Initiative and administered by the IDM Programme Office
- National Natural Science Foundation of China (61125106)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Database Systems and Queries
- Keywords
- big data analytics, big data systems, data acquisition, data storage, Hadoop, scalable data processing, unstructured data, real-time analytics, big data challenges
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
