The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index.
Utilizing big data analytics for information systems research: challenges, promises and guidelines
Open accessAbstract
This essay discusses the use of big data analytics (BDA) as a strategy of enquiry for advancing information systems (IS) research. In broad terms, we understand BDA as the statistical modelling of large, diverse, and dynamic data sets of user-generated content and digital traces. BDA, as a new paradigm for utilising big data sources and advanced analytics, has already found its way into some social science disciplines. Sociology and economics are two examples that have successfully harnessed BDA for scientific enquiry. Often, BDA draws on methodologies and tools that are unfamiliar for some IS researchers (e.g., predictive modelling, natural language processing). Following the phases of a typical research process, this article is set out to dissect BDA’s challenges and promises for IS research, and illustrates them by means of an exemplary study about predicting the helpfulness of 1.3 million online customer reviews. In order to assist IS researchers in planning, executing, and interpreting their own studies, and evaluating the studies of others, we propose an initial set of guidelines for conducting rigorous BDA studies in IS.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Qualitative Data Analysis: An Expanded Sourcebook
(1994) · Journal of Environmental Psychology · doi:10.1016/s0272-4944(05)80231-2 · Cited by 48,420
The Discovery of Grounded Theory: Strategies for Qualitative Research.
Wagner, H. R., Glaser, B. G., & Strauss, A. L. (1968) · Social Forces · doi:10.2307/2575405 · Cited by 46,361
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
Kuhn, M., & Johnson, K. (2013) · doi:10.1007/978-1-4614-6849-3 · Cited by 6,788 · cited 10×
“Such explanations can take many forms, including comparing and contrasting predictions with existing theories and related explanatory empirical studies (Shmueli & Koppius, 2011), ‘ replaying ’ predictions for extreme or typical cases (Robnik-Sikonja & Kononenko, 2008), performing a sensitivity analysis for selected mo…”
BackgroundMethodInfluentialBusiness intelligence and analytics: from big data to big impact
Chen, H., Chiang, R. H. L., & Storey, V. C. (2012) · MIS Quarterly · doi:10.2307/41703503 · Cited by 6,250
Information and Consumer Behavior
Nelson, P. (1970) · Journal of Political Economy · doi:10.1086/259630 · Cited by 5,758
Top 10 algorithms in data mining
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., et al. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0114-2 · Cited by 5,667
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Twitter mood predicts the stock market
Bollen, J., Mao, H., & Zeng, X. (2011) · Journal of Computational Science · doi:10.1016/j.jocs.2010.12.007 · Cited by 5,206
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340
Detecting influenza epidemics using search engine query data
Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009) · Nature · doi:10.1038/nature07634 · Cited by 4,295
Experimental evidence of massive-scale emotional contagion through social networks
Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1320040111 · Cited by 3,687
Silverman, D. (2024) · SAGE Publications Ltd eBooks · doi:10.4135/9781036233358 · Cited by 3,553
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365
The Nature of Theory in Information Systems1
Gregor, S. (2006) · MIS Quarterly · doi:10.2307/25148742 · Cited by 3,306 · cited 1×
“The literature on research methods and statistics traditionally distinguishes two broad types of approaches to modelling (see, e.g., Breiman, 2001b; Gregor, 2006).”
MethodQuantitative Analysis of Culture Using Millions of Digitized Books
Michel, J.-B., Shen, Y. K., Aiden, A. P., Veres, A., Gray, M. K., The Google Books Team, Pickett, J. P., Hoiberg, D., Clancy, D., Norvig, P., et al. (2011) · Science · doi:10.1126/science.1199644 · Cited by 3,077
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,866
The Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014) · Science · doi:10.1126/science.1248506 · Cited by 2,523
Shmueli, G. (2010) · Statistical Science · doi:10.1214/10-sts330 · Cited by 2,517
What Makes a Helpful Online Review? A Study of Customer Reviews on Amazon.Com1
Mudambi, S. M., & Schuff, D. (2010) · MIS Quarterly · doi:10.2307/20721420 · Cited by 2,490
Venkatesh, V., Brown, S. A., & Bala, H. (2013) · MIS Quarterly · doi:10.25300/misq/2013/37.1.02 · Cited by 2,363
Image-Based Recommendations on Styles and Substitutes
McAuley, J., Targett, C., Shi, Q., & van den Hengel, A. (2015) · Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/2766462.2767755 · Cited by 2,280
(2009) · Cited by 1,900
The Unreasonable Effectiveness of Data
Halevy, A., Norvig, P., & Pereira, F. (2009) · IEEE Intelligent Systems · doi:10.1109/mis.2009.36 · Cited by 1,767
The World’s Technological Capacity to Store, Communicate, and Compute Information
Hilbert, M., & López, P. (2011) · Science · doi:10.1126/science.1200970 · Cited by 1,643
Emotions and Information Diffusion in Social Media—Sentiment of Microblogs and Sharing Behavior
Stieglitz, S., & Dang-Xuan, L. (2013) · Journal of Management Information Systems · doi:10.2753/mis0742-1222290408 · Cited by 1,604
Communications of the Association for Information Systems
(2017) · Communications of the Association for Information Systems · doi:10.17705/1cais · Cited by 1,603
Fletcher, E. (2015) · International Journal of Research & Method in Education · doi:10.1080/1743727x.2015.1066173 · Cited by 1,599
The Fourth Paradigm – Data-Intensive Scientific Discovery
Hey, T. (2012) · Communications in Computer and Information Science · doi:10.1007/978-3-642-33299-9_1 · Cited by 1,585
The Data Revolution: Big Data, Open Data, Data Infrastructures & Their Consequences
Kitchin, R. (2014) · doi:10.4135/9781473909472 · Cited by 1,469 · cited 3×
“Studies agree (e.g., Hilbert & López, 2011; IDC, 2011, 2014) that the volume of data being generated and stored today is growing exponentially (Kitchin, 2014) – but, big data is not just about volume (Constantiou & Kallinikos, 2015; Yoo, 2015).”
BackgroundReproducible research in computational science
Peng, R. D. (2011) · Science · doi:10.1126/science.1213847 · Cited by 1,449 · cited 2×
“The next step would be to publicly provide ‘ linked and executable code and data ’ (Peng, 2011, p. 5), for example, in the form of an interactive computational environment (e.g., IPython Notebook) that allows readers to inspect and explore the original data and reproduce all data analysis steps.”
BackgroundThe Fourth Paradigm: Data-Intensive Scientific Discovery
Buchan, I. E., Winn, J. M., & Bishop, C. (2009) · Cited by 1,439
Ghose, A., & Ipeirotis, P. G. (2011) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2010.188 · Cited by 1,400
Predictive Analytics in Information Systems Research1
Shmueli, G., & Koppius, O. R. (2011) · MIS Quarterly · doi:10.2307/23042796 · Cited by 1,328
Statistical modeling: The two cultures
Breiman, L. (2001) · Cited by 1,314
What makes a helpful online review? a study of customer reviews on amazon.com
Mudambi, S. M., & Schuff, D. (2010) · MIS Quarterly · doi:10.5555/2017447.2017457 · Cited by 1,281
Goh, K.-Y., Heng, C.-S., & Lin, Z. (2013) · Information Systems Research · doi:10.1287/isre.1120.0469 · Cited by 1,228
Research Commentary —Too Big to Fail: Large Samples and the p -Value Problem
Lin, M., Lucas, H. C., & Shmueli, G. (2013) · Information Systems Research · doi:10.1287/isre.2013.0480 · Cited by 972
“But the data is already public”: on the ethics of research in Facebook
Zimmer, M. (2010) · Ethics and Information Technology · doi:10.1007/s10676-010-9227-5 · Cited by 900
Dhar, V. (2013) · Communications of the ACM · doi:10.1145/2500499 · Cited by 856
Goh, K. Y., Heng, C. S., & Lin, Z. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2048614 · Cited by 799
Systematic Analysis of Breast Cancer Morphology Uncovers Stromal Features Associated with Survival
Beck, A. H., Sangoi, A. R., Leung, S., Marinelli, R. J., Nielsen, T. O., van de Vijver, M. J., West, R. B., van de Rijn, M., & Koller, D. (2011) · Science Translational Medicine · doi:10.1126/scitranslmed.3002564 · Cited by 763
Algorithmic Accountability: Journalistic investigation of computational power structures
Diakopoulos, N. (2015) · Digital Journalism · doi:10.1080/21670811.2014.976411 · Cited by 733
Overcoming the J-shaped distribution of product reviews
Hu, N., Zhang, J., & Pavlou, P. A. (2009) · Communications of the ACM · doi:10.1145/1562764.1562800 · Cited by 686
Predictive Analytics in Information Systems Research
Shmueli, G., & Koppius, O. (2010) · doi:10.2139/ssrn.1606674 · Cited by 676
Cao, Q., Duan, W., & Gan, Q. (2011) · Decision Support Systems · doi:10.1016/j.dss.2010.11.009 · Cited by 640
Born Unequal: A Study of the Helpfulness of User-Generated Product Reviews
Pan, Y., & Zhang, J. Q. (2011) · Journal of Retailing · doi:10.1016/j.jretai.2011.05.002 · Cited by 610
Editorial—Big Data, Data Science, and Analytics: The Opportunity and Challenge for IS Research
Agarwal, R., & Dhar, V. (2014) · Information Systems Research · doi:10.1287/isre.2014.0546 · Cited by 610
(2014) · CERN Document Server (European Organization for Nuclear Research) · Cited by 608
Korfiatis, N., García-Bariocanal, E., & Sánchez-Alonso, S. (2012) · Electronic Commerce Research and Applications · doi:10.1016/j.elerap.2011.10.003 · Cited by 548
Bell, G., Hey, T., & Szalay, A. (2009) · Science · doi:10.1126/science.1170411 · Cited by 537
Explanations From Intelligent Systems: Theoretical Foundations and Implications for Practice1
Gregor, S., & Benbasat, I. (1999) · MIS Quarterly · doi:10.2307/249487 · Cited by 535
New games, new rules: big data and the changing context of strategy
Constantiou, I. D., & Kallinikos, J. (2015) · Journal of Information Technology · doi:10.1057/jit.2014.17 · Cited by 526
Sharma, R., Mithas, S., & Kankanhalli, A. (2014) · European Journal of Information Systems · doi:10.1057/ejis.2014.17 · Cited by 516
Economics in the age of big data
Einav, L., & Levin, J. (2014) · Science · doi:10.1126/science.1243089 · Cited by 498
Comprehensible credit scoring models using rule extraction from support vector machines
Martens, D., Baesens, B., Van Gestel, T., & Vanthienen, J. (2007) · European Journal of Operational Research · doi:10.1016/j.ejor.2006.04.051 · Cited by 448
Using sensitivity analysis and visualization techniques to open black box data mining models
Cortez, P., & Embrechts, M. J. (2013) · Information Sciences · doi:10.1016/j.ins.2012.10.039 · Cited by 443
‘Datafication’: making sense of (big) data in a complex world
Lycett, M. (2013) · European Journal of Information Systems · doi:10.1057/ejis.2013.10 · Cited by 421
What Makes a Helpful Review? A Study of Customer Reviews on Amazon.comPreprint
Mudambi, S. M., & Schuff, D. (2010) · SSRN Electronic Journal · Cited by 333
Recommendation Networks and the Long Tail of Electronic Commerce1
Oestreicher-Singer, G., & Sundararajan, A. (2012) · MIS Quarterly · doi:10.2307/41410406 · Cited by 278
Explaining Data-Driven Document Classifications1
Martens, D., & Provost, F. (2014) · MIS Quarterly · doi:10.25300/misq/2014/38.1.04 · Cited by 264
Explaining Classifications For Individual Instances
Robnik-Sikonja, M., & Kononenko, I. (2008) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.190734 · Cited by 254
'Natural' and 'contrived' data: a sustainable distinction?
Speer, S. (2002) · Discourse Studies · doi:10.1177/1461445602004004027 · Cited by 232
Algorithmic Accountability Reporting: On the Investigation of Black Boxes
Diakopoulos, N. A. (2014) · Columbia Academic Commons (Columbia University) · doi:10.7916/d8zk5tw2 · Cited by 221
Handbook of Mixed Membership Models and Their Applications
(2014) · Chapman & Hall/CRC eBooks · doi:10.1201/b17520 · Cited by 187
Social Ties and User Content Generation: Evidence from Flickr
Zeng, X., & Wei, L. (2013) · Information Systems Research · doi:10.1287/isre.1120.0464 · Cited by 177
Yu, C. H., Jannasch-Pennell, A. K., & DiGangi, S. A. (2014) · The Qualitative Report · doi:10.46743/2160-3715/2011.1085 · Cited by 157
The logistic transform for bounded outcome scores
Lesaffre, E., Rizopoulos, D., & Tsonaka, R. (2006) · Biostatistics · doi:10.1093/biostatistics/kxj034 · Cited by 157
Problem Solving: A Statistician's Guide.
Kafandaris, S. P., & Chatfield, C. (1996) · Journal of the Operational Research Society · doi:10.2307/3010293 · Cited by 141
Murray‐Rust, P. (2008) · Serials Review · doi:10.1016/j.serrev.2008.01.001 · Cited by 126
How Incorporating Feedback Mechanisms in a DSS Affects DSS Evaluations
Kayandé, U., Bruyn, A. D., Lilien, G. L., Rangaswamy, A., & van Bruggen, G. H. (2008) · Information Systems Research · doi:10.1287/isre.1080.0198 · Cited by 126
Product-Oriented Web Technologies and Product Returns: An Exploratory Study
De, P., Hu, Y. J., & Rahman, M. S. (2013) · Information Systems Research · doi:10.1287/isre.2013.0487 · Cited by 124
Chatfield, C. (1995) · Chapman & Hall/CRC eBooks · doi:10.1201/b15238 · Cited by 98
Inferring Networks of Substitutable and Complementary ProductsPreprint
McAuley, J., Pandey, R., & Leskovec, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.08839 · Cited by 97
Profiting from Knowledge Management: The Impact of Time and Experience
Ko, D., & Dennis, A. R. (2009) · Information Systems Research · doi:10.1287/isre.1090.0247 · Cited by 92
Does the Web Reduce Customer Service Cost? Empirical Evidence from a Call Center
Kumar, A., & Telang, R. (2011) · Information Systems Research · doi:10.1287/isre.1110.0390 · Cited by 88
Keeping up with the quants: your guide to understanding and using analytics
Davenport, T. H., & Kim, J. (2013) · Choice Reviews Online · doi:10.5860/choice.51-2174 · Cited by 80
Murray‐Rust, P. (2008) · Serials Review · doi:10.1080/00987913.2008.10765152 · Cited by 80
On collocations and topic models
Lau, J. H., Baldwin, T., & Newman, D. J. (2013) · ACM Transactions on Speech and Language Processing · doi:10.1145/2483969.2483972 · Cited by 78
Pigliucci, M. (2009) · EMBO Reports · doi:10.1038/embor.2009.111 · Cited by 60
Interpretability of Mixed Membership Models
Singer, B. H., & Castro, M. C. (2014) · Chapman & Hall/CRC eBooks · doi:10.1201/b17520-16 · Cited by 55
It is not about Size: A Further thought on Big Data
Yoo, Y. (2015) · Journal of Information Technology · doi:10.1057/jit.2014.30 · Cited by 52
How In-memory Technology Can Create Business Value: Insights from the Hilti Case
Brocke, J. V., Debortoli, S., Müller, O., & Reuter, N. (2014) · Communications of the Association for Information Systems · doi:10.17705/1cais.03407 · Cited by 50
Self-citationA comparison of nonlinear methods for predicting earnings surprises and returns
Dhar, V., & Chou, D. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.935099 · Cited by 48
Analyzing Online Customer Reviews - An Interdisciplinary Literature Review And Research Agenda
Trenz, M., & Berger, B. (2013) · Journal of the Association for Information Systems · doi:10.13140/2.1.1203.1361 · Cited by 42
Taming Uncertainty in Big Data: Evidence from Social Media in Urban Areas
Bendler, J., Wagner, S., Brandt, T., & Neumann, D. (2014) · Business & Information Systems Engineering · doi:10.1007/s12599-014-0342-4 · Cited by 36
Computer assessment of interview data using latent semantic analysis
Dam, G., & Kaufmann, S. C. (2008) · Behavior Research Methods · doi:10.3758/brm.40.1.8 · Cited by 35
DIGITAL TRACES OF INFORMATION SYSTEMS : SOCIOMATERIALITY MADE RESEARCHABLE
Hedman, J., Srinivasan, N., & Lindgren, R. (2013) · CBS Research Portal (Copenhagen Business School) · Cited by 32
Big Data & Inductive Theory Development: Towards Computational Grounded Theory?
Berente, N., & Seidel, S. (2014) · Journal of the Association for Information Systems · Cited by 11
Cited by
A Topic Modeling Comparison Between LDA, NMF, Top2Vec, and BERTopic to Demystify Twitter Posts
Egger, R., & Yu, J. (2022) · Frontiers in Sociology · doi:10.3389/fsoc.2022.886498 · Cited by 970
Big data analytics capabilities: A systematic literature review and research agenda
Mikalef, P., Pappas, I. O., Krogstie, J., & Giannakos, M. (2018) · Information Systems and e-Business Management · doi:10.1007/s10257-017-0362-y · Cited by 888
Mikalef, P., & Krogstie, J. (2020) · European Journal of Information Systems · doi:10.1080/0960085x.2020.1740618 · Cited by 331
When the Machine Meets the Expert: An Ethnography of Developing AI for Hiring
van den Broek, E., Sergeeva, A., & Huysman, M. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16559 · Cited by 330
Schmiedel, T., Müller, O., & vom Brocke, J. (2019) · Organizational Research Methods · doi:10.1177/1094428118773858 · Cited by 319
Self-citation
Metadata
- Authors
- Müller, O., Junglas, I., Brocke, J. V., & Debortoli, S.
- Venue
- European Journal of Information Systems
- Publisher
- Informa UK Limited
- Published
- 2016-07
- Volume
- 25
- Issue
- 4
- Pages
- 289-302
- ISSN
- 0960-085X (ISSN-L)
- 1476-9344
- OpenAlex
- W2258542398
- Cited by
- 281
- References
- 96
- FWCI
- 43.24
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.