The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
On being ‘Systematic’ in Literature Reviews in IS
Abstract
General guidelines for conducting literature reviews often do not address the question of literature searches and dealing with a potentially large number of identified sources. These issues are specifically addressed by so-called systematic literature reviews (SLRs) that propose a strict protocol for the search and appraisal of literature. Moreover, SLRs are claimed to be a ‘standardized method’ for literature reviews that is replicable, transparent, objective, unbiased and rigorous, and thus superior to other approaches for conducting literature reviews. These are significant and consequential claims that – despite increasing adoption of SLRs – remained largely unnoticed in the information systems (IS) literature. The objective of this debate is to draw attention of the IS community to SLR's claims, to question their justification and reveal potential risks of their adoption. This is achieved by first examining the origins of SLR and the prescribed SLR process and then by critically assessing their claims and implications. In this debate, we show that SLRs are applicable and useful for a very specific kind of literature review, a meta study that identifies and summarizes evidence from earlier research. We also demonstrate that the claims that SLRs provide superior quality are not justified. More importantly, we argue that SLR as a general approach to conducting literature reviews is highly questionable, concealing significant perils. The paper cautions that SLR could undermine critical engagement with literature and what it means to be scholarly in academic work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
Systematic Reviews in the Social Sciences: A Practical Guide
Petticrew, M., & Roberts, H. (2006) · doi:10.1002/9780470754887 · Cited by 3,586
Writing narrative literature reviews for peer-reviewed journals: secrets of the trade
Green, B. N., Johnson, C. D., & Adams, A. (2006) · Journal of Chiropractic Medicine · doi:10.1016/s0899-3467(07)60142-6 · Cited by 2,224 · cited 1×
“…is the generation of new insights, perspectives and understanding by problematizing knowledge claims, approaches and assumptions in earlier research Alvesson and Sandberg (2011); Green et al. (2006); Hart (1998); Khoo et al. (2011); MacLure (2005) specific kind of literature reviews can be useful.”
BackgroundGreenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045 · cited 1×
“Knipschild (1995) reports that only 36% of the literature was identified through database searches, and Greenhalgh and Peacock (2005) report only 25%.”
BackgroundWriting Narrative Literature Reviews
Baumeister, R. F., & Leary, M. R. (1997) · Review of General Psychology · doi:10.1037/1089-2680.1.3.311 · Cited by 1,482
Leonardi, P. M., & Barley, S. R. (2010) · Academy of Management Annals · doi:10.5465/19416521003654160 · Cited by 733
Comments on “What theory is not”
DiMaggio, P. J. (1995) · Administrative Science Quarterly · doi:10.2307/2393790 · Cited by 488 · cited 1×
“However, questions of ‘how’ and ‘why’ which are fundamental to academic endeavors, especially in the social sciences (Whetten, 1989; DiMaggio, 1995; Sutton and Staw, 1995), are not considered.”
BackgroundAnalyzing the past to prepare for the future
(2002) · MIS Quarterly · doi:10.5555/2017160.2017162 · Cited by 179
Dong, Y. R. (1996) · Research in the Teaching of English · doi:10.58680/rte199615303 · Cited by 113
Griffith University, Amrollahi, A., Ghapanchi, A. H., Griffith University, Talaei-Khoei, A. T.-K., & University Technology Sydney (2013) · Pacific Asia Journal of the Association for Information Systems · doi:10.17705/1pais.05203 · Cited by 45
Cited by
The emergence of circular economy: a new framing around prolonging resource productivity
Blomsma, F., & Brennan, G. (2017) · Journal of Industrial Ecology · doi:10.1111/jiec.12603 · Cited by 1,383
Debating big data: a literature review on realizing value from big data
Günther, W. A., Rezazade Mehrizi, M. H., Huysman, M., & Feldberg, F. (2017) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2017.07.003 · Cited by 880
The state of immersive technology research: A literature analysis
Suh, A., & Prophet, J. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.04.019 · Cited by 841
Data governance: A conceptual framework, structured review, and research agenda
Abraham, R., Schneider, J., & vom Brocke, J. (2019) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.07.008 · Cited by 645
Ali, O., Abdelbaki, W., Shrestha, A., Elbasi, E., Alryalat, M. A. A., & Dwivedi, Y. K. (2023) · Journal of Innovation & Knowledge · doi:10.1016/j.jik.2023.100333 · Cited by 568
Why do people play games? A meta-analysis
Hamari, J., & Keronen, L. (2017) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2017.01.006 · Cited by 368
Ali, O., Murray, P. A., Momin, M., Dwivedi, Y. K., & Malik, T. (2024) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.123076 · Cited by 317
Digitally-induced change in the public sector: a systematic review and research agenda
Haug, N., Dan, S., & Mergel, I. (2024) · Public Management Review · doi:10.1080/14719037.2023.2234917 · Cited by 266
Templier, M., & Paré, G. (2018) · European Journal of Information Systems · doi:10.1080/0960085x.2017.1398880 · Cited by 180
Review and Theory Symbiosis: An Introspective Retrospective
Baylor University and Lund University, & Leidner, D. (2018) · Journal of the Association for Information Systems · doi:10.17705/1jais.00501 · Cited by 102
Characteristics of systematic reviews in the social sciences
Chapman, K. (2021) · The Journal of Academic Librarianship · doi:10.1016/j.acalib.2021.102396 · Cited by 93
Measuring servitization progress and outcome: the case of ‘advanced services’
Ziaee Bigdeli, A., Baines, T., Schroeder, A., Brown, S., Musson, E., Guang Shi, V., & Calabrese, A. (2018) · Production Planning & Control · doi:10.1080/09537287.2018.1429029 · Cited by 79
Buckingham Shum, S., Lim, L.-A., Boud, D., Bearman, M., & Dawson, P. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00410-9 · Cited by 77
Mandják, T., Lavissière, A., Hofmann, J., Bouchery, Y., Lavissière, M. C., Faury, O., & Sohier, R. (2019) · Transport Policy · doi:10.1016/j.tranpol.2018.11.011 · Cited by 58
An overview of systematic literature reviews in social media marketing
Rowley, J., & Keegan, B. J. (2020) · Journal of Information Science · doi:10.1177/0165551519866544 · Cited by 42
Buck, C., Keweloh, C., Bouras, A., & Simoes, E. J. (2021) · JMIR mHealth and uHealth · doi:10.2196/20199 · Cited by 33
How Can Business Analytics Enhance Decision-Making in Oil and Gas Surface Facilities?
Prasetya, A., Wasesa, M., & Sunitiyoso, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3582939 · Cited by 3
Maxwell, G. S. (2021) · The enabling power of assessment · doi:10.1007/978-3-030-63539-8_1 · Cited by 1
Metadata
- Authors
- Boell, S. K., & Cecez-Kecmanovic, D.
- Venue
- Journal of Information Technology
- Publisher
- SAGE Publications
- Published
- 2015-06
- Volume
- 30
- Issue
- 2
- Pages
- 161-173
- ISSN
- 0268-3962 (ISSN-L)
- 1466-4437
- OpenAlex
- W1996700032
- Cited by
- 485
- References
- 10
- FWCI
- 17.30
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
