The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
On being ‘systematic’ in literature reviews
Abstract
General guidelines for conducting literature reviews often do not address the question of literature searches and dealing with a potentially large number of identified sources. These issues are specifically addressed by so-called systematic literature reviews (SLR) that propose a strict protocol for the search and appraisal of literature. Moreover, SLR are claimed to be a ‘standardized method’ for literature reviews, that is, replicable, transparent, objective, unbiased, and rigorous, and thus superior to other approaches for conducting literature reviews. These are significant and consequential claims that — 2014; despite increasing adoption of SLR — 2014; remained largely unnoticed in the information systems (IS) literature. The objective of this debate is to draw attention of the IS community to SLR’s claims, to question their justification and reveal potential risks of their adoption. This is achieved by first examining the origins of SLR and the prescribed systematic literature review process and then by critically assessing their claims and implications. In this debate, we show that SLR are applicable and useful for a very specific kind of literature review, a meta study that identifies and summarizes evidence from earlier research. We also demonstrate that the claims that SLR provide superior quality are not justified. More importantly, we argue that SLR as a general approach to conducting literature reviews is highly questionable, concealing significant perils. The paper cautions that SLR could undermine critical engagement with literature and what it means to be scholarly in academic work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
Systematic Reviews in the Social Sciences: A Practical Guide
Petticrew, M., & Roberts, H. (2006) · doi:10.1002/9780470754887 · Cited by 3,586
Writing narrative literature reviews for peer-reviewed journals: secrets of the trade
Green, B. N., Johnson, C. D., & Adams, A. (2006) · Journal of Chiropractic Medicine · doi:10.1016/s0899-3467(07)60142-6 · Cited by 2,224
Greenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045
Systematic Reviews: Rationale for systematic reviews
Mulrow, C. D. (1994) · BMJ · doi:10.1136/bmj.309.6954.597 · Cited by 1,677
Writing Narrative Literature Reviews
Baumeister, R. F., & Leary, M. R. (1997) · Review of General Psychology · doi:10.1037/1089-2680.1.3.311 · Cited by 1,482
Leonardi, P. M., & Barley, S. R. (2010) · Academy of Management Annals · doi:10.5465/19416521003654160 · Cited by 733
Systematic literature studies: database searches vs. backward snowballing
Jalali, S., & Wohlin, C. (2012) · Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement · doi:10.1145/2372251.2372257 · Cited by 659
Comments on “What theory is not”
DiMaggio, P. J. (1995) · Administrative Science Quarterly · doi:10.2307/2393790 · Cited by 488
Analyzing the past to prepare for the future
(2002) · MIS Quarterly · doi:10.5555/2017160.2017162 · Cited by 179
Cited by
Responsible Artificial Intelligence Governance: A Review and Research Framework
Papagiannidis, E., Mikalef, P., & Conboy, K. (2025) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2024.101885 · Cited by 565
Alexa, what do we know about conversational commerce? Insights from a systematic literature review.
Lim, W. M., Kumar, S., Verma, S., & Chaturvedi, R. (2022) · Psychology & Marketing · doi:10.1002/mar.21654 · Cited by 293
Consumer engagement in social media brand communities: A literature review
Santos, Z. R., Cheung, C. M. K., Coelho, P. S., & Rita, P. (2022) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2021.102457 · Cited by 265
Responsible AI for Digital Health: a Synthesis and a Research Agenda
Trocin, C., Mikalef, P., Papamitsiou, Z., & Conboy, K. (2023) · Information Systems Frontiers · doi:10.1007/s10796-021-10146-4 · Cited by 205
Graph Retrieval-Augmented Generation for Large Language Models: A Survey
Procko, T. T., & Ochoa, O. (2024) · 2024 Conference on AI, Science, Engineering, and Technology (AIxSET) · doi:10.1109/aixset62544.2024.00030 · Cited by 125
Metadata
- Authors
- Boell, S. K., & Cecez-Kecmanovic, D.
- Venue
- Formulating Research Methods for Information Systems
- Publisher
- Palgrave Macmillan UK
- Published
- 2015
- Pages
- 48-78
- OpenAlex
- W2501157161
- Cited by
- 159
- References
- 13
- FWCI
- 0.99
- Type
- book chapter
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: On being ‘systematic’ in literature reviews. https://app.cochat.ai/papers/10.1057/9781137509888_3
- The paper
- Boell, S. K., & Cecez-Kecmanovic, D. (2015). On being ‘systematic’ in literature reviews. Formulating Research Methods for Information Systems. https://doi.org/10.1057/9781137509888_3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.