The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index.
A Guide to Conducting a Standalone Systematic Literature Review
Open accessAbstract
Many scholars are not well trained in conducting a standalone literature review, a scholarly paper that in its entirety summarizes and synthesizes knowledge from a prior body of research. Numerous guides that exist for information systems (IS) research mainly concentrate on only certain parts of the process; few span the entire process. This paper introduces the rigorous, standardized methodology for the systematic literature review (also called systematic review) to IS scholars. This comprehensive guide extends the base methodology from the health sciences and other fields with numerous adaptations to meet the needs of methodologically diverse fields such as IS research, especially those that involve including and synthesizing both quantitative and qualitative studies. Moreover, this guide provides many examples from IS research and provides references to guides with further helpful details for conducting a rigorous and valuable literature review. Although tailored to IS research, it is sufficiently broad to be applicable and valuable to scholars from any social science field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For this review, the guidelines published by Okoli to conducting a systematic literature review for Information Systems Research were adapted (Okoli, 2015).”
From A systematic literature review of personalized learning terms (2020) · cites this paper as Okoli, 2015
In the citing paper
“The foundation of our guide was taken from the guideline developed by Okoli (2015).”
From Artificial intelligence in information systems research: A systematic literature review and research agenda (2021) · cites this paper as Okoli (2015)
In the citing paper
“Okoli (2015) propose a systematic review process that consists of 8 steps, namely planning (2 steps), selection (2 steps), extraction (2 steps) and execution (2 steps) that are completed across 4 phases (see Fig. 1.).”
From Artificial intelligence in information systems research: A systematic literature review and research agenda (2021) · cites this paper as Okoli (2015)
In the citing paper
“As this systematic study followed the eight-step guideline laid out by (Okoli, 2015), it is attributed to mitigating the threats to validity.”
From Artificial intelligence in information systems research: A systematic literature review and research agenda (2021) · cites this paper as Okoli, 2015
In the citing paper
“To conduct this study, the authors followed the essential steps of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) and Okoli’s [54] steps for conducting a systematic review.”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [54]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Alavi, M., & Leidner, D. E. (2001) · MIS Quarterly · doi:10.2307/3250961 · Cited by 10,176
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Analyzing the Past to Prepare for the Future: Writing a Literature Review
Webster, J., & Watson, R. T. (2002) · MIS Quarterly · doi:10.2307/4132319 · Cited by 7,449
A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems1
Klein, H. K., & Myers, M. D. (1999) · MIS Quarterly · doi:10.2307/249410 · Cited by 5,263
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537
Meta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348600 · Cited by 4,249
Systematic reviews in the social sciences: a practical guide
(2006) · Choice Reviews Online · doi:10.5860/choice.43-5664 · Cited by 4,081
Melville, N. P., Kraemer, & Gurbaxani, V. (2004) · MIS Quarterly · doi:10.2307/25148636 · Cited by 3,561
Realist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190
Wade, M., & Hulland, J. (2004) · MIS Quarterly · doi:10.2307/25148626 · Cited by 2,822
Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
Barbour, R. S. (2001) · BMJ · doi:10.1136/bmj.322.7294.1115 · Cited by 2,613
Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M., & Khalil, M. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2006.07.009 · Cited by 2,394
Organizational Theories: Some Criteria for Evaluation
Bacharach, S. B. (1989) · The Academy of Management Review · doi:10.5465/amr.1989.4308374 · Cited by 2,305
Investigating and dealing with publication and other biases in meta-analysis
Sterne, J. A. C., Egger, M., & Smith, G. D. (2001) · BMJ · doi:10.1136/bmj.323.7304.101 · Cited by 2,225
A Guide to Conducting a Systematic Literature Review of Information Systems Research
Okoli, C., & Schabram, K. (2011) · doi:10.2139/ssrn.1954824 · Cited by 1,849
Self-citationConducting research literature reviews: from the Internet to paper
(2005) · Choice Reviews Online · doi:10.5860/choice.42-6198 · Cited by 1,846
Meta-Analysis: Recent Developments in Quantitative Methods for Literature Reviews
Rosenthal, R., & DiMatteo, M. R. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.59 · Cited by 1,780
Levy, Y. M., & Ellis, T. J. (2006) · Informing Science The International Journal of an Emerging Transdiscipline · doi:10.28945/479 · Cited by 1,724
Leidner, D. E., & Kayworth, T. R. (2006) · MIS Quarterly · doi:10.2307/25148735 · Cited by 1,661
Employee Turnover: A Meta-Analysis and Review with Implications for Research
Cotton, J. L., & Tuttle, J. M. (1986) · Academy of Management Review · doi:10.5465/amr.1986.4282625 · Cited by 1,493
Erduran, S., Simon, S., & Osborne, J. (2004) · Science Education · doi:10.1002/sce.20012 · Cited by 1,408
Reconstructing the Giant: On the Importance of Rigour in Documenting the Literature Search Process
Brocke, J. V., Simons, A., Niehaves, B., Reimer, K., Plattfaut, R., & Cleven, A. (2009) · Alexandria (UniSG) (University of St.Gallen) · Cited by 1,383
Doing a literature review: releasing the social science research imagination.
Hart, C. (1998) · Sage eBooks · Cited by 1,378
Boote, D. N., & Beile, P. (2005) · Educational Researcher · doi:10.3102/0013189x034006003 · Cited by 1,217
Using grounded theory as a method for rigorously reviewing literature
Wolfswinkel, J. F., Furtmueller, E., & Wilderom, C. P. M. (2011) · European Journal of Information Systems · doi:10.1057/ejis.2011.51 · Cited by 1,212
Writing meta-analytic reviews.
Rosenthal, R. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.118.2.183 · Cited by 1,108
Rousseau, D. M., Manning, J., & Denyer, D. (2008) · Academy of Management Annals · doi:10.5465/19416520802211651 · Cited by 1,083
Dubé, L., & Paré, G. (2016) · SAGE Publications Ltd eBooks · doi:10.4135/9781473915480.n65 · Cited by 1,076
Rousseau, D. M., Manning, J., & Denyer, D. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.1309606 · Cited by 997
Dubé, L., & Paré, G. (2003) · MIS Quarterly · doi:10.2307/30036550 · Cited by 976
Marakas, G. M., Yi, M. Y., & Johnson, R. D. (1998) · Information Systems Research · doi:10.1287/isre.9.2.126 · Cited by 919
Dead Or Alive? The Development, Trajectory And Future Of Technology Adoption Research.
University of Arkansas, Venkatesh, V., Davis, F., University of Arkansas, Morris, M., & University of Virginia (2007) · Journal of the Association for Information Systems · doi:10.17705/1jais.00120 · Cited by 789
Piccoli, G., & Ives, B. (2005) · MIS Quarterly · doi:10.2307/25148708 · Cited by 665
Brocke, J. V., Simons, A., Riemer, K., Niehaves, B., Plattfaut, R., & Cleven, A. (2015) · Communications of the Association for Information Systems · doi:10.17705/1cais.03709 · Cited by 626
A Meta-Analysis of the Relationships Between Individual Job Satisfaction and Individual Performance
Petty, M. M., McGee, G. W., & Cavender, J. W. (1984) · Academy of Management Review · doi:10.5465/amr.1984.4277608 · Cited by 595
Employee Turnover: A Meta-Analysis and Review with Implications for Research
Cotton, J. L., & Tuttle, J. M. (1986) · Academy of Management Review · doi:10.2307/258331 · Cited by 590
A Review of MIS Research and Disciplinary Development
Alavi, M., & Carlson, P. A. (1992) · Journal of Management Information Systems · doi:10.1080/07421222.1992.11517938 · Cited by 569
Review: A Cognitive-Affective Model of Organizational Communication for Designing IT1,2
Te’eni, D. (2001) · MIS Quarterly · doi:10.2307/3250931 · Cited by 523
Joseph, D., Ng, K.-Y., Koh, C., & Ang, S. (2007) · MIS Quarterly · doi:10.2307/25148807 · Cited by 468
Review : Power and Information Technology Research: A Metatriangulation Review1, 2
Jasperson, J., Carte, T. A., Saunders, C., Butler, B. S., Croes, H. J. P., & Zheng, W. (2002) · MIS Quarterly · doi:10.2307/4132315 · Cited by 413
Alavi, M., & Joachimsthaler, E. A. (1992) · MIS Quarterly · doi:10.2307/249703 · Cited by 371
Achieving Rigor in Literature Reviews: Insights from Qualitative Data Analysis and Tool-Support
Bandara, W., Furtmueller, E., Gorbacheva, E., Miskon, S., & Beekhuyzen, J. (2015) · Communications of the Association for Information Systems · doi:10.17705/1cais.03708 · Cited by 367
Synthesizing qualitative and quantitative health evidence: a guide to methods
Pope, C. J., Mays, N., & Popay, J. (2007) · Cited by 336
Impression Management in the Feedback-Seeking Process: A Literaturereview and Research Agenda
Morrison, E. W., & Bies, R. J. (1991) · Academy of Management Review · doi:10.5465/amr.1991.4279472 · Cited by 335
Automatically refining the wikipedia infobox ontology
Wu, F. Q., & Weld, D. S. (2008) · International World Wide Web Conference (WWW) · doi:10.1145/1367497.1367583 · Cited by 305
Self‐citation and self‐reference: Credibility and promotion in academic publication
Hyland, K. (2003) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.10204 · Cited by 303
The literature review: a step-by-step guide for students
Graham, V. F. (2011) · Evaluation & Research in Education · doi:10.1080/09500790.2011.583140 · Cited by 291
Robey, D., Im, G. P., Wareham, J. D., & ESADE (2008) · Journal of the Association for Information Systems · doi:10.17705/1jais.00171 · Cited by 266
Theory Construction and Model-Building Skills: A Practical Guide for Social Scientists
Jaccard, J. J., & Jacoby, J. (2009) · Cited by 261
Writing a review article for Psychological Bulletin.
Bem, D. J. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.118.2.172 · Cited by 245
Scientific Collaboration Results in Higher Citation Rates of Published Articles
Figg, W. D., Dunn, L. A., Liewehr, D. J., Steinberg, S. M., Thurman, P. W., Barrett, J. C., & Birkinshaw, J. (2006) · Pharmacotherapy The Journal of Human Pharmacology and Drug Therapy · doi:10.1592/phco.26.6.759 · Cited by 242
Quantitative methods for literature reviews
Green, B. F., & Hall, J. A. (1984) · Annual Review of Psychology · doi:10.1146/annurev.ps.35.020184.000345 · Cited by 213
The Literature Review : A Step-by-Step Guide for Students
Ridley, D. (2012) · Cited by 209
A Meta-Analysis of the Relationships between Individual Job Satisfaction and Individual Performance
Petty, M. M., McGee, G. W., & Cavender, J. W. (1984) · Academy of Management Review · doi:10.2307/258493 · Cited by 209
A Theoretical Perspective of Negotiation Support Systems
Lim, L. H., & Benbasat, I. (1992) · Journal of Management Information Systems · doi:10.1080/07421222.1992.11517966 · Cited by 191
Beyond synthesis: re-presenting heterogeneous research literature
Sylvester, A., Tate, M., & Johnstone, D. (2011) · Behaviour and Information Technology · doi:10.1080/0144929x.2011.624633 · Cited by 174
Theory Construction and Model-Building Skills: A Practical Guide for Social Scientists
Aducci, C. J., & Johnson, M. D. (2011) · Journal of Family Theory & Review · doi:10.1111/j.1756-2589.2010.00083.x · Cited by 171
Task Design: A Literature Review
Pierce, J. L., & Dunham, R. B. (1976) · Academy of Management Review · doi:10.5465/amr.1976.4396611 · Cited by 163
Human Resources Management in International Joint Ventures: Directions for Research
Shenkar, O., & Zeira, Y. (1987) · Academy of Management Review · doi:10.5465/amr.1987.4306569 · Cited by 163
Are Articles in “Top” Management Journals Necessarily of Higher Quality?
Singh, G., Haddad, K. M., & Chow, C. W. (2007) · Journal of Management Inquiry · doi:10.1177/1056492607305894 · Cited by 145
Maxwell, J. A. (2006) · Educational Researcher · doi:10.3102/0013189x035009028 · Cited by 131
Melville, N. P., Kraemer, K., & Gurbaxani, V. (2003) · Cited by 127
What Experienced Collaborators Say About Collaborative Writing
Allen, N., Atkinson, D., Morgan, M., Moore, T., & Snow, C. (1987) · Journal of Business and Technical Communication · doi:10.1177/105065198700100206 · Cited by 124
Impression Management in the Feedback-Seeking Process: A Literature Review and Research Agenda
Morrison, E. W., & Bies, R. J. (1991) · Academy of Management Review · doi:10.2307/258916 · Cited by 107
Perceived Task Characteristics and Employee Performance: A Literature Review
Griffin, R. W., Welsh, A., & Moorhead, G. (1981) · Academy of Management Review · doi:10.5465/amr.1981.4285726 · Cited by 97
Wade, M., & Hulland, J. S. (2004) · Cited by 90
Green, R., & Bowser, M. (2006) · Journal of Library Administration · doi:10.1300/j111v45n01_10 · Cited by 70
Lee, D., Kang, J., Mitra, P., Giles, C. L., & On, B.-W. (2007) · Communications of the ACM · doi:10.1145/1323688.1323690 · Cited by 68
Abstracts, introductions and discussions: How far do they differ in style?
Hartley, J., Pennebaker, J. W., & Fox, C. (2003) · Scientometrics · doi:10.1023/a:1025008802657 · Cited by 63
Finding open access articles using Google, Google Scholar, OAIster and OpenDOAR
Norris, M. W., Oppenheim, C., & Rowland, F. (2008) · Online Information Review · doi:10.1108/14684520810923881 · Cited by 63
Strategies for Disseminating Qualitative Research Findings: Three Exemplars
Keen, S., & Todres, L. (2008) · Forum qualitative Sozialforschung · doi:10.17169/fqs-8.3.285 · Cited by 58
Shift Work: A Review and Theoretical Analysis
Dunham, R. B. (1977) · Academy of Management Review · doi:10.5465/amr.1977.4406742 · Cited by 56
Providing Information Literacy Instruction to Graduate Students through Literature Review Workshops
Rempel, H. G. (2008) · PDXScholar (Portland State University) · doi:10.5062/f44x55rg · Cited by 55
A Critical Realist Guide to Developing Theory with Systematic Literature ReviewsPreprint
Okoli, C. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2115818 · Cited by 51
Self-citationHuman Resources Management in International Joint Ventures: Directions for Research
Shenkar, O., & Zeira, Y. (1987) · Academy of Management Review · doi:10.2307/258520 · Cited by 51
Winter, S. J., & Taylor, S. L. (1996) · Information Systems Research · doi:10.1287/isre.7.1.5 · Cited by 47
Leidner, D. E., & Kayworth, T. R. (2006) · Journal of the Association for Information Systems · Cited by 47
Lessons Learnt Undertaking a Large-scale Systematic Literature Review
Turner, M., Kitchenham, B. A., Budgen, D., & Pearl Brereton, A. (2008) · Electronic workshops in computing · doi:10.14236/ewic/ease2008.12 · Cited by 45
Writing a review article for Psychological Bulletin.
Bem, D. J. (1995) · Psychological Bulletin · doi:10.1037//0033-2909.118.2.172 · Cited by 41
The genre(s) of student writing: developing writing models
Hüttner, J. (2008) · International Journal of Applied Linguistics · doi:10.1111/j.1473-4192.2008.00200.x · Cited by 34
Joseph, D., Ng, K.-Y., Koh, C. S. K., & Ang, S. (2007) · Journal of the Association for Information Systems · Cited by 33
Shift Work: A Review and Theoretical Analysis
Dunham, R. B. (1977) · Academy of Management Review · doi:10.2307/257514 · Cited by 29
Bragge, J., Relander, S., Sunikka, A., & Mannonen, P. (2007) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2007.209 · Cited by 25
The Researcher, the Topic, and the Literature: A Procedure for Systematizing Literature Searches
Helmericks, S. G., Nelsen, R. L., & Unnithan, N. P. (1991) · The Journal of Applied Behavioral Science · doi:10.1177/0021886391273004 · Cited by 25
Meaning in philosophy and meaning in information retrieval (IR)
Thornley, C., & Gibb, F. (2009) · Journal of Documentation · doi:10.1108/00220410910926158 · Cited by 22
Dawson, K., & Ferdig, R. E. (2006) · Cited by 17
The Importance of Resubmitting Rejected Papers
Donovan, S. K. (2007) · Journal of Scholarly Publishing · doi:10.3138/w706-5884-kg84-j282 · Cited by 15
The “dos and don'ts” of writing a journal article
Kekäle, T., de Weerd-Nederhof, P. C., Cervai, S., & Borelli, M. (2009) · Journal of Workplace Learning · doi:10.1108/13665620910924925 · Cited by 14
The Discard of Study Evidence by Literature Reviewers
Rodgers, R. C., & Hunter, J. E. (1994) · The Journal of Applied Behavioral Science · doi:10.1177/0021886394303005 · Cited by 14
Quantitative Methods for Literature Reviews
Green, B. (1984) · Annual Review of Psychology · doi:10.1146/annurev.psych.35.1.37 · Cited by 14
The dimensions of cited reference enhanced database subsets
Jacsó, P. (2007) · Online Information Review · doi:10.1108/14684520710832360 · Cited by 11
The Importance of Resubmitting Rejected Papers
Donovan, S. K. (2007) · Journal of Scholarly Publishing · doi:10.1353/scp.2007.0013 · Cited by 5
Flagged first, then by citations
Research works printed but not linked: 4
- Alavi, M., & Leidner, D. E. (2001). Review: Knowledge management and knowledge management systems: Conceptual foundations and research issues. MIS Quarterly, 25(1), 107-136.
- Alias, M., Zurinah, C., Reiska, A. J., Åhlberg, M., & Novak, J. D. (2008). Concept mapping: A tool for creating a literature review. In Proceedings of the 3 rd International Conference on Concept Mapping.
- Anonymous. (1998). "Gray literature" forces new role on librarians (sidebar). Chemical & Engineering News, 76(20), 12.
- Myers, P. M. D. (2008). Qualitative research in business & management (illustrated edition.). Thousand Oaks, CA: Sage.
Cited by
Sorted by citations
The art of crafting a systematic literature review in entrepreneurship research
Kraus, S., Breier, M., & Dasí-Rodríguez, S. (2020) · International Entrepreneurship and Management Journal · doi:10.1007/s11365-020-00635-4 · Cited by 1,319
Collins, C., Dennehy, D., Conboy, K., & Mikalef, P. (2021) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2021.102383 · Cited by 783
Hawlitschek, F., Notheisen, B., & Teubner, T. (2018) · Electronic Commerce Research and Applications · doi:10.1016/j.elerap.2018.03.005 · Cited by 768
Utilising learning analytics to support study success in higher education: a systematic review
Ifenthaler, D., & Yau, J. Y.-K. (2020) · Educational Technology Research and Development · doi:10.1007/s11423-020-09788-z · Cited by 1,063
The ABC of systematic literature review: the basic methodological guidance for beginners
Mohamed Shaffril, H. A., Samsuddin, S. F., & Abu Samah, A. (2021) · Quality & Quantity · doi:10.1007/s11135-020-01059-6 · Cited by 656
Metadata
- Authors
- Okoli, C.
- Affiliations
- Concordia University (CA)
- Corresponding author
- Okoli, Chitu
- ORCID
- Okoli, Chitu
- Venue
- Communications of the Association for Information Systems
- Publisher
- Association for Information Systems
- Published
- 2015
- Volume
- 37
- ISSN
- 1529-3181 (ISSN-L)
- OpenAlex
- W2173536757
- Cited by
- 1,413
- References
- 93
- Citations by year
- 2026: 150 · 2025: 241 · 2024: 250 · 2023: 230 · 2022: 220 · 2021: 162 · 2020: 78 · 2019: 34 · 2018: 30 · 2017: 12 · 2016: 4 · 2015: 2
- FWCI
- 2.63
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- other-oa
- Locations
- Communications of the Association for Information Systems (publishedVersion, other-oa)
- Funders
- Social Sciences and Humanities Research Council of Canada
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Social Sciences › Business, Management and Accounting › Strategy and Management › Business Strategies and Management Research
- Keywords
- systematic reviews, information systems research, research methodology, literature synthesis, mixed methods synthesis, research guidelines, meta-analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
