CoChat
Sign Up
Cited by 28Meta-analysis and systematic reviewsComputational and Text Analysis MethodsDiverse Interdisciplinary Research Innovations

A hybrid framework for creating artificial intelligence-augmented systematic literature reviews

Open access
Malik, Faisal Saeed, Terzidis, Orestis·2026·Management Review Quarterly
DOI 10.1007/s11301-025-00522-8
Open this paper in CoChat — free accountOpen PDF

Abstract

The integration of artificial intelligence (AI), particularly generative AI (GenAI) and large language models (LLMs), into systematic literature reviews (SLRs) represents a transformative advancement in research methodologies. This paper proposes a hybrid framework combining AI’s computational power with the epistemological rigor of human expertise, anchored in transparency, validity, reliability, comprehensiveness, and reflective agency. Through three interconnected phases—design, study collection, and interpretation—the framework employs AI model selection, knowledge base curation, and iterative prompt engineering to enhance scalability, uncover interdisciplinary connections, and ensure methodological integrity through robust human oversight. It addresses key SLR challenges, including handling vast datasets, ensuring reproducibility, and maintaining epistemic rigor while leveraging advanced AI capabilities. Key innovations include cyclical validation, inter-model comparisons, and sensitivity testing to enhance trustworthiness and mitigate biases. The framework aligns AI processes with ethical standards and research objectives by emphasizing domain-specific LLMs, reliability metrics, and standardized reporting protocols. It establishes SLRs as a foundation for advancing knowledge in complex, interdisciplinary research landscapes, harmonizing AI efficiency with human expertise.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s11301-025-00522-8

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLMs can be cautiously used to flag potential qualitative issues in study design.

    “A robust AI-assisted workflow would not rely on one tool but would use them in combination: employing specific, high-accuracy tools like Statcheck and GRIM to screen for numerical and reporting-level errors, while cautiously using LLMs to flag potential qualitative issues in the study design, all of which must be confirmed by an expert human reviewer.10,11,41”

    From Artificial Intelligence in Detecting Statistical Errors: Implications for Authors, Reviewers, and Editors (2025) · cites this paper as [41]

    Section 4, “Key levers for authors using LLMs in SLRs” (page unavailable), supports cautious use of LLMs to process study text. Section 6, “Reflection on epistemic practices in AI-augmented SLRs” (page unavailable), says AI can identify inconsistencies relevant to validity. Neither passage specifically says LLMs can flag qualitative flaws in an individual study’s design. (LLM, conf=0.79)

    Experimental — accuracy not yet measured

  • Statcheck and GRIM are specific, high-accuracy tools for screening numerical and reporting-level errors.

    “A robust AI-assisted workflow would not rely on one tool but would use them in combination: employing specific, high-accuracy tools like Statcheck and GRIM to screen for numerical and reporting-level errors, while cautiously using LLMs to flag potential qualitative issues in the study design, all of which must be confirmed by an expert human reviewer.10,11,41”

    From Artificial Intelligence in Detecting Statistical Errors: Implications for Authors, Reviewers, and Editors (2025) · cites this paper as [41]

    The copy discusses inter-model validation and human oversight in section 5, and AI-assisted literature screening in section 3.1.2. Searches for Statcheck, GRIM, and numerical or statistical-reporting error screening found no passage addressing those tools or their accuracy. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 85

  1. Bobko P, Roth PL, Buster MA (2007) The usefulness of unit weights in creating composite scores: A literature review, application to content validity, and Meta-Analysis. Organizational Res Methods 10(4):689-709
  2. Bonett DG, Wright TA (2014) Cronbach's alpha reliability: interval estimation, hypothesis testing, and sample size planning. J Organizational Behav 36(1):3-15
  3. Breslin D, Gatrell C (2023) Theorizing through literature reviews: the miner-prospector continuum. Orga- nizational Res Methods 26(1):139-167
  4. Cai Y, Mao S, Wu W, Wang Z, Liang Y, Ge T,… and, Wei F (2023) Low-code LLM: Graphical User Inter- face over Large Language Models. arXiv preprint arXiv:2304.08103 · arXiv
  5. Chintalapati S, Pandey SK (2022) Artificial intelligence in marketing: A systematic literature review. Int J Market Res 64(1):38-68
  6. Clark WR, Clark LA, Raffo DM, Williams RI (2021) Extending fisch and block's (2018) tips for a system- atic review in management and business literature. Manage Rev Q 71:215-231
  7. Cronin MA, George E (2023) The why and how of the integrative review. Organizational Res Methods 26(1):168-192
  8. Davison RM, Chughtai H, Nielsen P, Marabelli M, Iannacci F, van Offenbeek M et al (2024) The ethics of using generative AI for qualitative data analysis. Inform Syst J 34(5):1433-1439
  9. Deepa R, Sekar S, Malik A, Kumar J, Attri R (2024) Impact of AI-focussed technologies on social and technical competencies for HR managers-A systematic review and research agenda. Technol Fore- cast Soc Chang 202:123301
  10. Denyer D, Tranfield D (2009) Producing a systematic review. In: Buchanan DA, Bryman A (eds) The Sage handbook of organizational research methods. Sage Publications Ltd., pp 671-689
  11. Dergaa I, Chamari K, Zmijewski P, Ben Saad H (2023) From human writing to artificial intelligence gen- erated text: examining the prospects and potential threats of ChatGPT in academic writing. Biology Sport 40(2):615-622
  12. Duan Y, Ong VK, Xu M, Mathews BP (2012) Supporting decision making process with ideal software agents-What do business executives want? Expert Syst Appl 39(10):8724-8738
  13. Eisenhardt KM (1989) Building theories from case study research. Acad Manage Rev 14(4):532-550
  14. Else H (2023) Abstracts written by ChatGPT fool scientists. Nature 613(7945):423
  15. Endrigo Sordan J, Andersson R, Antony J, Lopes Pimenta M, Oprime PC (2024) How industry 4.0, arti- ficial intelligence and augmented reality can boost digital lean six Sigma. Total Qual Manage Bus Excellence 35(13-14):1542-1566
  16. Etsenake D, Nagappan M (2024) Understanding the Human-LLM Dynamic: A Literature Survey of LLM Use in Programming Tasks. arXiv preprint arXiv:2410.01026 · arXiv
  17. Gatrell C, Muzio D, Post C, Wickert C (2024) Here, there and everywhere: on the responsible use of artificial intelligence (AI) in management research and the peer-review process. J Manage Stud 61(3):739-751
  18. Ge Z, Huang H, Zhou M, Li J, Wang G, Tang S, Zhuang Y (2024, October) Worldgpt: empowering llm as multimodal world model. In: Proceedings of the 32nd ACM internationalconference on multimedia (pp 7346-7355).
  19. Gibbert M, Ruigrok W (2010) The ''what''and ''how''of case study rigor: three strategies based on pub- lished work. Organizational Res Methods 13(4):710-737
  20. Glaser VL, Sloan J, Gehman J (2024) Organizations as algorithms: A new metaphor for advancing man- agement theory. J Manage Stud 61(6):2748-2769
  21. Gond J-P, Mena S, Mosonyi S (2023) The performativity of literature reviewing: constituting the corporate social responsibility literature through Re-presentation and intervention. Organizational Res Meth- ods 26(2):195-228
  22. Grant MJ, Booth A (2009) A typology of reviews: an analysis of 14 review types and associated method- ologies. Health Inf Libr J 26(2):91-108
  23. Grimes M, von Krogh G, Feuerriegel S, Rink F, Gruber M (2023) From scarcity to abundance: scholars and scholarship in an age of generative artificial intelligence. Acad Manag J 66(6):1617-1624
  24. Harley B, Cornelissen J (2022) Rigor with or without templates? The pursuit of methodological rigor in qualitative research. Organizational Res Methods 25(2):239-261
  25. Hayden N, Gilbert S, Poisson LM, Griffith B, Klochko C, Moy L, Wolfe S (2024) Performance of GPT-4 with vision on text-and image-based ACR diagnostic radiology in-training examination questions. Radiology, 312(3), e240153
  26. Hiebl MR (2023) Sample selection in systematic literature reviews of management research. Organiza- tional Res Methods 26(2):229-261
  27. Hoon C (2013) Meta-synthesis of qualitative case studies: an approach to theory Building. Organizational Res Methods 16(4):522-556
  28. Jatobá MN, Ferreira JJ, Fernandes PO, Teixeira JP (2023) Intelligent human resources for the adop- tion of artificial intelligence: a systematic literature review. J Organizational Change Manage, (ahead-of-print).
  29. Köhler T, Smith A, Bhakoo V (2022) Templates in qualitative research methods: origins, limitations, and new directions. Organizational Res Methods 25(2):183-210
  30. Korzynski P, Mazurek G, Krzypkowska P, Kurasinski A (2023) Artificial intelligence prompt engineering as a new digital competence: analysis of generative AI technologies such as ChatGPT. Entrepreneur- ial Bus Econ Rev 11(3):25-37
  31. Kraus S, Bouncken RB, Yela Aránega A (2024) The burgeoning role of literature review articles in man- agement research: an introduction and outlook. RMS 18(2):299-314
  32. Kraus S, Breier M, Dasí-Rodríguez S (2020) The Art of crafting a systematic literature review in entrepre- neurship research. Int Entrepreneurship Manage J 16:1023-1042
  33. Kulkarni M, Mantere S, Vaara E, van den Broek E, Pachidi S, Glaser VL et al (2024) The future of research in an artificial intelligence-driven world. J Manage Inq 33(3):207-229
  34. Kunisch S, Denyer D, Bartunek JM, Menz M, Cardinal LB (2023) Review research as scientific inquiry. Organizational Res Methods 26(1):3-45
  35. Kunisch S, Menz M, Bartunek JM, Cardinal LB, Denyer D (2018) Feature topic at organizational research methods: how to conduct rigorous and impactful literature reviews?? Organizational Res Methods 21(3):519-523
  36. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Moher D (2009b) The PRISMA statement for reporting systematic reviews and meta-analyses. PLoS Med, 6(7), e1000097
  37. Li J, Li J (2024) A map of exploring human interaction patterns with LLM: insights into collaboration and creativity. ArXiv, abs/2404.04570. · arXiv
  38. Lindebaum D, Fleming P (2024) ChatGPT undermines human reflexivity, scientific responsibility and responsible management research. Br J Manag 35(2):566-575
  39. Lê JK, Schmid T (2022) The practice of innovating research methods. Organizational Res Methods 25(2):308-336
  40. Locke K, Feldman M, Golden-Biddle K (2022) Coding practices and iterativity: beyond templates for analyzing qualitative data. Organizational Res Methods 25(2):262-284
  41. Lu Y (2019) Artificial intelligence: A survey on evolution, models, applications and future trends. J Man- age Analytics 6(1):1-29
  42. Maaravi Y, Heller B, Shoham Y, Mohar S, Deutsch B (2020) Ideation in the digital age: literature review and integrative model for electronic brainstorming. RMS 14(3):561-591
  43. Maaravi Y, Heller B, Shoham Y, Mohar S, Deutsch B (2021) Ideation in the digital age: literature review and integrative model for electronic brainstorming. Rev Manag Sci 15:1431-1464.
  44. Marchionini G (2008) Human-information interaction research and development. Libr Inform Sci Res 30(3):165-174
  45. Mariani MM, Machado I, Magrelli V, Dwivedi YK (2023) Artificial intelligence in innovation research: A systematic review, conceptual framework, and future research directions. Technovation 122:102623
  46. Mariani M, Wirtz J (2023) A critical reflection on analytics and artificial intelligence based analytics in hospitality and tourism management research. Int J Contemp Hospitality Manage 35(8):2929-2943
  47. Mees-Buss J, Welch C, Piekkari R (2022) From templates to heuristics: how and why to move beyond the Gioia methodology. Organizational Res Methods 25(2):405-429
  48. Mollick E (2024) Co-intelligence: living and working with AI. Penguin Portfolio
  49. Najafi B, Najafi A, Farahmandian A (2023) The impact of artificial intelligence and blockchain on six sigma: a systematic literature review of the evidence and implications. IEEETrans Eng Manag 71:10261-10294.
  50. Nguyen N-T, Nguyen TV, Kashani A, Ngo T, Bordas SPA (2019) Deep neural network with high- order neuron for the prediction of foamed concrete strength. Computer-Aided Civ Infrastruct Eng 34(4):316-332
  51. Ngwenyama O, Rowe F (2024) Should we collaborate with AI to conduct literature reviews? Changing epistemic values in a flattening world. J Association Inform Syst 25(1):122-136
  52. Okoli C, Schabram K (2010) A guide to conducting a systematic literature review of information systems research. Sprouts: Working Papers Inform Syst 10(26):1-49
  53. Pan SL, Nishant R, Tuunanen T, Nah FF-H (2023) Literature review in the generative AI era-How to make a compelling contribution. J Strategic Inform Syst 32(3):4
  54. Porsdam Mann S, Earp BD, Nyholm S, Danaher J, Møller N, Bowman-Smart H et al (2023) Generative AI entails a credit-blame asymmetry. Nat Mach Intell 5(5):472-475
  55. Pratt MG (2008) Fitting oval pegs into round holes: tensions in evaluating and publishing qualitative research in top-tier North American journals. Organizational Res Methods 11(3):481-509
  56. Pratt MG, Sonenshein S, Feldman MS (2022) Moving beyond templates: A bricolage approach to conduct- ing trustworthy qualitative research. Organizational Res Methods 25(2):211-238
  57. Prikshat V, Islam M, Patel P, Malik A, Budhwar P, Gupta S (2023) AI-Augmented HRM: literature review and a proposed multilevel framework for future research. Technol Forecast Soc Chang 193:122645
  58. Najafi B, Najafi A, Farahmandian A (2023) The impact of artificial intelligence and blockchain on six Sigma: A systematic literature review of the evidence 71:10261-10294.
  59. Rammal HG (2023) Systematic literature reviews: steps and practical tips. Advancing methodologies of conducting literature review in management research. Emerald Publishing Limited, pp 27-35
  60. Redondo-Rodríguez MD, Díaz-Garrido E, Pérez-Bustamante Yábar DC, Ramón-Jerónimo M (2024) Á. Entrepreneurship and artificial intelligence: a bibliometric analysis. The Journal of Technology Transfer, 1-33
  61. Reyes-Menendez A, Clemente-Mediavilla J, Villagra N (2023) Understanding STI and SDG with artificial intelligence: A review and research agenda for entrepreneurial action. Technol Forecast Soc Chang 196:122785
  62. Russell SJ, Norvig P (2020) Artificial Intelligence: A Modern Approach (4th ed.).
  63. Sauer PC, Seuring S (2023) How to conduct systematic literature reviews in management research: A guide in 6 steps and 14 decisions. RMS 17(1):1-35
  64. Saura JR, Škare V, Dosen DO (2024) Is AI-based digital marketing ethical? Assessing a new data privacy paradox. J Innov Knowl 9(4):100597
  65. Schlagwein D, Willcocks L (2023) ChatGPT et al.': the ethics of using (generative) artificial intelligence in research and science. J Inform Technol 38(3):232-238
  66. Schryen G, Marrone M, Yang J (2024, January) Adopting generative AI for literature reviews: an episte- mological perspective. In HICSS (pp. 4020-4029)
  67. Schryen G, Wagner G, Benlian A (2015) Theory of knowledge for literature reviews: an epistemological model, taxonomy and empirical analysis of IS literature. In ICIS 2015 Proceedings, 2015
  68. Shahzadi G, Jia F, Chen L, John A (2024) AI adoption in supply chain management: a systematic literature review. J Manuf Technol Manage 35(6):1125-1150
  69. Sharma G, Bansal P (2023) Partnering up: including managers as research partners in systematic reviews. Organizational Res Methods 26(2):262-291
  70. Shen Y, Heacock L, Elias J, Hentel KD, Reig B, Shih G, Moy L (2023) ChatGPT and other large Language models are double-edged swords. Radiology, 307(2), e230163
  71. Simsek Z, Fox B, Heavey C (2023) Systematicity in organizational research literature reviews: A frame- work and assessment. Organizational Res Methods 26(2):292-321
  72. Snyder H (2019) Literature review as a research methodology: an overview and guidelines. J Bus Res 104:333-339
  73. Sumbal A, Sumbal R, Amir A (2024) Can ChatGPT-3.5 pass a medical exam? A systematic review of ChatGPT's performance in academic testing. J Med Educ Curriculum Dev 11:23821205241238640
  74. Tapeh TG, A., and, Naser MZ (2022) Artificial intelligence, machine learning, and deep learning in struc- tural engineering: A scientometrics review of trends and best practices. Arch Comput Methods Eng 29(6):3751-3796
  75. Tingelhoff F, Brugger M, Leimeister JM (2024) A guide for structured literature reviews in business research: the state-of-the-art and how to integrate generative artificial intelligence. J Inform Technol 39(4):489-505
  76. Tranfield D, Denyer D, Smart P (2003) Toward a methodology for developing evidence-informed manage- ment knowledge by means of systematic review. Br J Manag 14(3):207-222
  77. Villiger J, Schweiger SA, Baldauf A (2022) Making the invisible visible: guidelines for the coding process in meta-analyses. Organizational Res Methods 25(4):716-740
  78. von Krogh G, Roberson Q, Gruber M (2023) Recognizing and utilizing novel research opportunities with artificial intelligence. Acad Manag J 66(2):367-373
  79. Webster J, Watson RT (2002) Analyzing the past to prepare for the future: writing a literature review. MIS Q 26(2):xiii-xxiii
  80. Williams RI Jr, Clark LA, Clark WR, Raffo DM (2021) Re-examining systematic literature review in man- agement research: additional benefits and execution protocols. Eur Manag J 39(4):521-533
  81. Wittenborg T, Karras O, Auer S (2024) SWARM-SLR: Streamlined workflow automation for machine- actionable systematic literature reviews. arXiv preprint arXiv:2407.18657. · arXiv
  82. Wu S, Fung M, Qian C, Kim J, Hakkani-Tur D, Ji H (2024) Aligning LLMs with Individual Preferences via Interaction. arXiv preprint arXiv:2410.03642 · arXiv
  83. Wu T, He S, Liu J, Sun S, Liu K, Han QL, Tang Y (2023) A brief overview of ChatGPT: the history, status quo and potential future development. IEEE/CAA J Automatica Sinica 10(5):1122-1136
  84. Zagzebski LT (2020) Must knowers be agents? In Epistemic values: Collected papers in epistemology (online ed.). Oxford Academic
  85. Zupic I, Čater T (2015) Bibliometric methods in management and organization. Organizational Res Meth- ods 18(3):429-472

Cited by

Sorted by citations

Authors
Malik, F. S., & Terzidis, O.
Affiliations
Karlsruhe Institute of Technology (DE)
Corresponding author
Malik, Faisal Saeed
ORCID
Venue
Management Review Quarterly
Publisher
Springer Science and Business Media LLC
Published
2025-04-29
Volume
76
Issue
2
Pages
2031-2057
ISSN
  • 2198-1620 (ISSN-L)
  • 2198-1639
DOI
10.1007/s11301-025-00522-8
OpenAlex
W4409920855
Cited by
28
References
16
Citations by year
2026: 20 · 2025: 8
FWCI
5.39
Type
journal article
Language
en
Open access
Licence
Locations
  • Management Review Quarterly (publishedVersion, cc-by)
  • Repository KITopen (Karlsruhe Institute of Technology) (submittedVersion, cc-by)
  • Econstor (Econstor) (submittedVersion, cc-by)
  • KITopen (cc-by)
Funders
Karlsruher Institut für Technologie (KIT)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Diverse Interdisciplinary Research Innovations
Keywords
systematic reviews, generative AI, large language models, prompt engineering, knowledge base curation, human oversight, reproducibility, bias mitigation, model intercomparison, sensitivity experiments, AI ethics in research, research methodology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review