CoChat
Sign Up
Cited by 25Publishing and Scholarly CommunicationAcademic Writing and PublishingAcademic Publishing and Open Access

Democratizing Knowledge Creation Through Human-AI Collaboration in Academic Peer Review

Open access
Sarker, Suprateek, University of Virginia, Susarla, Anjana, Michigan State University, Gopal, Ram, University of Warwick, Thatcher, Jason Bennett, University of Colorado / University of Manchester·2024·Journal of the Association for Information Systems
DOI 10.17705/1jais.00872
Open this paper in CoChat — free accountOpen PDF

Abstract

In the rapidly evolving landscape of academic research, artificial intelligence (AI) is poised to revolutionize traditional academic peer review processes and knowledge evaluation systems. We believe that the growing collaboration between humans and AI will disrupt how academics assess scholarly manuscripts and disseminate published works in a way that facilitates the closing of gaps among diverse scholars as well as competing scholarly traditions. Such human-AI collaboration is not a distant reality but is unfolding before us, in part, through the development, application, and actual use of AI, including language learning models (LLMs). This opinion piece focuses on the academic peer review process. It offers preliminary ideas on how human-AI collaboration will likely change the peer review process, highlights the benefits, identifies possible bottlenecks, and underscores the potential for democratizing academic culture worldwide.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.17705/1jais.00872

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Current studies mainly investigate either generic AI reviewers or AI-human collaboration.

    “Current studies mainly investigate either generic AI reviewers or AI-human collaboration (Sarker et al. 2024;Chen et al. 2025;Ebadi et al. 2025).”

    From Who Reviews the Reviewer? A Multi‐Agent LLM Architecture With Meta‐Review Synthesis for Editorial Peer Review (2026) · cites this paper as Sarker et al. 2024

    The paper is an example of work on human–AI collaboration in peer review: the Values of the Academic Peer Review Process section outlines scenarios for that collaboration. It does not establish that current studies mainly investigate the two categories named in the claim, or characterize generic AI reviewers as a principal research focus. The supplied copy gives no page numbers for these passages. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Researchers increasingly perceive AI as an automation tool and as a technology capable of reshaping academic evaluation, editorial workflows and reviewer support systems.

    “Researchers increasingly perceive AI as an automation tool and as a technology capable of reshaping academic evaluation, editorial workflows and reviewer support systems”

    From Who Reviews the Reviewer? A Multi‐Agent LLM Architecture With Meta‐Review Synthesis for Editorial Peer Review (2026) · cites this paper as Sarker et al. 2024

  • This finding is consistent with research emphasizing collaborative human-AI review models and with evidence that AI-assisted reviewing may influence review scores and reproduce existing biases.

    “This finding is consistent with research emphasizing collaborative human-AI review models (Sarker et al. 2024;Ebadi et al. 2025) and with evidence that AIassisted reviewing may influence review scores and reproduce existing biases (Russo et al. 2025;Vasu et al. 2025).”

    From Who Reviews the Reviewer? A Multi‐Agent LLM Architecture With Meta‐Review Synthesis for Editorial Peer Review (2026) · cites this paper as Sarker et al. 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 39 · Cited by 25

Research works printed but not linked: 27

  1. Acemoglu, D. (2021). Harms of AI (NBER Working Paper No. 29247). https://doi.org/10.3386/ w29247 · DOI
  2. Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., Teevan, J., Kikin-Gil, R., & Horvitz, E. (2019). Guidelines for human-AI interaction. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300233 · DOI · Paper page
  3. Basson, I., Ni, C., Badia, G., Tufenkji, N., Sugimoto, C. R., & Larivière, V. (2023). Gender differences in submission behavior exacerbate publication disparities in elite journals. bioRxiv. at https://www.biorxiv.org/content/ 10.1101/2023.08.21.554192v1 Bauersfeld, L, Romero, A., Muglikar, M., & Scaramuzza, D. (2023). Cracking double-blind review: Authorship attribution with deep learning. PLOS ONE, 18(6), Article e0287611. · DOI
  4. Beane, M. (2019). Shadow learning: Building robotic surgical skill when approved means fail. Administrative Science Quarterly, 64(1), 87- 123.
  5. Checco, A., Bracciale, L., Loreti, P., Pinfield, S., & Bianchini, C. (2021). AI-assisted peer review. Humanities and Social Sciences Communications, 8, Article 25.
  6. Csiszar, A. (2016). Peer review: Troubled from the start. Nature, 532(7599), 306-308.
  7. Dignum, V. (2019). Responsible artificial intelligence: How to develop and use AI in a responsible way. Springer.
  8. Fugener, A., Grahl, J., Gupta, A., & Ketter, W. (2021). Cognitive challenges in human-artificial intelligence collaboration: Investigating the path toward productive delegation. Information Systems Research, 33(2), 678-696.
  9. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM Computing Surveys, 51(5), Article 93.
  10. Jain, H., Padmanabhan, B., Pavlou, P. A., & Raghu, T. S. (2021). Editorial for the special section on humans, algorithms, and augmented intelligence: The future of work, organizations, and society. Information Systems Research, 32(3), 675-687. https://doi.org/10.1287/isre. 2021.1046 · DOI
  11. Joshi, K. D., Sarker, S., & Sarker, S. (2007). Knowledge transfer among face-to-face information systems development team members. Decision Support Systems, 43(2), 322-335.
  12. Jussupow, E., Spohrer, K., Heinzl, A., & Gawlitza, J. (2021). Augmenting medical diagnosis decisions? An investigation into physicians' decision-making process with artificial intelligence. Information Systems Research, 32(3), 713-735.
  13. Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2022). Trustworthy artificial intelligence: A review. ACM Computing Surveys, 55(2), Article 39.
  14. Kogut, B., & Zander, U. (1992). Knowledge of the firm, combinative capabilities, and the replication of technology. Organization Science, 3(3), 383-397.
  15. Lamont, M. (2009). How professors think: Inside the curious world of academic judgment. Harvard University Press.
  16. Lebovitz, S., Levina, N., & Lifshitz-Assaf, H. (2021). Is AI ground truth really "true"? The dangers of training and evaluating AI tools based on experts' know-what. MIS Quarterly, 45(3), 1501-1525.
  17. Lipton, Z. C. (2018). The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3), 31-57.
  18. Newell, A., & Simon, H. A. (1956). The logic theory machine: A complex information processing system. IRE Transactions on information theory, 2(3), 61-79.
  19. Palmer, J. W., Speier, C., Wren, D., & Hahn, S. (2000). Electronic journals in business schools: Legitimacy, acceptance, and use. Journal of the Association for Information Systems, 1, Article
  20. Rawls, J. (1993). Political liberalism. Columbia University Press.
  21. Sahaym, A., Vithayathil, J., Sarker, S., & Bjørn- Andersen, N. (2023). Value destruction in information technology ecosystems: A mixed- method investigation with interpretive case study and analytical modeling. Information Systems Research, 34(2), 508-531.
  22. Sarker, S., Agarwal, R., Goes, P., Gregor, S., Henfridsson, O., Saunders, C., & Tan, B. (2015). Roles and responsibilities of a senior editor. Journal of the Association for Information Systems, 16(9), 707-728.
  23. Saunders, C. (2005). Editor's comments: Looking for diamond cutters. MIS Quarterly, 29(1), iii-viii. https://doi.org/10.2307/25148664 Scholarly Peer Review. (2023, October 25). In Wikipedia. https://en.wikipedia.org/wiki/ · DOI
  24. Simon, H. A. (2019). The sciences of the artificial (3rd ed.). MIT Press.
  25. Susarla, A., Gopal, R. D., Thatcher, J. B., & Sarker, S. (2023). The Janus effect of generative AI: Charting the path for responsible conduct of scholarly activities in information systems. Information Systems Research. Advance online publication. https://doi.org/10.1287/isre.2023. ed.v34.n2 · DOI
  26. Teye, M. T., Missah, Y. M., Ahene, E., & Frimpong, T. (2022). Evaluation of conversational agents: understanding culture, context and environment in emotion detection. IEEE Access, 10, 24976- 24984.
  27. Wanderley, S., Alcadipani, R., & Barros, A. (2021). Recentering the Global South in the Making of Business School Histories: Dependency Ambiguity in Action. Academy of Management Learning & Education, 20(3), 361-381.

Authors
Sarker, S., University of Virginia, Susarla, A., Michigan State University, Gopal, R., University of Warwick, Thatcher, J. B., & University of Colorado / University of Manchester
Affiliations
  • University of Virginia (US)
  • Michigan State University (US)
  • University of Warwick (GB)
  • University of Colorado System (US)
ORCID
Venue
Journal of the Association for Information Systems
Publisher
Association for Information Systems
Published
2024
Volume
25
Issue
1
Pages
158-171
ISSN
1536-9323 (ISSN-L)
DOI
10.17705/1jais.00872
OpenAlex
W4390491415
Cited by
25
References
39
Citations by year
2026: 12 · 2025: 9 · 2024: 4
FWCI
37.19
Type
journal article
Language
en
Open access
Locations
  • Journal of the Association for Information Systems (publishedVersion)
  • Journal of the Association for Information Systems (submittedVersion)
Funders
National Science Foundation
Topics
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Publishing and Scholarly Communication
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
  • Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
Keywords
academic peer review, human-AI collaboration, large language models, democratization of knowledge, scholarly publishing, academic tradition, knowledge assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review