Artificial Intelligence to Automate Network Meta-Analyses: Four Case Studies to Evaluate the Potential Application of Large Language Models
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index; 7 of the 18 sources its claims cite were checked for support.
Abstract
BACKGROUND: The emergence of artificial intelligence, capable of human-level performance on some tasks, presents an opportunity to revolutionise development of systematic reviews and network meta-analyses (NMAs). In this pilot study, we aim to assess use of a large-language model (LLM, Generative Pre-trained Transformer 4 [GPT-4]) to automatically extract data from publications, write an R script to conduct an NMA and interpret the results. METHODS: We considered four case studies involving binary and time-to-event outcomes in two disease areas, for which an NMA had previously been conducted manually. For each case study, a Python script was developed that communicated with the LLM via application programming interface (API) calls. The LLM was prompted to extract relevant data from publications, to create an R script to be used to run the NMA and then to produce a small report describing the analysis. RESULTS: The LLM had a > 99% success rate of accurately extracting data across 20 runs for each case study and could generate R scripts that could be run end-to-end without human input. It also produced good quality reports describing the disease area, analysis conducted, results obtained and a correct interpretation of the results. CONCLUSIONS: This study provides a promising indication of the feasibility of using current generation LLMs to automate data extraction, code generation and NMA result interpretation, which could result in significant time savings and reduce human error. This is provided that routine technical checks are performed, as recommend for human-conducted analyses. Whilst not currently 100% consistent, LLMs are likely to improve with time.
Overview
CoChat Summary
TL;DR
This pilot study assessed GPT-4 for automating data extraction, R script generation, and result interpretation for network meta-analyses across four case studies. It found >99% data extraction success and R scripts that often ran with no or minor human input, with NMA results closely matching manual analyses.
Key findings
- Across 20 runs per case study, LLM data extraction success was >99%: case study 1 had no extraction failures; case study 2 was 99.1% (35 items/run); case study 3 was 99.8% (77 items/run); case study 4 was 99.7% (35 items/run). The LLM did not report incorrect data, only intermittently failed to extract data from two trials.
- R scripts ran with no or very minor amendments in all 20 runs for case studies 1 and 2, 15 runs for case study 3, and 19 runs for case study 4. All errors caused the script to fail with an error message; no script ran and generated erroneous results.
- NMA results closely matched manual analyses: e.g., hidradenitis suppurativa adalimumab odds ratio 2.84 [2.06, 3.97] manual vs 2.84 [2.06, 3.90] GPT-4; NSCLC overall survival atezolizumab hazard ratio 0.78 [0.69, 0.88] for both. Differences were within expected variability (<1%) due to Monte Carlo error.
- Reports were clearly written, and interpretation was correct in all 20 runs for each case study, with the correct best treatment and statistical significance correctly stated.
Methods
- Four case studies used previously manually conducted NMAs: hidradenitis suppurativa clinical response (6 trials) and non-small cell lung cancer overall survival base case (5 trials), overall survival sensitivity analysis (12 trials), and progression-free survival (5 trials).
- A Python script communicated with GPT-4 via API calls to extract data from publications, generate an R script using the ‘multinma’ package and Stan, and produce a short report. Temperature was set to 0, and GPT-4 was assigned the role of statistician and medical researcher.
- Each Python script was run end to end 20 times per analysis (80 runs total). Performance was assessed by comparing extracted data with human-checked data, evaluating R script runnability and correctness against manual NMAs, and qualitatively assessing the generated reports.
Limitations
- The paper states the approach was tested on a single LLM (GPT-4) and a limited number of case studies; treatment networks were relatively simple, all trials included a common treatment, and the proportional hazards assumption was not checked for time-to-event analyses.
- The paper states LLM responses are not always consistent or reproducible, token limits required chunking publications, image-based publications could not be processed by GPT-4 at the time, data extraction from tables was out of scope, and the approach is not yet robust enough for HTA methodological rigour and lacks heterogeneity assessment.
- Evident from the text: manual feasibility and heterogeneity assessments were conducted before automation, so full NMA automation was not tested; some R script errors required domain knowledge and could not be fixed without prior network knowledge (five scripts in case study 3); and the generated report was not sufficiently detailed for an HTA appraisal.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
6 Valid · 3 Not valid · 7 Not checked · 75 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Despite these quality-control measures, the processes can still be prone to error [10].”
§ Artificial Intelligence to Automate Network Meta-Analyses: Four Case Studies to Evaluate the Potential Application of Large Language Models > Introduction
What the sources say
- Partly supportedcites [10] Mathes T, Klaßen P, Pieper D. Frequency of data extraction errors and methods to increase data extraction quality: a methodological review. BMC Med Res Methodol. 2017;17(1):152. https://doi.org/10.1186/s12874-017-0431-4. sourceExperimental — accuracy not yet measured
The Results report data-extraction errors in systematic reviews, and the Discussion says errors remain prevalent while comparing methods that use a second reviewer for quality assurance. This supports the point that a quality-controlled review step can still be error-prone, but the paper does not establish that HTA processes generally are prone to error. Page numbers were not available in our copy. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Whilst some studies have considered use of automation or AI within HEOR [12, 13, 16], we believe that this is the first study to consider the use of an LLM to automate data extraction and NMA end to end.”
§ Artificial Intelligence to Automate Network Meta-Analyses: Four Case Studies to Evaluate the Potential Application of Large Language Models > Discussion
What the sources say
- Not checked yetcites [12] Rueda JD, Cristancho RA, Slejko JF. Is artificial intelligence the next big thing in health economics and outcomes research? Value Outcomes Spotlight Magazine. 2019:22–4. https://www.ispor.org/docs/default-source/publications/value-outcomes-spotlight/march-april-2019/vos-heor-articles---rueda.pdf?sfvrsn=18cb16f5_0. Accessed 05 Jan 2024.
- Supportedcites [13] van Dinter R, Tekinerdogan B, Catal C. Automation of systematic literature reviews: a systematic literature review. Inf Softw Technol. 2021;136: 106589. https://doi.org/10.1016/j.infsof.2021.106589. source
The citation supports the preceding clause about automation or AI in health-related research, not the citing authors’ own assertion that their LLM study is the first to automate data extraction and NMA end to end. The source’s abstract (p. 2) describes a review of machine-learning and NLP approaches to automating systematic reviews, and its application-domains section (p. 7) reports that 60% of the evaluated tools used medical-review data. (LLM, conf=0.87)
- Not checked yetcites [16] Poldrack RA, Lu T, Beguš G. AI-assisted coding: experiments with GPT-4. 2023. https://arxiv.org/abs/2304.13187. Accessed 05 Jan 2024.
hedgedn = 4- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This approach worked well for the binary outcome, where the data required for the analysis in R is number at risk and number of patients affected in each arm.”
§ Artificial Intelligence to Automate Network Meta-Analyses: Four Case Studies to Evaluate the Potential Application of Large Language Models > Methods > Prompt Development and Key Learnings
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the analysis input, different treatments were given numbers in the R script (Fig. S7 [Online Resource]) but the LLM did not always use the same numbering for the same treatment.”
§ Artificial Intelligence to Automate Network Meta-Analyses: Four Case Studies to Evaluate the Potential Application of Large Language Models > Methods > Prompt Development and Key Learnings
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The LLM had a > 99% success rate of accurately extracting data across 20 runs for each case study and could generate R scripts that could be run end-to-end without human input.”
§ Artificial Intelligence to Automate Network Meta-Analyses: Four Case Studies to Evaluate the Potential Application of Large Language Models > Abstract > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
38 more claims
Create a free account to see all 91 claims and to verify the 11 references still open.
- It may state two separate claims as one. 29
- Probably not a substantial claim. 25
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs can automate critical tasks for SLRs, including screening abstracts and full texts, assessing risk of bias and extracting data for use in network meta-analyses with high accuracy, all while significantly reducing the time and costs required by using human review [2, 4–7].”
From Automated Mass Extraction of Over 680,000 PICOs from Clinical Study Abstracts Using Generative AI: A Proof-of-Concept Study. (2024) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“In our experience, extracting and summarising information that is already present in text is a task that LLMs excel at [7].”
From Automated Mass Extraction of Over 680,000 PICOs from Clinical Study Abstracts Using Generative AI: A Proof-of-Concept Study. (2024) · cites this paper as [7]
Citations
Cites
PD-1 and PD-L1 Immune Checkpoint Blockade to Treat Breast Cancer
Hartkopf, A. D., Taran, F.-A., Wallwiener, M., Walter, C. B., Krämer, B., Grischke, E.-M., & Brucker, S. Y. (2016) · Breast Care · doi:10.1159/000453569 · Cited by 30,845
Adverse Renal Effects of Immune Checkpoint Inhibitors: A Narrative Review
Wanchoo, R., Karam, S., Uppal, N. N., Barta, V. S., Deray, G., Devoe, C., Launay-Vacher, V., Jhaveri, K. D., & on behalf of Cancer and Kidney International Network Workgroup on Immune Checkpoint Inhibitors (2017) · American Journal of Nephrology · doi:10.1159/000455014 · Cited by 15,256
Stan : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754 · cited 1×
“Models were estimated in a Bayesian framework using Stan [20].”
MethodHerbst, R. S., Baas, P., Kim, D.-W., Felip, E., Pérez-Gracia, J. L., Han, J.-Y., Molina, J., Kim, J.-H., Arvis, C. D., Ahn, M.-J., et al. (2016) · The Lancet · doi:10.1016/s0140-6736(15)01281-7 · Cited by 6,449 · cited 1×
“3a. Photograph of poster GPT-4 was asked to extract all data from an older publication [33] and then was provided with the updated hazard ratio (hard-coded into the Python script), before R script generation Case study 3 (NSCLC OS sensitivity) GFPC 05-06 [34] Hazard ratio provided in a figure GPT-4…”
MethodChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · cited 1×
“Including worked examples has previously been shown to improve the performance of LLMs in multi-step reasoning tasks [21].”
MethodLlama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · cited 1×
“Alternatively, using open-source LLMs, for example through the ‘Ollama’ package [26, 27], would allow tighter LLM version control and allow better reproducibility of responses.”
MethodRay, P. P. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.04.003 · Cited by 2,463 · cited 1×
“The latest LLMs, including GPT-4 [14], have been trained using vast quantities of publicly available text data to understand, generate and manipulate human language by recognising patterns and relationships in language [15].”
MethodEvidence summaries: the evolution of a rapid review approach
Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-10 · Cited by 1,232 · cited 1×
“In addition, the HTA process is time-consuming, generally taking over a year to complete, with the SLR and NMA part of this process taking several months each [8, 11].”
Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0468-4 · Cited by 1,164
Research works printed but not linked: 1
- McKenzie J, Brennan S, Ryan R, Thomson H, Johnston R. Chapter 9: summarizing study characteristics and preparing for synthesis in Cochrane handbook for systematic reviews of interventions version 64. www.training.cochrane.org/handbook. Accessed 05 Jan 2024.
Cited by
Scherbakov, D., Hubig, N., Jansari, V., Bakumenko, A., & Lenert, L. A. (2024) · arXiv · doi:10.48550/arxiv.2409.04600 · Cited by 156
Scherbakov, D., Hubig, N., Jansari, V., Bakumenko, A., & Lenert, L. A. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf063 · Cited by 154
Data extraction methods for systematic review (semi)automation: a living systematic review
Schmidt, L., Olorisade, B. K., McGuinness, L. A., Thomas, J., & Higgins, J. P. T. (2021) · F1000Research · doi:10.12688/f1000research.51117.1 · Cited by 106
Collaborative large language models for automated data extraction in living systematic reviews
Khan, M. A., Ayub, U., Naqvi, S. A. A., Khakwani, K. Z. R., Sipra, Z. B. R., Raina, A., Zhou, S., He, H., Saeidi, A., Hasan, B., et al. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae325 · Cited by 50
Fleurence, R. L., Bian, J., Wang, X., Xu, H., Dawoud, D., Higashi, M., & Chhatwal, J. (2025) · Value in Health · doi:10.1016/j.jval.2024.10.3846 · Cited by 54
Trad, F. S., Yammine, R., Charafeddine, J., Chakhtoura, M., Rahme, M., Fuleihan, G. E., & Chehab, A. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02583-5 · Cited by 62
“Reason et al. evaluated the potential of LLMs to automate tasks such as data extraction, script creation, and report generation within SRs [18].”
Large Language Models and Their Applications in Drug Discovery and Development: A Primer
Lu, J. J.-Q., Choi, K., Eremeev, M., Gobburu, J. V. S., Goswami, S., Liu, Q., Mo, G., Musante, C. J., & Shahin, M. H. (2025) · Clinical and Translational Science · doi:10.1111/cts.70205 · Cited by 56
Artificial intelligence across the cancer care continuum
Riaz, I. B., Khan, M. A., & Osterman, T. J. (2025) · Cancer · doi:10.1002/cncr.70050 · Cited by 40
Gartlehner, G., Kahwati, L. C., Nußbaumer-Streit, B., Crotty, K., Hilscher, R., Kugley, S., Viswanathan, M., Thomas, I., Konet, A., Booth, G., et al. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2024-113199 · Cited by 49
Data extraction methods for systematic review (semi)automation: Update of a living systematic review
Schmidt, L., Finnerty Mutlu, A. N., Elmore, R., Olorisade, B. K., Thomas, J., & Higgins, J. P. T. (2025) · F1000Research · doi:10.12688/f1000research.51117.3 · Cited by 27
Fleurence, R., Dawoud, D., Bian, J., Higashi, M. K., Wang, X., Xu, H., Chhatwal, J., & Ayer, T. (2025) · Value in Health · doi:10.1016/j.jval.2025.06.018 · Cited by 41
Reason, T., Klijn, S. L., Rawlinson, W. D., Benbow, E., Langham, J., Teitsson, S., Johannesen, K. M., & Malcolm, B. (2025) · PharmacoEconomics - Open · doi:10.1007/s41669-025-00580-4 · Cited by 27
Self-citationLee, K., Paek, H., Ofoegbu, N., Rube, S., Higashi, M. K., Dawoud, D., Xu, H., Shi, L., & Wang, X. (2025) · Value in Health · doi:10.1016/j.jval.2025.08.002 · Cited by 23
Potential application of artificial intelligence in cancer therapy
Riaz, I. B., Khan, M. A., & Haddad, T. C. (2024) · Current Opinion in Oncology · doi:10.1097/cco.0000000000001068 · Cited by 21
Li, L., Mathrani, A., & Sušnjak, T. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2025.10065 · Cited by 29
“Recently, [87] demonstrated that LLMs (GPT-4) can efficiently extract data for NMA, reducing manual effort and accelerating analysis.”
Rashid, M., Yi, C. S., Sathapanasiri, T., Udayachalerm, S., Boonpattharatthiti, K., Insuk, S., Veettil, S. K., Lai, N. M., Chaiyakunapruk, N., Dhippayom, T., et al. (2025) · Value in Health · doi:10.1016/j.jval.2025.07.001 · Cited by 26
Reason, T., Langham, J., & Gimblett, A. (2024) · Pharmaceutical Medicine · doi:10.1007/s40290-024-00539-6 · Cited by 16
“LLMs can auto-mate critical tasks for SLRs, including screening abstracts and full texts, assessing risk of bias and extracting data for use in network meta-analyses with high accuracy, all while significantly reducing the time and costs required by using human review [2, 4–7].”
BackgroundMethodSelf-citationHilkenmeier, F., Pelzer, M., Stierle, C. M. G., & Fink‐Lamotte, J. (2025) · Social Science Computer Review · doi:10.1177/08944393251404052 · Cited by 22
Rodgers, E. M., Baldanzi, S., Collins, M., Dowd, W. W., Feugere, L., Mottola, G., Vermandele, F., & Isaza, D. F. G. (2025) · Conservation Physiology · doi:10.1093/conphys/coaf052 · Cited by 18
Doleman, B., Jakobsen, J. C., Mathiesen, O., Cooper, N. J., Sutton, A. J., & Hardman, J. G. (2025) · British Journal of Anaesthesia · doi:10.1016/j.bja.2024.12.039 · Cited by 14
Peng, Z., Wu, X., Qin, Z., Doi, S. A. R., Furuya‐Kanamori, L., Hong, Y., Lin, L., Chu, H., Xu, C., & Liu, M. (2025) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2025.102192 · Cited by 16
Novel AI applications in systematic review: gPT-4 assisted data extraction, analysis, review of bias
Kim, J. K., Chua, M. E., Li, T. G., Rickard, M., & Lorenzo, A. J. (2025) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2024-113066 · Cited by 11
Fleurence, R., Wang, X., Bian, J., Higashi, M. K., Ayer, T., Xu, H., Dawoud, D., & Chhatwal, J. (2025) · Value in Health · doi:10.1016/j.jval.2025.04.2167 · Cited by 21
Schmidt, L., Campbell, F., Craig, D., Walker, V. R., Rooney, A. A., Schmitt, C., Härtung, T., & Thomas, J. (2025) · Evidence-Based Toxicology · doi:10.1080/2833373x.2025.2546791 · Cited by 13
Pharmacometrics in the Age of Large Language Models: A Vision of the Future
Tosca, E. M., Aiello, L., Carlo, A. D., & Magni, P. (2025) · Pharmaceutics · doi:10.3390/pharmaceutics17101274 · Cited by 13
“In addition, as also suggested by Lu et al. [7], LLMs could support MBMA tasks such as harmonizing heterogeneous study endpoints, standardizing terminology across datasets, and generating executable analysis code [67].”
Collaborative Large Language Models for Automated Data Extraction in Living Systematic Reviews
Khan, M. A., Ayub, U., Naqvi, S. A. A., Khakwani, K. Z. R., Sipra, Z. B. R., Raina, A., Zou, S., He, H., Hossein, S. A., Hasan, B., et al. (2024) · doi:10.1101/2024.09.20.24314108 · Cited by 54
“[37] However, their approach relied only on publication text for data extraction and extracted data for a few variables, limiting its broader applicability.”
MethodAccuracy of LLMs to retrieve numeric data for meta-analysis in dentistry
Caponio, V. C. A., Lorenzo‐Pouso, A. I., Magalhães, M. A. O., Ali, A. A., Adamo, D., Cirillo, N., López‐Pintor, R. M., & Musella, G. (2025) · Journal of Dentistry · doi:10.1016/j.jdent.2025.106245 · Cited by 13
Integration of Generative AI with Human Expertise in HEOR: A Hybrid Intelligence Framework
Srivastava, T. E., Irfan, H., Babiy, V. V., & Swami, S. (2025) · Advances in Therapy · doi:10.1007/s12325-025-03273-w · Cited by 10
Bandara, E., Shetty, S. S., Mukkamala, R., Rahman, A., Foytik, P. B., Liang, X., Zoysa, K. D., & Ng, W. K. (2024) · IEEE Cloud Summit · doi:10.1109/cloud-summit61220.2024.00012 · Cited by 8
“The fine-tuned Llama2 LLM run on Ollam [19].”
MethodRawlinson, W. D., Teitsson, S., Reason, T., Malcolm, B., Gimblett, A., & Klijn, S. L. (2025) · Value in Health · doi:10.1016/j.jval.2025.05.020 · Cited by 8
Self-citationRonquillo, J. G., Ye, J., Gorman, D. N., Lemeshow, A. R., & WATT, S. J. (2024) · JMIR Medical Informatics · doi:10.2196/64143 · Cited by 11
“To our knowledge, this is one of the first studies investigating LLMs to accelerate the literature screening process for cardiovascular drug development applications [3,4,6,7].”
BackgroundReason, T., Wu, Y., Jones, C., Benbow, E., Johannesen, K. M., & Malcolm, B. (2025) · Value in Health · doi:10.1016/j.jval.2025.08.001 · Cited by 7
Self-citationDigital Product Passports: A Framework for Developing Adaptable and Compliant Data Models
Gianvincenzi, M., Marconi, M., Mosconi, E. M., Pennino, D., & Tola, F. (2025) · Lecture notes in mechanical engineering · doi:10.1007/978-3-031-76597-1_20 · Cited by 8
de Araújo, P. R., de Rezende, J. A. M., Faria, D. R. M., & Gomes, O. D. S. M. (2026) · Sensors · doi:10.3390/s26020747 · Cited by 7
“These observations align with recent evidence on the use of LLMs in systematic reviews, which report high levels of accuracy, although they still rely on human supervision for bias mitigation and quality control [47,48].”
Custom GPTs to aid in compliance checking for reporting standards in academic publishing
Mohapatra, R., & Härtung, T. (2026) · ALTEX · doi:10.14573/altex.2601011 · Cited by 5
Bandara, E., Bouk, S. H., Shetty, S. S., Roy, S., Mukkamala, R., Rahman, A., Foytik, P. B., Liang, X., Ng, W. K., & Zoysa, K. D. (2025) · IEEE Consumer Communications and Networking Conference · doi:10.1109/ccnc54725.2025.10976116 · Cited by 9
“The fine-tuned LLama2 LLM operates within Ollama [12], as depicted in Fig.”
Bandara, E., Bouk, S. H., Shetty, S. S., Gore, R. J., Kompella, S., Mukkamala, R., Rahman, A., Foytik, P. B., Liang, X., Ng, W. K., et al. (2025) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc65282.2025.11059510 · Cited by 6
Bandara, E., Gore, R. J., Shetty, S., Mukkamala, R., Rhea, C. K., Samulski, B., Hass, A., Yarlagadda, A., Kaushik, S., Silva, M. D., et al. (2026) · Biomechanics · doi:10.3390/biomechanics6010023 · Cited by 3
Brini, S., & Leung, T. I. (2026) · Journal of Medical Internet Research · doi:10.2196/92132 · Cited by 8
“%) in extracting predefined data points from primary studies and have even generated the necessary R scripts for network meta-analysis [64].”
Bandara, E., Foytik, P. B., Shetty, S. S., Mukkamala, R., Rahman, A., Liang, X., Ng, W. K., & Zoysa, K. D. (2024) · IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc61673.2024.10733674 · Cited by 3
“The fine-tuned Llama2 LLM run on Ollam [16], Figure 2.”
Empowering Meta-Analysis: Leveraging Large Language Models for Scientific Synthesis
Ahad, J. I., Sultan, R. M., Kaikobad, A., Rahman, F. A., Amin, M. R., Mohammed, N., & Rahman, S. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825310 · Cited by 11
“…is to address this limitation by efficiently managing large contextual data and segmenting it into smaller chunks to facilitate effective fine-tuning of LLMs. (b) Enhance information retrieval: Prior research has demonstrated that fine-tuning LLMs enhances their data extraction capabilities [23].”
Lee, K., Datta, S., Paek, H., Rastegar-Mojarad, M., Huang, L., He, L., Wang, S., Wang, J., & Wang, X. (2024) · medRxiv · doi:10.1101/2024.07.03.24309897 · Cited by 6
“…the LLM system has been employed to automate the extraction of Population, Intervention, Comparator, and Outcome (PICO) elements 25,26 , the generation of evidence via data extraction 27,28 , and the conduction of the network meta-analyses using generated R scripts and extracted data elements 29 .”
MethodBandara, E., Bouk, S. H., Shetty, S. S., Gore, R. J., Kompella, S., Mukkamala, R., Rahman, A., Foytik, P. B., Liang, X., Ng, W. K., et al. (2025) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc65282.2025.11059647 · Cited by 7
Large language model enhanced framework for systematic reviews and meta-analyses
Shen, J., Luo, Z.-Y., Jia, D., Wang, S.-R., Sun, F., & Wu, J.-G. (2025) · BMJ digital health & AI · doi:10.1136/bmjdhai-2025-000017 · Cited by 6
Model Context Contracts - MCP-Enabled Framework to Integrate LLMs With Blockchain
Bandara, E., Shetty, S. S., Mukkamala, R., Gore, R. J., Rahman, A., Foytik, P. B., Bouk, S. H., Liang, X., Ng, W. K., & Zoysa, K. D. (2025) · International Conference on Blockchain Computing and Applications (BCCA) · doi:10.1109/bcca66705.2025.11229749 · Cited by 2
Slice-MCP — Fine-Tuned Llama-4 LLM and MCP Enabled 5G Network Slice Orchestration Platform
Bandara, E., Bouk, S. H., Mukkamala, R., Rahman, A., Liang, X., Gore, R. J., & Shetty, S. S. (2025) · IEEE Military Communications Conference (MILCOM) · doi:10.1109/milcom64451.2025.11310417 · Cited by 5
“This layer is powered by a fine-tuned Llama-4 LLM, integrated through platforms such as Ollama [11], and tailored specifically for the domain of 5G network slicing and orchestration [4].”
Park, Y., Zhang, H.-S., & Bai, S. W. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02431-w · Cited by 3
Gaind, N., Badin, M., Jacob, J., Haugli-Stephens, T., Ivkovic, L., Pourrahmat, M., Fazeli, M. S., & Ting, E. (2026) · Value in Health Regional Issues · doi:10.1016/j.vhri.2026.101658 · Cited by 4
Bertolaccini, L., Guarize, J., Diotti, C., Donghi, S., Casiraghi, M., Mazzella, A., & Spaggiari, L. (2024) · Frontiers in Oncology · doi:10.3389/fonc.2024.1450568 · Cited by 3
Bandara, E., Foytik, P. B., Shetty, S. S., & Hassanzadeh, A. (2024) · IEEE International Conference on Mobile Ad-Hoc and Sensor Systems (MASS) · doi:10.1109/mass62177.2024.00026 · Cited by 2
“The fine-tuned Llama2 LLM run on Ollam [16].”
MethodOff‐the‐Shelf Large Language Models for Guiding Pharmacoepidemiological Study Design
Ompad, G. D., Wurst, K., Ramcharran, D., Hviid, A. P., Bate, A., & Sessa, M. (2025) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.70039 · Cited by 2
Bandara, E., Bouk, S. H., Shetty, S. S., Mukkamala, R., Rahman, A., Foytik, P. B., Gore, R. J., Liang, X., Ng, W. K., & Zoysa, K. D. (2025) · International Conference on Blockchain Computing and Applications (BCCA) · doi:10.1109/bcca66705.2025.11229660 · Cited by 1
“The interaction with Meta’s Llama-3 LLM is facilitated by the LLM smart contract through Ollam’s LLM API.”
MetaMind: A multi-agent transformer-driven framework for automated network meta-analyses
Livieratos, A., Kudela, M., Zhao, Y., Chen, A.-S., Luo, X., Lin, J., Zhang, D., Dharmarajan, S. H., Tsiodras, S., Rudrapatna, V. A., et al. (2026) · PLoS ONE · doi:10.1371/journal.pone.0342895 · Cited by 6
“AI-driven automation could alleviate the burdens associated with traditional NMA workflows, making the process faster, more accurate, and scalable [1–4].”
InfluentialGonzález-González, E. J., Castilla‐Rodríguez, I., & Aday, J. (2026) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462326103754 · Cited by 2
Public English Teaching System based on Large Language Model Meta-Artificial Intelligence
Li, T., & Li, D. (2025) · International Conference on Intelligent Systems and Computational Networks (ICISCN) · doi:10.1109/iciscn64258.2025.10934277 · Cited by 1
Engineering of Inquiry: The “Transformation” of Social Science through Generative AIPreprint
Broska, D., & McFarland, D. A. (2025) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/uy4c7 · Cited by 2
How Artificial Intelligence Can Revolutionize Evidence-Based Health Care: A Critical Commentary
Tamimi, F., & Jasim, K. H. (2025) · JDR Clinical & Translational Research · doi:10.1177/23800844251321839 · Cited by 1
Replicating Health-Economic Simulation Models for Alzheimer’s Disease Using Artificial IntelligencePreprint
Gilson, F., Osstyn, S. L., & Handels, R. L. H. (2025) · medRxiv · doi:10.1101/2025.10.08.25337408 · Cited by 2
Xia, Z., Ye, J., Hu, B., Qiang, Q., & Debnath, R. (2024) · Research Square · doi:10.21203/rs.3.rs-5063165/v1 · Cited by 1
Algorithms for Evidence Synthesis in Clinical Domain
Bhattacharya, A., Revathi, R., & Venkatesu, B. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-0194-5.ch001
Nielsen, C. S., Soetikno, B. T., Pollreisz, A., & Ting, D. S. W. (2025) · Current Opinion in Ophthalmology · doi:10.1097/icu.0000000000001173
Network Meta-Analysis and Umbrella Review: Complementary, Not Competing, Tools of Evidence Synthesis
Nair, A. S., Mistry, T., & Garg, R. (2026) · Cureus · doi:10.7759/cureus.102123
“…Generative Pre-trained Transformer (GPT-4) demonstrated the ability to perform structured data extraction, generate executable R scripts for Bayesian NMAs, and provide preliminary interpretation of forest plots and heterogeneity outputs, albeit with the need for careful technical verification [12].”
Cheng, W., Zhu, H., & Lian, Y. (2026) · Value in Health · doi:10.1016/j.jval.2025.12.019
RetractedLi, X., Dai, J., & Pang, H. (2026) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2026.2670522 · Cited by 1
Zhou, F., Zhong, T., He, S., Yang, G., Ding, X., Tian, G., Xiang, S., Tan, S., Zhao, S., & Yan, Y. (2026) · Research Square · doi:10.21203/rs.3.rs-9766806/v1
Irrigation Scheduling Dataset Extraction Using LLM-Mined Research Evidence
Manoj, S., & Bajpai, P. (2026) · IEEE Conference on Technologies for Sustainability · doi:10.1109/sustech67720.2026.11536359
“Other studies have also successfully used LLMs to extract data to solve similar problems in other fields [23], [24].”
Agent-Based Automation of SLRs via Large Language Models
Nafil, K., Boubakri, M., & Yamani, A. (2026) · International Conference on Information Technology Based Higher Education and Training (ITHET) · doi:10.1109/ithet69978.2026.11584987
“…visual elements, while some projects present bibliometric analyses or integrate visualization tools and data analysis reports [3], [16], [17], [20]–[22], this system includes automatic LaTeX chart and graph generation capabilities (leveraging packages like pgfplots , TikZ , pgfgantt , etc.),…”
InfluentialBandara, E., Moghim, N., Bouk, S. H., Shetty, S., Gore, R., Mukkamala, R., Rahman, A., Liang, X., Keong, N. W., & Zoysa, K. D. (2026) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc69287.2026.11580008
“Once fine-tuned, these quantized models run seamlessly on Ollama [18], a deployment framework optimized for the operation of vision-language models.”
InfluentialAutomated Meta-Analysis of Transcranial Magnetic Stimulation Based on GraphRAG: A Methodology StudyPreprint
Zhong, Y., Jin, J., Wang, X., He, W., & Yin, T. (2026) · Research Square · doi:10.21203/rs.3.rs-10200494/v1
Shrestha, B., Wabina, R. S., Numthavaj, P., Tantrakul, V., McKay, G. J., Attia, J. R., & Thakkinstian, A. (2026) · Scientific Reports · doi:10.1038/s41598-026-58657-x
The Rebirth of Ecological StudyPreprint
Ma, H. (2026) · Qeios · doi:10.32388/fbo0c7
The Rebirth of Ecological StudyPreprint
Ma, H. (2026) · Qeios · doi:10.32388/fbo0c7.2
Samaan, S., Devi, J., Vincent, M. L., Coombs, S. S., Sehgal, P., Mohamed, M. F. H., Rai, V., Johnson, A. M., Yarur, A. J., Barnes, E. L., et al. (2026) · doi:10.2196/preprints.107732 · Cited by 1
ENTGPT : Applying Large Language Models to Systematic Review Screening With the Novel STARR Protocol
Kapoor, A., Baranker, B., Alter, I. L., Bomback, M. J., Hamiter, M. J., & Lalwani, A. K. (2026) · The Laryngoscope · doi:10.1002/lary.70801 · Cited by 2
Polymenakos, K., Korasidis, N., & Teperikidis, L. (2026) · Research Square · doi:10.2196/preprints.111132
Adam, G. P., Davies, M. C., Caputo, E. L., Mai, H. J., Coppola, E. L., Holmer, H. K., Kuhn, E., Balk, E. M., Trikalinos, T. A., Ivlev, I., et al. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10118
Srivastava, T., Irfan, H., & Swami, S. (2026) · PharmacoEconomics - Open · doi:10.1007/s41669-026-00687-2
Başer, O. (2026) · Journal of health economics and outcomes research · doi:10.36469/001c.171937
Başer, O. (2026) · Journal of health economics and outcomes research · doi:10.36469/jheor.2026.171937
Pietrancosta, M., Wang, S., Wu, Y., Brulotte, V., Harutyunyan, R., Hemmerling, T. M., Morisson, L., & Laferrière-Langlois, P. (2026) · Research Square · doi:10.21203/rs.3.rs-10879359/v1
Shah, S. M. H., Hopfgartner, F., & Bleier, A. (2026) · Companion Proceedings of the ACM Web Conference 2026 · doi:10.1145/3774905.3795485
Metadata
- Authors
- Reason, T., Benbow, E., Langham, J., Gimblett, A., Klijn, S. L., & Malcolm, B.
- Affiliations
- Bristol-Myers Squibb (United States) (US)
- Bristol-Myers Squibb (United Kingdom) (GB)
- Corresponding author
- Reason, Tim
- Venue
- PharmacoEconomics - Open
- Published
- 2024-02-10
- Volume
- 8
- Issue
- 2
- Pages
- 205-220
- ISSN
- 2509-4254 (ISSN-L)
- 2509-4262
- PMC
- PMC10884375
- OpenAlex
- W4391723759
- Semantic Scholar
- 194e9a28e3b8edf5bc0c6a178a1173482ca1a2dc
- Cited by
- 71
- References
- 20
- Citations by year
- 2026: 23 · 2025: 36 · 2024: 12
- FWCI
- 19.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- creativecommons.org/licenses/by-nc/4.0 (tdm)
- creativecommons.org/licenses/by-nc/4.0 (vor)
- Locations
- PharmacoEconomics - Open (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- Bristol-Myers Squibb
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- network meta-analysis, large language models, GPT-4, automated data extraction, code generation, R code, systematic reviews, time-to-event outcomes, binary outcomes, report generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.