Implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: a qualitative study
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Objective: Many options are currently available for sepsis surveillance clinical decision support (CDS) from electronic medical record (EMR) vendors, third party, and homegrown models drawing on rule-based (RB) and machine learning (ML) algorithms. This study explores sepsis CDS implementation from the perspective of implementation leads by describing the motivations, tool choices, and implementation experiences of a diverse group of implementers. Materials and Methods: Semi-structured interviews were conducted with and a questionnaire was administered to 21 hospital leaders overseeing CDS implementation at 15 US medical centers. Participants were recruited via convenience sampling. Responses were coded by 2 coders with consensus approach and inductively analyzed for themes. Results: Use of sepsis CDS is motivated in part by quality metrics for sepsis patients. Choice of tool is driven by ease of integration, customization capability, and perceived predictive potential. Implementation processes for these CDS tools are complex, time-consuming, interdisciplinary undertakings resulting in heterogeneous choice of tools and workflow integration. To improve clinician acceptance, implementers addressed both optimization of the alerts as well as clinician understanding and buy in. More distrust and confusion was reported for ML models, as compared to RB models. Respondents described a variety of approaches to overcome implementation barriers; these approaches related to alert firing, content, integration, and buy-in. Discussion: While there are shared socio-technical challenges of implementing CDS for both RB and ML models, attention to user education, support, expectation management, and dissemination of effective practices may improve feasibility and effectiveness of ML models in quality improvement efforts. Conclusion: Further implementation science research is needed to determine real world efficacy of these tools. Clinician acceptance is a significant barrier to sepsis CDS implementation. Successful implementation of less clinically intuitive ML models may require additional attention to user confusion and distrust.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Limitations relate to the small number of empirical studies of deployed sepsis-prediction algorithms, under-reporting of post-implementation performance metrics, focus on adult hospital settings, and potential publication bias from under-reporting of other sepsis MLA implementation studies.18”
From Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework (2023) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Implementation barriers for sepsis AI cluster around trust and interpretability, workflow integration, technical readiness and generalizability, governance and safety, leadership alignment, and the resources required for sustainment (Table 4) [10,33,34,46,48,77,78].”
From Artificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis (2026) · cites this paper as [77]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Other studies have examined implementation issues when integrating machine learning models into clinical practice and reported numerous socio-technical factors such as the unfamiliarity of AI among clinicians and lack of AI education, trust in the accuracy and utility of predictive models, and IT support among other issues that affected this process (Joshi et al., 2022;Sandhu et al., 2020).”
From The application and use of artificial intelligence in cancer nursing: A systematic review (2024) · cites this paper as Joshi et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“Their study found that implementation was complex and resource-intensive, with major barriers including alert optimization, workflow integration, and especially clinician acceptance. There were greater distrust, confusion, and interpretability challenges for machine learning models, highlighting the need for improved user education, transparency, and implementation strategies.”
From Artificial Intelligence to Facilitate SEP-1 Measure Compliance and Fluid Management in Sepsis (2026) · cites this paper as [77]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
S., N. L. (2017) · doi:10.1177/1609406917733847 · Cited by 14,051
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Saturation in qualitative research: exploring its conceptualization and operationalization
Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018) · Quality & Quantity · doi:10.1007/s11135-017-0574-8 · Cited by 12,033
Seymour, C. W., Liu, V. X., Iwashyna, T. J., Brunkhorst, F. M., Rea, T. D., Scherag, A., Rubenfeld, G., Kahn, J. M., Shankar-Hari, M., Singer, M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0288 · Cited by 3,939 · cited 1×
“Some prospective studies have shown significant reduction in time to blood culture and antibiotics without significant reductions in mortality.(9,12,16) While others have largely been negative, including 2 randomized controlled trials.”
BackgroundExplainable machine-learning predictions for the prevention of hypoxaemia during surgery
Lundberg, S. M., Nair, B., Vavilala, M. S., Horibe, M., Eisses, M. J., Adams, T., Liston, D. E., Low, D. K.-W., Newman, S.-F., Kim, J., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0304-0 · Cited by 1,967
Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009-2014.
Rhee, C., Dantes, R., Epstein, L., Murphy, D. J., Seymour, C. W., Iwashyna, T. J., Kadri, S. S., Angus, D. C., Danner, R. L., Fiore, A. E., et al. (2017) · JAMA · doi:10.1001/jama.2017.13836 · Cited by 1,884
Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.2626 · Cited by 995 · cited 1×
“Furthermore, a recent large validation cohort study focusing on characterizing alert fatigue around the Epic sepsis detection algorithm reported low sensitivity and many missed cases of sepsis despite generation of a large number of alerts.(32) However, the argument can be made that further evaluation is needed and th…”
BackgroundFleuren, L. M., Klausch, T. L. T., Zwager, C. L., Schoonmade, L. J., Guo, T., Roggeveen, L. F., Swart, E. L., Girbes, A. R. J., Thoral, P., Ercole, A., et al. (2020) · Intensive Care Medicine · doi:10.1007/s00134-019-05872-y · Cited by 755 · cited 1×
“This approach was chosen as qualitative methods can provide “rich descriptions of complex phenomena” typical of healthcare organization and IT implementation.(20) A convenience sample was developed of hospital leaders from various hospitals nationwide with criteria intended to include implementors of CDS tools from 3…”
MethodIntroducing artificial intelligence training in medical education
Paranjape, K., Schinkel, M., Nannan Panday, R., Car, J., & Nanayakkara, P. (2019) · JMIR Medical Education · doi:10.2196/16048 · Cited by 634
Desautels, T., Calvert, J., Hoffman, J., Jay, M., Kerem, Y., Shieh, L., Shimabukuro, D., Chettipally, U., Feldman, M. D., Barton, C., et al. (2016) · JMIR Medical Informatics · doi:10.2196/medinform.5909 · Cited by 509
Taylor, R. A., Pare, J. R., Venkatesh, A. K., Mowafi, H., Melnick, E. R., Fleischman, W., & Hall, M. K. (2016) · Academic Emergency Medicine · doi:10.1111/acem.12876 · Cited by 491
Artificial intelligence and the implementation challenge
Shaw, J., Rudzicz, F., Jamieson, T., & Goldfarb, A. (2019) · Journal of Medical Internet Research · doi:10.2196/13659 · Cited by 389 · cited 1×
““Meaningful decision support” and “explainability” have been described as 2 key implementation science barriers specific to ML models, which carried additional buy in challenges in our study.(35) In order for decision support to be “meaningful,” users need to first trust individual predictions enough to act on them.”
BackgroundShimabukuro, D. W., Barton, C. W., Feldman, M. D., Mataraso, S. J., & Das, R. (2017) · BMJ Open Respiratory Research · doi:10.1136/bmjresp-2017-000234 · Cited by 382 · cited 1×
“Furthermore, there are data suggesting that such algorithms may enable earlier intervention and improve outcomes such as length of stay and mortality.(14,21) However, ML algorithms are new to clinical practice and raise practical, ethical, and face validity concerns.”
BackgroundAutomated Identification of Adults at Risk for In-Hospital Clinical Deterioration
Escobar, G. J., Liu, V. X., Schuler, A., Lawson, B., Greene, J. D., & Kipnis, P. (2020) · New England Journal of Medicine · doi:10.1056/nejmsa2001090 · Cited by 314 · cited 1×
“Another study, conducted at 21 hospitals involving 374 838 patients, demonstrated lower mortality, a lower incidence of ICU admission, and a shorter length of hospital stay using a full rapid response intervention program built around a validated deterioration prediction model.(34) This study emphasized an approach us…”
MethodGiannini, H. M., Ginestra, J. C., Chivers, C., Draugelis, M., Hanish, A., Schweickert, W. D., Fuchs, B. D., Meadows, L., Lynch, M., Donnelly, P. J., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003891 · Cited by 269 · cited 1×
“Furthermore, there are data suggesting that such algorithms may enable earlier intervention and improve outcomes such as length of stay and mortality.(14,21) However, ML algorithms are new to clinical practice and raise practical, ethical, and face validity concerns.”
BackgroundPrediction of sepsis patients using machine learning approach: a meta-analysis
Islam, M. M., Nasrin, T., Walther, B. A., Wu, C.-C., Yang, H.-C., & Li, Y.-C. (2019) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.12.027 · Cited by 259
Implementation of a real-time computerized sepsis alert in nonintensive care unit patients*
Sawyer, A. M., Deal, E. N., Labelle, A. J., Witt, C., Thiel, S. W., Heard, K., Reichley, R. M., Micek, S. T., & Kollef, M. H. (2011) · Critical Care Medicine · doi:10.1097/ccm.0b013e318205df85 · Cited by 169
Development, implementation, and impact of an automated early warning and response system for sepsis
Umscheid, C. A., Betesh, J., VanZandbergen, C., Hanish, A., Tait, G., Mikkelsen, M. E., French, B., & Fuchs, B. D. (2015) · Journal of Hospital Medicine · doi:10.1002/jhm.2259 · Cited by 168
Ginestra, J. C., Giannini, H. M., Schweickert, W. D., Meadows, L., Lynch, M. J., Pavan, K., Chivers, C. J., Draugelis, M., Donnelly, P. J., Fuchs, B. D., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003803 · Cited by 159 · cited 1×
“Consistent with our findings, existing literature specifically surveying clinician end users describes that alerts do not change perception of patient’s risk and alerts vary in their ability to alter management.(30,31) Furthermore, a recent large validation cohort study focusing on characterizing alert fatigue around…”
ResultClinical applications of artificial intelligence in sepsis: A narrative review
Schinkel, M., Paranjape, K., Nannan Panday, R. S., Skyttberg, N., & Nanayakkara, P. W. B. (2019) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2019.103488 · Cited by 130
Rozenblum, R., Jang, Y., Zimlichman, E., Salzberg, C., Tamblyn, M., Buckeridge, D., Forster, A., Bates, D. W., & Tamblyn, R. (2011) · Canadian Medical Association Journal · doi:10.1503/cmaj.100856 · Cited by 123
Self-citationProspective Trial of Real-Time Electronic Surveillance to Expedite Early Care of Severe Sepsis
Nelson, J. L., Smith, B. L., Jared, J. D., & Younger, J. G. (2011) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2010.12.008 · Cited by 122 · cited 1×
“Some prospective studies have shown significant reduction in time to blood culture and antibiotics without significant reductions in mortality.(9,12,16) While others have largely been negative, including 2 randomized controlled trials.”
BackgroundDowning, N. L., Rolnick, J., Poole, S. F., Hall, E., Wessels, A. J., Heidenreich, P., & Shieh, L. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008765 · Cited by 112 · cited 1×
“Some randomized studies support reduction in length of stay and in hospital mortality,(21,29) while others have shown no significant change in clinical outcomes.(2,8) There is however prospective data to suggest that these alerts may lead to faster interventions such as time to blood culture or antibiotics.”
ResultSamal, L., Dykes, P. C., Greenberg, J. O., Hasan, O., Venkatesh, A. K., Volk, L. A., & Bates, D. W. (2016) · BMC Health Services Research · doi:10.1186/s12913-016-1373-y · Cited by 101
Self-citationThe epic sepsis model falls short-the importance of external validation
Habib, A. R., Lin, A. L., & Grant, R. W. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.3333 · Cited by 101 · cited 1×
“However, the argument can be made that further evaluation is needed and that variability in results may be related to variability between hospitals and differential workflow creation.(33) One study, particularly highlighting significant change management effort and delivery of alerts via a mobile application in additi…”
BackgroundMakam, A. N., Nguyen, O. K., & Auerbach, A. D. (2015) · Journal of Hospital Medicine · doi:10.1002/jhm.2347 · Cited by 99 · cited 1×
“Traditional early warning systems using Systemic Inflammatory Response System (SIRS) criteria relying on vital sign and lab abnormalities have been criticized for producing excessive false positive alerts.(2,5,6) Existing data are inconsistent about whether these tools actually change the likelihood of bundle interven…”
BackgroundCase-based explanation of non-case-based learning methods.
Caruana, R., Kangarloo, H., Dionisio, J. D. N., Sinha, U., & Johnson, D. (1999) · PubMed · Cited by 94
Kollef, M. H., Chen, Y., Heard, K., LaRossa, G. N., Lu, C., Martin, N. R., Martin, N., Micek, S. T., & Bailey, T. (2014) · Journal of Hospital Medicine · doi:10.1002/jhm.2193 · Cited by 90
Manaktala, S., & Claypool, S. R. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw056 · Cited by 89 · cited 1×
“One study, particularly highlighting significant change management effort and delivery of alerts via a mobile application in addition to implementation of an algorithm, did demonstrate 53% decrease in mortality.(13) Another study, conducted at 21 hospitals involving 374 838 patients, demonstrated lower mortality, a lo…”
ResultSemler, M. W., Weavind, L., Hooper, M. H., Rice, T. W., Gowda, S. S., Nadas, A., Song, Y., Martin, J. B., Bernard, G. R., & Wheeler, A. P. (2015) · Critical Care Medicine · doi:10.1097/ccm.0000000000001020 · Cited by 86 · cited 3×
“Since early resuscitation and antibiotic administration can reduce mortality, sepsis recognition and care has become a nationwide priority.(2) In 2015, Centers for Medicare and Medicaid implemented a sepsis bundled payment (SEP-1), mandating placement of specific orders within 3 h of sepsis onset or hospital presentat…”
BackgroundResultA patient-centered longitudinal care plan: vision versus reality
Dykes, P. C., Samal, L., Donahue, M., Greenberg, J. O., Hurley, A. C., Hasan, O., O'Malley, T. A., Venkatesh, A. K., Volk, L. A., & Bates, D. W. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002454 · Cited by 75
Self-citationEffect of an electronic medical record alert for severe sepsis among ED patients
Narayanan, N., Gross, K., Pintens, M., Fee, C. R., & MacDougall, C. (2015) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2015.10.005 · Cited by 74 · cited 1×
“Some prospective studies have shown significant reduction in time to blood culture and antibiotics without significant reductions in mortality.(9,12,16) While others have largely been negative, including 2 randomized controlled trials.”
Automated electronic medical record sepsis detection in the emergency department
Nguyen, S. Q., Mwakalindile, E., Booth, J. S., Hogan, V., Morgan, J., Prickett, C. T., Donnelly, J. P., & Wang, H. E. (2014) · PeerJ · doi:10.7717/peerj.343 · Cited by 68 · cited 1×
“Traditional early warning systems using Systemic Inflammatory Response System (SIRS) criteria relying on vital sign and lab abnormalities have been criticized for producing excessive false positive alerts.(2,5,6) Existing data are inconsistent about whether these tools actually change the likelihood of bundle interven…”
BackgroundOrganizational strategies for promoting patient and provider uptake of personal health records
Wells, S., Rozenblum, R., Park, A., Dunn, M., & Bates, D. W. (2015) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2014-003055 · Cited by 67
Self-citationHealth Informatics on FHIR: How HL7's New API is Transforming Healthcare
Braunstein, M. L. (2018) · Springer eBooks · doi:10.1007/978-3-319-93414-3 · Cited by 60 · cited 1×
“With regards to explainability, it is well established in the literature that “black boxes” of ML in clinical medicine are difficult for clinicians to accept and that demystifying the “black box” is critical for establishing trust in the model.(36,37) RB models had more straightforward explanations since the rules are…”
BackgroundThe CMS Sepsis Mandate: Right Disease, Wrong Measure
Klompas, M., & Rhee, C. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0588 · Cited by 45 · cited 1×
“In 2015, Centers for Medicare and Medicaid implemented a sepsis bundled payment (SEP-1), mandating placement of specific orders within 3 h of sepsis onset or hospital presentation.(3) The bundle has led many hospitals to invest money, time, and energy into measuring quality of sepsis care.”
BackgroundGuidi, J. L., Clark, K., Upton, M. T., Faust, H., Umscheid, C. A., Lane-Fall, M. B., Mikkelsen, M. E., Schweickert, W. D., Vanzandbergen, C. A., Betesh, J., et al. (2015) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201503-129oc · Cited by 43 · cited 1×
“Consistent with our findings, existing literature specifically surveying clinician end users describes that alerts do not change perception of patient’s risk and alerts vary in their ability to alter management.(30,31) Furthermore, a recent large validation cohort study focusing on characterizing alert fatigue around…”
ResultThe current capabilities of health information technology to support care transitions.
Samal, L., Dykes, P. C., Greenberg, J. J., Hasan, O., Venkatesh, A. K., Volk, L. A., & Bates, D. W. (2013) · Europe PMC (PubMed Central) · Cited by 17
Self-citationKeeping It Simple in Sepsis Measures
Walkey, A. J., & Lindenauer, P. K. (2017) · Journal of Hospital Medicine · doi:10.12788/jhm.2873 · Cited by 10
Patient-Specific Explanations for Predictions of Clinical Outcomes
Tajgardoon, M., Samayamuthu, M. J., Calzoni, L., & Visweswaran, S. (2019) · ACI Open · doi:10.1055/s-0039-1697907 · Cited by 7
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv
Cited by
Hogg, H. D. J., Al-Zubaidy, M., Technology Enhanced Macular Services Study Reference Group, Talks, J., Denniston, A. K., Kelly, C. J., Malawana, J., Papoutsi, C., Teare, M. D., Keane, P. A., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/39742 · Cited by 163
Rahimi, A. K., Pienaar, O., Ghadimi, M., Canfell, O. J., Pole, J. D., Shrapnel, S., van der Vegt, A. H., & Sullivan, C. M. (2024) · Journal of Medical Internet Research · doi:10.2196/49655 · Cited by 164
“%) originated from the United States [20,32,33,36,37,39,43-45, Radiology was the clinical setting in 46% (12/26) of the studies [33,34,36,37,39-41, 43-45”
InfluentialTrust in and Acceptance of Artificial Intelligence Applications in Medicine: Mixed Methods Study
Shevtsova, D., Ahmed, A., Boot, I. W. A., Sanges, C., Hudecek, M., Jacobs, J. J. L., Hort, S., & Vrijhoef, H. J. M. (2024) · JMIR Human Factors · doi:10.2196/47031 · Cited by 124
“Therefore, it can be argued that acceptance of an AI application depends on trust people have toward this technology [11,15,16].”
BackgroundImplementation frameworks for end-to-end clinical AI: derivation of the SALIENT framework
van der Vegt, A. H., Scott, I. A., Dermawan, K., Schnetler, R. J., Kalke, V. R., & Lane, P. J. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad088 · Cited by 109
“Themes: (1) Regulatory challenges; (2) Oversight and regulation; (3) Safety and efficacy surveillance; (4) Accountability; (5) Liability; (6) Equity and inclusion; (7) Transparency; (8) Education; (9) Patient engagement; (10) Cybersecurity and privacy; (11) Ethics and fairness; (12) Financial incentives.”
Finkelstein, J., Gabriel, A. S., Schmer, S., Truong, T., & Dunn, A. S. (2024) · Journal of Medical Systems · doi:10.1007/s10916-024-02104-9 · Cited by 91
“Previous studies utilized qualitative interviews for assessing challenges to AI implementation in healthcare without aligning their analyses with existing implementation science frameworks [55–57].”
Method
Metadata
- Authors
- Joshi, M., Mecklai, K., Rozenblum, R., & Samal, L.
- Affiliations
- Stanford Medicine (US)
- Harvard University (US)
- Brigham and Women's Hospital (US)
- Corresponding author
- Joshi, Mugdha
- Venue
- JAMIA Open
- Publisher
- Oxford University Press (OUP)
- Published
- 2022-04-06
- Volume
- 5
- Issue
- 2
- Pages
- ooac022-ooac022
- ISSN
- 2574-2531 (ISSN-L)
- PMC
- PMC9030109
- OpenAlex
- W4223913070
- Cited by
- 46
- References
- 40
- Citations by year
- 2026: 9 · 2025: 17 · 2024: 12 · 2023: 7 · 2022: 1
- FWCI
- 6.93
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- JAMIA Open (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Sepsis Diagnosis and Treatment
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- sepsis risk prediction, clinical decision support, rule-based models, machine learning models, implementation barriers, clinician acceptance, electronic health records, sepsis surveillance, trust in AI, quality improvement, socio-technical challenges
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.