CoChat
Sign Up
Cited by 228Ethics and Social Impacts of AIAcademic integrity and plagiarismLaw, AI, and Intellectual Property

The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool

Open access
Resnik, David B., Hosseini, Mohammad·2025·AI and Ethics
DOI 10.1007/s43681-024-00493-8PMC PMC12057767
Open this paper in CoChat — free account

Abstract

Using artificial intelligence (AI) in research offers many important benefits for science and society but also creates novel and complex ethical issues. While these ethical issues do not necessitate changing established ethical norms of science, they require the scientific community to develop new guidance for the appropriate use of AI. In this article, we briefly introduce AI and explain how it can be used in research, examine some of the ethical issues raised when using it, and offer nine recommendations for responsible use, including: (1) Researchers are responsible for identifying, describing, reducing, and controlling AI-related biases and random errors; (2) Researchers should disclose, describe, and explain their use of AI in research, including its limitations, in language that can be understood by non-experts; (3) Researchers should engage with impacted communities, populations, and other stakeholders concerning the use of AI in research to obtain their advice and assistance and address their interests and concerns, such as issues related to bias; (4) Researchers who use synthetic data should (a) indicate which parts of the data are synthetic; (b) clearly label the synthetic data; (c) describe how the data were generated; and (d) explain how and why the data were used; (5) AI systems should not be named as authors, inventors, or copyright holders but their contributions to research should be disclosed and described; (6) Education and mentoring in responsible conduct of research should include discussion of ethical use of AI.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper argues that using AI in scientific research creates novel ethical issues but does not require a radical revision of established scientific norms; instead, the scientific community needs new guidance for responsible use. It reviews AI/ML capabilities and ethical challenges and offers nine recommendations covering bias and error, disclosure, stakeholder engagement, synthetic data, authorship, confidentiality, and research ethics education.

Key findings

  • The authors propose nine recommendations for ethical AI use in research, including that researchers are responsible for identifying, describing, reducing, and controlling AI-related biases and random errors; should disclose and explain AI use in non-expert language; should use AI only when they have sufficient expertise or judgment; should engage impacted communities; are liable for misconduct if they intentionally, knowingly, or recklessly use AI to fabricate/falsify data or plagiarize; should label and describe synthetic data; should not name AI as author, inventor, or copyright holder; should avoid AI where it risks unauthorized disclosure of confidential information; and should include AI ethics in responsible conduct of research education (Table 2).
  • The paper reports that ChatGPT 3.5-generated medical papers produced references that were 47% fabricated, 46% authentic but inaccurately used, and only 7% correct.
  • It cites AlphaFold’s protein-structure prediction accuracy as 92.4%, while noting human researchers still needed to test and refine outputs for 100% accuracy.
  • It notes ChatGPT scored in the 93rd percentile on the SAT reading exam and 89th percentile on the math exam, and that two US lawyers were fined $5000 for submitting a court filing with fake ChatGPT-generated citations.

Methods

  • Design: conceptual and ethical analysis, not an empirical study; the paper introduces AI/ML, examines ethical issues in research, and develops recommendations.
  • Data/sample: no primary data or human subjects; it draws on published literature, ethics codes and policies (e.g., European Code of Conduct, NIH, ICMJE, UNESCO), and illustrative cases.
  • Main analysis: identifies ethical challenges related to AI in science and derives nine recommendations, summarized in Table 2.

Limitations

  • Stated: the authors say their list of scientific norms is not original, exhaustive, or exclusive; they call one point about sanitizing LLM data “highly speculative”; they state more empirical research and ethical/policy analysis is needed on explainability; and they note their assumption that current AI lacks moral agency may need revision in the future.
  • Evident: the recommendations are normative and not empirically tested, and the paper does not use a systematic review or stakeholder-engagement methodology to validate them.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s43681-024-00493-8

Cites

145 of 145 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 170 cited sources checked · 0 supported · 0 flagged

  • Findingcitation

    Dealing with the black box problem has been a key issue in FDA approval of medical devices that use AI/ML.

    “Indeed, dealing with the black box problem has been a key issue in FDA approval of medical devices that use AI/ML [74, 183].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [74] Food and Drug Administration.: Artificial intelligence (AI) and machine learning (ML) in medical devices (2020). https://www.fda.gov/media/142998/download

      Not checked in the free run.

    • cites [183] Pew Charitable Trust.: How FDA regulates artificial intelligence in medical products. Issue brief (2021). https://www.pewtrusts.org/en/research-and-analysis/issue-briefs/2021/08/how-fda-regulates-artificial-intelligence-in-medical-products

      Not checked in the free run.

  • Findingcitation

    In 2021, Alexa, Amazon's LLM-based voice-assistant, instructed a 10-year-old girl to stick a penny into an electric outlet when she asked it for a challenge to do.

    “in 2021, Alexa, Amazon’s LLM-based voice-assistant, instructed a 10-year-old girl to stick a penny into an electric outlet when she asked it for a challenge to do [20].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20] BBC News.: Alexa tells 10-year-old girl to touch live plug with penny. BBC News (2021). https://www.bbc.com/news/technology-59810383

      Not checked in the free run.

  • Findingcitation

    An expert computer scientist or engineer will not be able to interpret the thousands of numbers used in the weightings and the billions of calculations from those numbers in an AI/ML system.

    “While an expert computer scientist or engineer could troubleshoot the code, they will not be able to interpret the thousands of numbers used in the weightings and the billions of calculations from those numbers [110, 151, 199].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [110] IBM.: Explainable AI (2023). https://www.ibm.com/topics/explainable-ai

      Not checked in the free run.

    • cites [151] Mitchell, M.: Artificial Intelligence. Picador, New York (2019)

      Not checked in the free run.

    • cites [199] Savage, N.: Breaking into the black box of artificial intelligence. Nature (2022). https://www.nature.com/articles/d41586-022-00858-1 source

      Not checked in the free run.

  • Findingcitation

    OpenAI, Google, and other generative AI developers have been using filters that prevent their systems from generating text that is outright racist, sexist, homophobic, pornographic, offensive, or dangerous.

    “OpenAI, Google, and other generative AI developers have been using filters that prevent their systems from generating text that is outright racist, sexist, homophobic, pornographic, offensive, or dangerous [93].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [93] Heikkilä, M.: How OpenAI is trying to make ChatGPT safer and less biased. The Atlantic (2023). https://www.technologyreview.com/2023/02/21/1068893/how-openai-is-trying-to-make-chatgpt-safer-and-less-biased/

      Not checked in the free run.

  • Findingcitation

    The use of computer software to analyze large sets of complex data has challenged traditional views about norms related to hypothesis testing.

    “The use of computer software to analyze large sets of complex data has challenged traditional views about norms related to hypothesis testing [193, 194].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [193] Resnik, D.B., Elliott, K.C., Soranno, P.A., Smith, E.M.: Data-intensive science and research integrity. Account. Res. 24(6), 344–358 (2017) source

      Not checked in the free run.

    • cites [194] Resnik, D.B., Smith, E.M., Chen, S.H., Goller, C.: What is recklessness in scientific research? The Frank Sauer case. Account. Res. 24(8), 497–502 (2017) source

      Not checked in the free run.

  • May combine two claims. 42
  • Classified as probably not a substantial claim. 49

  • May combine two claims.

    Findingcitation

    In a study by Bhattacharyya et al., ChatGPT 3.5 generated 30 short papers on medical topics; 47% of references were fabricated, 46% were authentic but inaccurately used, and only 7% were correct.

    “Bhattacharyya et al. [24] used ChatGPT 3.5 to generate 30 short papers (200 words or less) on medical topics. 47% of the references produced by the chatbot were fabricated, 46% were authentic but inaccurately used, and only 7% were correct.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [24] Bhattacharyya, M., Miller, V.M., Bhattacharyya, D., Miller, L.E.: High rates of fabricated and inaccurate references in ChatGPT-generated medical content. Cureus 15(5), e39238 (2023)

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    A recent study found that while ChatGPT can pass medical boards, it can give dangerous medical advice due to its tendency to make factual errors and its lack of understanding of the meaning and context of language.

    “A recent study found that while ChatGPT can pass medical boards, it can give dangerous medical advice due to its tendency to make factual errors and its lack of understanding of the meaning and context of language [51].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [51] Chen, S., Kann, B.H., Foote, M.B., Aerts, H.J.W.L., Savova, G.K., Mak, R.H., Bitterman, D.S.: Use of artificial intelligence chatbots for cancer treatment information. JAMA Oncol. 9(10), 1459–1462 (2023) source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Public trust in science is already in decline, due primarily to the politicization of topics with significant social implications, such as climate change, COVID-19 vaccines and public health measures.

    “which is already in decline, due primarily to the politicization of topics with significant social implications, such as climate change, COVID-19 vaccines and public health measures [123, 189].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [123] Kennedy, B., Tyson, A., Funk, C.: Americans’ trust in scientists, other groups declines. Pew Research Center (2022). https://www.pewresearch.org/science/2022/02/15/americans-trust-in-scientists-other-groups-declines/

      Not checked in the free run.

    • cites [189] Resnik, D.B.: Playing Politics with Science: Balancing Scientific Independence and Government Oversight. Oxford University Press, New York (2009) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The increased presence of women and racial and ethnic minorities in science has led to the development of policies for preventing sexual and other forms of harassment.

    “The increased presence of women and racial and ethnic minorities in science has led to the development of policies for preventing sexual and other forms of harassment [185].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [185] Rasmussen, L.: Why and how to incorporate issues of race/ethnicity and gender in research integrity education. Accountability in Research (2023) source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    ChatGPT is a highly opaque system that is yet to disclose its training data and it is unclear whether/when OpenAI would open up its technology to external scrutiny.

    “ChatGPT, for example, is a highly opaque system that is yet to disclose its training data and it is unclear whether/when OpenAI would open up its technology to external scrutiny [28, 66, 130].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [28] Blumauer, A.: How ChatGPT works and the problems with non-explainable AI. Pool Party (2023). https://www.poolparty.biz/blogposts/how-chat-gpt-works-non-explainable-ai#:~:text=ChatGPT%20is%20the%20antithesis%20of,and%20explainability%20are%20critical%20requirements

      Not checked in the free run.

    • cites [66] Eliot, L.: Nobody can explain for sure why ChatGPT is so good at what it does, troubling AI ethics and AI Law. Forbes (2023). https://www.forbes.com/sites/lanceeliot/2023/04/17/nobody-can-explain-for-sure-why-chatgpt-is-so-good-at-what-it-does-troubling-ai-ethics-and-ai-law/?sh=334c95685041

      Not checked in the free run.

    • cites [130] La Malfa, E., Petrov, A., Frieder, S., Weinhuber, C., Burnell, R., Cohn, A.G., Shadbolt, N., Woolridge, M.: The ARRT of language-models-as-a-service: overview of a new paradigm and its challenges (2023). arXiv: 2309.16573 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Norms such as empiricism, objectivity, and consistency originated in ancient Greek science; others, such as reproducibility and openness, developed during the 1500s; and many, such as protection of research subjects and social responsibility, did not emerge as formalized norms until the twentieth century.

    “While norms such as empiricism, objectivity, and consistency originated in ancient Greek science, others, such as reproducibility and openness, developed during the 1500s; and many, such as protection of research subjects and social responsibility, did not emerge as formalized norms until the twentieth century.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The funding of science by private companies and their requirements concerning data access and release policies have led to changes in norms related to open sharing of data and materials.

    “the funding of science by private companies and their requirements concerning data access and release policies have led to changes in norms related to open sharing of data and materials [188].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [188] Resnik, D.B.: The Price of Truth: How Money Affects the Norms of Science. Oxford University Press, New York (2007) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Findingnone given

    There seems to be an emerging consensus on the utility and importance of making AI explainable.

    “While there seems to be an emerging consensus on the utility and importance of making AI explainable”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    LLMs are impacted by biases inherent in their training data and can propagate biases related to race, ethnicity, nationality, gender, sexuality, age, and politics when used in generative AI models like ChatGPT.

    “LLMs are also impacted by various biases inherent in their training data, and when used in generative AI models like ChatGPT, can propagate biases related to race, ethnicity, nationality, gender, sexuality, age, and politics [25, 171].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [25] Biddle, S.: The internet’s new favorite AI proposes torturing Iranians and surveilling mosques. The Intercept (2022). https://theintercept.com/2022/12/08/openai-chatgpt-ai-bias-ethics/

      Not checked in the free run.

    • cites [171] Omiye, J.A., Lester, J.C., Spichak, S., Rotemberg, V., Daneshjou, R.: Large language models propagate race-based medicine. NPJ Digit. Med. 6, 195 (2023) source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    AI tools have been used to perform or augment a variety of scientific tasks, including analyzing data and images, interpreting data and images, generating hypotheses, modelling complex phenomena, designing molecules and materials, generating data for validation, searching and reviewing literature, writing and editing scientific papers and related documents, and reviewing scientific papers andother

    “AI tools have been used to perform or augment a variety of scientific tasks, including1:Analyzing data and images [34, 43, 65, 88, 106, 115, 122, 124, 149, 161].Interpreting data and images [13, 14, 21, 41].Generating hypotheses [32, 37, 41, 107, 149].Modelling complex phenomena [32, 41, 43, 122, 129].Designing molecules and materials [15, 37, 43, 205].Generating data for use in validation of hy”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    ChatGPT 4.0 performs significantly better than ChatGPT 3.5 but still produces fabricated and inaccurate citations.

    “Although ChatGPT 4.0 performs significantly better than ChatGPT 3.5, it still produces fabricated and inaccurate citations [230].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [230] Walters, W.H., Wilder, E.I.: Fabrication and errors in the bibliographic citations generated by ChatGPT. Sci. Rep. 13, 14045 (2023) source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Codes of AI ethics, such as UNESCO's Ethics of Artificial Intelligence and the Office of Science and Technology Policy's Blueprint for an AI Bill of Rights, provide useful guidance for the development and use of AI in general without including specific guidance concerning the development and use of AI in scientific research.

    “Codes of AI ethics, such as UNESCO’s [223] Ethics of Artificial Intelligence and the Office of Science and Technology Policy’s [168, 169] Blueprint for an AI Bill of Rights, provide useful guidance for the development and use of AI in general without including specific guidance concerning the development and use of AI in scientific research [215].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [223] UNESCO.: Ethics of Artificial Intelligence (2024). https://www.unesco.org/en/artificial-intelligence/recommendation-ethics

      Not checked in the free run.

    • cites [168] Office of Science and Technology Policy: Federal research misconduct policy. Fed. Reg. 65(235), 76260–76264 (2000)

      Not checked in the free run.

    • cites [169] Office and Science and Technology Policy.: Blueprint for an AI Bill of Rights (2022). https://www.whitehouse.gov/ostp/ai-bill-of-rights/

      Not checked in the free run.

    • cites [215] Tambornino, L., Lanzerath, D., Rodrigues, R., Wright, D.: SIENNA D4.3: survey of REC approaches and codes for Artificial Intelligence & Robotics (2019). https://zenodo.org/records/4067990

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The 2023 European Code of Conduct for Research Integrity briefly discusses transparency and considers hiding AI use in content creation or drafting publications a violation of research integrity.

    “the 2023 revision of the European Code of Conduct for Research Integrity [4] briefly discusses the importance of transparency. The code stipulates that researchers should report “their results and methods including the use of external services or AI and automated tools” (Ibid., p. 7) and considers “hiding the use of AI or automated tools in the creation of content or drafting of publications” as a”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4] All European Academies.: The European Code of Conduct for Research Integrity, Revised Edition 2023 (2023). https://allea.org/code-of-conduct/

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The National Institutes of Health Guidelines and Policies for the Conduct of Research provides guidance for using AI to write and edit manuscripts but not for other tasks.

    “the National Institutes of Health Guidelines and Policies for the Conduct of Research provides guidance for using AI to write and edit manuscripts but not for other tasks [158].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [158] National Institutes of Health.: Guidelines for the Conduct of Research in the Intramural Program of the NIH (2023). https://oir.nih.gov/system/files/media/file/2023-11/guidelines-conduct_research.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationcitation

    This response to the black box problem does not solve the problem but simply tells us not to worry about it.

    “This response to the black box problem does not solve the problem but simply tells us not to worry about it [63].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [63] Durán, J.M., Jongsma, K.R.: Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI. J. Med. Ethics 47(5), 329–335 (2021)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The correct output of an AI system is ultimately defined by its users and others who may be affected.

    “The correct output of an AI system is ultimately defined by its users and others who may be affected.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    ChatGPT has passed licensing exams for medicine and law and scored in the 93rd percentile on the Scholastic Aptitude Test reading exam and in the 89th percentile on the math exam.

    “ChatGPT has passed licensing exams for medicine and law and scored in the 93rd percentile on the Scholastic Aptitude Test reading exam and in the 89th percentile on the math exam [133, 138, 232].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [133] Leswing, K.: OpenAI announces GPT-4, claims it can beat 90% of humans on the SAT. CNBC (2023). https://www.cnbc.com/2023/03/14/openai-announces-gpt-4-says-beats-90percent-of-humans-on-sat.html

      Not checked in the free run.

    • cites [138] Lubell, J.: ChatGPT passed the USMLE. What does it mean for med ed? AMA (2023). https://www.ama-assn.org/practice-management/digital/chatgpt-passed-usmle-what-does-it-mean-med-ed

      Not checked in the free run.

    • cites [232] Weiss, D.C.: Latest version of ChatGPT aces bar exam with score nearing 90th percentile. ABA J. (2023). https://www.abajournal.com/web/article/latest-version-of-chatgpt-aces-the-bar-exam-with-score-in-90th-percentile

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    AI systems cannot be held morally or legally responsible or accountable.

    “even though AI cannot be held morally or legally responsible or accountable.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    LLMs do not form mental models based on implicit facts and assumptions about the natural world, human psychology, society, and culture; they only process symbols and predict the most likely string of symbols from linguistic prompts.

    “LLMs do not do this,they only process symbols and predict the most likely string of symbols from linguistic prompts.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    AlphaFold can predict 3-D protein structures from amino acid sequences with 92.4% accuracy.

    “Tremendous progress on the protein-folding problem was made in 2022, when scientists demonstrated that an ML system, DeepMind’s AlphaFold, can predict 3-D structures from amino acid sequences with 92.4% accuracy [118, 204].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [118] Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S.A.A., Ballard, A.J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., Back, T., Petersen, S., Reiman, D., Clancy, E., Zielinski, M., Steinegger, M., Pacholska, M., Berghammer, T., Bodenstein, S., Silver, D., Vinyals, O., Senior, A.W., Kavukcuoglu, K., Kohli, P., Hassabis, D.: Highly accurate protein structure prediction with AlphaFold. Nature 596(7873), 583–589 (2021) source

      Not checked in the free run.

    • cites [204] Service RF: The game has changed. AI trumphs at protein folding. Science 370(6521), 1144–1145 (2022) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Scientists have dealt with bias in research for years and have developed methods and strategies for minimizing and controlling bias in experimental design, data analysis, model building, and theory construction.

    “Scientists have dealt with bias in research for years and have developed methods and strategies for minimizing and controlling bias in experimental design, data analysis, model building, and theory construction [79, 89, 104].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [79] Giere, R., Bickle, J., Maudlin, R.F.: Understanding Scientific Reasoning, 5th edn. Wadsworth, Belmont (2005)

      Not checked in the free run.

    • cites [89] Haack, S.: Defending Science within Reason. Prometheus Books, New York (2007)

      Not checked in the free run.

    • cites [104] Howson, C., Urbach, P.: Scientific Reasoning: A Bayesian Approach, 3rd edn. Open Court, New York (2005)

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Human beings are ultimately responsible for determining whether AI outputs are correct.

    “While AI systems can play a key role in providing information that is used to define correct outputs (for example, when a system is used to discover new chemical compounds or solve complex math problems), human beings are ultimately responsible for determining whether outputs are correct (see discussion of moral agency in Sect. 7.6).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: AI systems are not moral agents.

  • May combine two claims.

    Interpretationcitation

    The opacity of AI systems is ethically problematic because we cannot trust them if even the best experts do not completely understand how they work.

    “The opacity of AI systems is ethically problematic because one might argue that we should not use these devices if we cannot trust them, and we cannot trust them if even the best experts do not completely understand how they work [6, 7, 39, 47, 63, 186].”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [6] Alvarado, R.: Should we replace radiologists with deep learning? Bioethics 36(2), 121–133 (2022) source

      Not checked in the free run.

    • cites [7] Alvarado, R.: What kind of trust does AI deserve, if any? AI Ethics (2022). https://doi.org/10.1007/s43681-022-00224-x source

      Not checked in the free run.

    • cites [39] Burrell, J.: How the machine ‘thinks’: understanding opacity in machine learning algorithms. Big Data Soc. 3(1), 2053951715622512 (2016) source

      Not checked in the free run.

    • cites [47] Chan, B.: Black-box assisted medical decisions: AI power vs. ethical physician care. Med. Health Care Philos. 26, 285–292 (2023) source

      Not checked in the free run.

    • cites [63] Durán, J.M., Jongsma, K.R.: Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI. J. Med. Ethics 47(5), 329–335 (2021)

      Not checked in the free run.

    • cites [186] Ratti, E., Graves, M.: Explainable machine learning practices: opening another black box for reliable medical AI. AI Ethics 2, 801–814 (2022) source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    AI systems make errors in reading, classifying, and reconstructing radiological images, but the error rate is decreasing, and AI systems will soon outperform humans in terms of speed and accuracy of image reading.

    “AI systems also make errors in reading, classifying, and reconstructing radiological images, but the error rate is decreasing, and AI systems will soon outperform humans in terms of speed and accuracy of image reading [12, 17, 103, 228].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [12] Antun, V., Renna, F., Poon, C., Adcock, B., Hansen, A.C.: On instabilities of deep learning in image reconstruction and the potential costs of AI. Proc. Natl. Acad. Sci. U.S.A. 117(48), 30088–30095 (2020) source

      Not checked in the free run.

    • cites [17] Barrera, F.J., Brown, E.D.L., Rojo, A., Obeso, J., Plata, H., Lincango, E.P., Terry, N., Rodríguez-Gutiérrez, R., Hall, J.E., Shekhar, S.: Application of machine learning and artificial intelligence in the diagnosis and classification of polycystic ovarian syndrome: a systematic review. Front. Endocrinol. (2023). https://doi.org/10.3389/fendo.2023.1106625 source

      Not checked in the free run.

    • cites [103] Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L.H., Aerts, H.J.W.L.: Artificial intelligence in radiology. Nat. Rev. Cancer 18(8), 500–510 (2018) source

      Not checked in the free run.

    • cites [228] Varoquaux, G., Cheplygina, V.: Machine learning for medical imaging: methodological failures and recommendations for the future. NPJ Digit. Med. 5, 48 (2022) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Some researchers have used ChatGPT to write scientific papers and have even named them as authors.

    “Some researchers have used ChatGPT to write scientific papers and have even named them as authors [48, 53, 54, 167].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [48] ChatGPT, Zhavoronkov, A.: Rapamycin in the context of Pascal’s Wager: generative pre-trained transformer perspective. Oncoscience 9, 82–84 (2022) source

      Not checked in the free run.

    • cites [53] Conroy, G.: Scientists used ChatGPT to generate an entire paper from scratch—but is it any good? Nature 619, 443–444 (2023) source

      Not checked in the free run.

    • cites [54] Conroy, G.: How ChatGPT and other AI tools could disrupt scientific publishing. Nature (2023). https://www.nature.com/articles/d41586-023-03144-w

      Not checked in the free run.

    • cites [167] O’Connor, S., ChatGPT: Open artificial intelligence platforms in nursing education: tools for academic progress or abuse? Nurse Educ. Pract. 66, 103537 (2023) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationcitation

    Scientists can take steps to understand sources of bias and control them, especially those that can lead to discrimination, stigmatization, harm, or injustice.

    “Nevertheless, scientists can (and should) take steps to understand sources of bias and control them, especially those that can lead to discrimination, stigmatization, harm, or injustice [89, 154, 188].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [89] Haack, S.: Defending Science within Reason. Prometheus Books, New York (2007)

      Not checked in the free run.

    • cites [154] Mittermaier, M., Raza, M.M., Kvedar, J.C.: Bias in AI-based models for medical applications: challenges and mitigation strategies. NPJ Digit. Med. 6, 113 (2023) source

      Not checked in the free run.

    • cites [188] Resnik, D.B.: The Price of Truth: How Money Affects the Norms of Science. Oxford University Press, New York (2007) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisenone given

    Trust in a technology is partially based on understanding that technology.

    “Trust in a technology is partially based on understanding that technology.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    Private companies uninterested in sharing information about their systems may not comply with explainable AI requirements or they may "game" the requirements to resemble compliance without actually complying.

    “private companies uninterested in sharing information about their systems may not comply with explainable AI requirements or they may “game” the requirements to resemble compliance without actually complying.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    AI biases in medical research have generated considerable concern, since biases related to race, ethnicity, gender, sexuality, age, nationality, and socioeconomic status in health-related datasets can perpetuate health disparities by supporting biased hypotheses, models, theories, and policies.

    “AI biases in medical research have generated considerable concern, since biases related to race, ethnicity, gender, sexuality, age, nationality, and socioeconomic status in health-related datasets can perpetuate health disparities by supporting biased hypotheses, models, theories, and policies [177, 198, 211].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [177] Parikh, R.B., Teeple, S., Navathe, A.S.: Addressing bias in artificial intelligence in health care. J. Am. Med. Assoc. 322(24), 2377–2378 (2019) source

      Not checked in the free run.

    • cites [198] Röösli, E., Rice, B., Hernandez-Boussard, T.: Bias at warp speed: how AI may contribute to the disparities gap in the time of COVID-19. J. Am. Med. Inform. Assoc. 28(1), 190–192 (2021) source

      Not checked in the free run.

    • cites [211] Straw, I., Callison-Burch, C.: Artificial Intelligence in mental health and the biases of language based models. PLoS ONE 15(12), e0240376 (2020) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Premisereasoning

    Ethical norms promote trust among scientists, which is essential for collaboration, peer review, publication, sharing of data and resources, mentoring, education, and other scientific activities.

    “ethical norms promote trust among scientists, which is essential for collaboration, peer review, publication, sharing of data and resources, mentoring, education, and other scientific activities.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Science's ethical norms have changed over time, and they are likely to continue to evolve.

    “science’s ethical norms have changed over time, and they are likely to continue to evolve [80, 128, 147, 237].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [80] Gillispie, C.C.: The Edge of Objectivity. Princeton University Press, Princeton (1960)

      Not checked in the free run.

    • cites [128] Kuhn, T.S.: The Essential Tension. University of Chicago Press, Chicago (1977) source

      Not checked in the free run.

    • cites [147] Merton, R.: The Sociology of Science. University of Chicago Press, Chicago (1973)

      Not checked in the free run.

    • cites [237] Ziman, J.: Real Science. Cambridge University Press, Cambridge (2000) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Model-building in science may involve tradeoffs among various epistemic norms, including generality, precision, realism, simplicity, and explanatory power.

    “model-building in science may involve tradeoffs among various epistemic norms, including generality, precision, realism, simplicity, and explanatory power [143].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [143] Matthewson, J.: Trade-offs in model-building: a more target-oriented approach. Stud. Hist. Philos. Sci. Part A 42(2), 324–333 (2011) source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    AI can reproduce and amplify biases inherent in the data and generate results that lend support to policies that are discriminatory, unfair, harmful, or ineffective.

    “AI can reproduce and amplify biases inherent in the data and generate results that lend support to policies that are discriminatory, unfair, harmful, or ineffective [16, 202].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [16] Ball, P.: Is AI leading to a reproducibility crisis in science? Nature 624, 22–25 (2023) source

      Not checked in the free run.

    • cites [202] Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., Hal, P.: Towards a standard for identifying and managing bias in artificial intelligence. National Institute of Standards and Technology (2022). https://view.ckcest.cn/AllFiles/ZKBG/Pages/264/c914336ac0e68a6e3e34187adf9dd83bb3b7c09f.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    If we want to use LLMs to make sound moral and political judgments, sanitizing their data processing and output may hinder their ability to excel at this task, because the ability to make sound moral judgements or anticipate harm may depend, in part, on some familiarity with immoral choices and the darker side of humanity.

    “If we want to use LLMs to make sound moral and political judgments, sanitizing their data processing and output may hinder their ability to excel at this task, because the ability to make sound moral judgements or anticipate harm may depend, in part, on some familiarity with immoral choices and the darker side of humanity.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Strategies for reducing errors in science include time-honored quality assurance and quality improvement techniques, such as auditing data, instruments, and systems; validating and testing instruments that analyze or process data; and investigating and analyzing errors.

    “Strategies for reducing errors in science include time-honored quality assurance and quality improvement techniques, such as auditing data, instruments, and systems; validating and testing instruments that analyze or process data; and investigating and analyzing errors [1].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] Aboumatar, H., Thompson, C., Garcia-Morales, E., Gurses, A.P., Naqibuddin, M., Saunders, J., Kim, S.W., Wise, R.: Perspective on reducing errors in research. Contemp. Clin. Trials Commun. 23, 100838 (2021) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Premisecitation

    Honesty is the hallmark of good science, and there are very few situations in which scientists are justified in deviating from this norm.

    “honesty is the hallmark of good science, and there are very few situations in which scientists are justified in deviating from this norm.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Openness can be deemphasized to protect research participants' privacy, intellectual property, classified information, or unpublished research.

    “Openness, on the other hand, can be deemphasized to protect research participants’ privacy, intellectual property, classified information, or unpublished research [207].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Given the accessibility of AI systems and the fact that they can exploit our cognitive shortcomings, they are creating an illusion of understanding.

    “given the accessibility of AI systems and the fact that they can exploit our cognitive shortcomings, they are creating an illusion of understanding [148].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [148] Messeri, L., Crockett, M.J.: Artificial intelligence and illusions of understanding in scientific research. Nature (2024). https://doi.org/10.1038/s41586-024-07146-0 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    This evolution is in response to changes in science's social, institutional, economic, and political environment and advancements in scientific instruments, tools, and methods.

    “This evolution is in response to changes in science’s social, institutional, economic, and political environment and advancements in scientific instruments, tools, and methods [100].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [100] Hosseini, M., Senabre Hidalgo, E., Horbach, S.P.J.M., Güttinger, S., Penders, B.: Messing with Merton: the intersection between open science practices and Mertonian values. Accountability in Research, pp. 1–28 (2022) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    An excessive amount of random errors when using AI could be considered carelessness or recklessness when using AI.

    “an excessive amount when using AI could be considered carelessness or recklessness when using AI”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Explainability helps to promote trust in AI because it allows users and other stakeholders to make rational and informed decisions about it.

    “Explainability, according to proponents of this approach, helps to promote trust in AI because it allows users and other stakeholders to make rational and informed decisions about it [77, 83, 110, 186].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [77] Fuhrman, J.D., Gorre, N., Hu, Q., Li, H., El Naqa, I., Giger, M.L.: A review of explainable and interpretable AI with applications in COVID-19 imaging. Med. Phys. 49(1), 1–14 (2022) source

      Not checked in the free run.

    • cites [83] Google.: Responsible AI practices (2023). https://ai.google/responsibility/responsible-ai-practices/

      Not checked in the free run.

    • cites [110] IBM.: Explainable AI (2023). https://www.ibm.com/topics/explainable-ai

      Not checked in the free run.

    • cites [186] Ratti, E., Graves, M.: Explainable machine learning practices: opening another black box for reliable medical AI. AI Ethics 2, 801–814 (2022) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    Ethical norms are not rigid rules; norms sometimes conflict, and when they do, scientists must make decisions concerning epistemic or moral priorities.

    “Although vital for conducting science, ethical norms are not rigid rules. Norms sometimes conflict, and when they do, scientists must make decisions concerning epistemic or moral priorities [191].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [191] Resnik, D.B., Elliott, K.C.: Value-entanglement and the integrity of scientific research. Stud. Hist. Philos. Sci. 75, 1–11 (2019) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Placebo control groups are not used in clinical trials when receiving a placebo instead of an effective treatment would cause serious harm to the participant.

    “placebo control groups are not used in clinical trials when receiving a placebo instead of an effective treatment would cause serious harm to the participant [207].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Research with human subjects often involves tradeoffs between rigor and protection of participants.

    “Research with human subjects often involves tradeoffs between rigor and protection of participants.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning

    It is nearly impossible to understand how a natural or social process works or make accurate predictions about it without standards pertaining to honesty, logical consistency, empirical support, and reproducibility of data and results.

    “It is nearly impossible to understand how a natural or social process works or make accurate predictions about it without standards pertaining to honesty, logical consistency, empirical support, and reproducibility of data and results.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    If we try to weed undesirable features of data used to train LLMs, we will eliminate parts of our language and culture, and ultimately, parts of us.

    “If we try to weed undesirable features of this data, we will eliminate parts of our language and culture, and ultimately, parts of us.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    It is only by understanding evil that we can freely and rationally choose the good.

    “It is only by understanding evil that we can freely and rationally choose the good [40].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Calder, T.: The concept of evil. Stanford Encyclopedia of Philosophy (2022). https://plato.stanford.edu/entries/concept-evil/#KanTheEvi

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    By taking machines' disinterestedness in findings as a necessary and sufficient condition of objectivity, users of AI in research may overestimate the objectivity of their findings.

    “by taking machines’ disinterestedness in findings as a necessary and sufficient condition of objectivity, users of AI in research may overestimate the objectivity of their findings.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Transparency about potential limitations and AI-related errors is important for making research trustworthy and reproducible.

    “Transparency about these is important for making research trustworthy and reproducible [16].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [16] Ball, P.: Is AI leading to a reproducibility crisis in science? Nature 624, 22–25 (2023) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The use of synthetic data may help to protect confidentiality.

    “Also, as mentioned in Sect. 7.3, the use of synthetic data may help to protect confidentiality.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    Using synthetic data in research, even appropriately, may blur the line between real and fake data and undermine data integrity.

    “using synthetic data in research, even appropriately, may blur the line between real and fake data and undermine data integrity.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given

    When we hold humans accountable, we expect them to explain their behavior in clear and intelligible language.

    “Regarding the second reason, when we hold humans accountable, we expect them to explain their behavior in clear and intelligible language.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    AI systems do not have interests, values, reputation and feelings in the same way that humans do and cannot be punished by law enforcement.

    “These systems do not have interests, values, reputation and feelings in the same way that humans do and cannot be punished by law enforcement.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given

    Researchers can use synthetic data to develop and validate models and enhance statistical analysis.

    “researchers can use synthetic data to develop and validate models and enhance statistical analysis.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Some US courts now require lawyers to disclose their use of generative AI in preparing legal documents and to assure they have taken appropriate steps to protect confidentiality.

    “Some US courts now require lawyers to disclose their use of generative AI in preparing legal documents and make assurances that they have taken appropriate steps to protect confidentiality [146].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [146] Merken, S.: Another US judge says lawyers must disclose AI use. Reuters (2023). https://www.reuters.com/legal/transactional/another-us-judge-says-lawyers-must-disclose-ai-use-2023-06-08/

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Electronic medical records have security of the type provided by a local LLM behind a secure firewall.

    “Electronic medical records, for example, have this type of security [127].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [127] Kruse, C.S., Smith, B., Vanderlinden, H., Nealand, A.: Security techniques for the electronic health records. J. Med. Syst. 41(8), 127 (2017) source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Responsible conduct of research (RCR) education is widely recognized as essential to promoting ethical judgment, reasoning, and behavior in science.

    “The last topic we will address in this section has to do with education and mentoring in responsible conduct of research (RCR), which is widely recognized as essential to promoting ethical judgment, reasoning, and behavior in science [207].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    It is only a matter of time until scientists are charged with research misconduct for presenting synthetic data as real data.

    “it is only a matter of time until this happens, given the pressures to produce results, publish, and obtain grants, and the temptations to cheat or cut corners.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Authorship on scientific papers is based not only on making a substantial contribution, but also on being accountable for the work.

    “Authorship on scientific papers is based not only on making a substantial contribution, but also on being accountable for the work [207].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In the US, the NIH and NSF require RCR education for funded students and trainees, and many academic institutions require some form of RCR training for all research faculty.

    “In the US, the NIH and National Science Foundation (NSF) require RCR education for funded students and trainees, and many academic institutions require some form of RCR training for all research faculty [190].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [190] Resnik, D.B., Dinse, G.E.: Do U.S. research institutions meet or exceed federal mandates for instruction in responsible conduct of research? A national survey. Acad. Med. 87, 1237–1242 (2012) source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    AI systems cannot suffer consequences nor be sanctioned.

    “(3) they cannot suffer consequences nor can be sanctioned.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    There is an emerging consensus that AI systems can be used in preparing manuscripts but that their use should be appropriately disclosed and discussed.

    “Currently, there seems to be an emerging consensus that falls in between these two extremes position, namely, that AI systems can be used in preparing manuscripts but that their use should be appropriately disclosed and discussed, [4, 102].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [4] All European Academies.: The European Code of Conduct for Research Integrity, Revised Edition 2023 (2023). https://allea.org/code-of-conduct/

      Not checked in the free run.

    • cites [102] Hosseini, M., Resnik, D.B., Holmes, K.: The ethics of disclosing the use of artificial intelligence in tools writing scholarly manuscripts. Res. Ethics (2023). https://doi.org/10.1177/17470161231180449 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Material submitted to generative AI for editing or review will not be kept confidential.

    “Researchers who use generative AI to edit or review a document should assume that the material contained in it will not be kept confidential”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Making AI explainable does not completely solve the black box problem but it can play a key role in promoting transparency, accountability, and trust.

    “making AI explainable does not completely solve the black box problem but it can play a key role in promoting transparency, accountability, and trust [7, 9].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [7] Alvarado, R.: What kind of trust does AI deserve, if any? AI Ethics (2022). https://doi.org/10.1007/s43681-022-00224-x source

      Not checked in the free run.

    • cites [9] Alvarado, R.: AI as an epistemic technology. Sci. Eng. Ethics 29, 32 (2023) source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    There is general agreement that moral agency requires the capacity to perform intentional (or purposeful) actions, understand moral norms, and make decisions based on moral norms.

    “While the concept of moral agency, like the concept of consciousness, is controversial, there is general agreement that moral agency requires the capacity to perform intentional (or purposeful) actions, understand moral norms, and make decisions based on moral norms.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [46] Cervantes, J.A., López, S., Rodríguez, L.F., Cervantes, S., Cervantes, F., Ramos, F.: Artificial moral agents: a survey of the current status. Sci. Eng. Ethics 26(2), 501–532 (2020) source

      Not checked in the free run.

    • cites [95] Himma, K.E.: Artificial agency, consciousness, and the criteria for moral agency: what properties must an artificial agent have to be a moral agent? Ethics Inf. Technol. 11, 19–29 (2009) source

      Not checked in the free run.

    • cites [213] Talbert, M.: Moral responsibility. Stanford Encyclopedia of Philosophy (2019). https://plato.stanford.edu/entries/moral-responsibility/

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Scientists may still be liable for misconduct due to recklessness, which may have consequences for using AI.

    “scientists may still be liable for misconduct due to recklessness [42, 193, 194], which may have consequences for using AI.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [42] Caron, M.M., Dohan, S.B., Barnes, M., Bierer, B.E.: Defining "recklessness" in research misconduct proceedings. Accountability in Research, pp. 1–23 (2023) source

      Not checked in the free run.

    • cites [193] Resnik, D.B., Elliott, K.C., Soranno, P.A., Smith, E.M.: Data-intensive science and research integrity. Account. Res. 24(6), 344–358 (2017) source

      Not checked in the free run.

    • cites [194] Resnik, D.B., Smith, E.M., Chen, S.H., Goller, C.: What is recklessness in scientific research? The Frank Sauer case. Account. Res. 24(8), 497–502 (2017) source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In the winter of 2022-2023, some researchers listed ChatGPT as authors on papers, making AI moral agency a major issue for ethical norms of science.

    “This became a major issue for the ethical norms of science in the winter of 2022–2023, when some researchers listed ChatGPT as authors on papers [102].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [102] Hosseini, M., Resnik, D.B., Holmes, K.: The ethics of disclosing the use of artificial intelligence in tools writing scholarly manuscripts. Res. Ethics (2023). https://doi.org/10.1177/17470161231180449 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    If an organization operates a local application of an LLM behind a secure firewall, its members can use the technology safely.

    “For example, if an organization operates a local application of an LLM and places the technology behind a secure firewall, its members can use the technology safely.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [127] Kruse, C.S., Smith, B., Vanderlinden, H., Nealand, A.: Security techniques for the electronic health records. J. Med. Syst. 41(8), 127 (2017) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    The black box problem presents significant challenges to the trustworthiness and transparency of research that use AI because some of the steps in the scientific process will not be fully open and understandable to humans, including AI experts.

    “The black box problem presents significant challenges to the trustworthiness and transparency of research that use AI because some of the steps in the scientific process will not be fully open and understandable to humans, including AI experts.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Some universities have already begun experimenting with operating their own AI systems for use by students, faculty, and administrators.

    “Some universities have already begun experimenting with operating their own AI systems for use by students, faculty, and administrators [225].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [225] University of Michigan.: Generative artificial intelligence (2023). https://genai.umich.edu/

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    The use of AI in research has a direct bearing on most RCR topics, especially authorship, rigor and reproducibility, peer review, and social responsibility.

    “As demonstrated in this paper, the use of AI in research has a direct bearing on most of these topics, but especially on authorship, rigor and reproducibility, peer review, and social responsibility.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationnone given

    Scientists do not need to radically revise their ethical norms to deal with AI issues, but they need new guidance for the appropriate use of AI in research.

    “Although scientists do not need to radically revise their ethical norms to deal with these issues, they do need new guidance for the appropriate use of AI in research.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    Using AI in research benefits science and society but also creates novel and complex ethical issues that affect accountability, responsibility, transparency, trustworthiness, reproducibility, fairness, objectivity, and other important values in research.

    “Using AI in research benefits science and society but also creates some novel and complex ethical issues that affect accountability, responsibility, transparency, trustworthiness, reproducibility, fairness, and objectivity, and other important values in research.”
    Quote in the paperNot a substantial claim
73 of 73

  • The paper relies on this without support.

    Bias reduction and management is essential to promoting public trust in AI.

    “Indeed, bias reduction and management is essential to promoting public trust in AI (discussed in Sects. 5.5 and 5.7).”

  • The paper relies on this without support.

    AI systems are not moral agents.

    “If we set aside questions related to intent on the grounds that AI systems are not moral agents (see discussion in Sect. 7.6), we can think of AI error as the difference between the output of an AI system and the correct output.”

  • The paper relies on this without support.

    The ethical norms of science listed in Table 1 are the relevant norms for evaluating AI in research.

    “These are some commonly discussed norms. We do not claim that our list is original, exhaustive, or exclusive”

  • The paper relies on this without support.

    AI systems are currently used in scientific research in ways that raise ethical issues.

    “With this overview of AI in mind, we can now consider how using AI in research impacts the ethical norms of science.”

  • The authors state this as an assumption.

    Today's AI systems lack the capacities necessary for moral agency.

    “In closing, we will observe that many scholars, including ourselves, assume that today’s AI systems lack the capacities necessary for moral agency.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The literature review that forms the basis of research conception may include facts or citations that are inaccurate, irrelevant, or fake, i.e., hallucinations.

    “First, the literature review that forms the basis of research conception may include facts or citations that are inaccurate, irrelevant, or fake, i.e., “hallucinations” [5, 34–36].”

    From Autonomous artificial intelligence, scientific research, and human values (2026) · cites this paper as [5]

  • AI systems may produce outputs that are skewed by biases related to race, ethnicity, sex, religion, politics, and other factors.

    “AI systems may also produce outputs that are skewed by biases related to race, ethnicity, sex, religion, politics, and other factors. [5, 23, 53–56]”

    From Autonomous artificial intelligence, scientific research, and human values (2026) · cites this paper as [5]

  • The integration of AI into the research workflow raises ethical issues and concerns pertaining to accountability, transparency, trust, bias, and confidentiality.

    “While the integration of AI into the research workflow may significantly benefit science and society, it also raises ethical issues and concerns pertaining to accountability, transparency, trust, bias, confidentiality, among others [5].”

    From Autonomous artificial intelligence, scientific research, and human values (2026) · cites this paper as [5]

  • For more than a decade, non-autonomous AI systems have been assisting with various tasks that involve only the manipulation of information, such as processing and analyzing data.

    “For more than a decade, non-autonomous AI systems have been assisting with various tasks that involve only the manipulation of information, such as processing and analyzing data [5].”

    From Autonomous artificial intelligence, scientific research, and human values (2026) · cites this paper as [5]

  • Uploading data into an AI system could compromise the confidentiality of the data because other unauthorized users or hackers could access the data.

    “As mentioned in Sect. 2.4, one concern with uploading data into an AI system is that this could compromise the confidentiality of the data because other (unauthorized) users of the system or hackers could access the data [5, 118].”

    From Autonomous artificial intelligence, scientific research, and human values (2026) · cites this paper as [5]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Two reasons are stated, but three are listed

    Minor

    The sentence says AI systems cannot be held accountable for two reasons, then enumerates three distinct reasons.

  • Reference 168 does not identify the cited AI Bill of Rights

    Minor

    The text cites references 168 and 169 for the Blueprint for an AI Bill of Rights, but reference 168 is listed as the Federal research misconduct policy.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 24

  1. All European Academies.: The European Code of Conduct for Research Integrity, Revised Edition 2023 (2023). https://allea.org/code-of-conduct/
  2. American Society of Microbiology.: Code of Conduct (2021). https://asm.org/Articles/Ethics/COEs/ASM-Code-of-Ethics-and-Conduct
  3. Bevier, C.: ChatGPT broke the Turing test—the race is on for new ways to assess AI. Nature (2023). https://www.nature.com/articles/d41586-023-02361-7
  4. Brainard, J.: As scientists face a flood of papers, AI developers aim to help. Science (2023). https://www.science.org/content/article/scientists-face-flood-papers-ai-developers-aim-help
  5. Cyrus, L.: How to fold graciously. In: Mossbauer Spectroscopy in Biological Systems: Proceedings of a Meeting Held at Allerton House, Monticello, Illinois, pp. 22–24 (1969)
  6. Einstein, A.: Letter to Max Born. Walker and Company, New York (1926). Published in: Irene Born (translator), The Born-Einstein Letters (1971)
  7. Gillispie, C.C.: The Edge of Objectivity. Princeton University Press, Princeton (1960)
  8. Kahneman, D.: Thinking, Fast and Slow. Farrar, Straus, Giroux, New York (2011)
  9. Kitcher, P.: The Advancement of Knowledge. Oxford University Press, New York (1993)
  10. Lee, N.T., Resnick, P., Barton, G.: Algorithmic Bias Detection and Mitigation: Best Practices and Policies to Reduce Consumer Harms. Brookings Institute, Washington, DC (2019)
  11. Lubell, J.: ChatGPT passed the USMLE. What does it mean for med ed? AMA (2023). https://www.ama-assn.org/practice-management/digital/chatgpt-passed-usmle-what-does-it-mean-med-ed
  12. Martins, C., Padovan, P., Reed, C.: The role of explainable AI (XAI) in addressing AI liability. SSRN (2020). https://ssrn.com/abstract=3751740
  13. Matta, V., Bansal, G., Akakpo, F., Christian, S., Jain, S., Poggemann, D., Rousseau, J., Ward, E.: Diverse perspectives on bias in AI. J. Inf. Technol. Case Appl. Res. 24(2), 135–143 (2022)
  14. Merton, R.: The Sociology of Science. University of Chicago Press, Chicago (1973)
  15. Mitchell, M.: Artificial Intelligence. Picador, New York (2019)
  16. Office of Science and Technology Policy: Federal research misconduct policy. Fed. Reg. 65(235), 76260–76264 (2000)
  17. Parrilla, J.M.: ChatGPT use shows that the grant-application system is broken. Nature (2023). https://www.nature.com/articles/d41586-023-03238-5
  18. Pearson, J.: Scientific Journal Publishes AI-Generated Rat with Gigantic Penis In Worrying Incident [Internet]. Vice (2024). https://www.vice.com/en/article/dy3jbz/scientific-journal-frontiers-publishes-ai-generated-rat-with-gigantic-penis-in-worrying-incident
  19. Rodgers, W., Murray, J.M., Stefanidis, A., Degbey, W.Y., Tarba, S.: An artificial intelligence algorithmic approach to ethical decision-making in human resource management processes. Hum. Resour. Manag. Rev. 33(1), 100925 (2023)
  20. Service R.: Materials-predicting AI from DeepMind could revolutionize electronics, batteries, and solar cells. Science (2023). https://www.science.org/content/article/materials-predicting-ai-deepmind-could-revolutionize-electronics-batteries-and-solar
  21. Solomon, M.: Social Empiricism. MIT Press, Cambridge (2007)
  22. Southern, M.G.: ChatGPT update: Improved math capabilities. Search Engine Journal (2023). https://www.searchenginejournal.com/chatgpt-update-improved-math-capabilities/478057/
  23. US Copyright Office: Copyright registration guidance: works containing material generated by artificial intelligence. Fed. Reg. 88(51), 16190–16194 (2023)
  24. World Conference on Research Integrity.: Singapore Statement (2010). http://www.singaporestatement.org/statement.html

Cited by

Sorted by citations

Related

Authors
Resnik, D. B., & Hosseini, M.
Affiliations
  • National Institute of Environmental Health Sciences (US)
  • Northwestern University (US)
Corresponding author
Resnik, David B.
ORCID
Venue
AI and Ethics
Publisher
Springer Science and Business Media LLC
Published
2024-05-27
Volume
5
Issue
2
Pages
1499-1521
ISSN
  • 2730-5953 (ISSN-L)
  • 2730-5961
DOI
10.1007/s43681-024-00493-8
PMC
PMC12057767
OpenAlex
W4399054066
Semantic Scholar
36b88327acc36b282c34955e20e3d789c48ba24d
Cited by
228
References
135
Citations by year
2026: 131 · 2025: 76 · 2024: 19 · 2023: 2
FWCI
93.36
Type
journal article
Language
en
Open access
Licence
Locations
  • AI and Ethics (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion, cc-by-sa)
  • SocArXiv (OSF Preprints) (submittedVersion)
Funders
  • Foundation for the National Institutes of Health (ziaes102646-10)
  • National Institutes of Health
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
Keywords
artificial intelligence in research, research ethics, responsible AI use, synthetic data, AI disclosure, stakeholder engagement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review