The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool

In the paper

“Although human researchers still needed to test and refine AlphaFold’s output to ensure that the proposed structure is 100% accurate, the ML system greatly improves the efficiency of protein chemistry research [216].”

What the sources say

• Supported

  The Conclusions state that AlphaFold predictions can greatly accelerate experimental protein-structure determination and change how structural hypotheses are generated and tested. This supports the attributed claim that AlphaFold improves research efficiency. (LLM, conf=0.90)

  cites [216] Terwilliger, T.C., Liebschner, D., Croll, T.I., Williams, C.J., McCoy, A.J., Poon, B.K., Afonine, P.V., Oeffner, R.D., Richardson, J.S., Read, R.J., Adams, P.D.: AlphaFold predictions are valuable hypotheses and accelerate but do not replace experimental structure determination. Nat. Methods (2023). https://doi.org/10.1038/s41592-023-02087-4 source

In the paper

“it is unclear whether making AI explainable will satisfy non-experts because considerable expertise in computer science and/or data analytics may be required to understand what is being explained [120, 186].”

What the sources say

• Not checked yetcites [120] Kahn, J.: What wrong with “explainable A.I.” Fortune (2022). https://fortune.com/2022/03/22/ai-explainable-radiology-medicine-crisis-eye-on-ai/
• ?Could not verify

  The Introduction and Section 2 discuss opacity, explainability, and clinicians’ trust, but do not address whether explanations satisfy non-experts or require computer-science or data-analytics expertise to understand. Searches for non-experts, technical expertise, and understanding explanations found no passage making that point in our copy. (LLM, conf=0.74)

  cites [186] Ratti, E., Graves, M.: Explainable machine learning practices: opening another black box for reliable medical AI. AI Ethics 2, 801–814 (2022) sourceExperimental — accuracy not yet measured

In the paper

“Even in the case of generative AI models where some filtering has happened, changing the inputted prompt may simply confuse and push a system to generate biased content anyway [98].”

What the sources say

• Supported

  In “Roles of reviewers in peer review” (page unavailable), the authors say ChatGPT appears trained to avoid judgmental language but can still produce biased views; they show a biased response to a prompt requesting a cynical review. In “Peer review and reproducibility” (page unavailable), they report that small prompt changes can produce significantly different responses. Together, these passages support the claim’s hedged point. (LLM, conf=0.86)

  cites [98] Hosseini, M., Horbach, S.P.J.M.: Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review. Res. Integr. Peer Rev. 8(1), 4 (2023) source

In the paper

“because synthetic data are similar to but not the same as real data, they can be used to eliminate or mask personal identifiers and protect the confidentiality of human participants [31, 81, 200].”

What the sources say

• Partly supported

  Section 4 says synthetic trial participants can statistically replace enrolled patients to preserve privacy and enable data sharing, and that these synthetic entities are not linked to real patients. It does not specifically say that synthetic data eliminate or mask personal identifiers, so that mechanism is not established by this source. (LLM, conf=0.82)

  cites [31] Bordukova, M., Makarov, N., Rodriguez-Esteban, P., Schmich, F., Menden, M.P.: Generative artificial intelligence empowers digital twins in drug discovery and clinical trials. Expert Opin. Drug Discov. 19(1), 33–42 (2024) sourceExperimental — accuracy not yet measured
• Partly supported

  The ‘Synthetic data generation and types’ section says partially synthetic data can protect patient privacy while remaining useful for analysis. The ‘Regulatory blind spots and proposals’ section describes removing direct identifiers as a de-identification method, but does not say that synthetic data itself eliminates or masks them; the ‘Privacy concerns and regulatory agencies’ section also warns that synthetic data can disclose identifiable details. The claim’s privacy benefit is supported, but its specific mechanism is not. (LLM, conf=0.84)

  cites [81] Giuffrè, M., Shung, D.L.: Harnessing the power of synthetic data in healthcare: innovation, application, and privacy. NPJ Digit. Med. 6, 186 (2023) sourceExperimental — accuracy not yet measured
• Not checked yetcites [200] Savage, N.: Synthetic data could be better than real data. Nature (2023). https://www.nature.com/articles/d41586-023-01445-8 source

In the paper

“Indeed, dealing with the black box problem has been a key issue in FDA approval of medical devices that use AI/ML [74, 183].”

What the sources say

• Not checked yetcites [74] Food and Drug Administration.: Artificial intelligence (AI) and machine learning (ML) in medical devices (2020). https://www.fda.gov/media/142998/download
• Not checked yetcites [183] Pew Charitable Trust.: How FDA regulates artificial intelligence in medical products. Issue brief (2021). https://www.pewtrusts.org/en/research-and-analysis/issue-briefs/2021/08/how-fda-regulates-artificial-intelligence-in-medical-products

In the paper

“in 2021, Alexa, Amazon’s LLM-based voice-assistant, instructed a 10-year-old girl to stick a penny into an electric outlet when she asked it for a challenge to do [20].”

What the sources say

• Not checked yetcites [20] BBC News.: Alexa tells 10-year-old girl to touch live plug with penny. BBC News (2021). https://www.bbc.com/news/technology-59810383

In the paper

“While an expert computer scientist or engineer could troubleshoot the code, they will not be able to interpret the thousands of numbers used in the weightings and the billions of calculations from those numbers [110, 151, 199].”

What the sources say

• Not checked yetcites [110] IBM.: Explainable AI (2023). https://www.ibm.com/topics/explainable-ai
• Not checked yetcites [151] Mitchell, M.: Artificial Intelligence. Picador, New York (2019)
• Not checked yetcites [199] Savage, N.: Breaking into the black box of artificial intelligence. Nature (2022). https://www.nature.com/articles/d41586-022-00858-1 source

In the paper

“OpenAI, Google, and other generative AI developers have been using filters that prevent their systems from generating text that is outright racist, sexist, homophobic, pornographic, offensive, or dangerous [93].”

What the sources say

• Not checked yetcites [93] Heikkilä, M.: How OpenAI is trying to make ChatGPT safer and less biased. The Atlantic (2023). https://www.technologyreview.com/2023/02/21/1068893/how-openai-is-trying-to-make-chatgpt-safer-and-less-biased/

In the paper

“The use of computer software to analyze large sets of complex data has challenged traditional views about norms related to hypothesis testing [193, 194].”

What the sources say

• Not checked yetcites [193] Resnik, D.B., Elliott, K.C., Soranno, P.A., Smith, E.M.: Data-intensive science and research integrity. Account. Res. 24(6), 344–358 (2017) source
• Not checked yetcites [194] Resnik, D.B., Smith, E.M., Chen, S.H., Goller, C.: What is recklessness in scientific research? The Frank Sauer case. Account. Res. 24(8), 497–502 (2017) source

In the paper

“Synthetic data are playing an increasingly important role in some areas of science.”

In the paper

“there is very little agreement about what explainability means in practice, because what makes AI explainable depends on the context of its use [58].”

What the sources say

• Not checked yetcites [58] de Bruijn, H., Warnier, M., Janssen, M.: The perils and pitfalls of explainable AI: strategies for explaining algorithmic decision-making. Gov. Inf. Q. 39(2), 101666 (2022) source

In the paper

“a small proportion of scientists deliberately fabricate or falsify data at some point in their careers [73, 140].”

What the sources say

• Not checked yetcites [73] Fanelli, D.: How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS ONE 4(5), e5738 (2009) source
• Not checked yetcites [140] Martinson, B.C., Anderson, M.S., de Vries, R.: Scientists behaving badly. Nature 435(7043), 737–738 (2005) source

In the paper

“Studies have shown that racial and ethnic biases impact the use of AI/ML in medical imaging, diagnosis, and prognosis due to biases in healthcare databases [78, 154].”

What the sources say

• Not checked yetcites [78] Garin, S.P., Parekh, V.S., Sulam, J., Yi, P.H.: Medical imaging data science competitions should report dataset demographics and evaluate for bias. Nat. Med. 29(5), 1038–1039 (2023) source
• Not checked yetcites [154] Mittermaier, M., Raza, M.M., Kvedar, J.C.: Bias in AI-based models for medical applications: challenges and mitigation strategies. NPJ Digit. Med. 6, 113 (2023) source

In the paper

“There is therefore a gap in ethical and policy guidance concerning AI use in scientific research that needs to be filled to promote its appropriate use.”

In the paper

“However, bias related to using AI in science can be subtle and difficult to detect due to the size and complexity of research data and interactions between data, algorithms, and applications [131].”

What the sources say

• Not checked yetcites [131] Larkin, Z.: AI bias—what Is it and how to avoid it? Levity (2022). https://levity.ai/blog/ai-bias-how-to-avoid

In the paper

“Moreover, the need for guidance is urgent because using AI raises novel epistemological and ethical issues related to objectivity, reproducibility, transparency, accountability, responsibility, and trust in science [9, 102].”

What the sources say

• Not checked yetcites [9] Alvarado, R.: AI as an epistemic technology. Sci. Eng. Ethics 29, 32 (2023) source
• Not checked yetcites [102] Hosseini, M., Resnik, D.B., Holmes, K.: The ethics of disclosing the use of artificial intelligence in tools writing scholarly manuscripts. Res. Ethics (2023). https://doi.org/10.1177/17470161231180449 source

In the paper

“Bias is also a problem in using AI systems to find relationships between genomics and disease due to racial and ethnic prejudices in genomic databases [55].”

What the sources say

• Not checked yetcites [55] Dai, B., Xu, Z., Li, H., Wang, B., Cai, J., Liu, X.: Racial bias can confuse AI for genomic studies. Oncologie 24(1), 113–130 (2022) source

In the paper

“Because scientific theories, hypotheses, and models are based on human perceptual categories, concepts, and assumptions, bias-free research is not possible [121, 125, 137].”

What the sources say

• Not checked yetcites [121] Kahneman, D.: Thinking, Fast and Slow. Farrar, Straus, Giroux, New York (2011)
• Not checked yetcites [125] Kitcher, P.: The Advancement of Knowledge. Oxford University Press, New York (1993)
• Not checked yetcites [137] Longino, H.: Science as Social Knowledge. Princeton University Press, Princeton (1990) source

In the paper

“ChatGPT, for example, is prone to making random factual and citation errors.”

In the paper

“AI systems can make random errors even after extensive training [51, 151].”

What the sources say

• Not checked yetcites [51] Chen, S., Kann, B.H., Foote, M.B., Aerts, H.J.W.L., Savova, G.K., Mak, R.H., Bitterman, D.S.: Use of artificial intelligence chatbots for cancer treatment information. JAMA Oncol. 9(10), 1459–1462 (2023) source
• Not checked yetcites [151] Mitchell, M.: Artificial Intelligence. Picador, New York (2019)

In the paper

“With the exception of using AI to draft or edit scientific papers (see discussion in Sect. 7.6), most codes and policies do not explicitly address ethical issues related to using AI in scientific research.”

In the paper

“As will be discussed below, AI systems can make systemic and random errors [70, 174].”

What the sources say

• Not checked yetcites [70] European Agency for Fundamental Rights.: Data quality and Artificial Intelligence—Mitigating Bias and Error to Protect Fundamental Rights (2019). https://fra.europa.eu/sites/default/files/fra_uploads/fra-2019-data-quality-and-ai_en.pdf
• Not checked yetcites [174] Osoba, O., Welser, W.: An Intelligence in Our Image: The Risks of Bias and Errors in Artificial Intelligence. Rand Corporation (2017). https://www.rand.org/content/dam/rand/pubs/research_reports/RR1700/RR1744/RAND_RR1744.pdf source

In the paper

“Since AI systems are designed to accurately represent the data on which they are trained, they can reproduce or even amplify racial, ethnic, gender, political, or other biases in the training data and subsequent data received [131].”

What the sources say

• Not checked yetcites [131] Larkin, Z.: AI bias—what Is it and how to avoid it? Levity (2022). https://levity.ai/blog/ai-bias-how-to-avoid

In the paper

“Although using AI in scientific research has steadily grown, ethical guidance has lagged far behind.”

In the paper

“The black box problem raises important ethical issues for science (discussed further in Sect. 7.4), because it can undermine public trust in science”

In the paper

“Although human researchers still needed to test and refine AlphaFold’s output to ensure that the proposed structure is 100% accurate, the ML system greatly improves the efficiency of protein chemistry research [216].”

What the sources say

• Not checked yetcites [216] Terwilliger, T.C., Liebschner, D., Croll, T.I., Williams, C.J., McCoy, A.J., Poon, B.K., Afonine, P.V., Oeffner, R.D., Richardson, J.S., Read, R.J., Adams, P.D.: AlphaFold predictions are valuable hypotheses and accelerate but do not replace experimental structure determination. Nat. Methods (2023). https://doi.org/10.1038/s41592-023-02087-4 source

In the paper

“Thus, to perform optimally, LLMs often need human supervision and input to provide the necessary meaning and context for language [61].”

What the sources say

• Not checked yetcites [61] Dickson, B.: LLMs can’t self-correct in reasoning tasks, DeepMind study finds. Tech Talks (2023). https://bdtechtalks.com/2023/10/09/llm-self-correction-reasoning-failures

In the paper

“End-users may become highly dissatisfied with a technology when it fails to perform its function, especially when not even the experts can explain why.”

In the paper

“Systemic errors are often more detrimental to science and society than random ones, because they may negatively affect many different decisions involving people, projects, and paradigms.”

In the paper

“Trying to understand the meaning of the weightings and calculations in ML is very different from trying to understand other types of computer programs, such as those used in most cell phones or personal computers, in which an expert could examine the system (as a whole) to determine what it is doing and why [151, 199].”

What the sources say

• Not checked yetcites [151] Mitchell, M.: Artificial Intelligence. Picador, New York (2019)
• Not checked yetcites [199] Savage, N.: Breaking into the black box of artificial intelligence. Nature (2022). https://www.nature.com/articles/d41586-022-00858-1 source

In the paper

“Some of the factual and reasoning errors made by LLM-based systems occur because they lack human-like understanding of language [29, 135, 152, 153].”

What the sources say

• Not checked yetcites [29] Bogost, I.: ChatGPT is dumber than you think. The Atlantic (2022). https://www.theatlantic.com/technology/archive/2022/12/chatgpt-openai-artificial-intelligencewriting-ethics/672386/
• Not checked yetcites [135] Lipenkova, J.: Overcoming the limitations of large language models: how to enhance LLMs with human-like cognitive skills. Towards Data Science (2023). https://towardsdatascience.com/overcoming-the-limitations-of-large-language-models-9d4e92ad9823
• Not checked yetcites [152] Mitchell, M.: What does it mean for AI to understand? Quanta Magazine (2021). https://www.quantamagazine.org/what-does-it-mean-for-ai-to-understand-20211216/
• Not checked yetcites [153] Mitchell, M.: AI’s challenge of understanding the world. Science 382(6671), eadm8175 (2023) source

In the paper

“narrow AI systems, such as LLMs, lack the capacities regarded as essential for moral agency, such as consciousness, self-concepts, personal memory, life experiences, goals, and emotions [18, 139, 151].”

What the sources say

• Not checked yetcites [18] Bartosz, B.B., Bartosz, J.: Can artificial intelligences be moral agents? New Ideas Psychol. 54, 101–106 (2019) source
• Not checked yetcites [139] Martinho, A., Poulsen, A., Kroesen, M., Chorus, C.: Perspectives about artificial moral agents. AI Ethics 1, 477–490 (2021) source
• Not checked yetcites [151] Mitchell, M.: Artificial Intelligence. Picador, New York (2019)

In the paper

“if something goes wrong with a tool or technology, regulatory agencies, injured parties, insurers, politicians, and others want to know precisely how it works to prevent similar problems in the future and hold people and organizations legally accountable [141].”

What the sources say

• Not checked yetcites [141] Martins, C., Padovan, P., Reed, C.: The role of explainable AI (XAI) in addressing AI liability. SSRN (2020). https://ssrn.com/abstract=3751740

In the paper

“The lack of moral agency, when combined with other AI limitations, such as lack of a meaningful and human-like connection to the physical world, can produce dangerous results.”

In the paper

“when regulatory agencies, such as the Food and Drug Administration (FDA), make decisions concerning the approval of new products, they want to know how the products work, so they can make well-informed, publicly-defendable decisions and inform the consumers about risks.”

In the paper

“Public dissatisfaction with responses to the black box problem may undermine the adoption of AI/ML technologies, especially when these technologies cause harm, invade privacy, or produce biased claims and results [60, 85, 134, 175].”

What the sources say

• Not checked yetcites [60] Dhinakaran, A.: Overcoming AI’s transparency paradox. Forbes (2021). https://www.forbes.com/sites/aparnadhinakaran/2021/09/10/overcoming-ais-transparency-paradox/?sh=6c6b18834b77
• Not checked yetcites [85] Grad, P.: Trick prompts ChatGPT to leak private data. TechXplore (2023). https://techxplore.com/news/2023-12-prompts-chatgpt-leak-private.html
• Not checked yetcites [134] Licht, K., Licht, J.: Artificial intelligence, transparency, and public decision-making: Why explanations are key when trying to produce perceived legitimacy. AI Soc. 35, 917–926 (2020) source
• Not checked yetcites [175] Othman, K.: Public acceptance and perception of autonomous vehicles: a comprehensive review. AI Ethics 1, 355–387 (2021) source

In the paper

“That said, the disconnect between AI/ML and human information processing can also make it difficult to anticipate, understand, control, and reduce errors produced by ML systems.”

In the paper

“One reason why LLM-based systems, such as ChatGPT produce fake, but realistic-looking citations is that they process text data differently from human beings.”

In the paper

“Despite problems like the one discovered by Roberts et al. [195], the fact that ML systems process and analyze data differently from human beings can be a great benefit to science and society because these systems may be able to identify useful and innovative structures, properties, patterns, and relationships that human beings would not recognize.”

What the sources say

• Not checked yetcites [195] Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A.I., Etmann, C., McCague, C., Beer, L., Weir-McCall, J.R., Teng, Z., Gkrania-Klotsas, E., AIX-COVNET, Rudd, J.H.F., Sala, E., Schönlieb, C.B.: Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nat. Mach. Intell. 3, 199–217 (2021) source

In the paper

“Reduction of random errors, like reduction of bias, is widely recognized as essential to good scientific methodology and practice [207].”

What the sources say

• Not checked yetcites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

In the paper

“AI systems may also have political and social biases that are difficult to identify or control [19].”

What the sources say

• Not checked yetcites [19] Baum, J., Villasenor, J.: The politics of AI: ChatGPT and political biases. Brookings (2023). https://www.brookings.edu/articles/the-politics-of-ai-chatgpt-and-political-bias/

In the paper

“Explainable AI, at least in its current form, may not address the informational needs of laypeople, politicians, professionals, or scientists because the information is too technical [58].”

What the sources say

• Not checked yetcites [58] de Bruijn, H., Warnier, M., Janssen, M.: The perils and pitfalls of explainable AI: strategies for explaining algorithmic decision-making. Gov. Inf. Q. 39(2), 101666 (2022) source

In the paper

“by removing, reducing and controlling some biases, AI developers may create other biases, which are difficult to anticipate, identify or describe at this point.”

In the paper

“The public is not likely to financially, legally, or socially support research that is perceived as corrupt, incompetent, untrustworthy, or unethical [191].”

What the sources say

• Not checked yetcites [191] Resnik, D.B., Elliott, K.C.: Value-entanglement and the integrity of scientific research. Stud. Hist. Philos. Sci. 75, 1–11 (2019) source

In the paper

“these norms help the scientific community achieve its epistemic and practical goals, such as understanding, predicting, and controlling nature.”

In the paper

“the biases inherent in data are so pervasive that no amount of filtering can remove all of them [44, 69].”

What the sources say

• Not checked yetcites [44] CBS News.: ChatGPT and large language model bias. CBS News (2023). https://www.cbsnews.com/news/chatgpt-large-language-model-bias-60-minutes-2023-03-05/
• Not checked yetcites [69] Euro News.: Man ends his life after an AI chatbot 'encouraged' him to sacrifice himself to stop climate change. Euro News (2023). https://www.euronews.com/next/2023/03/31/man-ends-his-life-after-an-ai-chatbot-encouraged-him-to-sacrifice-himself-to-stop-climate

In the paper

“Bias can undermine the quality and trustworthiness of science and its social impacts [207].”

What the sources say

• Not checked yetcites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

In the paper

“errors related to the modeling of the transmission of an infectious disease could undermine efforts to control an epidemic.”

In the paper

“random errors can undermine the validity and reliability of scientific knowledge and have disastrous consequences for public health, safety, and social policy [207].”

What the sources say

• Not checked yetcites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

In the paper

“ethical norms are important for fostering public support for science.”

In the paper

“Replication of results by independent researchers, journal peer review, and post-publication peer review also play a major role in error reduction [207].”

What the sources say

• Not checked yetcites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

In the paper

“it is unclear whether explainable AI completely solves issues related to accountability and legal liability because we are yet to witness how legal systems will deal with AI lawsuits in which information pertaining to explainability (or lack thereof) is used as evidence in a court [141].”

What the sources say

• Not checked yetcites [141] Martins, C., Padovan, P., Reed, C.: The role of explainable AI (XAI) in addressing AI liability. SSRN (2020). https://ssrn.com/abstract=3751740

In the paper

“Biases also negatively impact areas of science outside the health sphere, including ecology, forestry, urban planning, economics, wildlife management, geography, and agriculture [142, 164, 165].”

What the sources say

• Not checked yetcites [142] Matta, V., Bansal, G., Akakpo, F., Christian, S., Jain, S., Poggemann, D., Rousseau, J., Ward, E.: Diverse perspectives on bias in AI. J. Inf. Technol. Case Appl. Res. 24(2), 135–143 (2022)
• Not checked yetcites [164] Nost, E., Colven, E.: Earth for AI: a political ecology of data-driven climate initiatives. Geoforum 130, 23–34 (2022) source
• Not checked yetcites [165] Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M.E., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., Kompatsiaris, I., Kinder-Kurlanda, K., Wagner, C., Karimi, F., Fernandez, M., Alani, H., Berendt, B., Kruegel, T., Heinze, C., Broelemann, K., Kasneci, K., Tiropanis, T., Staab, S.: Bias in data-driven artificial intelligence systems—an introductory survey. Wires (2020). https://doi.org/10.1002/widm source

In the paper

“random errors in the processing of radiologic images in a clinical trial of a new cancer drug could harm patients in the trial and future patients who take an approved drug”

In the paper

“Even if an AI can intelligibly explain itself in the future, this does not imply that it can be morally responsible.”

In the paper

“given that content generated by AI systems is not always reproducible [98], identifying and adopting measures to reduce errors is extremely complicated.”

What the sources say

• Not checked yetcites [98] Hosseini, M., Horbach, S.P.J.M.: Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review. Res. Integr. Peer Rev. 8(1), 4 (2023) source

In the paper

“AI systems cannot be held accountable for their actions for two reasons: (1) they cannot provide intelligible explanations for what they did”

In the paper

“These capacities also presuppose additional capacities, such as consciousness, self-awareness, personal memory, perception, general intelligence, and emotions [46, 95, 213].”

What the sources say

• Not checked yetcites [46] Cervantes, J.A., López, S., Rodríguez, L.F., Cervantes, S., Cervantes, F., Ramos, F.: Artificial moral agents: a survey of the current status. Sci. Eng. Ethics 26(2), 501–532 (2020) source
• Not checked yetcites [95] Himma, K.E.: Artificial agency, consciousness, and the criteria for moral agency: what properties must an artificial agent have to be a moral agent? Ethics Inf. Technol. 11, 19–29 (2009) source
• Not checked yetcites [213] Talbert, M.: Moral responsibility. Stanford Encyclopedia of Philosophy (2019). https://plato.stanford.edu/entries/moral-responsibility/

In the paper

“Using AI in research, especially generative AI models, raises ethical issues related to data privacy and confidentiality.”

In the paper

“While computer scientists are making some progress on developing AI systems that have quasi-moral agency, that is, AI systems that can make decisions based on moral norms [71, 196, 203], they are still a long way from developing AGI or AC (see definitions of these terms in Sect. 2), which would seem to be required for genuine moral agency.”

What the sources say

• Not checked yetcites [71] Evans, K., de Moura, N., Chauvier, S., Chatila, R., Dogan, E.: Ethical decision making in autonomous vehicles: the AV ethics project. Sci. Eng. Ethics 26, 3285–3312 (2020) source
• Not checked yetcites [196] Rodgers, W., Murray, J.M., Stefanidis, A., Degbey, W.Y., Tarba, S.: An artificial intelligence algorithmic approach to ethical decision-making in human resource management processes. Hum. Resour. Manag. Rev. 33(1), 100925 (2023)
• Not checked yetcites [203] Semler, J.: Artificial quasi moral agency. In: AIES '22: Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (2022). https://doi.org/10.1145/3514094.3539549 source

In the paper

“If an AI system makes a mistake which harms others, it cannot be sanctioned.”

In the paper

“a person who uses ChatGPT to write a paper without carefully checking its output for errors or plagiarism could be liable for research misconduct for reckless use of AI.”

In the paper

“The data could potentially include personal and confidential information, such as information contained in drafts of scientific papers, grant proposals, experimental protocols, or institutional policies; computer code; legal strategies; business plans; and private information about human research participants [67, 85].”

What the sources say

• Not checked yetcites [67] Eliot, L.: Generative AI ChatGPT can disturbingly gobble up your private and confidential data, forewarns AI ethics and AI law. Forbes (2023). https://www.forbes.com/sites/lanceeliot/2023/01/27/generative-ai-chatgpt-can-disturbingly-gobble-up-your-private-and-confidential-data-forewarns-ai-ethics-and-ai-law/?sh=592b16547fdb
• Not checked yetcites [85] Grad, P.: Trick prompts ChatGPT to leak private data. TechXplore (2023). https://techxplore.com/news/2023-12-prompts-chatgpt-leak-private.html

In the paper

“It is also possible that some scientists will use AI to fabricate data or images presented in scientific papers, grant proposals, or other documents.”

In the paper

“Unless users opt out, this information could be used for training and other purposes.”

What the sources say

• Not checked yetcites [67] Eliot, L.: Generative AI ChatGPT can disturbingly gobble up your private and confidential data, forewarns AI ethics and AI law. Forbes (2023). https://www.forbes.com/sites/lanceeliot/2023/01/27/generative-ai-chatgpt-can-disturbingly-gobble-up-your-private-and-confidential-data-forewarns-ai-ethics-and-ai-law/?sh=592b16547fdb
• Not checked yetcites [85] Grad, P.: Trick prompts ChatGPT to leak private data. TechXplore (2023). https://techxplore.com/news/2023-12-prompts-chatgpt-leak-private.html

In the paper

“Second, AI systems cannot be copyright holders, because to own a copyright, one must be a person [49].”

What the sources say

• Not checked yetcites [49] Chatterjee, M.: AI cannot hold copyright, federal judge rules. Politico (2023). https://www.politico.com/news/2023/08/21/ai-cannot-hold-copyright-federal-judge-rules-00111865#:~:text=Friday's%20ruling%20will%20be%20a%20critical%20component%20in%20future%20legal%20fights.&text=Artificial%20intelligence%20cannot%20hold%20a,a%20federal%20judge%20ruled%20Friday

In the paper

“AI systems cannot be held accountable for their actions for two reasons: (1) they cannot provide intelligible explanations for what they did, (2) they cannot be held morally responsible for their actions, (3) they cannot suffer consequences nor can be sanctioned.”

In the paper

“(2) they cannot be held morally responsible for their actions”

In the paper

“First, AI systems cannot be named as inventors on patents, because inventorship also implies moral agency [62].”

What the sources say

• Not checked yetcites [62] Dunlap, T.: Artificial intelligence (AI) as an inventor? Dunlap, Bennett and Ludwig (2023). https://www.dbllawyers.com/artificial-intelligence-as-an-inventor/

In the paper

“Failure to appropriately control AI-related errors could make scientists liable for research misconduct, if they intentionally, knowingly, or recklessly disseminate false data or plagiarize [207].”

What the sources say

• Not checked yetcites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

In the paper

“Due to concerns about breaches of confidentiality, the National Institutes of Health (NIH) recently prohibited the use of generative AI technologies, such as LLMs, in grant peer review [159].”

What the sources say

• Not checked yetcites [159] National Institutes of Health.: The use of generative artificial intelligence technologies is prohibited for the NIH peer review process. NOT-OD-23-149 (2023). https://grants.nih.gov/grants/guide/notice-files/NOT-OD-23-149.html

In the paper

“ChatGPT, for example, stores the information submitted by users, including data submitted in initial prompts and subsequent interactions.”

What the sources say

• Not checked yetcites [67] Eliot, L.: Generative AI ChatGPT can disturbingly gobble up your private and confidential data, forewarns AI ethics and AI law. Forbes (2023). https://www.forbes.com/sites/lanceeliot/2023/01/27/generative-ai-chatgpt-can-disturbingly-gobble-up-your-private-and-confidential-data-forewarns-ai-ethics-and-ai-law/?sh=592b16547fdb
• Not checked yetcites [85] Grad, P.: Trick prompts ChatGPT to leak private data. TechXplore (2023). https://techxplore.com/news/2023-12-prompts-chatgpt-leak-private.html

In the paper

“This unethical use of AI is becoming increasingly likely since generative models can be used to create synthetic datasets from scratch or make alternative versions of existing datasets [50, 155, 200, 214].”

What the sources say

• Not checked yetcites [50] Chen, R.J., Lu, M.Y., Chen, T.Y., Williamson, D.F., Mahmood, F.: Synthetic data in machine learning for medicine and healthcare. Nat. Biomed. Eng. 5, 493–497 (2021) source
• Not checked yetcites [155] Naddaf, M.: ChatGPT generates fake data set to support scientific hypothesis. Nature (2023). https://www.nature.com/articles/d41586-023-03635-w#:~:text=Researchers%20say%20that%20the%20model,doesn't%20pass%20for%20authentic
• Not checked yetcites [200] Savage, N.: Synthetic data could be better than real data. Nature (2023). https://www.nature.com/articles/d41586-023-01445-8 source
• Not checked yetcites [214] Taloni, A., Scorcia, V., Giannaccre, G.: Large language model advanced data analysis abuse to create a fake data set in medical research. JAMA Ophthalmol. (2023). https://jamanetwork.com/journals/jamaophthalmology/fullarticle/2811505 source

In the paper

“In some cases, someone else may be able to provide an explanation of how they work and what they do, but this not the same as the AI providing the explanation, which is a prerequisite for accountability.”

In the paper

“Copyrights, under US law, are granted only to people [224].”

What the sources say

• Not checked yetcites [224] US Copyright Office: Copyright registration guidance: works containing material generated by artificial intelligence. Fed. Reg. 88(51), 16190–16194 (2023)

In the paper

“Patents are granted to individuals, i.e., persons, but since AI systems lack moral agency, they do not qualify as persons under the patent laws adopted by most countries.”

What the sources say

• Not checked yetcites [62] Dunlap, T.: Artificial intelligence (AI) as an inventor? Dunlap, Bennett and Ludwig (2023). https://www.dbllawyers.com/artificial-intelligence-as-an-inventor/

In the paper

“Although current proposals for making AI explainable may help to deal with this issue, they still fall far short of humanlike accountability, because these proposals do not require that the AI system, itself, should provide an explanation.”