The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool

In the paper

“Bhattacharyya et al. [24] used ChatGPT 3.5 to generate 30 short papers (200 words or less) on medical topics. 47% of the references produced by the chatbot were fabricated, 46% were authentic but inaccurately used, and only 7% were correct.”

What the sources say

• Not checked yetcites [24] Bhattacharyya, M., Miller, V.M., Bhattacharyya, D., Miller, L.E.: High rates of fabricated and inaccurate references in ChatGPT-generated medical content. Cureus 15(5), e39238 (2023)

In the paper

“A recent study found that while ChatGPT can pass medical boards, it can give dangerous medical advice due to its tendency to make factual errors and its lack of understanding of the meaning and context of language [51].”

What the sources say

• Not checked yetcites [51] Chen, S., Kann, B.H., Foote, M.B., Aerts, H.J.W.L., Savova, G.K., Mak, R.H., Bitterman, D.S.: Use of artificial intelligence chatbots for cancer treatment information. JAMA Oncol. 9(10), 1459–1462 (2023) source

In the paper

“which is already in decline, due primarily to the politicization of topics with significant social implications, such as climate change, COVID-19 vaccines and public health measures [123, 189].”

What the sources say

• Not checked yetcites [123] Kennedy, B., Tyson, A., Funk, C.: Americans’ trust in scientists, other groups declines. Pew Research Center (2022). https://www.pewresearch.org/science/2022/02/15/americans-trust-in-scientists-other-groups-declines/
• Not checked yetcites [189] Resnik, D.B.: Playing Politics with Science: Balancing Scientific Independence and Government Oversight. Oxford University Press, New York (2009) source

In the paper

“The increased presence of women and racial and ethnic minorities in science has led to the development of policies for preventing sexual and other forms of harassment [185].”

What the sources say

• Not checked yetcites [185] Rasmussen, L.: Why and how to incorporate issues of race/ethnicity and gender in research integrity education. Accountability in Research (2023) source

In the paper

“ChatGPT, for example, is a highly opaque system that is yet to disclose its training data and it is unclear whether/when OpenAI would open up its technology to external scrutiny [28, 66, 130].”

What the sources say

• Not checked yetcites [28] Blumauer, A.: How ChatGPT works and the problems with non-explainable AI. Pool Party (2023). https://www.poolparty.biz/blogposts/how-chat-gpt-works-non-explainable-ai#:~:text=ChatGPT%20is%20the%20antithesis%20of,and%20explainability%20are%20critical%20requirements
• Not checked yetcites [66] Eliot, L.: Nobody can explain for sure why ChatGPT is so good at what it does, troubling AI ethics and AI Law. Forbes (2023). https://www.forbes.com/sites/lanceeliot/2023/04/17/nobody-can-explain-for-sure-why-chatgpt-is-so-good-at-what-it-does-troubling-ai-ethics-and-ai-law/?sh=334c95685041
• Not checked yetcites [130] La Malfa, E., Petrov, A., Frieder, S., Weinhuber, C., Burnell, R., Cohn, A.G., Shadbolt, N., Woolridge, M.: The ARRT of language-models-as-a-service: overview of a new paradigm and its challenges (2023). arXiv: 2309.16573 source

In the paper

“While norms such as empiricism, objectivity, and consistency originated in ancient Greek science, others, such as reproducibility and openness, developed during the 1500s; and many, such as protection of research subjects and social responsibility, did not emerge as formalized norms until the twentieth century.”

In the paper

“the funding of science by private companies and their requirements concerning data access and release policies have led to changes in norms related to open sharing of data and materials [188].”

What the sources say

• Not checked yetcites [188] Resnik, D.B.: The Price of Truth: How Money Affects the Norms of Science. Oxford University Press, New York (2007) source

In the paper

“While there seems to be an emerging consensus on the utility and importance of making AI explainable”

In the paper

“LLMs are also impacted by various biases inherent in their training data, and when used in generative AI models like ChatGPT, can propagate biases related to race, ethnicity, nationality, gender, sexuality, age, and politics [25, 171].”

What the sources say

• Not checked yetcites [25] Biddle, S.: The internet’s new favorite AI proposes torturing Iranians and surveilling mosques. The Intercept (2022). https://theintercept.com/2022/12/08/openai-chatgpt-ai-bias-ethics/
• Not checked yetcites [171] Omiye, J.A., Lester, J.C., Spichak, S., Rotemberg, V., Daneshjou, R.: Large language models propagate race-based medicine. NPJ Digit. Med. 6, 195 (2023) source

In the paper

“AI tools have been used to perform or augment a variety of scientific tasks, including1:Analyzing data and images [34, 43, 65, 88, 106, 115, 122, 124, 149, 161].Interpreting data and images [13, 14, 21, 41].Generating hypotheses [32, 37, 41, 107, 149].Modelling complex phenomena [32, 41, 43, 122, 129].Designing molecules and materials [15, 37, 43, 205].Generating data for use in validation of hy”

In the paper

“Although ChatGPT 4.0 performs significantly better than ChatGPT 3.5, it still produces fabricated and inaccurate citations [230].”

What the sources say

• Not checked yetcites [230] Walters, W.H., Wilder, E.I.: Fabrication and errors in the bibliographic citations generated by ChatGPT. Sci. Rep. 13, 14045 (2023) source

In the paper

“Codes of AI ethics, such as UNESCO’s [223] Ethics of Artificial Intelligence and the Office of Science and Technology Policy’s [168, 169] Blueprint for an AI Bill of Rights, provide useful guidance for the development and use of AI in general without including specific guidance concerning the development and use of AI in scientific research [215].”

What the sources say

• Not checked yetcites [223] UNESCO.: Ethics of Artificial Intelligence (2024). https://www.unesco.org/en/artificial-intelligence/recommendation-ethics
• Not checked yetcites [168] Office of Science and Technology Policy: Federal research misconduct policy. Fed. Reg. 65(235), 76260–76264 (2000)
• Not checked yetcites [169] Office and Science and Technology Policy.: Blueprint for an AI Bill of Rights (2022). https://www.whitehouse.gov/ostp/ai-bill-of-rights/
• Not checked yetcites [215] Tambornino, L., Lanzerath, D., Rodrigues, R., Wright, D.: SIENNA D4.3: survey of REC approaches and codes for Artificial Intelligence & Robotics (2019). https://zenodo.org/records/4067990

In the paper

“the 2023 revision of the European Code of Conduct for Research Integrity [4] briefly discusses the importance of transparency. The code stipulates that researchers should report “their results and methods including the use of external services or AI and automated tools” (Ibid., p. 7) and considers “hiding the use of AI or automated tools in the creation of content or drafting of publications” as a”

What the sources say

• Not checked yetcites [4] All European Academies.: The European Code of Conduct for Research Integrity, Revised Edition 2023 (2023). https://allea.org/code-of-conduct/

In the paper

“the National Institutes of Health Guidelines and Policies for the Conduct of Research provides guidance for using AI to write and edit manuscripts but not for other tasks [158].”

What the sources say

• Not checked yetcites [158] National Institutes of Health.: Guidelines for the Conduct of Research in the Intramural Program of the NIH (2023). https://oir.nih.gov/system/files/media/file/2023-11/guidelines-conduct_research.pdf

In the paper

“This response to the black box problem does not solve the problem but simply tells us not to worry about it [63].”

What the sources say

• Not checked yetcites [63] Durán, J.M., Jongsma, K.R.: Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI. J. Med. Ethics 47(5), 329–335 (2021)

In the paper

“The correct output of an AI system is ultimately defined by its users and others who may be affected.”

In the paper

“ChatGPT has passed licensing exams for medicine and law and scored in the 93rd percentile on the Scholastic Aptitude Test reading exam and in the 89th percentile on the math exam [133, 138, 232].”

What the sources say

• Not checked yetcites [133] Leswing, K.: OpenAI announces GPT-4, claims it can beat 90% of humans on the SAT. CNBC (2023). https://www.cnbc.com/2023/03/14/openai-announces-gpt-4-says-beats-90percent-of-humans-on-sat.html
• Not checked yetcites [138] Lubell, J.: ChatGPT passed the USMLE. What does it mean for med ed? AMA (2023). https://www.ama-assn.org/practice-management/digital/chatgpt-passed-usmle-what-does-it-mean-med-ed
• Not checked yetcites [232] Weiss, D.C.: Latest version of ChatGPT aces bar exam with score nearing 90th percentile. ABA J. (2023). https://www.abajournal.com/web/article/latest-version-of-chatgpt-aces-the-bar-exam-with-score-in-90th-percentile

In the paper

“even though AI cannot be held morally or legally responsible or accountable.”

In the paper

“LLMs do not do this,they only process symbols and predict the most likely string of symbols from linguistic prompts.”

In the paper

“Tremendous progress on the protein-folding problem was made in 2022, when scientists demonstrated that an ML system, DeepMind’s AlphaFold, can predict 3-D structures from amino acid sequences with 92.4% accuracy [118, 204].”

What the sources say

• Not checked yetcites [118] Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S.A.A., Ballard, A.J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., Back, T., Petersen, S., Reiman, D., Clancy, E., Zielinski, M., Steinegger, M., Pacholska, M., Berghammer, T., Bodenstein, S., Silver, D., Vinyals, O., Senior, A.W., Kavukcuoglu, K., Kohli, P., Hassabis, D.: Highly accurate protein structure prediction with AlphaFold. Nature 596(7873), 583–589 (2021) source
• Not checked yetcites [204] Service RF: The game has changed. AI trumphs at protein folding. Science 370(6521), 1144–1145 (2022) source

In the paper

“Scientists have dealt with bias in research for years and have developed methods and strategies for minimizing and controlling bias in experimental design, data analysis, model building, and theory construction [79, 89, 104].”

What the sources say

• Not checked yetcites [79] Giere, R., Bickle, J., Maudlin, R.F.: Understanding Scientific Reasoning, 5th edn. Wadsworth, Belmont (2005)
• Not checked yetcites [89] Haack, S.: Defending Science within Reason. Prometheus Books, New York (2007)
• Not checked yetcites [104] Howson, C., Urbach, P.: Scientific Reasoning: A Bayesian Approach, 3rd edn. Open Court, New York (2005)

In the paper

“While AI systems can play a key role in providing information that is used to define correct outputs (for example, when a system is used to discover new chemical compounds or solve complex math problems), human beings are ultimately responsible for determining whether outputs are correct (see discussion of moral agency in Sect. 7.6).”

In the paper

“The opacity of AI systems is ethically problematic because one might argue that we should not use these devices if we cannot trust them, and we cannot trust them if even the best experts do not completely understand how they work [6, 7, 39, 47, 63, 186].”

What the sources say

• Not checked yetcites [6] Alvarado, R.: Should we replace radiologists with deep learning? Bioethics 36(2), 121–133 (2022) source
• Not checked yetcites [7] Alvarado, R.: What kind of trust does AI deserve, if any? AI Ethics (2022). https://doi.org/10.1007/s43681-022-00224-x source
• Not checked yetcites [39] Burrell, J.: How the machine ‘thinks’: understanding opacity in machine learning algorithms. Big Data Soc. 3(1), 2053951715622512 (2016) source
• Not checked yetcites [47] Chan, B.: Black-box assisted medical decisions: AI power vs. ethical physician care. Med. Health Care Philos. 26, 285–292 (2023) source
• Not checked yetcites [63] Durán, J.M., Jongsma, K.R.: Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI. J. Med. Ethics 47(5), 329–335 (2021)
• Not checked yetcites [186] Ratti, E., Graves, M.: Explainable machine learning practices: opening another black box for reliable medical AI. AI Ethics 2, 801–814 (2022) source

In the paper

“AI systems also make errors in reading, classifying, and reconstructing radiological images, but the error rate is decreasing, and AI systems will soon outperform humans in terms of speed and accuracy of image reading [12, 17, 103, 228].”

What the sources say

• Not checked yetcites [12] Antun, V., Renna, F., Poon, C., Adcock, B., Hansen, A.C.: On instabilities of deep learning in image reconstruction and the potential costs of AI. Proc. Natl. Acad. Sci. U.S.A. 117(48), 30088–30095 (2020) source
• Not checked yetcites [17] Barrera, F.J., Brown, E.D.L., Rojo, A., Obeso, J., Plata, H., Lincango, E.P., Terry, N., Rodríguez-Gutiérrez, R., Hall, J.E., Shekhar, S.: Application of machine learning and artificial intelligence in the diagnosis and classification of polycystic ovarian syndrome: a systematic review. Front. Endocrinol. (2023). https://doi.org/10.3389/fendo.2023.1106625 source
• Not checked yetcites [103] Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L.H., Aerts, H.J.W.L.: Artificial intelligence in radiology. Nat. Rev. Cancer 18(8), 500–510 (2018) source
• Not checked yetcites [228] Varoquaux, G., Cheplygina, V.: Machine learning for medical imaging: methodological failures and recommendations for the future. NPJ Digit. Med. 5, 48 (2022) source

In the paper

“Some researchers have used ChatGPT to write scientific papers and have even named them as authors [48, 53, 54, 167].”

What the sources say

• Not checked yetcites [48] ChatGPT, Zhavoronkov, A.: Rapamycin in the context of Pascal’s Wager: generative pre-trained transformer perspective. Oncoscience 9, 82–84 (2022) source
• Not checked yetcites [53] Conroy, G.: Scientists used ChatGPT to generate an entire paper from scratch—but is it any good? Nature 619, 443–444 (2023) source
• Not checked yetcites [54] Conroy, G.: How ChatGPT and other AI tools could disrupt scientific publishing. Nature (2023). https://www.nature.com/articles/d41586-023-03144-w
• Not checked yetcites [167] O’Connor, S., ChatGPT: Open artificial intelligence platforms in nursing education: tools for academic progress or abuse? Nurse Educ. Pract. 66, 103537 (2023) source

In the paper

“Nevertheless, scientists can (and should) take steps to understand sources of bias and control them, especially those that can lead to discrimination, stigmatization, harm, or injustice [89, 154, 188].”

What the sources say

• Not checked yetcites [89] Haack, S.: Defending Science within Reason. Prometheus Books, New York (2007)
• Not checked yetcites [154] Mittermaier, M., Raza, M.M., Kvedar, J.C.: Bias in AI-based models for medical applications: challenges and mitigation strategies. NPJ Digit. Med. 6, 113 (2023) source
• Not checked yetcites [188] Resnik, D.B.: The Price of Truth: How Money Affects the Norms of Science. Oxford University Press, New York (2007) source

In the paper

“Trust in a technology is partially based on understanding that technology.”

In the paper

“private companies uninterested in sharing information about their systems may not comply with explainable AI requirements or they may “game” the requirements to resemble compliance without actually complying.”

In the paper

“AI biases in medical research have generated considerable concern, since biases related to race, ethnicity, gender, sexuality, age, nationality, and socioeconomic status in health-related datasets can perpetuate health disparities by supporting biased hypotheses, models, theories, and policies [177, 198, 211].”

What the sources say

• Not checked yetcites [177] Parikh, R.B., Teeple, S., Navathe, A.S.: Addressing bias in artificial intelligence in health care. J. Am. Med. Assoc. 322(24), 2377–2378 (2019) source
• Not checked yetcites [198] Röösli, E., Rice, B., Hernandez-Boussard, T.: Bias at warp speed: how AI may contribute to the disparities gap in the time of COVID-19. J. Am. Med. Inform. Assoc. 28(1), 190–192 (2021) source
• Not checked yetcites [211] Straw, I., Callison-Burch, C.: Artificial Intelligence in mental health and the biases of language based models. PLoS ONE 15(12), e0240376 (2020) source

In the paper

“ethical norms promote trust among scientists, which is essential for collaboration, peer review, publication, sharing of data and resources, mentoring, education, and other scientific activities.”

In the paper

“science’s ethical norms have changed over time, and they are likely to continue to evolve [80, 128, 147, 237].”

What the sources say

• Not checked yetcites [80] Gillispie, C.C.: The Edge of Objectivity. Princeton University Press, Princeton (1960)
• Not checked yetcites [128] Kuhn, T.S.: The Essential Tension. University of Chicago Press, Chicago (1977) source
• Not checked yetcites [147] Merton, R.: The Sociology of Science. University of Chicago Press, Chicago (1973)
• Not checked yetcites [237] Ziman, J.: Real Science. Cambridge University Press, Cambridge (2000) source

In the paper

“model-building in science may involve tradeoffs among various epistemic norms, including generality, precision, realism, simplicity, and explanatory power [143].”

What the sources say

• Not checked yetcites [143] Matthewson, J.: Trade-offs in model-building: a more target-oriented approach. Stud. Hist. Philos. Sci. Part A 42(2), 324–333 (2011) source

In the paper

“AI can reproduce and amplify biases inherent in the data and generate results that lend support to policies that are discriminatory, unfair, harmful, or ineffective [16, 202].”

What the sources say

• Not checked yetcites [16] Ball, P.: Is AI leading to a reproducibility crisis in science? Nature 624, 22–25 (2023) source
• Not checked yetcites [202] Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., Hal, P.: Towards a standard for identifying and managing bias in artificial intelligence. National Institute of Standards and Technology (2022). https://view.ckcest.cn/AllFiles/ZKBG/Pages/264/c914336ac0e68a6e3e34187adf9dd83bb3b7c09f.pdf

In the paper

“If we want to use LLMs to make sound moral and political judgments, sanitizing their data processing and output may hinder their ability to excel at this task, because the ability to make sound moral judgements or anticipate harm may depend, in part, on some familiarity with immoral choices and the darker side of humanity.”

In the paper

“Strategies for reducing errors in science include time-honored quality assurance and quality improvement techniques, such as auditing data, instruments, and systems; validating and testing instruments that analyze or process data; and investigating and analyzing errors [1].”

What the sources say

• Not checked yetcites [1] Aboumatar, H., Thompson, C., Garcia-Morales, E., Gurses, A.P., Naqibuddin, M., Saunders, J., Kim, S.W., Wise, R.: Perspective on reducing errors in research. Contemp. Clin. Trials Commun. 23, 100838 (2021) source

In the paper

“honesty is the hallmark of good science, and there are very few situations in which scientists are justified in deviating from this norm.”

In the paper

“Openness, on the other hand, can be deemphasized to protect research participants’ privacy, intellectual property, classified information, or unpublished research [207].”

What the sources say

• Not checked yetcites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

In the paper

“given the accessibility of AI systems and the fact that they can exploit our cognitive shortcomings, they are creating an illusion of understanding [148].”

What the sources say

• Not checked yetcites [148] Messeri, L., Crockett, M.J.: Artificial intelligence and illusions of understanding in scientific research. Nature (2024). https://doi.org/10.1038/s41586-024-07146-0 source

In the paper

“This evolution is in response to changes in science’s social, institutional, economic, and political environment and advancements in scientific instruments, tools, and methods [100].”

What the sources say

• Not checked yetcites [100] Hosseini, M., Senabre Hidalgo, E., Horbach, S.P.J.M., Güttinger, S., Penders, B.: Messing with Merton: the intersection between open science practices and Mertonian values. Accountability in Research, pp. 1–28 (2022) source

In the paper

“an excessive amount when using AI could be considered carelessness or recklessness when using AI”

In the paper

“Explainability, according to proponents of this approach, helps to promote trust in AI because it allows users and other stakeholders to make rational and informed decisions about it [77, 83, 110, 186].”

What the sources say

• Not checked yetcites [77] Fuhrman, J.D., Gorre, N., Hu, Q., Li, H., El Naqa, I., Giger, M.L.: A review of explainable and interpretable AI with applications in COVID-19 imaging. Med. Phys. 49(1), 1–14 (2022) source
• Not checked yetcites [83] Google.: Responsible AI practices (2023). https://ai.google/responsibility/responsible-ai-practices/
• Not checked yetcites [110] IBM.: Explainable AI (2023). https://www.ibm.com/topics/explainable-ai
• Not checked yetcites [186] Ratti, E., Graves, M.: Explainable machine learning practices: opening another black box for reliable medical AI. AI Ethics 2, 801–814 (2022) source

In the paper

“Although vital for conducting science, ethical norms are not rigid rules. Norms sometimes conflict, and when they do, scientists must make decisions concerning epistemic or moral priorities [191].”

What the sources say

• Not checked yetcites [191] Resnik, D.B., Elliott, K.C.: Value-entanglement and the integrity of scientific research. Stud. Hist. Philos. Sci. 75, 1–11 (2019) source

In the paper

“placebo control groups are not used in clinical trials when receiving a placebo instead of an effective treatment would cause serious harm to the participant [207].”

What the sources say

• Not checked yetcites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

In the paper

“Research with human subjects often involves tradeoffs between rigor and protection of participants.”

In the paper

“It is nearly impossible to understand how a natural or social process works or make accurate predictions about it without standards pertaining to honesty, logical consistency, empirical support, and reproducibility of data and results.”

In the paper

“If we try to weed undesirable features of this data, we will eliminate parts of our language and culture, and ultimately, parts of us.”

In the paper

“It is only by understanding evil that we can freely and rationally choose the good [40].”

What the sources say

• Not checked yetcites [40] Calder, T.: The concept of evil. Stanford Encyclopedia of Philosophy (2022). https://plato.stanford.edu/entries/concept-evil/#KanTheEvi

In the paper

“by taking machines’ disinterestedness in findings as a necessary and sufficient condition of objectivity, users of AI in research may overestimate the objectivity of their findings.”

In the paper

“Transparency about these is important for making research trustworthy and reproducible [16].”

What the sources say

• Not checked yetcites [16] Ball, P.: Is AI leading to a reproducibility crisis in science? Nature 624, 22–25 (2023) source

In the paper

“Also, as mentioned in Sect. 7.3, the use of synthetic data may help to protect confidentiality.”

In the paper

“using synthetic data in research, even appropriately, may blur the line between real and fake data and undermine data integrity.”

In the paper

“Regarding the second reason, when we hold humans accountable, we expect them to explain their behavior in clear and intelligible language.”

In the paper

“These systems do not have interests, values, reputation and feelings in the same way that humans do and cannot be punished by law enforcement.”

In the paper

“researchers can use synthetic data to develop and validate models and enhance statistical analysis.”

In the paper

“Some US courts now require lawyers to disclose their use of generative AI in preparing legal documents and make assurances that they have taken appropriate steps to protect confidentiality [146].”

What the sources say

• Not checked yetcites [146] Merken, S.: Another US judge says lawyers must disclose AI use. Reuters (2023). https://www.reuters.com/legal/transactional/another-us-judge-says-lawyers-must-disclose-ai-use-2023-06-08/

In the paper

“Electronic medical records, for example, have this type of security [127].”

What the sources say

• Not checked yetcites [127] Kruse, C.S., Smith, B., Vanderlinden, H., Nealand, A.: Security techniques for the electronic health records. J. Med. Syst. 41(8), 127 (2017) source

In the paper

“The last topic we will address in this section has to do with education and mentoring in responsible conduct of research (RCR), which is widely recognized as essential to promoting ethical judgment, reasoning, and behavior in science [207].”

What the sources say

• Not checked yetcites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

In the paper

“it is only a matter of time until this happens, given the pressures to produce results, publish, and obtain grants, and the temptations to cheat or cut corners.”

In the paper

“Authorship on scientific papers is based not only on making a substantial contribution, but also on being accountable for the work [207].”

What the sources say

• Not checked yetcites [207] Shamoo, A.E., Resnik, D.B.: Responsible Conduct of Research, 4th edn. Oxford University Press, New York (2022) source

In the paper

“In the US, the NIH and National Science Foundation (NSF) require RCR education for funded students and trainees, and many academic institutions require some form of RCR training for all research faculty [190].”

What the sources say

• Not checked yetcites [190] Resnik, D.B., Dinse, G.E.: Do U.S. research institutions meet or exceed federal mandates for instruction in responsible conduct of research? A national survey. Acad. Med. 87, 1237–1242 (2012) source

In the paper

“(3) they cannot suffer consequences nor can be sanctioned.”

In the paper

“Currently, there seems to be an emerging consensus that falls in between these two extremes position, namely, that AI systems can be used in preparing manuscripts but that their use should be appropriately disclosed and discussed, [4, 102].”

What the sources say

• Not checked yetcites [4] All European Academies.: The European Code of Conduct for Research Integrity, Revised Edition 2023 (2023). https://allea.org/code-of-conduct/
• Not checked yetcites [102] Hosseini, M., Resnik, D.B., Holmes, K.: The ethics of disclosing the use of artificial intelligence in tools writing scholarly manuscripts. Res. Ethics (2023). https://doi.org/10.1177/17470161231180449 source

In the paper

“Researchers who use generative AI to edit or review a document should assume that the material contained in it will not be kept confidential”

In the paper

“making AI explainable does not completely solve the black box problem but it can play a key role in promoting transparency, accountability, and trust [7, 9].”

What the sources say

• Not checked yetcites [7] Alvarado, R.: What kind of trust does AI deserve, if any? AI Ethics (2022). https://doi.org/10.1007/s43681-022-00224-x source
• Not checked yetcites [9] Alvarado, R.: AI as an epistemic technology. Sci. Eng. Ethics 29, 32 (2023) source

In the paper

“While the concept of moral agency, like the concept of consciousness, is controversial, there is general agreement that moral agency requires the capacity to perform intentional (or purposeful) actions, understand moral norms, and make decisions based on moral norms.”

What the sources say

• Not checked yetcites [46] Cervantes, J.A., López, S., Rodríguez, L.F., Cervantes, S., Cervantes, F., Ramos, F.: Artificial moral agents: a survey of the current status. Sci. Eng. Ethics 26(2), 501–532 (2020) source
• Not checked yetcites [95] Himma, K.E.: Artificial agency, consciousness, and the criteria for moral agency: what properties must an artificial agent have to be a moral agent? Ethics Inf. Technol. 11, 19–29 (2009) source
• Not checked yetcites [213] Talbert, M.: Moral responsibility. Stanford Encyclopedia of Philosophy (2019). https://plato.stanford.edu/entries/moral-responsibility/

In the paper

“scientists may still be liable for misconduct due to recklessness [42, 193, 194], which may have consequences for using AI.”

What the sources say

• Not checked yetcites [42] Caron, M.M., Dohan, S.B., Barnes, M., Bierer, B.E.: Defining "recklessness" in research misconduct proceedings. Accountability in Research, pp. 1–23 (2023) source
• Not checked yetcites [193] Resnik, D.B., Elliott, K.C., Soranno, P.A., Smith, E.M.: Data-intensive science and research integrity. Account. Res. 24(6), 344–358 (2017) source
• Not checked yetcites [194] Resnik, D.B., Smith, E.M., Chen, S.H., Goller, C.: What is recklessness in scientific research? The Frank Sauer case. Account. Res. 24(8), 497–502 (2017) source

In the paper

“This became a major issue for the ethical norms of science in the winter of 2022–2023, when some researchers listed ChatGPT as authors on papers [102].”

What the sources say

• Not checked yetcites [102] Hosseini, M., Resnik, D.B., Holmes, K.: The ethics of disclosing the use of artificial intelligence in tools writing scholarly manuscripts. Res. Ethics (2023). https://doi.org/10.1177/17470161231180449 source

In the paper

“For example, if an organization operates a local application of an LLM and places the technology behind a secure firewall, its members can use the technology safely.”

What the sources say

• Not checked yetcites [127] Kruse, C.S., Smith, B., Vanderlinden, H., Nealand, A.: Security techniques for the electronic health records. J. Med. Syst. 41(8), 127 (2017) source

In the paper

“The black box problem presents significant challenges to the trustworthiness and transparency of research that use AI because some of the steps in the scientific process will not be fully open and understandable to humans, including AI experts.”

In the paper

“Some universities have already begun experimenting with operating their own AI systems for use by students, faculty, and administrators [225].”

What the sources say

• Not checked yetcites [225] University of Michigan.: Generative artificial intelligence (2023). https://genai.umich.edu/

In the paper

“As demonstrated in this paper, the use of AI in research has a direct bearing on most of these topics, but especially on authorship, rigor and reproducibility, peer review, and social responsibility.”

In the paper

“Although scientists do not need to radically revise their ethical norms to deal with these issues, they do need new guidance for the appropriate use of AI in research.”

In the paper

“Using AI in research benefits science and society but also creates some novel and complex ethical issues that affect accountability, responsibility, transparency, trustworthiness, reproducibility, fairness, and objectivity, and other important values in research.”