Friend or foe? Exploring the implications of large language models on the science system

Fecher, Benedikt; Hebing, Marcel; Laufer, Melissa; Pohle, Jörg; Sofsky, Fabian

doi:10.1007/s00146-023-01791-1

Cited by 78Ethics and Social Impacts of AIArtificial Intelligence ApplicationsArtificial Intelligence in Healthcare and Education

Friend or foe? Exploring the implications of large language models on the science system

Open access

Fecher, Benedikt, Hebing, Marcel, Laufer, Melissa, Pohle, Jörg, Sofsky, Fabian·2025·AI & SOCIETY

DOI 10.1007/s00146-023-01791-1

Open this paper in CoChat — free account Open PDF

Abstract

The advent of ChatGPT by OpenAI has prompted extensive discourse on its potential implications for science and higher education. While the impact on education has been a primary focus, there is limited empirical research on the effects of large language models (LLMs) and LLM-based chatbots on science and scientific practice. To investigate this further, we conducted a Delphi study involving 72 researchers specializing in AI and digitization. The study focused on applications and limitations of LLMs, their effects on the science system, ethical and legal considerations, and the required competencies for their effective use. Our findings highlight the transformative potential of LLMs in science, particularly in administrative, creative, and analytical tasks. However, risks related to bias, misinformation, and quality assurance need to be addressed through proactive regulation and science education. This research contributes to informed discussions on the impact of generative AI in science and helps identify areas for future action.

AI snapshot

Written from the paper’s full text.

TL;DR

A two-round Delphi study with 72 researchers specializing in AI and digitization finds that experts expect large language models to transform science mainly as time-saving tools for administrative, creative, and analytical tasks, while also posing risks around bias, misinformation, and quality assurance. Most experts see the benefits as outweighing the drawbacks but call for proactive regulation and science education.

Key findings

Experts identified six main applications of LLMs in science: text improvement, text summary, text analysis, code writing, idea generation, and text translation. In round two, text improvement ranked first, text summary second, and code writing third.
Most experts (59.6%) already use or intend to use LLMs in their own work, and 86.5% perceive LLMs as valuable for administrative tasks. A large majority (82.7%) disagrees that LLMs could replace researchers.
The highest-ranked limitation was incorrectness, followed by non-transparency and unspecificity. The highest-ranked risk was bias, followed by disinformation and overburdening academic quality assurance mechanisms; 75.0% regard LLMs as a catalyst for disinformation.
A large majority (63.5%) believe LLMs should be subject to stronger regulations. Experts ranked a reflective mindset as the top required competency, followed by the ability to contextualize results and ethical understanding.

Methods

The study used a two-stage Delphi method with researchers working at the intersection of science, technology, and society, recruited via convenience sampling through professional networks and institutional mailing lists.
The first survey contained 12 open questions and yielded 72 responses; the second survey contained 11 questions, mostly ranking and Likert-scale items, and yielded 52 responses (72% of first-round participants).
Responses were analyzed with inductive and deductive coding; ranking questions were scored by frequency and weighted rank, and Likert-scale agreement combined “agree” and “strongly agree.”

Limitations

The paper states that it could not track long-term implications because the interval between data collections was relatively short.
The paper notes that the sampling strategy may have involved technology-proficient experts, which could explain the generally optimistic view that benefits outweigh drawbacks.
The paper states that ethical and legal implications were not ranked in the second Delphi round. Evident from the text: the findings are expert opinions and expectations rather than observed effects, and the study is time-bound to early ChatGPT versions and their known limitations.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s00146-023-01791-1

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 26 cited sources checked · 0 supported · 0 flagged

Findingown data
86.5% of experts perceive LLMs as valuable for administrative tasks.
“A significant portion (86.5%) of the experts perceive LLMs as valuable for administrative tasks, confirming the assumption that time savings are expected for researchers through LLM utilization (Fig. 2).”
Quote in the paperNot a substantial claim
Findingown data
59.6% of experts either already use or express their intention to use LLMs in their own work.
“Most (59.6%) of the experts either already use or express their intention to use LLMs in their own work (Fig. 2).”
Quote in the paperNot a substantial claim
Findingown data
The incorrectness of LLMs was a dominant and recurring issue mentioned by the experts.
“The incorrectness of LLMs was a dominant and recurring issue mentioned by the experts.”
Quote in the paperSubstantial claim
Findingown data
The initial round of the Delphi survey revealed that the ethical implications discussed frequently underscore the significance of the human element in scientific endeavors.
“The initial round of the Delphi survey revealed that the ethical implications discussed frequently underscore the significance of the human element in scientific endeavors.”
Quote in the paperNot a substantial claim
Findingown data
A large majority of experts (63.5%) believe that LLMs should be subject to stronger regulations.
“A large majority of the experts (63.5%) believe that LLMs should be subject to stronger regulations (Fig. 2).”
Quote in the paperSubstantial claim

Findingreasoning
A majority of respondents assume that ChatGPT and other LLMs will transform scientific practice.
“Taken together, it is not surprising that a majority of the respondents assume that ChatGPT and other LLMs will transform scientific practice, although this might—at this stage—relate primarily to the textuality of academic work.”
Quote in the paperNot a substantial claim
Findingown data
The majority of experts saw no danger that LLMs will replace researchers in the foreseeable future.
“It is important to note the majority of the experts saw no danger that LLMs will replace researchers in the foreseeable future.”
Quote in the paperSubstantial claim
Findingown data
In the second phase, experts ranked a reflective mindset first, followed by the ability to contextualize results and ethical understanding.
“In the second phase, they ranked a reflective mindset first, followed by the ability to contextualize results and ethical understanding.”
Quote in the paperSubstantial claim
Findingown data
Experts anticipate feedback effects on science.
“The results indicate that the experts anticipate feedback effects on science, while also suggesting that the responsible application of knowledge will become even more paramount in the future.”
Quote in the paperSubstantial claim
Findingown data
Text improvement (rephrasing and optimization of textual content) was considered the most important application.
“For example, text improvement as in the rephrasing and optimization of textual content was considered the most important application.”
Quote in the paperNot a substantial claim
Findingown data
The majority of experts perceive the benefits of LLMs to outweigh the drawbacks.
“Notwithstanding the gravity of the aforementioned risks, the majority of experts perceive the benefits of LLMs to outweigh the drawbacks (Fig. 2), which explains why most of them already use or intend to use LLMs in their work.”
Quote in the paperSubstantial claim
Findingown data
The majority of experts deem ChatGPT unfit for authorship due to its inability to assume responsibility for the results.
“The comments show clearly that the majority deem ChatGPT unfit for authorship due to its inability to assume responsibility for the results.”
- Also stated in: § 5 Discussion · plain
  “The majority of the experts deem that LLMas cannot claim authorship due to its inability to assume responsibility for its actions.”
Quote in the paperSubstantial claim
Findingreasoning
Many respondents identify privacy and data protection law issues as considerable legal challenges.
“certainly explains why many respondents identify privacy and data protection law issues as considerable legal challenges.”
Quote in the paperNot a substantial claim
- Rests on an assumption: There is a lack of transparency regarding the personal data on which LLMs were trained and the further possible uses of personal data generated by the use of the tools.
Findingreasoning
A large majority of experts identify issues with copyright law as a pressing legal implication.
“it is not surprising that a large majority of the experts identify issues with copyright law as a pressing legal implication.”
Quote in the paperNot a substantial claim
- Rests on an assumption: LLMs are trained on works produced by others and produce (or co-produce) works, both of which almost certainly fall under copyright law.
Findingown data
Experts in our study were more cautious with revolutionary claims, seeing changes as more incremental and pragmatic.
“In contrast, experts in our study were more cautious with such claims seeing these changes as more incremental and pragmatic.”
Quote in the paperSubstantial claim
Findingown data
ChatGPT and other LLMs were collectively understood as potential 'time-savers' to be used to improve and streamline the writing process, especially academic writing.
“In general, ChatGPT and other LLMs were collectively understood as potential 'time-savers' to be used to improve and streamline the writing process, especially academic writing.”
Quote in the paperNot a substantial claim
Premisecitation
There is limited empirical research on the effects of large language models (LLMs) and LLM-based chatbots on science and scientific practice.
“While the impact on education has been a primary focus, there is limited empirical research on the effects of large language models (LLMs) and LLM-based chatbots on science and scientific practice.”
- Also stated in: § 1 Introduction · plain
  “there is much less-especially empirical research-on the implications of LLMs as well as LLM-based chatbots or prompts on scholarly practices and the science system”
Quote in the paperSubstantial claim0 of 2 sources checked
- cites Ribeiro et al. 2023 source
  Not checked in the free run.
- cites Chubb et al. 2022 source
  Not checked in the free run.
Premisecitation
The literature on knowledge work and the transformative effects of AI cannot account for the complexities of specific practices.
“However, the literature on knowledge work and the transformative effects of AI cannot account for the complexities of specific practices (Jiang et al. 2022).”
Quote in the paperSubstantial claim0 of 1 sources checked
- cites Jiang et al. 2022
  Not checked in the free run.
Interpretationown data
Experts anticipate that the utilization of LLMs will have a transformative and largely positive impact on science and scientific practice.
“Our findings indicate that experts anticipate that the utilization of LLMs will have a transformative and largely positive impact on science and scientific practice.”
Quote in the paperSubstantial claim
- The authors state this as an assumption: Many limitations of existing LLMs can be addressed in newer models.
- Rests on an assumption: The recruited experts are sufficiently knowledgeable to provide informed assessments about the impact of LLMs on science.
Interpretationown data
Our study indicates that LLMs have the potential to reshape the science system.
“In addition, our study indicates that LLMs have the potential to reshape the science system.”
Quote in the paperSubstantial claim
Premisecitation
To date, scholars have primarily focused on the implications of LLMs on education, with limited attention to their impact on science and scientific practices.
“To date, scholars have primarily focused on the implications of LLMs on education (e.g., Perkins 2023; Fyfe 2023) with limited attention being paid to their impact on science and scientific practices (for exceptions, see Chubb et al. 2022;Ribeiro et al. 2023).”
Quote in the paperSubstantial claim0 of 3 sources checked
- cites Perkins 2023 source
  Not checked in the free run.
- cites Fyfe 2023 source
  Not checked in the free run.
- cites Chubb et al. 2022 source
  Not checked in the free run.
Study design and conductreasoning
The finding that most experts use or intend to use LLMs may be attributed to the convenience sampling strategy, possibly involving technology-proficient experts.
“This, however, can also be attributed to the sampling strategy employed in this study, possibly involving technology-proficient experts.”
Quote in the paperSubstantial claim
Interpretationown data
There is evidence that LLMs are relevant for ideation, conception, and programming.
“There is also evidence to suggest that LLMs are relevant for ideation, conception, and programming, the latter of which is an increasingly important scientific practice.”
Quote in the paperNot a substantial claim
Interpretationown data
Researchers see LLMs primarily as a tool to relieve and simplify their workload.
“These results indicate that researchers see LLMs primarily as a tool to relieve and simplify their workload.”
Quote in the paperSubstantial claim

May combine two claims. 31
Classified as probably not a substantial claim. 54
The quote only partly states this claim. 1

May combine two claims.
Findingown data
Text improvement is considered the most important application, followed by text summary as the second most important, and code writing as the third most important application.
“Our results show that text improvement is considered the most important application, followed by text summary as the second most important, and code writing as the third most important application.”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim. May combine two claims.
Findingown data
The experts highlighted key shortcomings such as AI-produced texts may have incorrect information, their origin and referencing is non-transparent, and they lack specificity, which are at odds with principles of good scientific practice.
“The experts highlighted key shortcomings such as AI-produced texts may have incorrect information, their origin and referencing is non-transparent and that they lack specificity, shortcomings which are at odds with the principles of good scientific practice.”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim.
Findingown data
Experts propose that generative AI could automate administrative and generic tasks, freeing up time for critical reflection, analysis and innovation.
“The experts propose that generative AI could automate administrative and generic tasks, freeing up time for critical reflection, analysis and innovation.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data
Experts mention the amount of energy necessary for training models and running inference and the CO2 footprint as primary examples of ecological sustainability issues presented by ChatGPT and similar tools.
“The amount of energy that is necessary both for training models and running inference and the CO 2 footprint are mentioned as primary examples for the ecological sustainability issues ChatGPT and the like present.”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim.
Findingown data
It is the prevailing perception of the experts in our study that suggests that the positive transformative scenario is more likely to occur.
“It is the prevailing perception of the experts in our study that suggests that this scenario is more likely to occur.”
Quote in the paperSubstantial claim
May combine two claims.
Findingown data
Overall, the experts in our study were optimistic and in agreement that the advantages of this technology outweigh their disadvantages.
“Overall, the experts in our study were optimistic and in agreement that the advantages of this technology outweigh their disadvantages.”
Quote in the paperSubstantial claim
May combine two claims.
Findingown data
Experts in our study were aware of the limitations of LLMs and cited similar apprehensions to those raised in the literature.
“At the same time, experts in our study were aware of the limitations of LLMs and cited similar apprehensions to those raised in the literature (Dwivedi et al. 2023;OpenAI 2022;Perkins 2023).”
Quote in the paperNot a substantial claim0 of 3 sources checked
- cites Dwivedi et al. 2023 source
  Not checked in the free run.
- cites OpenAI 2022
  Not checked in the free run.
- cites Perkins 2023 source
  Not checked in the free run.
May combine two claims.
Findingown data
The reduction of administrative tasks was ranked first, followed by more efficiency and inclusiveness.
“In the second phase of the Delphi study, the experts ranked these, with the reduction of administrative tasks ranked first, followed by more efficiency and inclusiveness (see Table 4).”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown data
Bias is seen as the biggest threat, followed by disinformation and overburdening academic quality assurance mechanisms.
“In the second phase, the experts ranked these, indicating that bias is seen as the biggest threat, followed by disinformation and overburdening academic quality assurance mechanisms (see Online appendix Table 5).”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim.
Findingown data
Experts recognize significant potential for LLMs in administrative, creative, and analytical tasks.
“In LLMs, they recognize significant potential for administrative, creative, and analytical tasks.”
Quote in the paperSubstantial claim
- Rests on an assumption: The recruited experts are sufficiently knowledgeable to provide informed assessments about the impact of LLMs on science.
May combine two claims.
Findingnone given
The most pressing fear identified is that LLMs perpetuate disinformation and will overburden quality assurance mechanisms in academia.
“The most pressing fear we identified is that LLMs perpetuate disinformation and will overburden quality assurance mechanisms in academia.”
Quote in the paperSubstantial claim
May combine two claims.
Findingown data
The study reveals two possible scenarios: a utopian transformative scenario and a dystopian deformative scenario.
“Based on the answers to these questions, our study reveals two possible scenarios, namely (1) a utopian transformative scenario and a (2) dystopian deformative scenario.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Findingnone given
The issue of access was highlighted as an ethical dilemma, that is, not all researchers will have equal access to such technologies, potentially furthering inequalities.
“In addition, the issue of access was highlighted as an ethical dilemma, that is, not all researchers will have equal access to such technologies, potentially furthering inequalities.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable).
Findingnone given
This study aided in disentangling the ethical and legal implications of ChatGPT and other LLMs.
“This study aided in disentangling the ethical and legal implications of ChatGPT and other LLMs.”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim. May combine two claims. The quote only partly states this claim.
Findingown data
According to the experts, the use of LLMs provides the science system with opportunities: efficiency, reflection, reduced administrative workload, inclusiveness, and productivity.
“According to the experts, the use of LLMs provides the science system with four opportunities: (1) LLMs can promote efficiency by automating and supporting text work, (2) LLMs may promote reflection by identifying biases and new research areas, (3) LLMs may reduce administrative workload, (4) LLMs can promote inclusiveness by leveling the playing field between researchers from different”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim.
Findingown data
Experts pointed out that the researcher's role in the writing process will shift from being the originator of ideas and texts to being required to contextualize and reflect on AI-generated results.
“Furthermore, the experts pointed out that the researcher's role in the writing process will shift from being the originator of ideas and texts to being required to contextualize and reflect on AI-generated results.”
Quote in the paperSubstantial claim
May combine two claims.
Findingown data
Overall, there are significantly more indications (in terms of number of codes) for a positive scenario, confirmed by the opinion battery in Phase 2.
“However, overall, there are significantly more indications (in terms of the number of codes) for a positive scenario, which was also confirmed by the opinion battery in Phase 2.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown data
The non-transparency in the training data remains problematic and was viewed by some as inconsistent with scientific principles of quality.
“However, the non-transparency in the training data remains problematic and was viewed by some as inconsistent with scientific principles of quality.”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim. May combine two claims.
Findingown data
The human element includes responsibility and accountability of individuals, the value of creativity and generating novel ideas, ensuring equitable access to science, and addressing the potential risk of dependency on LLM-based tools that may hinder individual skills and capabilities in scientific work.
“This includes the responsibility and accountability of individuals for their contributions, the value of creativity and generating novel ideas, ensuring equitable access to science and the scientific community, and addressing the potential risk of dependency on LLM-based tools that may hinder individual skills and capabilities in scientific work.”
Quote in the paperSubstantial claim
May combine two claims.
Findingown data
The analysis identifies three key aspects of generative AI's impact on science in the utopian scenario: streamlining repetitive tasks, promoting inclusivity, and facilitating interdisciplinary research.
“Our analysis identifies three key aspects of its impact on science: (1) streamlining repetitive tasks, (2) promoting inclusivity, and (3) facilitating interdisciplinary research.”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim.
Findingcitation
Experts in our study voiced concerns that ChatGPT and other LLMs have the potential to widen the digital divide between researchers who possess technical know-how and researchers who do not.
“In line with scholars such as Teubner et al. (2023), experts in our study voiced concerns that ChatGPT and other LLMs have the potential to widen the digital divide between researchers who possess technical know-how and researchers who do not.”
Quote in the paperSubstantial claim0 of 1 sources checked
- cites Teubner et al. (2023) source
  Not checked in the free run.
Classified as probably not a substantial claim.
Findingown data
Our study provided insights into the competencies researchers need to be able to utilize LLMs.
“Moreover, our study provided insights into the competencies researchers need to be able to utilize LLMs.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone given
The negative scenario is almost a negation of the positive scenario and vice versa.
“It is noteworthy that the negative scenario is almost a negation of the positive scenario and vice versa.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Findingnone given
Dependence on commercial providers for generative AI tools raises concerns among experts about the lack of independence and control over scientific research, potentially leading to conflicts of interest and biases in research results.
“Additionally, dependence on commercial providers for generative AI tools raises concerns among experts about the lack of independence and control over scientific research, potentially leading to conflicts of interest and biases in research results.”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim.
Findingown data
Experts perceive legal implications regarding copyright, data protection, and liability.
“The experts perceive legal implications regarding (1) copyright, due to the unclear infringement of intellectual property by LLMs, (2) data protection, due to the ambiguity of the data used and how OpenAI utilizes input data, and (3) liability, due to the uncertainty of the extent to which LLMs can be held responsible for criminal errors.”
Quote in the paperSubstantial claim
May combine two claims.
Findingown data
Experts anticipate that LLMs will lead to more efficient workflows, with the reduction of administrative tasks being ranked the highest anticipated change.
“The experts anticipate that they will lead to more efficient workflows, with the reduction of administrative tasks being ranked the highest anticipated change.”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim.
Findingown data
Our study presents potential future pathways for AI and its impact on the science system and society in the form of future scenarios constructed from our data.
“Lastly, our study presents potential future pathways for AI and its impact on the science system and society in the form of future scenarios constructed from our data.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S1 (asserted). May combine two claims.
Findingown data
Issues such as decreased scientific rigor, reproducibility, and a potential homogenization of science were highlighted.
“Issues such as decreased scientific rigor, reproducibility, and a potential homogenization of science were highlighted.”
Quote in the paperNot a substantial claim
May combine two claims.
Findingown data
The main risks associated with LLMs pertain to issues of bias, misinformation, and overburdening of the scientific quality assurance system.
“The main risks associated with LLMs pertain to issues of bias, misinformation, and overburdening of the scientific quality assurance system.”
Quote in the paperSubstantial claim
- Rests on an assumption: The recruited experts are sufficiently knowledgeable to provide informed assessments about the impact of LLMs on science.
Classified as probably not a substantial claim.
Findingreasoning
The experts centered on distilling the role humans play in being accountable for their usage of LLMs, taking into consideration issues such as plagiarism, copyright and data protection.
“In this light, the experts centered on distilling the role humans play in being accountable for their usage of LLMs, taking into consideration issues such as plagiarism, copyright and data protection.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Findingcitation
The findings provide additional clarity on the matter of authorship concerning AI utilization.
“The findings provide additional clarity on the matter of authorship concerning AI utilization, a subject also explored by other scholars (Lucey and Dowling 2023;Tomlinson et al. 2023).”
Quote in the paperNot a substantial claim0 of 2 sources checked
- cites Lucey and Dowling 2023
  Not checked in the free run.
- cites Tomlinson et al. 2023
  Not checked in the free run.
Classified as probably not a substantial claim. May combine two claims.
Findingreasoning
The analysis reveals three crucial aspects of the dystopian scenario: decline in research quality due to plausible yet flawed results, loss of research diversity through amplifying mainstream voices, and decrease in scientific integrity due to AI-generated content raising risks of predatory publishing and false information.
“In a dystopian scenario, the anticipated positive impacts of generative AI are largely negated, as our analysis reveals three crucial aspects: (1) a decline in research quality due to plausible yet flawed results, compromising reliability and validity; (2) a loss of research diversity through amplifying mainstream voices, resulting in missed opportunities for novel perspectives; and (3) a decrease”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim. May combine two claims.
Findingcitation
Experts underlined that human responsibility in AI usage is both a legal and ethical challenge.
“Thus, they underlined that human responsibility in AI usage is both a legal and ethical challenge, a sentiment that echoes the arguments of critics who postulate that chatbots cannot take responsibility for their actions (Corless 2023;Liebrenz et al. 2023).”
Quote in the paperSubstantial claim0 of 2 sources checked
- cites Corless 2023
  Not checked in the free run.
- cites Liebrenz et al. 2023 source
  Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown data
Experts expressed the importance of researchers having an ethical understanding, e.g., using AI in a responsible manner.
“Our experts also expressed the importance of researchers having an ethical understanding, e.g., using AI in a responsible manner.”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim.
Findingown data
We were able to identify patterns emerging regarding potential opportunities and risks.
“We were also able to identify patterns emerging regarding potential opportunities and risks.”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim.
Findingnone given
Experts raised concerns about the impact of generative AI on scientific quality, integrity, and the scientific ecosystem.
“Experts raised concerns about the impact of generative AI on scientific quality, integrity, and the scientific ecosystem.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given
The open questions were analyzed with a combination of inductive and deductive coding, carried out jointly by the authors.
“The open questions were analyzed with a combination of inductive and deductive coding, carried out jointly by the authors.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning
The Delphi method is a fitting technique to investigate the impact of ChatGPT and LLMs on scientific practices and the science system.
“In this light, the Delphi method is a fitting technique to investigate our objective of exploring the impact of ChatGPT and LLMs on scientific practices and the science system.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown data
The second survey yielded 52 responses, which is 72% of the participants from the first round.
“The survey was sent to the same experts, yielding 52 responses (72% of the participants from the first round).”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim. May combine two claims.
Premisecitation
The Delphi method is a technique used to establish consensus among a group of experts on complex issues and in some cases used to forecast future developments.
“First developed in the 1960s, the Delphi method is a technique used to establish consensus among a group of experts on complex issues (Landeta 2006) and in some cases used to forecast future developments (Linstone and Turoff 1975).”
Quote in the paperSubstantial claim0 of 2 sources checked
- cites Landeta 2006 source
  Not checked in the free run.
- cites Linstone and Turoff 1975
  Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning
These risks are significant as they touch on fundamental pillars of scientific ethics and good practice.
“These risks are significant as they touch on fundamental pillars of scientific ethics and good practice, such as scientific freedom regarding the dependence on commercial publishers, scientific quality assurance concerning the handling of highly generic publications, as well as the public legitimation of science, which could be put into question by plausible and seemingly scientific nonsense”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationown data
Our study presents a consistent picture from experts which can further our understanding of future expectations of LLMs.
“Nevertheless, our study presents a consistent picture from experts which can further our understanding of future expectations of LLMs.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation
CO2 emissions generated by the use of AI technologies pose environmental risks.
“Furthermore, the CO 2 emissions generated by these use of AI technologies poses environmental risks (Hao 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites Hao 2019
  Not checked in the free run.
May combine two claims.
Interpretationreasoning
The potential benefits of LLMs lie primarily in text-based work, which is significant because scientific value creation in most disciplines is text based.
“The results indicate that the potential benefits of LLMs lie not only but primarily in text-based work, which is significant because scientific value creation in most disciplines is text based.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning
Responsible application of knowledge will become even more paramount in the future.
“The results indicate that the experts anticipate feedback effects on science, while also suggesting that the responsible application of knowledge will become even more paramount in the future.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Interpretationreasoning
Reflexivity highlights the ethical implications of AI on scientific practices and ways to proactively address them, while contextuality focuses on the practical use of AI-supported findings and strategies for maximizing their utility.
“It can be argued that reflexivity highlights the ethical implications of AI on scientific practices and ways to proactively address them, while contextuality focuses on the practical use of AI-supported findings and strategies for maximizing their utility.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). May combine two claims.
Premisecitation
Other scholars argue that LLMs will help automate mundane tasks and free up space for creative thinking.
“This forecast supports claims made by other scholars, who argue that LLMs will help automate mundane tasks and free up space for creative thinking (Lund et al. 2023;Dwivedi et al. 2023).”
Quote in the paperNot a substantial claim0 of 2 sources checked
- cites Lund et al. 2023 source
  Not checked in the free run.
- cites Dwivedi et al. 2023 source
  Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning
The perpetuation of plausible nonsense could further have negative consequences for society when policy decisions or public opinions rely on unreliable information.
“The perpetuation of plausible nonsense could further have negative consequences for society when policy decisions or public opinions rely on unreliable information.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation
Researchers in the field of AI and work, in their optimistic scenarios, expect a shift in tasks, the creation of new tasks and an emergence of new work profiles rather than the replacement of workers.
“They share this expectation with researchers in the field of AI and work, who in their optimistic scenarios expect a shift in tasks, the creation of new tasks and an emergence of new work profiles rather than the replacement of workers (Deranty and Corbin 2022).”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites Deranty and Corbin 2022 source
  Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation
The scholarly discourse highlights how generative AI can be used to enhance texts, such as with brainstorming, crafting literature reviews, and improving text clarity.
“This outcome resonates with the scholarly discourse which highlights how generative AI can be used to enhance texts, such as with brainstorming (Staiman 2023), crafting literature reviews (Lucey and Dowling 2023), and improving text clarity (Lund et al. 2023).”
Quote in the paperNot a substantial claim0 of 3 sources checked
- cites Staiman 2023
  Not checked in the free run.
- cites Lucey and Dowling 2023
  Not checked in the free run.
- cites Lund et al. 2023 source
  Not checked in the free run.
May combine two claims.
Interpretationown data
The result that the majority of experts perceive benefits outweighing drawbacks supports the hypothesis that generative AI will change scientific work in the long run.
“This result is noteworthy nonetheless and supports the hypothesis that generative AI will change scientific work in the long run.”
Quote in the paperSubstantial claim
May combine two claims.
Interpretationreasoning
Our findings point to a double-edged sword embedded within the LLM constellation: this technology could serve to both promote inclusion and reinforce biases.
“For example, our findings point to a double-edged sword embedded within the LLM constellation: this technology could serve to both promote inclusion and reinforce biases.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning
Our study highlights the great potential generative AI has for transforming the science system.
“Our study highlights the great potential generative AI has for transforming the science system.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning
This change will entail a new way of thinking about key scientific practices and the role the individual academic plays in them.
“This change will entail a new way of thinking about key scientific practices and the role the individual academic plays in them.”
Quote in the paperNot a substantial claim
May combine two claims.
Interpretationown data
LLMs can level the playing field for non-English speakers by providing editorial support, but can also increase inequalities by drawing on mainstream opinions and widening the gap between those who have access and those who do not.
“On the one hand, LLMs can level the playing field for non-English speakers as they can provide editorial support, but on the other hand, they can also increase inequalities by drawing on mainstream opinions and widening the gap between those who have access to these technologies and those who do not.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Interpretationreasoning
Reliance on generative AI models without proper validation may lead to a decrease in critical thinking and creativity.
“In addition, the reliance on generative AI models without proper validation may lead to a decrease in critical thinking and creativity.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning
Underlying these findings is the understanding that it is up to the individual academic to ensure that they have the skills and knowledge needed to navigate these technological changes.
“Underlying these findings is the understanding that it is up to the individual academic to ensure that they have the skills and knowledge needed to navigate these technological changes.”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim.
Interpretationreasoning
Such a stance can contribute to furthering digital divides due to preexisting uneven digital literacy between academics, institutions, and higher education systems.
“Such a stance can contribute to furthering digital divides due to preexisting uneven digital literacy between academics, institutions, and higher education systems.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Premisecitation
This adds a new and fresh direction to the LLM discussion, which has focused heavily on the impact of this technology for individual academics and learners.
“This adds a new and fresh direction to the LLM discussion, which has focused heavily on the impact of this technology for individual academics and learners (Fyfe 2023;Perkins 2023).”
Quote in the paperNot a substantial claim0 of 2 sources checked
- cites Fyfe 2023 source
  Not checked in the free run.
- cites Perkins 2023 source
  Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationreasoning
These tools offer the prospect of ameliorating administrative burdens on researchers, thus reorienting focus toward analytical facets of scientific endeavors.
“Furthermore, these tools offer the prospect of ameliorating administrative burdens on researchers, thus reorienting focus toward analytical facets of scientific endeavors.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning
Citation-based metrics could conceivably diminish in value if scientific articles could be predominantly generated through automation.
“For instance, regarding the reputation system, citation-based metrics, pivotal for scholarly progress, could conceivably diminish in value if scientific articles could be predominantly generated through automation.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone given
We did not ask the participants to rank the ethical and legal implications in the second round of the Delphi survey.
“We did not ask the participants to rank the ethical and legal implications in the second round of the Delphi survey.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationreasoning
This theoretical promise is counterbalanced by the instantaneous generation of potentially misleading content, raising concerns about scientific integrity and the nature of evidence.
“However, this theoretical promise is counterbalanced by the instantaneous generation of potentially misleading content, raising concerns about scientific integrity and the nature of evidence.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S4 (based).
Study design and conductnone given
At the time of the second round of our Delphi method, less than half a year had passed since the first preview of ChatGPT.
“At the time of the second round of our Delphi method, less than half a year had passed since the first preview of ChatGPT.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable).
Study design and conductreasoning
It is difficult to make concrete predictions about the potential capabilities of future versions of LLMs like ChatGPT.
“Accordingly, it is difficult to make concrete predictions about the potential capabilities of future versions of LLMs like ChatGPT.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning
LLMs' text generation capabilities have implications for scholarly communication and knowledge dissemination, potentially redefining conventional norms such as the academic reputation or quality control system.
“LLMs' text generation capabilities have implications for scholarly communication and knowledge dissemination, potentially redefining conventional norms such as the academic reputation or quality control system.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation
The importance of researchers having an ethical understanding was only marginally addressed in the literature.
“A point that was only marginally addressed in the literature (Dwivedi et al. 2023).”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites Dwivedi et al. 2023 source
  Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning
Generative AI, exemplified by LLMs, presents a transformative prospect for the social organization of science.
“Generative AI, exemplified by LLMs, presents a transformative prospect for the social organization of science.”
Quote in the paperNot a substantial claim

68 of 68

The authors state this as an assumption.
Many limitations of existing LLMs can be addressed in newer models.
“it can be assumed that many of these limitations can be addressed in newer models, as some respondents pointed out.”
The paper relies on this without support.
The recruited experts are sufficiently knowledgeable to provide informed assessments about the impact of LLMs on science.
“As generative AI continues to advance, it is crucial to gather expert knowledge and informed assessments regarding its potential impact on science.”
The paper relies on this without support.
LLMs are trained on works produced by others and produce (or co-produce) works, both of which almost certainly fall under copyright law.
“Taking into account that LLMs are trained on works produced by others and produce (or co-produce) works, both of which almost certainly fall under copyright law, it is not surprising that a large majority of the experts identify issues with copyright law as a pressing legal implication.”
The paper relies on this without support.
There is a lack of transparency regarding the personal data on which LLMs were trained and the further possible uses of personal data generated by the use of the tools.
“The lack of transparency regarding the personal data on which the LLMs were trained, but also the further possible uses of personal data generated by the use of the tools, certainly explains why many respondents identify privacy and data protection law issues as considerable legal challenges.”
The paper relies on this without support.
ChatGPT experienced overnight popularity since its debut in November 2022.
“The overnight popularity ChatGPT experienced since its debut in November 2022 stressed even more the necessity to evaluate the implications of LLMs for science and scientific practice.”

5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

LLMs are already starting to permeate the academic landscape.
“Finding answers to this question is urgent as LLMs are already starting to permeate the academic landscape (10–17).”
From How should the advancement of large language models affect the practice of science? (2025) · cites this paper as [16]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

Five opportunities are described as four
Minor
The sentence says the experts identified four opportunities, but the numbered list contains five.

1 of 1

On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1× · see citing works
“This is considered a structural issue of how these systems are trained and cannot be resolved by simply creating bigger models as size does not guarantee diversity (Bender et al. 2021).”
Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., et al. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102642 · Cited by 4,584 · see citing works
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519 · cited 1× · see citing works
“…some people, especially non-native speakers, the focus in academic work is expected to shift from text writing to conducting research, which requires researchers to formulate interesting research questions and carry out research to find answers (Dwivedi et al. 2023) (Floridi and Chiriatti 2020).”
Current validity of the Delphi method in social sciences
Landeta, J. (2006) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2005.09.002 · Cited by 1,353 · cited 1× · see citing works
“First developed in the 1960s, the Delphi method is a technique used to establish consensus among a group of experts on complex issues (Landeta 2006) and in some cases used to forecast future developments (Linstone and Turoff 1975).”
<scp>ChatGPT</scp> and a new academic reality: <scp>Artificial Intelligence‐written</scp> research papers and the ethics of the large language models in scholarly publishing
Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24750 · Cited by 919 · cited 15× · see citing works
“Errors may also occur in interpreting meaning, in particular if terms are ambiguous, have multiple meanings or consist of compound words (Lund et al. 2023).”
Influential
ChatGPT listed as author on research papers: many scientists disapprove
Stokel-Walker, C. (2023) · Nature · doi:10.1038/d41586-023-00107-z · Cited by 868 · cited 1× · see citing works
“…comes as no surprise that publishers like Springer Nature have banned ChatGPT and similar software from being given authorship on papers, a position shared by many scientists, according to Stokel-Walker (2023), and Science editors even have prohibited the use of any text generated by those tools.”
Academic Integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond
Perkins, M. (2023) · Journal of University Teaching and Learning Practice · doi:10.53761/1.20.02.07 · Cited by 839 · cited 7× · see citing works
“To date, scholars have primarily focused on the implications of LLMs on education (e.g., Perkins 2023; Fyfe 2023) with limited attention being paid to their impact on science and scientific practices (for exceptions, see Chubb et al. 2022; Ribeiro et al. 2023).”
Influential
GPT detectors are biased against non-native English writers
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023) · Patterns · doi:10.1016/j.patter.2023.100779 · Cited by 673 · cited 1× · see citing works
“A recent study by Liang et al. (2023) evaluating the performance of several widely used GPT detectors found that they consistently misclassify non-native English writing samples as AI-generated, whereas native writing samples are accurately identified.”
ChatGPT for (Finance) research: The Bananarama Conjecture
Dowling, M., & Lucey, B. M. (2023) · Finance research letters · doi:10.1016/j.frl.2023.103662 · Cited by 611 · see citing works
Generating scholarly content with ChatGPT: Ethical challenges for medical publishing
Liebrenz, M., Schleifer, R., Buadze, A., Bhugra, D., & Smith, A. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00019-5 · Cited by 594 · cited 6× · see citing works
“This multifaceted concern was also echoed by other scholars (Corless 2023; Liebrenz et al. 2023).”
Influential
Regulating ChatGPT and other Large Generative AI Models
Hacker, P., Engel, A., & Mauer, M. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594067 · Cited by 466 · cited 1× · see citing works
“Furthermore, Hacker et al. (2023) call for specific regulation of LLM-based tools, “large generative AI models”, under the EU Digital Services Act and provide four concrete, workable suggestions that include transparency obligations, mandatory yet limited risk management, non-discrimination data…”
Nonhuman “Authors” and Implications for the Integrity of Scientific Publication and Medical Knowledge
Flanagin, A., Bibbins-Domingo, K., Berkwits, M., & Christiansen, S. L. (2023) · JAMA · doi:10.1001/jama.2023.1344 · Cited by 420 · cited 3× · see citing works
“…2023), transforming notes into text (Buruk 2023), creating a first draft of a paper (Dwivedi et al. 2023), assisting with grammar and language (Flanagin et al. 2023), e.g., to improve clarity (Lund et al. 2023), especially for non-native speakers (Perkins 2023), but also stylistic issues,…”
Can AI language models replace human participants?
Dillion, D., Tandon, N., Gu, Y., & Gray, K. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.04.008 · Cited by 405 · cited 1× · see citing works
“They may even be used as a replacement for human participants in psychological experiments (Dillion et al. 2023).”
Welcome to the Era of ChatGPT et al.: The Prospects of Large Language Models
Teubner, T., Flath, C. M., Weinhardt, C., van der Aalst, W., & Hinz, O. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00795-x · Cited by 384 · cited 4× · see citing works
“Prompt engineering, i.e., developing and producing prompts for conversational AI systems like ChatGPT, is often discussed as a new competence that is required from researchers (e.g., Teubner et al. 2023).”
Influential
Using AI to write scholarly publications
Hosseini, M., Rasmussen, L. M., & Resnik, D. B. (2024) · Accountability in Research · doi:10.1080/08989621.2023.2168535 · Cited by 286 · cited 1× · see citing works
“…considered to lack transparency and explainability (Dwivedi et al. 2023), and reproduce or even amplify biases inherent in the information that was used to train them (Corless 2023; Hosseini et al. 2023), reproducing an “of the same old trivialities and stereo-types” (Teubner et al. 2023, p. 99).”
Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other large language models in scholarly peer review
Hosseini, M., & Horbach, S. P. J. M. (2023) · Research Integrity and Peer Review · doi:10.1186/s41073-023-00133-5 · Cited by 243 · cited 1× · see citing works
“They are expected to assist editors in screening submission for issues such as plagiarism or image manipulation, triaging, validating references, editing and formatting (Flanagin et al. 2023; Hosseini and Horbach 2023).”
Speeding up to keep up: exploring the use of AI in the research process
Chubb, J., Cowling, P., & Reed, D. (2022) · AI & SOCIETY · doi:10.1007/s00146-021-01259-0 · Cited by 218 · cited 3× · see citing works
“…empirical research—on the implications of LLMs as well as LLM-based chatbots or prompts on scholarly practices and the science system, which we understand as a collective body of all academic disciplines, including the sciences and humanities (Ribeiro et al. 2023; Chubb et al. 2022).”
How to cheat on your final paper: Assigning AI for student writing
Fyfe, P. (2023) · AI & SOCIETY · doi:10.1007/s00146-022-01397-z · Cited by 205 · cited 1× · see citing works
“This adds a new and fresh direction to the LLM discussion, which has focused heavily on the impact of this technology for individual academics and learners (Fyfe 2023; Perkins 2023).”
ChatGPT Needs SPADE (Sustainability, PrivAcy, Digital divide, and Ethics) Evaluation: A Review
Khowaja, S. A., Khuwaja, P., Dev, K., Wang, W., & Nkenyereye, L. (2023) · doi:10.36227/techrxiv.22619932.v2 · Cited by 183 · see citing works
ChatGPT and the AI Act
Helberger, N., & Diakopoulos, N. (2023) · Internet Policy Review · doi:10.14763/2023.1.1682 · Cited by 123 · cited 1× · see citing works
“At the same time, the European Parliament called for expanding the potential reach of the proposed EU AI Act by including ChatGPT-like systems to the list of high-risk categories of AI systems (Helberger and Diakopoulos 2023).”
Artificial intelligence and work: a critical review of recent research from the social sciences
Deranty, J.-P., & Corbin, T. (2024) · AI & SOCIETY · doi:10.1007/s00146-022-01496-x · Cited by 118 · cited 1× · see citing works
“They share this expectation with researchers in the field of AI and work, who in their optimistic scenarios expect a shift in tasks, the creation of new tasks and an emergence of new work profiles rather than the replacement of workers (Deranty and Corbin 2022).”
How Nature readers are using ChatGPT
Owens, B. (2023) · Nature · doi:10.1038/d41586-023-00500-8 · Cited by 109 · cited 1× · see citing works
“Scientists may also use LLM-based tools for non-scholarly tasks, as a recent Nature poll has shown: while eighty per cent of respondents have used AI chatbots, more than half say they use them for ‘creative fun’ (Owens 2023).”
AI et al.: Machines Are About to Change Scientific Publishing Forever
Grimaldi, G., & Ehrler, B. (2023) · ACS Energy Letters · doi:10.1021/acsenergylett.2c02828 · Cited by 98 · cited 3× · see citing works
“Similar thoughts are discussed by other scholars (Grimaldi and Ehrler 2023; Lund et al. 2023), with these changes being described in revolutionary terms in which LLMs are positioned as the great ‘game-changers’ of academia.”
ChatGPT: deconstructing the debate and moving it forward
Coeckelbergh, M., & Gunkel, D. J. (2023) · AI & Society · doi:10.1007/s00146-023-01710-4 · Cited by 88 · cited 1× · see citing works
“Even if our perspectives on authorship and language are not to be understood as outdated, as some scholars argue (e.g., Coeckelbergh and Gunkel 2023), they should at least be reconsidered in light of the potential impact of ChatGPT and similar LLM-based tools on our conventional under-standings…”
The Notorious GPT: science communication in the age of artificial intelligence
Schäfer, M. S. (2023) · Journal of Science Communication · doi:10.22323/2.22020402 · Cited by 67 · cited 1× · see citing works
“…of a text or even translate research output to multiple languages, both of which would level the field for researchers who speak English as a foreign language (Cor-less 2023; Liebrenz et al. 2023), or provide it in multimodal ways, including dialogical science communication at scale (Schäfer 2023).”
AI language models cannot replace human research participants
Harding, J., D’Alessandro, W., Laskowski, N. G., & Long, R. (2023) · AI & Society · doi:10.1007/s00146-023-01725-x · Cited by 53 · cited 1× · see citing works
“Their limitations in simulating human comprehension, reasoning, and evolving views make them unsuitable as substitutes for human participants in psychological studies (Harding et al. 2023).”
A publishing infrastructure for AI-assisted academic authoring
Pividori, M., & Greene, C. S. (2023) · doi:10.1101/2023.01.21.525030 · Cited by 52 · cited 1× · see citing works
“There is an expectation that using these tools to improve their texts, researchers “can focus more on what to communicate to others, rather than on how to write it” (Pividori and Greene 2023, p. 15).”
The digitalisation paradox of everyday scientific labour: How mundane knowledge work is amplified and diversified in the biosciences
Ribeiro, B., Meckin, R., Balmer, A., & Shapira, P. (2023) · Research Policy · doi:10.1016/j.respol.2022.104607 · Cited by 51 · cited 2× · see citing works
“…empirical research—on the implications of LLMs as well as LLM-based chatbots or prompts on scholarly practices and the science system, which we understand as a collective body of all academic disciplines, including the sciences and humanities (Ribeiro et al. 2023; Chubb et al. 2022).”
Creating meaningful work in the age of AI: explainable AI, explainability, and why it matters to organizational designers
Wulff, K., & Finnestrand, H. O. (2023) · AI & Society · doi:10.1007/s00146-023-01633-0 · Cited by 38 · see citing works
ChatGPT and Works Scholarly: Best Practices and Legal Pitfalls in Writing with AI
Tomlinson, B., Torrance, A. W., & Black, R. W. (2023) · SMU Law Review Forum · doi:10.25172/slrf.76.1.5 · Cited by 24 · see citing works
How language-generation AIs could transform science
Noorden, R. V. (2022) · Nature · doi:10.1038/d41586-022-01191-3 · Cited by 23 · cited 1× · see citing works
“…(2023) express apprehension regarding the utilization of LLM-based tools in academia, contending that this employment not only engenders apprehensions regarding research reproducibility and transparency but also has the potential to erode trust in the scientific process (see also Van Noorden 2022).”
Academic Writing with GPT-3.5Preprint
Buruk, O. (2023) · doi:10.31224/2861 · Cited by 13 · see citing works
AI ethics discourse: a call to embrace complexity, interdisciplinarity, and epistemic humility
Gellers, J. C. (2023) · AI & Society · doi:10.1007/s00146-023-01708-y · Cited by 10 · cited 1× · see citing works
“…perspectives on authorship and language are not to be understood as outdated, as some scholars argue (e.g., Coeckelbergh and Gunkel 2023), they should at least be reconsidered in light of the potential impact of ChatGPT and similar LLM-based tools on our conventional under-standings (Gellers 2023).”

Flagged first, then by citations

Research works printed but not linked: 4

Bazeley P (2009) Analysing qualitative data: more than ‘identifying themes.’ Malays J Qual Res 2(2):6–22
Jiang M, Breidbach C, Karanasios S (2022) How does artificial intelligence transform knowledge work? PACIS 2022 Proceedings, 312. https://aisel.aisnet.org/pacis2022/312
Kirchenbauer J, Geiping J, Wen Y, Katz J, Miers I, Goldstein T (2023).A watermark for large language models. arXiv https://arxiv.org/abs/2301.10226 · arXiv
Linstone HA, Turoff M (1975) The Delphi method. Addison-Wesley, Reading

Sorted by citations

The impact of large language models on higher education: exploring the connection between AI and Education 4.0
Peláez‐Sánchez, I. C., Velarde-Camaqui, D., & Glasserman‐Morales, L. D. (2024) · Frontiers in Education · doi:10.3389/feduc.2024.1392091 · Cited by 182 · see citing works
On the creativity of large language models
Franceschelli, G., & Musolesi, M. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-02127-3 · Cited by 113 · see citing works
“In the same way, LLMs also represent a valuable tool in scientific research (Fecher et al., 2023), especially for hypothesis generation (Gero et al., 2022).”
Background
How should the advancement of large language models affect the practice of science?
Binz, M., Alaniz, S., Roskies, A., Aczel, B., Bergstrom, C. T., Allen, C., Schad, D., Wulff, D., West, J. D., Zhang, Q., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2401227121 · Cited by 74 · see citing works
Exploring the frontiers of LLMs in psychological applications: a comprehensive review
Ke, L., Tong, S., Cheng, P., & Peng, K. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11297-5 · Cited by 92 · see citing works
Artificial intelligence in academic writing and clinical pharmacy education: consequences and opportunities
Weidmann, A. E. (2024) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-024-01705-1 · Cited by 42 · see citing works
Open Science at the generative AI turn: An exploratory analysis of challenges and opportunities
Hosseini, M., Horbach, S. P. J. M., Holmes, K., & Ross-Hellauer, T. (2025) · Quantitative Science Studies · doi:10.1162/qss_a_00337 · Cited by 40 · see citing works
“Another study has highlighted the potential of GenAI in scientific research, “ particularly in administrative, creative, and analytical tasks ” (Fecher, Hebing et al., 2023, p. 2).”
Beyond principlism: practical strategies for ethical AI use in research practices
Lin, Z. (2024) · AI and Ethics · doi:10.1007/s43681-024-00585-5 · Cited by 56 · see citing works
“Threatened and empty selves following AI-based virtual influencers”: comparison between followers and non-followers of virtual influencers in AI-driven digital marketing
Jin, S. A., & Viswanathan, V. (2024) · AI & Society · doi:10.1007/s00146-023-01832-9 · Cited by 40 · see citing works
“While AI-based virtual influencers today communicate with significant human support and intervention in the backend, it would be interesting to see how generative AI (GAI) algorithms such as ChatGPT (Fecher et al. 2023) embedded in AI-based virtual influencers enable large-scale personalization and diffusion of conten…”
Future Shock: Generative AI and the International AI Policy and Governance Crisis
Leslie, D., & Perini, A. M. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.88b4cc98 · Cited by 33 · see citing works
Trust in science, trust in ChatGPT? How Germans think about generative AI as a source in science communication
Schäfer, M. S., Kremer, B., Mede, N. G., & Fischer, L. (2024) · Journal of Science Communication · doi:10.22323/2.23090204 · Cited by 27 · see citing works
Responsible artificial intelligence in public health: a Delphi study on risk communication, community engagement and infodemic management
Mahl, D., Schäfer, M. S., Voinea, Ş. A., Adib, K., Duncan, B., Salvi, C., & Novillo-Ortiz, D. (2025) · BMJ Global Health · doi:10.1136/bmjgh-2024-018545 · Cited by 22 · see citing works
Prompt-to-SQL Injections in LLM-Integrated Web Applications: Risks and Defenses
Pedro, R., Coimbra, M. E., Castro, D., Carreira, P., & Santos, N. (2025) · International Conference on Software Engineering · doi:10.1109/icse55347.2025.00007 · Cited by 56 · see citing works
How generative artificial intelligence portrays science: Interviewing ChatGPT from the perspective of different audience segments
Volk, S. C., Schäfer, M. S., Lombardi, D., Mahl, D., & Yan, X. (2024) · Public Understanding of Science · doi:10.1177/09636625241268910 · Cited by 18 · see citing works
Artificial Intelligence in the 21st Century
奥宮, 正. (2023) · Research on Intelligent Manufacturing and Assembly · doi:10.25082/rima.2023.01.002 · Cited by 18 · see citing works
ChatGPT as an Academic Writing Tool: Factors Influencing Researchers’ Intention to Write Manuscripts Using Generative Artificial Intelligence
Garcia, M. B. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2499158 · Cited by 24 · see citing works
“Albeit not a common practice, it is technically feasible for researchers to incorporate AI-assisted text generation into their manuscript development workflows (Fecher et al., 2025).”
Navigating the AI era: university communication strategies and perspectives on generative AI tools
Henke, J. (2024) · Journal of Science Communication · doi:10.22323/2.23030205 · Cited by 16 · see citing works
Mapping the individual, social and biospheric impacts of Foundation Models
Hernández, A. D., Krishna, S., Perini, A., Katell, M. A., Bennett, S., Borda, A., Hashem, Y., Hadjiloizou, S., Mahomed, S., Jayadeva, S., et al. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658939 · Cited by 20 · see citing works
The construction of the role of AI in qualitative data analysis in the social sciences
Paulus, T. M., Lester, J. N., & Davis, C. (2025) · AI & Society · doi:10.1007/s00146-025-02488-3 · Cited by 22 · see citing works
A Bibliometric Analysis of Artificial Intelligence Applications in Global Higher Education
Li, M., & Rohayati, M. I. (2024) · International Journal of Information System Modeling and Design · doi:10.4018/ijismd.365913 · Cited by 21 · see citing works
“Fecher et al. (2023) highlight the need for proactive regulation and quality assurance frameworks to ensure the reliability and accountability of AI in education.”
Large language models in peer review: challenges and opportunities
Sun, Z. (2025) · Scientometrics · doi:10.1007/s11192-025-05440-w · Cited by 15 · see citing works
“…emergence and rapid development of large language models (LLMs), including ChatGPT and its contemporaries, new approaches to improve the peer review process are being explored (Couto et al., 2024; Fecher et al., 2023; Hosseini & Horbach, 2023; Li et al., 2024a; Thelwall, 2024a; Zhao et al., 2024).”
Prompt engineering for generative artificial intelligence chatbots in health research: A practical guide for traditional, complementary, and integrative medicine researchers
Ng, J. Y. (2025) · Integrative Medicine Research · doi:10.1016/j.imr.2025.101222 · Cited by 18 · see citing works
“80 , 116 , 117 Issues such as informed consent, responsible disclosure, and safeguarding cultural knowledge require more attention.”
Artificial Intelligence and content analysis: the large language models (LLMs) and the automatized categorization
Carius, A. C., & Teixeira, A. J. (2024) · AI & Society · doi:10.1007/s00146-024-01988-y · Cited by 13 · see citing works
Teaching UML using a RAG-based LLM
Ardimento, P., Bernardi, M., & Cimitile, M. (2024) · IEEE International Joint Conference on Neural Network · doi:10.1109/ijcnn60899.2024.10651492 · Cited by 16 · see citing works
Generative AI in Science Communication: Fostering Scientists’ Good Working Habits for Ethical and Effective Use
Hendriks, F., David, Y. B.-B., Banse, L., Fick, J., Greussing, E., Klein-Avraham, I., Rakedzon, T., Taddicken, M., & Baram‐Tsabari, A. (2025) · Science Communication · doi:10.1177/10755470251343486 · Cited by 10 · see citing works
Generative AI as the New Frontier in Science Education: A Systematic Review of Web of Science Articles
Aydın, S., Durukan, A., & Günbatar, M. S. (2025) · Science & Education · doi:10.1007/s11191-025-00677-6 · Cited by 15 · see citing works
“Among those, copyright infringement (Cooper, 2023), bias risk quality, and misinformation (Fecher et al., 2023) are the most crucial.”
Navigating opportunities and challenges of generative AI in higher education: towards responsible, equitable, and human-centered integration
Coman, C., Gherheș, V., Bucs, A., Coman, E., Cioca, V.-R., Bódi, D.-C., & Șerbănescu, R.-M. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1750978 · Cited by 16 · see citing works
Ethical AI Integration in Academia
van Rensburg, Z. J., & van der Westhuizen, S. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-1054-0.ch002 · Cited by 10 · see citing works
Implementation of generative AI tools in accounting education context
Al-Okaily, M. (2025) · Journal of Applied Research in Higher Education · doi:10.1108/jarhe-08-2024-0422 · Cited by 9 · see citing works
Open Science at the Generative AI Turn: An Exploratory Analysis of Challenges and OpportunitiesPreprint
Hosseini, M., Horbach, S. P. J. M., Holmes, K., et al. (2024) · doi:10.31235/osf.io/zns7g · Cited by 7 · see citing works
Quality in science communication with communicative artificial intelligence: A principle-based framework
Luna, D. S., Broer, I., Bilandzić, H., Taddicken, M., Schuller, B. W., & Bürger, M. (2025) · Public Understanding of Science · doi:10.1177/09636625251328854 · Cited by 6 · see citing works
“While ComAI offers unprecedented capabilities, it risks creating and disseminating inaccurate, incomplete, biased, and unverified scientific information, and potentially harming epistemic trust and the public perception of science (Fecher et al., 2023a).”
Influential
Ten Natural Language Processing Tasks with Generative Artificial Intelligence
Golec, J., & Hachaj, T. (2025) · Applied Sciences · doi:10.3390/app15169057 · Cited by 13 · see citing works
Beyond principlism: Practical strategies for ethical AI use in research practicesPreprint
Lin, Z. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4576581 · Cited by 5 · see citing works
Design of CO 2 -philic molecular units with large language models
Vogiatzis, K. D. (2025) · Chemical Communications · doi:10.1039/d5cc02652k · Cited by 5 · see citing works
Unveiling the dynamics of generative AI adoption: A business intelligence analysis through topic modeling-based bibliometric study
Aldreabi, H., Alhur, M., Al‐Okaily, M., Qasim, D., Dahdoul, N. K. S., & Shiyyab, F. S. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100456 · Cited by 5 · see citing works
How should the advent of large language models affect the practice of science?Preprint
Binz, M., Alaniz, S., Roskies, A. L., Aczél, B., Bergstrom, C. T., Allen, C., Schad, D. J., Wulff, D. U., West, J. D., Zhang, Q., et al. (2023) · doi:10.31219/osf.io/yr9xb · Cited by 8 · see citing works
Large Language Models in Psychology: Application in the Context of a Systematic Literature Review.Preprint
Uittenhove, K. L., Martinelli, P., & Roquet, A. (2024) · doi:10.31234/osf.io/nq4d2 · Cited by 4 · see citing works
Human languaging and large language models
Gahrn-Andersen, R. (2025) · AI & Society · doi:10.1007/s00146-025-02599-x · Cited by 3 · see citing works
“Just in an academic context, ChatGPT has proven useful for compiling academic literature, brainstorming ideas, writing up notes into coherent portions of text, proofreading and much more (cf. Fecher et al. 2025).”
Process Mindlessness: When we Lose Sight of What AI is Supposed to Improve
van der Aalst, W. M. P., Reijers, H. A., Hinz, O. F., & Pfeiffer, J. (2025) · Business & Information Systems Engineering · doi:10.1007/s12599-025-00972-0 · Cited by 3 · see citing works
“Generative AI is increasingly used by authors to assist in writing papers, such as summarizing prior work, reﬁning language, and even generating results sections or related work based on a few keywords (Fecher et al. 2025).”
“An Assistant in Your Pocket”: How Generative AI Shapes the Publishing Practices of Russian Postgraduate Students
Коваль, Е. А., Ушкин, С. Г., Николаевна, А. О., & Zhadunova, N. V. (2026) · Высшее образование в России · doi:10.31992/0869-3617-2025-34-12-107-126 · Cited by 3 · see citing works
A structured framework for effective and responsible generative artificial intelligence chatbot prompt engineering throughout the scientific process: a comprehensive guide for the health and medical researcher
Ng, J. Y. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1745928 · Cited by 5 · see citing works
“Additionally, the use of AI must be transparently reported to maintain integrity and accountability in the research process [World Health Organization (WHO), 2025; Minssen et al., 2023; Fecher et al., 2025; World Health Organization (WHO), 2021; Sartor and Lagioia, 2020; Bouderhem, 2024].”
Investigating How Computer Science Researchers Design Their Co-Writing Experiences With AI
Roffarello, A. M., Calò, T., Scibetta, L., & Russis, L. D. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713205 · Cited by 5 · see citing works
Conspiracy theories and misinformation in digital media: An international expert assessment of challenges, trends, and interventions
Mahl, D., Kessler, S. H., Schäfer, M. S., Jobin, A., Zeng, J., & Georgi, F. (2025) · Communications · doi:10.1515/commun-2024-0123 · Cited by 3 · see citing works
“In communication science, Delphi studies have been utilized to explore topics such as the impact of large language models on the science system (Fecher et al., 2023) and strategies for enhancing the quality of digital science communication (Fähnrich et al., 2023).”
Artificial Intelligence in Social Sciences
Rosmayanti, V. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-2612-2.ch012 · Cited by 3 · see citing works
The Bot Delusion. Large language models and anticipated consequences for academics’ publication and citation behavior
Wieczorek, O., Steinhardt, I., Schmidt, R., Mauermeister, S., & Schneijderberg, C. (2024) · Futures · doi:10.1016/j.futures.2024.103537 · Cited by 3 · see citing works
Strategic Interactions in Science Communication: A Complex Adaptive Systems Framework
Henke, J. (2025) · Science Communication · doi:10.1177/10755470251363825 · Cited by 2 · see citing works
¿Cómo está transformando la inteligencia artificial la comunicación científica? Desafíos, oportunidades y el papel de los actores involucrados: una revisión de alcance
Buitrago, J., Campos, E. M., & Romero, C. L. V. (2025) · Investigación Bibliotecológica Archivonomía Bibliotecología e Información · doi:10.22201/iibi.24488321xe.2025.104.59032 · Cited by 8 · see citing works
A linguistic comparison between human- and AI-generated content
Rodrigues, F. A., Sturm, N. F., & Pinheiro, F. L. (2026) · iScience · doi:10.1016/j.isci.2026.114976 · Cited by 2 · see citing works
Can large language models assess the quality of peer review? An empirical study
Tang, Y., Kang, Y., Wu, S., Zhang, R., & Sun, Z. (2026) · Scientometrics · doi:10.1007/s11192-026-05622-0 · Cited by 4 · see citing works
“Generative AI literacy across education and business: competencies, obstacles, and benefits—a systematic literature review”
Reicho, M., Otrel-Cass, K., Ebner, M., Brünner, B., Freinhofer, D., & Jahić, I. (2026) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-026-00596-8 · Cited by 16 · see citing works
“Nevertheless, significant challenges persist, including concerns about academic integrity, socio-economic disparities relating to access, professional development requirements, and implications for critical thinking development (Ganiyu, 2025; Zawacki-Richter, 2019).”
Introduction
Sudmann, A., Echterhölter, A., Ramsauer, M., et al. (2023) · AI critique · doi:10.14361/9783839467664-002 · Cited by 1 · see citing works
Wissenschaftliche Abschlussarbeiten und generative KI – Aspekte einer kompetenzorientierten Gestaltung
Schaper, N. (2025) · Springer eBooks · doi:10.1007/978-3-658-48999-1_28 · Cited by 1 · see citing works
Identifying Key Competencies for Education PhD Students with Generative AI Integration: A Scoping Review
de Lira, Y. R., Burneo, P. S., & Costa, A. P. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-02999-7_9 · Cited by 2 · see citing works
Hybrid epistemic practices and the transformation of academic assemblages: generative AI and epistemic messiness in the qualitative social sciences and humanities
Bareither, C., & Griessl, L. (2026) · AI & Society · doi:10.1007/s00146-026-02988-w · Cited by 3 · see citing works
“Our study benefited from various quantitative surveys (Fecher et al. 2023; Hadan et al. 2024; Formosa et al. 2024), which we extended with a previously published survey on the role of GenAI in the QSSH at the University of Tübin-gen (Griessl et al. 2024).”
The uses of generative artificial intelligence in social research north and south: Comparing empirical evidence from surveys in Denmark and Mexico
Schulz, M. S., Fenollosa, L. T., Dussauge‐Laguna, M. I., Fontanelli, Ó., Nedelcu, D. Q., Udave, J. A., & Andersen, J. W. (2026) · Methodological Innovations · doi:10.1177/20597991261454555 · Cited by 1 · see citing works
“All of this has been accompanied by discussions on how (if at all) to regulate the development and application of AI algorithms and tools (Dabis and Csák, 2024; Fecher et al., 2025; Goyanes et al., 2025; Law and McCall, 2024; Scherer, 2015; Yeung and Lodge, 2019).”
Goblin's Challenge to ChatGPT: Exploring AI's Dilemma Resolution and Mentalization through Riddle TalesPreprint
Bhattacharya, K., & Nandi, A. K. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4476837 · Cited by 1 · see citing works
Chatting with Pythons: using ChatGPT and Python computer code to screen abstracts for systematic literature reviews in complex disciplinesPreprint
Mudd, A., Conroy, T., Voldbjerg, S. L., Goldschmied, A. Z., & Schuwirth, L. (2024) · Research Square · doi:10.21203/rs.3.rs-4767916/v1 · Cited by 1 · see citing works
Putting the AI into social science
Breuer, J. (2023) · AI critique · doi:10.1515/9783839467664-014 · Cited by 4 · see citing works
DIFFERENT YET SAME: INTENTIONALITY ASCRIPTIONS TO ROUTINES ACROSS PRACTICE AND CAPABILITIES SCHOOLS. LARGE LANGUAGE MODEL-BASED APPROACHPreprint
Makowski, P. T., Hensel, A., & Zaei, M. E. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4813964
Detecting the Usage of Large Language Models Exploiting Generative Adversarial Networks
Aversano, L., Bernardi, M. L., & Cimitile, M. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-67351-1_9
Making Adequate Use of Generative Language Models in Scientific ResearchPreprint
Bustio-Martínez, L., Herrera-Semenets, V., Feregrino-Uribe, C., & van den Berg, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5367076
Personality Traits and Political Dispositions, Ideology, and Sentiment Toward Political Leaders of 14 Artificial Intelligence Large Language Models
Goyanes, M., Domínguez‐Díaz, A., & de-Marcos, L. (2025) · Human Behavior and Emerging Technologies · doi:10.1155/hbe2/5761832
The sophist in the server
Colangelo, M. T., & Galli, C. (2026) · EMBO Reports · doi:10.1038/s44319-026-00711-w
Understanding People’s Willingness to Participate in Human-LLM Conversational Interaction Research Studies - A Scenario StudyPreprint
Alzughbi, H., Park, J., & Knijnenburg, B. P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6235653
The diffusion of Artificial Intelligence into Neuroscience. A multiscale approach of the genericity of a research-technology
Fontaine, S. (2024) · theses.fr (ABES) · doi:10.70675/97af6b47z72c7z477azba31z98a1c6435222
Fluency Without Fidelity: Errors in Citation-Attributed Claims in Large Language Model-Generated Literature Reviews in Mental Health
Linardon, J., Messer, M., Anderson, C., Soliman, O. M., Liu, C., Firth, J., & Torous, J. B. (2026) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-026-00630-6 · Cited by 1 · see citing works
Tracking policy-relevant narratives of democratic resilience at scale: From experts and machines, to AI & the transformer revolution
Angus, S. D. (2026) · Data & Policy · doi:10.1017/dap.2026.10063 · Cited by 1 · see citing works
“…et al., 2025), with some authors arguing that the impact of LLMs will go beyond efficiency and opportunity to start re-shaping the scientific method itself (Fecher et al., 2023), and exacerbate existing resource and access inequalities in the political economy of ideas (Luitse and Denkena, 2021).”
An exploration of safety analytics for proactivity in workplace safety using large language models
Bayramova, A., Edwards, D. J., & Rillie, I. (2026) · Quality & Quantity · doi:10.1007/s11135-026-02842-7
“…use and benefits for generation of lecture notes and assignment checking (Küchemann et al. 2025); whereas others raise concerns for plagiarism inducement or serving as a breeding ground for breaching ethical, legal and academic standards and normative performance (Sætra 2023; Fecher et al. 2025).”
Artificial Intelligence in Higher Education Governance: Integrating AI into Quality Assurance and Strategic Management
Şenyapar, H. N. D., & Bayındır, R. (2026) · Üniversite Araştırmaları Dergisi · doi:10.32329/uad.1868361
Prompt injection compromises large language model-based peer review: evidence from randomised controlled trial abstracts from anaesthesia journals
CASSAI, A. D., Dost, B., Pistollato, E., et al. (2026) · British Journal of Anaesthesia · doi:10.1016/j.bja.2026.06.041
Using LLMs without trust: the adoption–trust paradox as institutional governance failure in academic ecosystems
Kringen, P., Gallucci, A., Hildt, E., Johansen, F. R., Westerlund, M., & Tigerstedt, C. (2026) · AI & Society · doi:10.1007/s00146-026-03323-z

Authors: Fecher, B., Hebing, M., Laufer, M., Pohle, J., & Sofsky, F.
Affiliations: Alexander von Humboldt Institute for Internet and Society (DE)
Corresponding author: Fecher, Benedikt
ORCID: Fecher, Benedikt
Laufer, Melissa
Pohle, Jörg
Venue: AI & SOCIETY
Publisher: Springer Science and Business Media LLC
Published: 2023-10-26
Volume: 40
Issue: 2
Pages: 447-459
ISSN: 0951-5666 (ISSN-L)
1435-5655
DOI: 10.1007/s00146-023-01791-1
OpenAlex: W4388202310
Cited by: 78
References: 33
Citations by year: 2026: 19 · 2025: 28 · 2024: 24 · 2023: 7
FWCI: 19.56
Type: journal article
Language: en
Open access: hybrid
doi.org
Licence: cc-by
creativecommons.org/licenses/by/4.0 (tdm)
creativecommons.org/licenses/by/4.0 (vor)
Locations: AI & Society (publishedVersion, cc-by)
arXiv (Cornell University) (submittedVersion, cc-by)
arXiv (Cornell University) (cc-by)
Topics: Social Sciences › Safety Research › Ethics and Social Impacts of AI
Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords: large language models, scientific practice, ChatGPT, AI ethics, AI regulation, bias, misinformation, quality assurance, AI competencies, science education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Friend or foe? Exploring the implications of large language models on the science system

Abstract

AI snapshot

TL;DR

Key findings

Methods

Limitations

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks