The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
GenAI against humanity: nefarious applications of generative artificial intelligence and large language models
Open accessAbstract
Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) are marvels of technology; celebrated for their prowess in natural language processing and multimodal content generation, they promise a transformative future. But as with all powerful tools, they come with their shadows. Picture living in a world where deepfakes are indistinguishable from reality, where synthetic identities orchestrate malicious campaigns, and where targeted misinformation or scams are crafted with unparalleled precision. Welcome to the darker side of GenAI applications. This article is not just a journey through the meanders of potential misuse of GenAI and LLMs, but also a call to recognize the urgency of the challenges ahead. As we navigate the seas of misinformation campaigns, malicious content generation, and the eerie creation of sophisticated malware, we’ll uncover the societal implications that ripple through the GenAI revolution we are witnessing. From AI-powered botnets on social media platforms to the unnerving potential of AI to generate fabricated identities, or alibis made of synthetic realities, the stakes have never been higher. The lines between the virtual and the real worlds are blurring, and the consequences of potential GenAI’s nefarious applications impact us all. This article serves both as a synthesis of rigorous research presented on the risks of GenAI and misuse of LLMs and as a thought-provoking vision of the different types of harmful GenAI applications we might encounter in the near future, and some ways we can prepare for them.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 1×
“" The distortion of the information ecosystem, including the spread of misinformation, fake news, and other forms of deceptive content [28], is categorized as "Information Manipulation.”
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1×
“, hallucinated [13]) information to an inattentive reader (see Table 1N).”
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842 · cited 1×
“Their potential seems endless, promising innovations that could redefine the way humans and machines interact with each other or partner to work together [10, 26].”
Generative AI and ChatGPT: applications, challenges, and AI-human collaboration
Fui-Hoon Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023) · Journal of Information Technology Case and Application Research · doi:10.1080/15228053.2023.2233814 · Cited by 1,363
Jagatic, T. N., Johnson, N. A., Jakobsson, M., & Menczer, F. (2007) · Communications of the ACM · doi:10.1145/1290958.1290968 · Cited by 1,156
Telling humans and computers apart automatically
von Ahn, L., Blum, M., & Langford, J. (2004) · Communications of the ACM · doi:10.1145/966389.966390 · Cited by 895
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2023) · arXiv · doi:10.48550/arxiv.2305.03514 · Cited by 839
Art and the science of generative AI
Epstein, Z., Hertzmann, A., the Investigators of Human Creativity, Akten, M., Farid, H., Fjeld, J., Frank, M. R., Groh, M., Herman, L., Leach, N., et al. (2023) · Science · doi:10.1126/science.adh4451 · Cited by 788
From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy
Gupta, M., Akiri, C., Aryal, K., Parker, E., & Praharaj, L. (2023) · IEEE Access · doi:10.1109/access.2023.3300381 · Cited by 786
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00502 · Cited by 585
Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P. S., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2303.04226 · Cited by 463
Baeza-Yates, R. (2018) · Communications of the ACM · doi:10.1145/3209581 · Cited by 432 · cited 1×
“From the subtle perpetuation of biases to the blatant reinforcement of stereotypes [2, 6, 21], GenAI can become mirrors reflecting and amplifying the imperfections of our society [1, 8].”
Establishing the rules for building trustworthy AI
Floridi, L. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0055-y · Cited by 394
Large pre-trained language models contain human-like biases of what is right and wrong to do
Schramowski, P., Turan, C., Andersen, N., Rothkopf, C. A., & Kersting, K. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00458-8 · Cited by 291 · cited 1×
“From the subtle perpetuation of biases to the blatant reinforcement of stereotypes [2, 7, 21], GenAI can become mirrors reflecting and amplifying the imperfections of our society [1, 6].”
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models
Ferrara, E. (2023) · arXiv · doi:10.48550/arxiv.2304.03738 · Cited by 282
Self-citationShould ChatGPT be biased? Challenges and risks of bias in large language models
Ferrara, E. (2023) · First Monday · doi:10.5210/fm.v28i11.13346 · Cited by 282 · cited 1×
“From the subtle perpetuation of biases to the blatant reinforcement of stereotypes [2, 7, 21], GenAI can become mirrors reflecting and amplifying the imperfections of our society [1, 6].”
Self-citationThe Butterfly Effect in artificial intelligence systems: Implications for AI bias and fairness
Ferrara, E. (2024) · Machine Learning with Applications · doi:10.1016/j.mlwa.2024.100525 · Cited by 208
Self-citationAddressing fairness in artificial intelligence for medical imaging
Ricci Lara, M. A., Echeveste, R., & Ferrante, E. (2022) · Nature Communications · doi:10.1038/s41467-022-32186-3 · Cited by 204
Bad machines corrupt good morals
Köbis, N., Bonnefon, J.-F., & Rahwan, I. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01128-2 · Cited by 202
Sison, A. J. G., Daza, M. T., Gozalo-Brizuela, R., & Garrido-Merchán, E. C. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2225931 · Cited by 169
Ferrara, E. (2019) · Communications of the ACM · doi:10.1145/3299768 · Cited by 108
Self-citationSocial bot detection in the age of ChatGPT: Challenges and opportunities
Ferrara, E. (2023) · First Monday · doi:10.5210/fm.v28i6.13185 · Cited by 84
Self-citation(2019) · Nature Machine Intelligence · doi:10.1038/42256.2522-5839 · Cited by 83
Anatomy of an AI-powered malicious social botnet
Yang, K., & Menczer, F. (2024) · Journal of Quantitative Description: Digital Media · doi:10.51685/jqd.2024.icwsm.7 · Cited by 71 · cited 1×
“Imagine a world where AI-powered botnets dominate social media [7, 29], where harmful or radicalizing content is churned out by algorithms [23], and where the lines between reality and AI-generated content blur [3].”
Scams, Frauds, and Crimes in the Nonfungible Token Market
Kshetri, N. (2022) · Computer · doi:10.1109/mc.2022.3144763 · Cited by 66
Addressing the harms of AI-generated inauthentic content
Menczer, F., Crandall, D., Ahn, Y.-Y., & Kapadia, A. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00690-w · Cited by 42
The Future of Cybercrime: AI and Emerging Technologies Are Creating a Cybercrime TsunamiPreprint
Treleaven, P. C., Barnett, J., Brown, D., Bud, A., Fenoglio, E., Kerrigan, C., Koshiyama, A., Sfeir-Tait, S., & Schoernig, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4507244 · Cited by 37
Social media, extremism, and radicalization
Shaw, A. (2023) · Science Advances · doi:10.1126/sciadv.adk2031 · Cited by 32
Disinformation 2.0 in the Age of AI: A Cybersecurity Perspective
Mazurczyk, W., Lee, D., & Vlachos, A. (2024) · Communications of the ACM · doi:10.1145/3624721 · Cited by 26
Seymour, M., Riemer, K., Yuan, L., & Dennis, A. R. (2023) · Communications of the ACM · doi:10.1145/3584973 · Cited by 10
Cited by
Ferrara, E. (2023) · Sci · doi:10.3390/sci6010003 · Cited by 875
Self-citationKurtz, G., Amzalag, M., Shaked, N., Zaguri, Y., Kohen-Vacs, D., Gal, E., Zailer, G., & Barak-Medina, E. (2024) · Education Sciences · doi:10.3390/educsci14050503 · Cited by 181
“…mass surveillance or data mining [54,55]; utilizing GenAI to develop sophisticated malware that is difficult to detect using security software [53,56], and utilizing GenAI capabilities to identify and exploit security vulnerabilities, allowing attackers to gain unauthorized access to systems…”
BackgroundMethodInfluentialZhang, X., Zhang, P., Shen, Y., Liu, M., Wang, Q., Gašević, D., & Fan, Y. (2024) · Frontiers of Digital Education · doi:10.1007/s44366-024-0028-5 · Cited by 83
University Students’ Insights of Generative Artificial Intelligence (AI) Writing Tools
Gasaymeh, A.-M. M., Beirat, M. A., & Abu Qbeita, A. A. (2024) · Education Sciences · doi:10.3390/educsci14101062 · Cited by 74
Conceptualizing generative AI as style engines: Application archetypes and implications
Riemer, K., & Peter, S. (2024) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2024.102824 · Cited by 63
Metadata
- Authors
- Ferrara, E.
- Venue
- Journal of Computational Social Science
- Published
- 2024-02-22
- Volume
- 7
- Issue
- 1
- Pages
- 549-569
- ISSN
- 2432-2717 (ISSN-L)
- 2432-2725
- OpenAlex
- W4392058134
- Cited by
- 259
- References
- 32
- FWCI
- 106.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.