GenAI Against Humanity: Nefarious Applications of Generative Artificial Intelligence and Large Language Models
Open accessAbstract
Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) are marvels of technology; celebrated for their prowess in natural language processing and multimodal content generation, they promise a transformative future. But as with all powerful tools, they come with their shadows. Picture living in a world where deepfakes are indistinguishable from reality, where synthetic identities orchestrate malicious campaigns, and where targeted misinformation or scams are crafted with unparalleled precision. Welcome to the darker side of GenAI applications. This article is not just a journey through the meanders of potential misuse of GenAI and LLMs, but also a call to recognize the urgency of the challenges ahead. As we navigate the seas of misinformation campaigns, malicious content generation, and the eerie creation of sophisticated malware, we'll uncover the societal implications that ripple through the GenAI revolution we are witnessing. From AI-powered botnets on social media platforms to the unnerving potential of AI to generate fabricated identities, or alibis made of synthetic realities, the stakes have never been higher. The lines between the virtual and the real worlds are blurring, and the consequences of potential GenAI's nefarious applications impact us all. This article serves both as a synthesis of rigorous research presented on the risks of GenAI and misuse of LLMs and as a thought-provoking vision of the different types of harmful GenAI applications we might encounter in the near future, and some ways we can prepare for them.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In generative AI, addressing bias requires a holistic strategy.
“In the realm of generative AI, addressing bias is even more challenging as it requires a holistic strategy [14,21,22].”
From Fairness and Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, and Mitigation Strategies (2023) · cites this paper as [21]
Cites
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · see citing works
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · see citing works
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · see citing works
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842 · see citing works
Generative AI and ChatGPT: applications, challenges, and AI-human collaboration
Fui-Hoon Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023) · Journal of Information Technology Case and Application Research · doi:10.1080/15228053.2023.2233814 · Cited by 1,363 · see citing works
Jagatic, T. N., Johnson, N. A., Jakobsson, M., & Menczer, F. (2007) · Communications of the ACM · doi:10.1145/1290958.1290968 · Cited by 1,156 · see citing works
Telling humans and computers apart automatically
von Ahn, L., Blum, M., & Langford, J. C. (2004) · Communications of the ACM · doi:10.1145/966389.966390 · Cited by 907 · see citing works
Art and the science of generative AI
Epstein, Z., Hertzmann, A., the Investigators of Human Creativity, Akten, M., Farid, H., Fjeld, J., Frank, M. R., Groh, M., Herman, L., Leach, N., et al. (2023) · Science · doi:10.1126/science.adh4451 · Cited by 788 · see citing works
From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy
Gupta, M., Akiri, C., Aryal, K., Parker, E., & Praharaj, L. (2023) · IEEE Access · doi:10.1109/access.2023.3300381 · Cited by 714 · see citing works
Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P. S., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2303.04226 · Cited by 463 · see citing works
Baeza-Yates, R. (2018) · Communications of the ACM · doi:10.1145/3209581 · Cited by 432 · see citing works
Establishing the rules for building trustworthy AI
Floridi, L. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0055-y · Cited by 394 · see citing works
Large pre-trained language models contain human-like biases of what is right and wrong to do
Schramowski, P., Turan, C., Andersen, N., Rothkopf, C. A., & Kersting, K. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00458-8 · Cited by 291 · see citing works
Should ChatGPT be biased? Challenges and risks of bias in large language models
Ferrara, E. (2023) · First Monday · doi:10.5210/fm.v28i11.13346 · Cited by 282 · see citing works
Self-citationBad machines corrupt good morals
Köbis, N., Bonnefon, J., & Rahwan, I. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01128-2 · Cited by 209 · see citing works
Addressing fairness in artificial intelligence for medical imaging
Lara, M. A. R., Echeveste, R., & Ferrante, E. (2022) · Nature Communications · doi:10.1038/s41467-022-32186-3 · Cited by 208 · see citing works
Sison, A. J. G., Daza, M. T., Gozalo-Brizuela, R., & Garrido-Merchán, E. C. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2225931 · Cited by 169 · see citing works
Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and VulnerabilitiesPreprint
Mozes, M., He, X., Kleinberg, B., & Griffin, L. (2023) · arXiv.org · doi:10.48550/arxiv.2308.12833 · Cited by 134 · see citing works
The Butterfly Effect in Artificial Intelligence Systems: Implications for AI Bias and Fairness
Ferrara, E. (2023) · doi:10.2139/ssrn.4614234 · Cited by 129 · see citing works
Self-citationFerrara, E. (2019) · Communications of the ACM · doi:10.1145/3299768 · Cited by 101 · see citing works
Self-citationSocial bot detection in the age of ChatGPT: Challenges and opportunities
Ferrara, E. (2023) · First Monday · doi:10.5210/fm.v28i6.13185 · Cited by 84 · see citing works
Self-citationAnatomy of an AI-powered malicious social botnet
Yang, K., & Menczer, F. (2024) · Journal of Quantitative Description: Digital Media · doi:10.51685/jqd.2024.icwsm.7 · Cited by 71 · see citing works
Scams, Frauds, and Crimes in the Nonfungible Token Market
Kshetri, N. (2022) · Computer · doi:10.1109/mc.2022.3144763 · Cited by 66 · see citing works
Addressing the harms of AI-generated inauthentic content
Menczer, F., Crandall, D., Ahn, Y.-Y., & Kapadia, A. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00690-w · Cited by 42 · see citing works
The Future of Cybercrime: AI and Emerging Technologies Are Creating a Cybercrime TsunamiPreprint
Treleaven, P. C., Barnett, J., Brown, D., Bud, A., Fenoglio, E., Kerrigan, C., Koshiyama, A., Sfeir-Tait, S., & Schoernig, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4507244 · Cited by 24 · see citing works
Social media, extremism, and radicalization
Shaw, A. (2023) · Science Advances · doi:10.1126/sciadv.adk2031 · Cited by 20 · see citing works
Sisón, A. J. G., Daza, M. T., Gozalo-Brizuela, R., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.11215 · Cited by 12 · see citing works
Seymour, M., Riemer, K., Yuan, L., & Dennis, A. R. (2023) · Communications of the ACM · doi:10.1145/3584973 · Cited by 10 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Ferrara, E. (2023) · Sci · doi:10.3390/sci6010003 · Cited by 875 · see citing works
Self-citationThe illusion of reality: How AI-generated images (AIGIs) are fooling social media users
Miskolczi, M. (2026) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108876 · Cited by 5 · see citing works
- Authors
- Ferrara, E.
- Affiliations
- University of Southern California (US)
- Corresponding author
- Ferrara, Emilio
- ORCID
- Ferrara, Emilio
- Venue
- Journal of Computational Social Science
- Publisher
- Elsevier BV
- Published
- 2023
- Volume
- 7
- Issue
- 1
- Pages
- 549-569
- ISSN
- 2432-2717 (ISSN-L)
- OpenAlex
- W4392058134
- Cited by
- 266
- References
- 29
- Citations by year
- 2026: 91 · 2025: 117 · 2024: 55 · 2023: 3
- FWCI
- 108.95
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Journal of Computational Social Science (publishedVersion, cc-by)
- SSRN Electronic Journal (acceptedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- University of Southern California
- Defense Advanced Research Projects Agency (HR001121C0169)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- generative AI, large language models, AI misuse, deepfakes, synthetic identity, misinformation campaigns, AI-generated malware, social media, scams, societal risks of AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
