The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
GenAI against humanity: nefarious applications of generative artificial intelligence and large language models
Open accessAbstract
Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) are marvels of technology; celebrated for their prowess in natural language processing and multimodal content generation, they promise a transformative future. But as with all powerful tools, they come with their shadows. Picture living in a world where deepfakes are indistinguishable from reality, where synthetic identities orchestrate malicious campaigns, and where targeted misinformation or scams are crafted with unparalleled precision. Welcome to the darker side of GenAI applications. This article is not just a journey through the meanders of potential misuse of GenAI and LLMs, but also a call to recognize the urgency of the challenges ahead. As we navigate the seas of misinformation campaigns, malicious content generation, and the eerie creation of sophisticated malware, we’ll uncover the societal implications that ripple through the GenAI revolution we are witnessing. From AI-powered botnets on social media platforms to the unnerving potential of AI to generate fabricated identities, or alibis made of synthetic realities, the stakes have never been higher. The lines between the virtual and the real worlds are blurring, and the consequences of potential GenAI’s nefarious applications impact us all. This article serves both as a synthesis of rigorous research presented on the risks of GenAI and misuse of LLMs and as a thought-provoking vision of the different types of harmful GenAI applications we might encounter in the near future, and some ways we can prepare for them.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 1×
“" The distortion of the information ecosystem, including the spread of misinformation, fake news, and other forms of deceptive content [28], is categorized as "Information Manipulation.”
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1×
“, hallucinated [13]) information to an inattentive reader (see Table 1N).”
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842 · cited 1×
“Their potential seems endless, promising innovations that could redefine the way humans and machines interact with each other or partner to work together [10, 26].”
Generative AI and ChatGPT: applications, challenges, and AI-human collaboration
Fui-Hoon Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023) · Journal of Information Technology Case and Application Research · doi:10.1080/15228053.2023.2233814 · Cited by 1,363
Jagatic, T. N., Johnson, N. A., Jakobsson, M., & Menczer, F. (2007) · Communications of the ACM · doi:10.1145/1290958.1290968 · Cited by 1,156
Telling humans and computers apart automatically
von Ahn, L., Blum, M., & Langford, J. (2004) · Communications of the ACM · doi:10.1145/966389.966390 · Cited by 895
Can Large Language Models Transform Computational Social Science?
Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., & Yang, D. (2023) · arXiv · doi:10.48550/arxiv.2305.03514 · Cited by 839
Art and the science of generative AI
Epstein, Z., Hertzmann, A., the Investigators of Human Creativity, Akten, M., Farid, H., Fjeld, J., Frank, M. R., Groh, M., Herman, L., Leach, N., et al. (2023) · Science · doi:10.1126/science.adh4451 · Cited by 788 · cited 3×
“Generative AI has shown promise in recreating or restoring damaged historical artifacts and paintings, breathing new life into our shared cultural heritage [4].”
Cited by
Ferrara, E. (2023) · Sci · doi:10.3390/sci6010003 · Cited by 875
Self-citationKurtz, G., Amzalag, M., Shaked, N., Zaguri, Y., Kohen-Vacs, D., Gal, E., Zailer, G., & Barak-Medina, E. (2024) · Education Sciences · doi:10.3390/educsci14050503 · Cited by 181
“…mass surveillance or data mining [54,55]; utilizing GenAI to develop sophisticated malware that is difficult to detect using security software [53,56], and utilizing GenAI capabilities to identify and exploit security vulnerabilities, allowing attackers to gain unauthorized access to systems…”
BackgroundMethodInfluentialThe Butterfly Effect in Artificial Intelligence Systems: Implications for AI Bias and Fairness
Ferrara, E. (2023) · doi:10.2139/ssrn.4614234 · Cited by 129
Self-citationZhang, X., Zhang, P., Shen, Y., Liu, M., Wang, Q., Gašević, D., & Fan, Y. (2024) · Frontiers of Digital Education · doi:10.1007/s44366-024-0028-5 · Cited by 83
University Students’ Insights of Generative Artificial Intelligence (AI) Writing Tools
Gasaymeh, A.-M. M., Beirat, M. A., & Abu Qbeita, A. A. (2024) · Education Sciences · doi:10.3390/educsci14101062 · Cited by 74
Metadata
- Authors
- Ferrara, E.
- Venue
- Journal of Computational Social Science
- Published
- 2024-02-22
- Volume
- 7
- Issue
- 1
- Pages
- 549-569
- ISSN
- 2432-2717 (ISSN-L)
- 2432-2725
- OpenAlex
- W4392058134
- Cited by
- 259
- References
- 33
- FWCI
- 106.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: GenAI against humanity: nefarious applications of generative artificial intelligence and large language models. https://app.cochat.ai/papers/10.1007/s42001-024-00250-1
- The paper
- Ferrara, E. (2024). GenAI against humanity: nefarious applications of generative artificial intelligence and large language models. Journal of Computational Social Science. https://doi.org/10.1007/s42001-024-00250-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.