Generative AI and criminal law
Open accessAbstract
Several criminal offenses can originate from or culminate with the creation of content. Sexual abuse can be committed by producing intimate materials without the subject’s consent, while incitement to violence or self-harm can begin with a conversation. When the task of generating content is entrusted to artificial intelligence (AI), it becomes necessary to explore the risks of this technology. AI changes criminal affordances because it creates new kinds of harmful content, it amplifies the range of recipients, and it can exploit cognitive vulnerabilities to manipulate user behavior. Given this evolving landscape, the question is whether policies aimed at fighting Generative AI-related harms should include criminal law. The bulk of criminal law scholarship to date would not criminalize AI harms on the theory that AI lacks moral agency. Even so, the field of AI might need criminal law, precisely because it entails a moral responsibility. When a serious harm occurs, responsibility needs to be distributed considering the guilt of the agents involved, and, if it is lacking, it needs to fall back because of their innocence. Thus, legal systems need to start exploring whether and how guilt can be preserved when the actus reus is completely or partially delegated to Generative AI.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cases described by participants sit primarily within Panattoni's malicious dimension and suggest that generative AI extends existing offence categories rather than constituting wholly new ones.
“The cases described by participants sit primarily within Panattoni's malicious dimension and, on this reading, suggest that generative AI extends existing offence categories rather than constituting wholly new ones.”
From Generative AI in child sexual exploitation and abuse: views from UK law enforcement (2026) · cites this paper as Panattoni (2025)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security
Chesney, R., & Citron, D. K. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3213954 · Cited by 880 · cited 2× · see citing works
“Generative AI systems can be used for disinformation campaigns at scale (Chesney & Citron, 2019; Kreps et al., 2022), and, more broadly, for covert or deceptive efforts to influence the opinions of a target audience, defined as “influence operations” (Goldstein et al., 2023), able to cause…”
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · see citing works
Moral Responsibility of Public Officials: The Problem of Many Hands
Thompson, D. F. (1980) · American Political Science Review · doi:10.2307/1954312 · Cited by 651 · see citing works
The social control of technology
T.P. (1981) · Design Studies · doi:10.1016/0142-694x(81)90102-2 · Cited by 547 · see citing works
Four Responsibility Gaps with Artificial Intelligence: Why they Matter and How to Address them
Santoni de Sio, F., & Mecacci, G. (2021) · Philosophy & Technology · doi:10.1007/s13347-021-00450-x · Cited by 490 · cited 1× · see citing works
“The many-hands-problem (Nissenbaum, 1996; Santoni de Sio & Mecacci, 2021; Thompson, 1980), wherein responsibility is distributed among numerous agents, combined with the “black box” nature of AI, complicates accountability.”
AI as Agency Without Intelligence: on ChatGPT, Large Language Models, and Other Generative Models
Floridi, L. (2023) · Philosophy & Technology · doi:10.1007/s13347-023-00621-y · Cited by 404 · see citing works
All the News That’s Fit to Fabricate: AI-Generated Text as a Tool of Media Misinformation
Kreps, S., McCain, R. M., & Brundage, M. (2022) · Journal of Experimental Political Science · doi:10.1017/xps.2020.37 · Cited by 360 · see citing works
Accountability in a computerized society
Nissenbaum, H. (1996) · Science and Engineering Ethics · doi:10.1007/bf02639315 · Cited by 346 · see citing works
Online manipulation: hidden influences in a digital world
Susser, D., Roessler, B., & Nissenbaum, H. F. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3306006 · Cited by 341 · see citing works
(2018) · Harvard journal of law & technology · Cited by 295 · see citing works
Robotics and the Lessons of Cyberlaw
Calo, R. (2014) · doi:10.2139/ssrn.2402972 · Cited by 269 · see citing works
Machine learning in protein structure prediction
AlQuraishi, M. (2021) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2021.04.005 · Cited by 251 · see citing works
The Oxford Handbook of Criminal Law
(2014) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199673599.001.0001 · Cited by 223 · see citing works
If Robots cause harm, Who is to blame? Self-driving Cars and Criminal Liability
Gleß, S., Silverman, E. D., & Weigend, T. (2016) · New Criminal Law Review · doi:10.1525/nclr.2016.19.3.412 · Cited by 175 · see citing works
Prevention and the Limits of the Criminal Law
(2013) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199656769.001.0001 · Cited by 172 · see citing works
The EU Artificial Intelligence Act
Neuwirth, R. (2022) · Social Science Research Network · doi:10.4324/9781003319436 · Cited by 169 · see citing works
Goldstein, J. A., Sastry, G., Musser, M., DiResta, R., Gentzel, M., & Sedova, K. (2023) · arXiv · doi:10.48550/arxiv.2301.04246 · Cited by 152 · see citing works
Husak, D. (2007) · doi:10.1093/acprof:oso/9780195328714.001.0001 · Cited by 93 · see citing works
Kelly, E. I. (2018) · Harvard University Press eBooks · doi:10.4159/9780674989436 · Cited by 82 · see citing works
An AI-Based Medical Chatbot Model for Infectious Disease Prediction
Chakraborty, S., Paul, H., Ghatak, S., Pandey, S. K., Kumar, A., Singh, K. U., & Shah, M. A. (2022) · IEEE Access · doi:10.1109/access.2022.3227208 · Cited by 80 · see citing works
The Preventive Turn in Criminal Law
Carvalho, H. (2017) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198737858.001.0001 · Cited by 78 · see citing works
Probing Toxic Content in Large Pre-Trained Language Models
Ousidhoum, N. D., Zhao, X., Fang, T., Song, Y., & Yeung, D. (2021) · doi:10.18653/v1/2021.acl-long.329 · Cited by 74 · see citing works
The Boundaries of the Criminal Law
Farmer, L. (2010) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199600557.001.0001 · Cited by 72 · see citing works
Positive Obligations in Criminal Law
(2013) · Hart Publishing eBooks · doi:10.5040/9781472561442 · Cited by 72 · see citing works
Oxford Handbook of Online Intermediary Liability
Cogo, A. E., & Ricolfi, M. (2020) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198837138.001.0001 · Cited by 64 · see citing works
Prevention and Criminalization
Ashworth, A. J., & Zedner, L. (2012) · New Criminal Law Review · doi:10.1525/nclr.2012.15.4.542 · Cited by 63 · see citing works
Simmler, M., & Markwalder, N. (2018) · Criminal Law Forum · doi:10.1007/s10609-018-9360-0 · Cited by 55 · see citing works
Comparative Concepts of Criminal Law
Keiler, J., & Roef, D. (2015) · Cited by 53 · see citing works
Sex, Lies, and Videotape: Deep Fakes and Free Speech Delusions
Franks, M. A., & Waldman, A. E. (2019) · Digital Commons at University of Maryland Carey Law (University of Maryland Francis King Carey School of Law) · Cited by 51 · see citing works
Three Liability Regimes for Artificial Intelligence
Beckers, A., & Teubner, G. (2021) · Hart Publishing eBooks · doi:10.5040/9781509949366 · Cited by 49 · see citing works
Core Concepts in Criminal Law and Criminal Justice
Dyson, M., & Meyer, F. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108649742 · Cited by 39 · see citing works
Ebers, M. (2022) · Cambridge University Press eBooks · doi:10.1017/9781009072168.030 · Cited by 38 · see citing works
Algorithms Acting Badly: A Solution from Corporate LawPreprint
Diamantis, M. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3545436 · Cited by 27 · see citing works
Tort Liability for Human Rights Abuses
(2008) · Hart Publishing eBooks · doi:10.5040/9781472564535 · Cited by 26 · see citing works
Where's the Liability in Harmful AI Speech?Preprint
Lemley, M. A., Henderson, P., & Hashimoto, T. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4531029 · Cited by 20 · see citing works
Socio-Digital VulnerabilityPreprint
DiPaola, D., & Calo, R. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4686874 · Cited by 17 · see citing works
Simmler, M. (2023) · Information Communication & Society · doi:10.1080/1369118x.2023.2239895 · Cited by 15 · see citing works
Where's the Liability in Harmful AI Speech?Preprint
Henderson, P., Hashimoto, T., & Lemley, M. A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.04635 · Cited by 14 · see citing works
Donini, M. (2008) · IRIS UNIMORE (University of Modena and Reggio Emilia) · Cited by 9 · see citing works
Large Libel Models? Liability for AI Output
Volokh, E. (2023) · eScholarship (California Digital Library) · Cited by 9 · see citing works
Lemoine, L., & Vermeulen, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4702422 · Cited by 6 · see citing works
Adversarial Machine Learning: The Rise in AI-Enabled Crime and its Role in Spam Filter EvasionPreprint
Sivaram, J., Narrain, J. M., Honnavalli, P. B., & Eswaran, S. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4155496 · Cited by 5 · see citing works
Negligent AI Speech: Some Thoughts About DutyPreprint
Bambauer, J. R. Y. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4432822 · Cited by 3 · see citing works
Stortoni, L., & Foffani, L. (2004) · Cited by 1 · see citing works
Criminal behavior and accountability of artificial intelligence systems
Giannini, A. (2023) · doi:10.26481/dis.20231124ag · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 9
- Askell, A., Brundage, M, and Hadfield, G K. (2019). The Role of Cooperation in Responsible AI Development. https://api. semanticscholar.org/CorpusID:195874137
- Beck, S., & Gerndt, S. (2023). German report on traditional criminal law categories and AI. Revue Internationale de Droit Pénal, 94(1), 195-223.
- Flor, R. (2024). Cybersecurity for artificial intelligence, medical device security and criminal law: First thoughts in the prism of European Law. In P. Magagna, & R. Magagna (Eds.), Advanced diagnostic tools in aortic pathology (pp. 142-149). Edizioni Minerva Medica.
- Gehman, S. (2020). Real toxicity prompts: Evaluating neural toxic degeneration in language models. In T. Cohn, Y. He, Y. Liu (Eds.). Findings of the association for computational linguistics (pp. 3356-3369). The Association for Computational Linguistics.
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative Adversarial Nets. In Z. Ghahramani (Eds.), Advances in Neural Information Processing Systems (pp. 1-9).
- Miró-Llinares, F., Duvac, C., Toader, T., et al. (Eds.) (2024). Criminalisation of AI-related offence [Monograph]. Revue Internationale de Droit Pénal, 1, 1-445.
- Mongillo, V. (2023). Corporate criminal liability for AI-related crimes: Possible legal techniques and obstacles. International Review of Penal Law, 1, 77-90.
- Stewart, J. G. (2015). Complicity. In M. D. Dubber & T. H ö rnle (Eds.), The Oxford handbook of criminal law (pp. 534-559). Oxford University Press.
- Surden, H. (2024). ChatGPT, artificial intelligence (AI) large language models, and law. Fordham Law Review, 92, 1940-1970.
Cited by
Sorted by citations
Platform-Facilitated Grooming and AI Chatbots: Rethinking Criminal Liability and Regulation
Chawki, M. (2026) · Laws · doi:10.3390/laws15040093 · Cited by 1 · see citing works
Generative AI in child sexual exploitation and abuse: views from UK law enforcement
Demeocq, C., Taylor, A., Ross, B., & McFeeters, A. (2026) · AI & SOCIETY · doi:10.1007/s00146-026-03176-6 · Cited by 1 · see citing works
Potential and Risk in the Intersection of AI and the Criminal Justice System
Stobbs, N., Hunter, D., & Bagaric, M. (2025) · Oxford Research Encyclopedia of Criminology and Criminal Justice · doi:10.1093/acrefore/9780190264079.013.714
Generative AI jailbreaks as emerging cyberthreats: Ethical, legal, and societal implications
Teichmann, F. (2026) · International Cybersecurity Law Review · doi:10.1365/s43439-026-00171-x
Guo, K., & Qiu, Y. (2026) · Crime Law and Social Change · doi:10.1007/s10611-026-10299-w
- Authors
- Panattoni, B.
- Affiliations
- University of Verona (IT)
- Corresponding author
- Panattoni, Beatrice
- ORCID
- Panattoni, Beatrice
- Venue
- Cambridge Forum on AI: Law and Governance
- Publisher
- Cambridge University Press (CUP)
- Published
- 2025
- Volume
- 1
- ISSN
- 3033-3733 (ISSN-L)
- OpenAlex
- W4406078431
- Cited by
- 5
- References
- 45
- Citations by year
- 2026: 4 · 2025: 1
- FWCI
- 31.30
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Cambridge Forum on AI Law and Governance (publishedVersion, cc-by)
- Open Repository and Bibliography (University of Luxembourg) (submittedVersion, cc-by)
- Topics
- Social Sciences › Law › Digitalization, Law, and Regulation
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
- Keywords
- generative AI, criminal law, criminal liability, moral agency, moral responsibility, sexual abuse, incitement to violence, self-harm, harmful content, cognitive manipulation, guilt, actus reus
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
