CoChat
Sign Up
Cited by 271Ethics and Social Impacts of AIDigitalization, Law, and RegulationCybercrime and Law Enforcement Studies

Artificial Intelligence Crime: An Interdisciplinary Analysis of Foreseeable Threats and Solutions

Open access
King, Thomas C., Aggarwal, Nikita, Taddeo, Mariarosaria, Floridi, Luciano·2020·Science and Engineering Ethics
DOI 10.1007/s11948-018-00081-0PMC PMC6978427
Open this paper in CoChat — free accountOpen PDF

Abstract

Artificial intelligence (AI) research and regulation seek to balance the benefits of innovation against any potential harms and disruption. However, one unintended consequence of the recent surge in AI research is the potential re-orientation of AI technologies to facilitate criminal acts, term in this article AI-Crime (AIC). AIC is theoretically feasible thanks to published experiments in automating fraud targeted at social media users, as well as demonstrations of AI-driven manipulation of simulated markets. However, because AIC is still a relatively young and inherently interdisciplinary area-spanning socio-legal studies to formal science-there is little certainty of what an AIC future might look like. This article offers the first systematic, interdisciplinary literature analysis of the foreseeable threats of AIC, providing ethicists, policy-makers, and law enforcement organisations with a synthesis of the current problems, and a possible solution space.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s11948-018-00081-0

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Literature [41, 42, 92] has collected and studied representative AI misuse cases, and summarized high-stake patterns that AI failures tend to follow [92], such as vulnerability to attacks, underperformance on noises, and biased prediction.

    “Literature such as [41, 42, 92] has collected and studied representative AI misuse cases, and summarized high-stake patterns that AI failures tend to follow [92], such as vulnerability to attacks, underperformance on noises, and biased prediction.”

    From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [42]

    The Conclusions (Security; page unavailable) discuss AI systems’ resilience to attacks and the exploitation of vulnerabilities, supporting the attack-related example. The paper does not summarize the claimed three-part pattern of AI failures: searches for noise-related underperformance and biased prediction found no corresponding discussion. Its Conclusions instead describe a literature analysis of AI-enabled crime threats and solutions. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • The mens rea may comprise an intention to commit the actus reus using an AI-based application, or knowledge that deploying an autonomous agent will or could cause it to perform a criminal action or omission.

    “The mens rea may comprise an intention to commit the actus reus using an AI-based application, or knowledge that deploying an autonomous agent will or could cause it to perform a criminal action or omission [250].”

    From Applications and Challenges of Artificial Intelligence in Space Missions (2024) · cites this paper as [250]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 123 · Cited by 271

Research works printed but not linked: 60

  1. Alvisi L Clement A Epasto A Lattanzi S Panconesi A SoK: The evolution of sybil defense via social networks Proceedings—IEEE Symposium on Security and Privacy 2013 2 382 396
  2. Andrighetto, G., Governatori, G., Noriega, P., & van der Torre, L. (2013). Normative multi-agent systems. In Dagstuhl follow-ups. Vol. 4. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.
  3. Arkin, R. C. (2008). Governing lethal behavior: Embedding ethics in a hybrid deliberative/reactive robot architecture part I: Motivation and philosophy. In Proceedings of the 3rd international conference on human robot interaction—HRI’08, 10.1145/1349822.1349839. · DOI
  4. Arkin, R. C., & Ulam, P. (2012). Overriding ethical constraints in lethal autonomous systems. Technical report GIT-MRL-12-01 (pp. 1–8). https://pdfs.semanticscholar.org/d232/4a80d870e01db4ac02ed32cd33a8edf2bbb7.pdf.
  5. Bilge, L., Strufe, T., Balzarotti, D., Kirda, K., & Antipolis, S. (2009). All your contacts are belong to us: Automated identity theft attacks on social networks. In WWW’09 proceedings of the 18th international conference on the world wide web (pp. 551–560). http://doi.acm.org/10.1145/1526709.1526784. · DOI
  6. Boshmaf, Y., Muslukhov, I., Beznosov, K., & Ripeanu, M. (2012b). Key challenges in defending against malicious socialbots. In Proceedings of the 5th USENIX workshop on large-scale exploits and emergent threats (pp. 1–5). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.382.8607.
  7. Cath C Wachter S Mittelstadt B Taddeo M Floridi L Artificial intelligence and the ‘Good Society’: The US, EU, and UK approach Science and Engineering Ethics 2017 24 2 505 528 28353045
  8. Chen, Y. P., Chen, P., Song, R., & Korba, L. (2004). Online gaming crime and security issues—cases and countermeasures from Taiwan. In Proceedings of the 2nd annual conference on privacy, security and trust. https://nrc-publications.canada.ca/eng/view/object/?id=a4a70b1a-332b-4161-bab5-e690de966a6b.
  9. Chu, Z., Gianvecchio, S., Wang, H., & Jajodia, S. (2010). Who is tweeting on Twitter: Human, bot, or cyborg? In ACSAC’10, proceedings of the 26th annual computer security applications conference (pp. 21–30). 10.1145/1920261.1920265. · DOI
  10. Cliff Dave Northrop Linda The Global Financial Markets: An Ultra-Large-Scale Systems Perspective Large-Scale Complex IT Systems. Development, Operation and Management 2012 Berlin, Heidelberg Springer Berlin Heidelberg 29 70
  11. Darling K Lin P Bekey G Abney K Jenkins R Who’s Johnny?’ Anthropomorphic framing in human–robot interaction, integration, and policy Robot ethics 2.0 2017 Oxford Oxford University Press
  12. De Angeli A Ethical implications of verbal disinhibition with conversational agents Psychology Journal 2009 7 1 49 57
  13. De Lima Salge CA Berente N Is that social bot behaving unethically? Communications of the ACM 2017 60 9 29 31
  14. Delamaire L Abdou H Pointon J Credit card fraud and detection techniques: A review Banks and Bank Systems 2009 4 2 57 68
  15. Dennett DC The intentional stance 1987 Cambridge, MA MIT Press
  16. Ezrachi, A., & Stucke, M. E. (2016). Two artificial neural networks meet in an online hub and change the future (of competition, market dynamics and society). Oxford Legal Studies Research Paper, No. 24/2017, University of Tennessee Legal Studies Research Paper, No. 323, 10.2139/ssrn.2949434. · DOI
  17. Ferrara, E. (2015). Manipulation and abuse on social media, 10.1145/2749279.2749283. · DOI
  18. Ferrara E Varol O Davis C Menczer F Flammini A The rise of social bots Communications of the ACM 2014 59 7 96 104
  19. Floridi L The ethics of information 2013 Oxford Oxford University Press
  20. Floridi L Taddeo M What is data ethics? Philosophical Transactions of the Royal Society A: Mathematical Physical and Engineering Sciences 2016 PMC5124072 28336805
  21. Freier, N. (2008). Children attribute moral standing to a personified agent. In Proceedings of the 26th annual SIGCHI conference on human factors in computing systems (CHI’08) (pp. 343–352). 10.1145/1357054.1357113. · DOI
  22. Freitas, P. M., Andrade, F., & Novais, P. (2014). Criminal liability of autonomous agents: From the unthinkable to the plausible. In: P. Casanovas, U. Pagallo, M. Palmirani, G. Sartor (Eds.), AI approaches to the complexity of legal systems. AICOL 2013. Lecture notes in computer science (Vol. 8929). Berlin: Springer.
  23. Gauci, M., Chen, J., Li, W., Dodd, T. J., & Gross, R. (2014). Clustering objects with robots that do not compute. Proceedings of the 2014 international conference on autonomous agents and multi-agent systems (AAMAS 2014) (pp. 421–428). https://dl.acm.org/citation.cfm?id=2615800.
  24. Gless S Silverman E Weigend T If robots cause harm, who is to blame? Self-driving cars and criminal liability New Criminal Law Review 2016 19 3 412 436
  25. Gogarty B Hagger M The laws of man over vehicles unmanned: The legal response to robotic revolution on sea, land and air Journal of Law, Information and Science 2008 19 73 145
  26. Hallevy G Unmanned vehicles—Subordination to criminal law under the modern concept of criminal liability Journal of Law, Information and Science 2012 21 200 211
  27. Haugen, G. M. S. (2017). Manipulation and deception with social bots: Strategies and indicators for minimizing impact. http://hdl.handle.net/11250/2448952. Accessed 5 Jan 2018.
  28. Hay GA Kelley D An empirical survey of price fixing conspiracies The Journal of Law and Economics 1974 17 1 13 38
  29. Jagatic TN Johnson NA Jakobsson M Menczer F Social phishing Communications of the ACM 2007 50 10 94 100
  30. Janoff-Bulman R Erroneous assumptions: Popular belief in the effectiveness of torture interrogation Peace and Conflict: Journal of Peace Psychology 2007 13 4 429
  31. Joh EE Policing police robots UCLA Law Review Discourse 2016 64 516
  32. Kerr IR Bots, babes and the californication of commerce University of Ottawa Law and Technology Journal 2004 1 284 324
  33. Kolosnjaji, B., Demontis, A., Biggio, B., Maiorca, D., Giacinto, G., Eckert, C., & Roli, F. (2018). Adversarial malware binaries: Evading deep learning for malware detection in executables. http://arxiv.org/abs/1803.04173. · arXiv
  34. Lin TCW The new market manipulation Emory Law Journal 2017 66 1253
  35. Luhmann N Social systems 1995 Stanford Stanford University Press
  36. Mackey TK Kalyanam J Katsuki T Lanckriet G Machine learning to detect prescription opioid abuse promotion and access via Twitter American Journal of Public Health 2017 107 12 e1 e6 PMC5678375 29048960
  37. Martínez-Miranda, E., McBurney, P., & Howard, M. J. (2016). Learning unfair trading: A market manipulation analysis from the reinforcement learning perspective. In Proceedings of the 2016 IEEE conference on evolving and adaptive intelligent systems, EAIS 2016 (pp. 103–109). 10.1109/EAIS.2016.7502499. · DOI
  38. McAllister A Stranger than science fiction: The rise of AI interrogation in the dawn of autonomous robots and the need for an additional protocol to the UN convention against torture Minnesota Law Review 2017 101 2527 2573
  39. McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (1955). A proposal for the Dartmouth summer research project on artificial intelligence.10.1609/aimag.v27i4.1904. · DOI
  40. Meneguzzi, F., & Luck, M. (2009). Norm-based behaviour modification in BDI agents. In Proceedings of the 8th international joint conference on autonomous agents and multi-agent systems (AAMAS 2009) (pp. 177–184).
  41. Neff G Nagy P Talking to bots: Symbiotic agency and the case of tay International Journal of Communication 2016 10 4915 4931
  42. Pagallo, U. (2017a). From automation to autonomous systems: A legal phenomenology with problems of accountability. In Proceedings of the 26th international joint conference on artificial intelligence (IJCAI-17) (pp. 17–23).
  43. Ratkiewicz, J., Conover, M., Meiss, M., Gonçalves, B., Patil, S., Flammini, A., & Menczer, F. (2011). Truthy: Mapping the spread of astroturf in Microblog streams. In Proceedings of the 20th international conference companion on world wide web (WWW’11) (pp. 249–252). 10.1145/1963192.1963301. · DOI
  44. Searle JR Intentionality: An essay in the philosophy of mind 1983 Cambridge Cambridge University Press
  45. Sharkey N Goodman M Ross N The coming robot crime wave IEEE Computer Magazine 2010 43 8 6 8
  46. Solis GD The law of armed conflict: International humanitarian law in war 2016 2 Cambridge Cambridge University Press
  47. Spatt C Security market manipulation Annual Review of Financial Economics 2014 6 1 405 418
  48. Taddeo M Floridi L Solving the symbol grounding problem: A critical review of fifteen years of research Journal of Experimental and Theoretical Artificial Intelligence 2005 17 4 419 445
  49. Taddeo M Floridi L Regulate artificial intelligence to avert cyber arms race Nature 2018 556 296 298 29662138
  50. Tonti, G., Bradshaw, J. M., & Jeffers, R. (2003). Semantic web languages for policy representation and reasoning: A comparison of KAoS, Rei, and Ponder. In Proceedings of international semantic web conference (pp. 419–437).
  51. Uszok, A. J., Bradshaw, R. J., Suri, N., Hayes, P., Breedy, M., Bunch, L., Johnson, M., Kulkarni, S., & Lott, J. (2003). KAoS policy and domain services: Toward a description-logic approach to policy representation, deconfliction, and enforcement. In Proceedings of IEEE policy 2003 (pp. 93–98). Los Amitos, CA: IEEE Computer Society.
  52. Van de Poel I Fahlquist JN Doorn N Zwart S Royakkers L The problem of many hands: Climate change as an example Science and Engineering Ethics 2012 18 49 67 21533835 PMC3275719
  53. Van Lier B From high frequency trading to self-organizing moral machines International Journal of Technoethics 2016 7 1 34 50
  54. Van Riemsdijk, M. B., Dennis, L., & Fisher, M. (2015). A semantic framework for socially adaptive agents towards strong norm compliance. In Proceedings of the 14th international conference on autonomous agents and multiagent systems (AAMAS 2015) (pp. 423–432). https://dl.acm.org/citation.cfm?id=2772935.
  55. Van Riemsdijk, M. B., Dennis, L. A., Fisher, M., & Hindriks, K. V. (2013). Agent reasoning for norm compliance: A semantic approach. In Proceedings of the 12th international conference on autonomous agents and multiagent systems (AAMAS 2013) (pp. 499–506). https://dl.acm.org/citation.cfm?id=2485000.
  56. Vanderelst, D., & Winfield, A. (2016b). The dark side of ethical robots. https://arxiv.org/abs/1606.02583. · arXiv
  57. Wang Y Kosinski M Deep neural networks can detect sexual orientation from faces Journal of Personality and Social Psychology 2017 114 2 246 257 29389215
  58. Wang, G., Mohanlal, M., Wilson, C., Wang, X., Metzger, M., Zheng, H., & Zhao, B. Y. (2012). Social turing tests: Crowdsourcing sybil detection. http://arxiv.org/abs/1205.3856. · arXiv
  59. Weizenbaum J Computer power and human reason: From judgment to calculation 1976 Oxford W. H. Freeman & Co.
  60. Yang GZ Bellingham J Dupont PE Fischer P Floridi L Full R Jacobstein N Kumar V McNutt M Merrifield R Nelson BJ Scassellati B Taddeo M Taylor R Veloso M Wang ZL Wood R The grand challenges of science robotics Science Robotics 2018 3 14 eaar7650 33141701

Authors
King, T. C., Aggarwal, N., Taddeo, M., & Floridi, L.
Affiliations
University of Oxford (GB)
Corresponding author
Floridi, Luciano
ORCID
Venue
Science and Engineering Ethics
Publisher
Springer Science and Business Media LLC
Published
2019-02-14
Volume
26
Issue
1
Pages
89-120
ISSN
  • 1353-3452 (ISSN-L)
  • 1471-5546
DOI
10.1007/s11948-018-00081-0
PMC
PMC6978427
OpenAlex
W2808102992
Cited by
271
References
123
Citations by year
2026: 23 · 2025: 64 · 2024: 51 · 2023: 53 · 2022: 24 · 2021: 28 · 2020: 23 · 2019: 4 · 2018: 1
FWCI
5.19
Type
journal article
Language
en
Open access
Licence
Locations
  • Science and Engineering Ethics (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
Engineering and Physical Sciences Research Council (EP/N023013/1)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Law › Digitalization, Law, and Regulation
  • Physical Sciences › Computer Science › Information Systems › Cybercrime and Law Enforcement Studies
Keywords
social media fraud, AI regulation, AI ethics, law enforcement
MeSH terms
Interdisciplinary Research, Artificial Intelligence, Commerce, Crime, Forecasting, Fraud, Humans, Sex Offenses, Liability, Legal, Social Media, Drug Trafficking

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review