Monitoring AI Services for Misuse
Open accessAbstract
Given the surge in interest in AI, we now see the emergence of Artificial Intelligence as a Service (AIaaS). AIaaS entails service providers offering remote access to ML models and capabilities at arms-length', through networked APIs. Such services will grow in popularity, as they enable access to state-of-the-art ML capabilities, 'on demand', 'out of the box', at low cost and without requiring training data or ML expertise. However, there is much public concern regarding AI. AIaaS raises particular considerations, given there is much potential for such services to be used to underpin and drive problematic, inappropriate, undesirable, controversial, or possibly even illegal applications. A key way forward is through service providers monitoring their AI services to identify potential situations of problematic use. Towards this, we elaborate the potential for 'misuse indicators' as a mechanism for uncovering patterns of usage behaviour warranting consideration or further investigation. We introduce a taxonomy for describing these indicators and their contextual considerations, and use exemplars to demonstrate the feasibility analysing AIaaS usage to highlight situations of possible concern. We also seek to draw more attention to AI services and the issues they raise, given AIaaS' increasing prominence, and the general calls for the more responsible and accountable use of AI.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Providers increasingly offer Artificial Intelligence solutions 'as a service' (AIaaS).
“Recognising the opportunity, providers increasingly ofer Artifcial Intelligence solutions”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [72]
There is a risk that algorithmic biases or other undesirable behaviours in AIaaS systems could be reproduced on a far greater scale across organisations.
“First, given that these services aim to ofer sophisticated AI capabilities at low cost and on demand to potentially anyone, often requiring little to no technical expertise, there is a risk that any algorithmic biases or other undesirable behaviours in the AIaaS system could be reproduced on a far greater scale across organisations [29,72].”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [72]
The concern about appropriateness of an AI service is relevant for any ML model development, but it is particularly pertinent in the AIaaS context where potential harms might achieve a far larger scale.
“While this concern is relevant for any ML model development, it is particularly pertinent in the AIaaS context where potential harms might achieve a far larger scale [29,72].”
From Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service” (2023) · cites this paper as [72]
Cites
(1996) · Cited by 20,179 · see citing works
Least squares quantization in PCM
Lloyd, S. (1982) · IEEE Transactions on Information Theory · doi:10.1109/tit.1982.1056489 · Cited by 15,836 · see citing works
A few useful things to know about machine learning
Domingos, P. (2012) · Communications of the ACM · doi:10.1145/2347736.2347755 · Cited by 3,326 · see citing works
An Introduction to Statistical Learning
James, G. M., Witten, D., Hastie, T. J., Tibshirani, R. J., & Taylor, J. (2023) · Springer texts in statistics · doi:10.1007/978-3-031-38747-0 · Cited by 1,765 · see citing works
Intrusion detection system: A comprehensive review
Liao, H.-J., Lin, C. R., Lin, Y.-C., & Tung, K.-Y. (2012) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2012.09.004 · Cited by 1,640 · see citing works
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · see citing works
Vaccari, C., & Chadwick, A. (2020) · Social Media + Society · doi:10.1177/2056305120903408 · Cited by 978 · cited 1× · see citing works
“speech generation for impersonation [19, 46], speech-to-text in surveillance [33, 47], object recognition for tracking movements [5, 32], and various tools that ultimately support producing ‘deepfakes’ [21].”
BackgroundA survey of anomaly detection techniques in financial domain
Ahmed, M., Mahmood, A. N., & Islam, R. (2015) · Future Generation Computer Systems · doi:10.1016/j.future.2015.01.001 · Cited by 553 · see citing works
A comprehensive survey on network anomaly detection
Fernandes, G., Rodrigues, J. J. P. C., Carvalho, L. F., Al-Muhtadi, J. F., & Proença, M. L. (2018) · Telecommunication Systems · doi:10.1007/s11235-018-0475-8 · Cited by 396 · see citing works
MLaaS: Machine Learning as a Service
Ribeiro, M., Grolinger, K., & Capretz, M. A. M. (2015) · IEEE International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla.2015.152 · Cited by 395 · see citing works
Wieringa, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372833 · Cited by 321 · see citing works
The FERET evaluation methodology for face-recognition algorithms
Phillips, P. J., Moon, H., Rizvi, S. A., & Rauss, P. J. (1998) · doi:10.6028/nist.ir.6264 · Cited by 273 · see citing works
From ethics washing to ethics bashing: a view on tech ethics from within moral philosophy
Bietti, E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372860 · Cited by 261 · see citing works
What's Next for AI Ethics, Policy, and Governance? A Global Overview
Schiff, D. S., Biddle, J. B., Borenstein, J., & Laas, K. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375804 · Cited by 204 · see citing works
Building Trust in Artificial Intelligence
Rossi, F. (2018) · Journal of international affairs · Cited by 103 · see citing works
Amazon: Surveillance as a Service
West, E. (2019) · Surveillance & Society · doi:10.24908/ss.v17i1/2.13008 · Cited by 89 · see citing works
Deepfakes: False Pornography Is Here and the Law Cannot Protect You
Harris, D. C. (2019) · Duke Law Scholarship Repository (Duke University) · Cited by 84 · see citing works
A Survey on Image Segmentation Methods using Clustering Techniques
Dhanachandra, N., & Chanu, Y. J. (2017) · European Journal of Engineering and Technology Research · doi:10.24018/ejers.2017.2.1.237 · Cited by 74 · see citing works
Will deepfakes do deep damage?
Greengard, S. (2019) · Communications of the ACM · doi:10.1145/3371409 · Cited by 44 · see citing works
Righi, R. D. R., Lehmann, M. B., Gomes, M. M., Nobre, J. C., da Costa, C. A., Rigo, S. J., Lena, M., Mohr, R. F., & de Oliveira, L. R. B. (2019) · Journal of Grid Computing · doi:10.1007/s10723-018-09471-x · Cited by 34 · see citing works
Monitoring Misuse for Accountable 'Artificial Intelligence as a Service'
Javadi, S. A., Cloete, R., Cobbe, J., Lee, M. S. A., & Singh, J. P. (2020) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3375627.3375873 · Cited by 29 · see citing works
Self-citationAlexa, What Should We Do about Privacy? Protecting Privacy for Users of Voice-Activated Devices
Pfeifle, A. (2018) · Washington law review · Cited by 24 · see citing works
Strengthening the U.S. AI Workforce
Zwetsloot, R., Heston, R., & Arnold, Z. (2019) · doi:10.51593/20190003 · Cited by 17 · see citing works
Artificial Intelligence as a Service: Legal Responsibilities, Liabilities, and Policy ChallengesPreprint
Cobbe, J., & Singh, J. P. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3824736 · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Understanding accountability in algorithmic supply chains
Cobbe, J., Veale, M., & Singh, J. P. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594073 · Cited by 141 · see citing works
Self-citationWidder, D. G., & Nafus, D. (2023) · Big Data & Society · doi:10.1177/20539517231177620 · Cited by 119 · see citing works
Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges
Wang, Q., Madaio, M., Kane, S. K., Kapania, S., Terry, M. A., & Wilcox, L. G. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581278 · Cited by 94 · see citing works
Steimers, A., & Schneider, M. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19063641 · Cited by 97 · see citing works
Chan, A., Ezell, C., Kaufmann, M., Wei, K., Hammond, L., Bradley, H., Bluemke, E., Rajkumar, N., Krueger, D., Kolt, N., et al. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658948 · Cited by 84 · see citing works
Cheung, K. K. C., Yun, L., Liu, Q., & Chan, H. (2024) · Science & Education · doi:10.1007/s11191-024-00511-5 · Cited by 100 · see citing works
Toward Label-Efficient Emotion and Sentiment Analysis
Zhao, S., Hong, X., Yang, J., Zhao, Y., & Ding, G. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3309299 · Cited by 61 · see citing works
Lewicki, K., Lee, M. S. A., Cobbe, J., & Singh, J. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581463 · Cited by 30 · see citing works
“Recognising the opportunity, providers increasingly offer Artificial Intelligence solutions “as a service” (AIaaS) [28, 71].”
BackgroundMethodInfluentialSelf-citationPosition is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)
Neumann, A., Kirsten, E., Zafar, M. B., & Singh, J. P. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732038 · Cited by 28 · see citing works
Self-citationMadaio, M., Chen, J., Wallach, H., & Vaughan, J. W. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653705 · Cited by 13 · see citing works
Artificial Intelligence as a Service: Legal Responsibilities, Liabilities, and Policy ChallengesPreprint
Cobbe, J., & Singh, J. P. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3824736 · Cited by 11 · see citing works
Self-citationNavigating the Audit Landscape: A Framework for Developing Transparent and Auditable XR
Norval, C., Cloete, R., & Singh, J. P. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594090 · Cited by 9 · see citing works
Self-citationBorines, A., Teckle, P., & Turi, A. N. (2025) · Future of business and finance · doi:10.1007/978-3-031-73639-1_1 · Cited by 9 · see citing works
Towards AI as a Service for Small and Medium-Sized Enterprises (SME)
Griesch, L., Rittelmeyer, J. D., & Sandkuhl, K. (2023) · Lecture notes in business information processing · doi:10.1007/978-3-031-48583-1_3 · Cited by 8 · see citing works
Accountability challenges of AI in smart grid services
Volkova, A., Patil, A. D., Javadi, S. A., & de Meer, H. (2022) · ACM International Conference on Future Energy Systems (e-Energy) · doi:10.1145/3538637.3539636 · Cited by 7 · see citing works
Self-citationThe What, So What, and What Now of the AI Landscape in Emerging Economies
Borines, V. A. M., Turi, A. N., & Hedru, P. (2024) · IEEE Region 10 Conference (TENCON) · doi:10.1109/tencon61640.2024.10902844 · Cited by 7 · see citing works
Auditable Augmented/Mixed/Virtual Reality
Cloete, R., Norval, C., & Singh, J. P. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3495001 · Cited by 4 · see citing works
Self-citationSuperscalar Time-Triggered Versatile-Tensor Accelerator
Bebawy, Y., Ezekiel, A. M., & Obermaisser, R. (2025) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2025.3528355 · Cited by 1 · see citing works
K-Space Bispectrum Steganography for Robust Unlearnable Data
Li, J., Chen, Y., Xing, Y., Gu, Y., & Lan, X. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755349
Digitale Güter, digitale Dienstleistungen und digitale Arbeit als Leistungsergebnisse
Leimeister, J. M. (2025) · Springer eBooks · doi:10.1007/978-3-662-70646-6_7
The Paradox of ‘Algorithmic’ Accountability and its Implications for Design
Ramesh, D., Parthasarathy, S., & Banović, N. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812374
- Authors
- Javadi, S. A., Norval, C., Cloete, R., & Singh, J.
- Affiliations
- University of Cambridge (GB)
- Venue
- Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2021-07-21
- Pages
- 597-607
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W3190162743
- Cited by
- 21
- References
- 25
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 4 · 2023: 7 · 2022: 2 · 2021: 2
- FWCI
- 3.39
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion, cc-by)
- Funders
- EPSRC (EP/P024394/1, EP/R033501/1)
- Microsoft (Microsoft Cloud Computing Research Centre)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- AI-as-a-Service, responsible AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
