CoChat
Sign Up
Cited by 11Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Social Robot Interaction and HRI

Towards Interactive Evaluations for Interaction Harms in Human-AI Systems

Open access
Ibrahim, Lujain, Huang, Saffron, Ahmad, Lama, Bhatt, Umang, Anderljung, Markus·2025·Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society
DOI 10.1609/aies.v8i2.36631
Open this paper in CoChat — free accountOpen PDF

Abstract

Current AI evaluation methods, which rely on static, model-only tests, fail to account for harms that emerge through sustained human-AI interaction. As AI systems proliferate and are increasingly integrated into real-world applications, this disconnect between evaluation approaches and actual usage becomes more significant. In this paper, we propose a shift towards evaluation based on interactional ethics, which focuses on interaction harms—issues like inappropriate parasocial relationships, social manipulation, and cognitive overreliance that develop over time through repeated interaction, rather than through isolated outputs. First, we discuss the limitations of current evaluation methods, which (1) are static, (2) assume a universal user experience, and (3) have limited construct validity. Drawing on research from human-computer interaction, natural language processing, and the social sciences, we present practical principles for designing interactive evaluations. These include ecologically valid interaction scenarios, human impact metrics, and diverse human participation approaches. Finally, we explore implementation challenges and open research questions for researchers, practitioners, and regulators aiming to integrate interactive evaluations into AI governance frameworks. This work lays the groundwork for developing more effective evaluation methods that better capture the complex dynamics between humans and AI systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1609/aies.v8i2.36631

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Measures of AI dependence consist either of adaptations from other scales of behavioral addiction to technologies (e.g., social media, smartphone, or internet addiction) or self-developed measures.

    “Measures of “AI dependence” consist either of adaptations from other scales of behavioral addiction to technologies (e.g., social media, smartphone, or internet addiction)100–102, or self-developed measures103–106.”

    From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [104]

    The citation falls in the group offered for self-developed measures of AI dependence. Section 2.3 calls for quantifying dependency levels, and Section 4.2 discusses measures of human impact, but neither describes a self-developed AI-dependence measure. Searches for such a measure and for adapted addiction scales found no supporting passage in our copy; page numbers were not provided. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 51

  1. Agnew, W.; Bergman, A. S.; Chien, J.; Díaz, M.; El- Sayed, S.; Pittman, J.; Mohamed, S.; and McKee, K. R. 2024. The illusion of artificial inclusion. arXiv preprint arXiv:2401.08572. · arXiv
  2. Alberts, L.; Keeling, G.; and McCroskery, A. 2024. Should agentic conversational AI change how we think about ethics? Characterising an interactional ethics centred on re- spect. arXiv preprint arXiv:2401.09082. · arXiv
  3. Anthis, J. R.; Liu, R.; Richardson, S. M.; Kozlowski, A. C.; Koch, B.; Evans, J.; Brynjolfsson, E.; and Bernstein, M. 2025. LLM Social Simulations Are a Promising Research Method. arXiv preprint arXiv:2504.02234. · arXiv
  4. Blodgett, S. L.; Lopez, G.; Olteanu, A.; Sim, R.; and Wal- lach, H. 2021. Stereotyping Norwegian salmon: An inven- tory of pitfalls in fairness benchmark datasets. In Proceed- ings of the 59th Annual Meeting of the Association for Com- putational Linguistics and the 11th International Joint Con- ference on Natural Language Processing (Volume 1: Long Papers), 1004-1015.
  5. Brehaut, J.; O'Connor, A.; Wood, T.; Hack, T.; Siminoff, L.; Gordon, E.; and Feldman-Stewart, D. 2003. Validation of a Decision Regret Scale. Medical decision making : an inter- national journal of the Society for Medical Decision Mak- ing, 23: 281-92.
  6. Chang, Y.; Wang, X.; Wang, J.; Wu, Y.; Yang, L.; Zhu, K.; Chen, H.; Yi, X.; Wang, C.; Wang, Y.; et al. 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3): 1-45.
  7. Chen, Y.; Wu, A.; DePodesta, T.; Yeh, C.; Li, K.; Marin, N. C.; Patel, O.; Riecke, J.; Raval, S.; Seow, O.; et al. 2024. Designing a dashboard for transparency and control of con- versational AI. arXiv preprint arXiv:2406.07882. · arXiv
  8. Collins, K. M.; Jiang, A. Q.; Frieder, S.; Wong, L.; Zilka, M.; Bhatt, U.; Lukasiewicz, T.; Wu, Y.; Tenenbaum, J. B.; Hart, W.; et al. 2023. Evaluating language models for mathematics through interactions. arXiv preprint arXiv:2306.01694. · arXiv
  9. Coronado, E.; Kiyokawa, T.; Ricardez, G. A. G.; Ramirez- Alpizar, I. G.; Venture, G.; and Yamanobe, N. 2022. Evaluat- ing quality in human-robot interaction: A systematic search and classification of performance and human-centered fac- tors, measures and metrics towards an industry 5.0. Journal of Manufacturing Systems, 63: 392-410.
  10. Damacharla, P.; Javaid, A. Y.; Gallimore, J. J.; and Dev- abhaktuni, V. K. 2018. Common Metrics to Benchmark Human-Machine Teams (HMT): A Review. IEEE Access, 6: 38637-38655.
  11. Dev, S.; Sheng, E.; Zhao, J.; Amstutz, A.; Sun, J.; Hou, Y.; Sanseverino, M.; Kim, J.; Nishi, A.; Peng, N.; et al. 2021. On measures of biases and harms in NLP. arXiv preprint arXiv:2108.03362. · arXiv
  12. Gao, J.; Gebreegziabher, S. A.; Choo, K. T. W.; Li, T. J. J.; Perrault, S. T.; and Malone, T. W. 2024. A Taxonomy for Human-LLM Interaction Modes: An Initial Exploration. arXiv preprint arXiv:2404.00405. · arXiv
  13. Ghandeharioun, A.; Yuan, A.; Guerard, M.; Reif, E.; Lepori, M.; and Dixon, L. 2025. Who's asking? User personas and the mechanics of latent misalignment. Advances in Neural Information Processing Systems, 37: 125967-126003.
  14. Gordon, M. L.; Zhou, K.; Patel, K.; Hashimoto, T.; and Bernstein, M. S. 2021. The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI '21. New York, NY, USA: Association for Computing Machinery. ISBN 9781450380966.
  15. Hardy, A.; Reuel, A.; Meimandi, K. J.; Soder, L.; Griffith, A.; Asmar, D. M.; Koyejo, S.; Bernstein, M. S.; and Kochen- derfer, M. J. 2024. More than Marketing? On the Infor- mation Value of AI Benchmarks for Practitioners. arXiv preprint arXiv:2412.05520. · arXiv
  16. Hariton, E.; and Locascio, J. J. 2018. Randomised controlled trials -the gold standard for effectiveness research: Study design: randomised controlled trials. BJOG, 125(13): 1716. Epub 2018 Jun 19
  17. Hartvigsen, T.; Gabriel, S.; Palangi, H.; Sap, M.; Ray, D.; and Kamar, E. 2022. Toxigen: A large-scale machine- generated dataset for adversarial and implicit hate speech detection. arXiv preprint arXiv:2203.09509. · arXiv
  18. Hofmann, V.; Kalluri, P. R.; Jurafsky, D.; and King, S. 2024. AI generates covertly racist decisions about people based on their dialect. Nature, 633(8028): 147-154.
  19. Ibrahim, L.; Akbulut, C.; Elasmar, R.; Rastogi, C.; Kahng, M.; Morris, M. R.; McKee, K. R.; Rieser, V.; Shanahan, M.; and Weidinger, L. 2025. Multi-turn Evaluation of Anthro- pomorphic Behaviours in Large Language Models. arXiv preprint arXiv:2502.07077. · arXiv
  20. Ibrahim, L.; Rocher, L.; and Valdivia, A. 2024. Characteriz- ing and modeling harms from interactions with design pat- terns in AI interfaces. arXiv:2404.11370. · arXiv
  21. Jin, Z.; Heil, N.; Liu, J.; Dhuliawala, S.; Qi, Y.; Schölkopf, B.; Mihalcea, R.; and Sachan, M. 2024. Implicit person- alization in language models: A systematic study. arXiv preprint arXiv:2405.14808. · arXiv
  22. Kuniavsky, M. 2003. In Observing the User Experience, xiii-xvi. San Francisco: Morgan Kaufmann. ISBN 978-1- 55860-923-5.
  23. Lee, M.; Srivastava, M.; Hardy, A.; Thickstun, J.; Durmus, E.; Paranjape, A.; Gerard-Ursin, I.; Li, X. L.; Ladhak, F.; Rong, F.; et al. 2022. Evaluating human-language model interaction. arXiv preprint arXiv:2212.09746. · arXiv
  24. Li, N.; Pan, A.; Gopal, A.; Yue, S.; Berrios, D.; Gatti, A.; Li, J. D.; Dombrowski, A.-K.; Goel, S.; Phan, L.; et al. 2024. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403.03218. · arXiv
  25. Liao, Q.; and Sundar, S. S. 2022. Designing for Respon- sible Trust in AI Systems: A Communication Perspective. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT '22, 1257-1268.
  26. Liao, Q. V.; and Xiao, Z. 2023. Rethinking model evalu- ation as narrowing the socio-technical gap. arXiv preprint arXiv:2306.03100. · arXiv
  27. McCoy, R. T.; Yao, S.; Friedman, D.; Hardy, M.; and Grif- fiths, T. L. 2023. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638. · arXiv
  28. McKee, K. R. 2024. Human participants in AI research: Ethics and transparency in practice. IEEE Transactions on Technology and Society. METR. 2023. METR Autonomy Evaluations Resources.
  29. Ouyang, S.; Wang, S.; Liu, Y.; Zhong, M.; Jiao, Y.; Iter, D.; Pryzant, R.; Zhu, C.; Ji, H.; and Han, J. 2023. The shifted and the overlooked: a task-oriented investigation of user-gpt interactions. arXiv preprint arXiv:2310.12418. · arXiv
  30. Park, J. S.; O'Brien, J.; Cai, C. J.; Morris, M. R.; Liang, P.; and Bernstein, M. S. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and tech- nology, 1-22.
  31. Park, J. S.; Zou, C. Q.; Shaw, A.; Hill, B. M.; Cai, C.; Morris, M. R.; Willer, R.; Liang, P.; and Bernstein, M. S. 2024. Gen- erative agent simulations of 1,000 people. arXiv preprint arXiv:2411.10109. · arXiv
  32. Parrish, A.; Chen, A.; Nangia, N.; Padmakumar, V.; Phang, J.; Thompson, J.; Htut, P. M.; and Bowman, S. R. 2021. BBQ: A hand-built bias benchmark for question answering. arXiv preprint arXiv:2110.08193. · arXiv
  33. Perez, E.; Huang, S.; Song, F.; Cai, T.; Ring, R.; Aslanides, J.; Glaese, A.; McAleese, N.; and Irving, G. 2022. Red team- ing language models with language models. arXiv preprint arXiv:2202.03286. · arXiv
  34. Phang, J.; Lampe, M.; Ahmad, L.; Agarwal, S.; Fang, C. M.; Liu, A. R.; Danry, V.; Lee, E.; Chan, S. W.; Pataranutaporn, P.; et al. 2025. Investigating Affective Use and Emotional Well-being on ChatGPT. arXiv preprint arXiv:2504.03888. · arXiv
  35. Phuong, M.; Aitchison, M.; Catt, E.; Cogan, S.; Kaskasoli, A.; Krakovna, V.; Lindner, D.; Rahtz, M.; Assael, Y.; Hod- kinson, S.; et al. 2024. Evaluating Frontier Models for Dan- gerous Capabilities. arXiv preprint arXiv:2403.13793. · arXiv
  36. Raji, I. D.; Bender, E. M.; Paullada, A.; Denton, E.; and Hanna, A. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366. · arXiv
  37. Raji, I. D.; Kumar, I. E.; Horowitz, A.; and Selbst, A. 2022. The fallacy of AI functionality. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Trans- parency, 959-972.
  38. Reuel, A.; Bucknall, B.; Casper, S.; Fist, T.; Soder, L.; Aarne, O.; Hammond, L.; Ibrahim, L.; Chan, A.; Wills, P.; et al. 2024. Open problems in technical ai governance. arXiv preprint arXiv:2407.14981. · arXiv
  39. Röttger, P.; Pernisi, F.; Vidgen, B.; and Hovy, D. 2024. Safe- tyPrompts: a Systematic Review of Open Datasets for Eval- uating and Improving Large Language Model Safety. arXiv preprint arXiv:2404.05399. · arXiv
  40. Shelby, R.; Rismani, S.; Henne, K.; Moon, A.; Ros- tamzadeh, N.; Nicholas, P.; Yilla-Akbari, N.; Gallegos, J.; Smart, A.; Garcia, E.; et al. 2023. Sociotechnical harms of algorithmic systems: Scoping a taxonomy for harm reduc- tion. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, 723-741.
  41. Staab, R.; Vero, M.; Balunović, M.; and Vechev, M. 2023. Beyond memorization: Violating privacy via inference with large language models. arXiv preprint arXiv:2310.07298. · arXiv
  42. Subramonyam, H.; Pea, R.; Pondoc, C. L.; Agrawala, M.; and Seifert, C. 2024. Bridging the Gulf of Envi- sioning: Cognitive Design Challenges in LLM Interfaces. arXiv:2309.14459. · arXiv
  43. Tamkin, A.; McCain, M.; Handa, K.; Durmus, E.; Lovitt, L.; Rathi, A.; Huang, S.; Mountfield, A.; Hong, J.; Ritchie, S.; et al. 2024. Clio: Privacy-Preserving Insights into Real- World AI Use. arXiv preprint arXiv:2412.13678. · arXiv
  44. Viégas, F.; and Wattenberg, M. 2023. The system model and the user model: Exploring AI dashboard design. arXiv preprint arXiv:2305.02469. · arXiv
  45. Wallach, H.; Desai, M.; Cooper, A. F.; Wang, A.; Atalla, C.; Barocas, S.; Blodgett, S. L.; Chouldechova, A.; Corvi, E.; Dow, P. A.; et al. 2025. Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge. arXiv preprint arXiv:2502.00561. · arXiv
  46. Weidinger, L.; Rauh, M.; Marchal, N.; Manzini, A.; Hen- dricks, L. A.; Mateos-Garcia, J.; Bergman, S.; Kay, J.; Griffin, C.; Bariach, B.; et al. 2023. Sociotechnical safety evaluation of generative ai systems. arXiv preprint arXiv:2310.11986. · arXiv
  47. Weidinger, L.; Uesato, J.; Rauh, M.; Griffin, C.; Huang, P.- S.; Mellor, J.; Glaese, A.; Cheng, M.; Balle, B.; Kasirzadeh, A.; et al. 2022. Taxonomy of risks posed by language mod- els. In Proceedings of the 2022 ACM Conference on Fair- ness, Accountability, and Transparency, 214-229.
  48. Wouters, O. J.; McKee, M.; and Luyten, J. 2020. Estimated Research and Development Investment Needed to Bring a New Medicine to Market, 2009-2018. JAMA, 323(9): 844- 853.
  49. Zhang, Z.; Jia, M.; Lee, H.-P. H.; Yao, B.; Das, S.; Lerner, A.; Wang, D.; and Li, T. 2024. "It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI '24. ACM.
  50. Zhao, W.; Ren, X.; Hessel, J.; Cardie, C.; Choi, Y.; and Deng, Y. 2024. WildChat: 1M ChatGPT Interaction Logs in the Wild. arXiv:2405.01470. · arXiv
  51. Zhou, X.; Kim, H.; Brahman, F.; Jiang, L.; Zhu, H.; Lu, X.; Xu, F.; Lin, B. Y.; Choi, Y.; Mireshghallah, N.; et al. 2024. Haicosystem: An ecosystem for sandboxing safety risks in human-ai interactions. arXiv preprint arXiv:2409.16427. · arXiv

Cited by

Sorted by citations

Authors
Ibrahim, L., Huang, S., Ahmad, L., Bhatt, U., & Anderljung, M.
Affiliations
  • University of Oxford (GB)
  • The Collective Intelligence Project (US)
  • OpenAI (United States) (US)
  • New York University (US)
  • Centre for the Governance of AI (GB)
ORCID
Venue
Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Published
2025-10-15
Volume
8
Issue
2
Pages
1302-1310
ISSN
3065-8365 (ISSN-L)
DOI
10.1609/aies.v8i2.36631
OpenAlex
W4415230017
Cited by
11
References
0
Citations by year
2026: 9 · 2025: 2
FWCI
11.36
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
interactive assessment, AI evaluation, human-AI interaction, parasocial relationships, social manipulation, AI governance, construct validity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review