Guidelines for Human-AI Interaction
The DOI matches its Crossref record; 35 of the 36 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Advances in artificial intelligence (AI) frame opportunities and challenges for user interface design. Principles for human-AI interaction have been discussed in the human-computer interaction community for over two decades, but more study and innovation are needed in light of advances in AI and the growing uses of AI technologies in human-facing applications. We propose 18 generally applicable design guidelines for human-AI interaction. These guidelines are validated through multiple rounds of evaluation including a user study with 49 design practitioners who tested the guidelines against 20 popular AI-infused products. The results verify the relevance of the guidelines over a spectrum of interaction scenarios and reveal gaps in our knowledge, highlighting opportunities for further research. Based on the evaluations, we believe the set of design guidelines can serve as a resource to practitioners working on the design of applications and features that harness AI technologies, and to researchers interested in the further development of human-AI interaction design principles.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Horvitz et al. [71] defined mixed initiative principles that optimize interaction based on utility and uncertainty, while Amershi et al. [7] provided validated general human-AI interaction guidelines.”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [7]
In the citing paper
“In effort-regulation domains-coding assistants, writing tools, office automation-affective cues such as confident completion and apologetic repair lower the perceived cost of acting on suggestions, reducing verification [8, 58, 87].”
From Caring Without Feeling: Affective Dynamics as the Control Layer of Human-AI Agent Collaboration (2026) · cites this paper as [8]
In the citing paper
“Amershi et al. [3] acknowledge the tradeoff between generality and specialization, and most of the general-purpose tools SIP interns explored were found to be lacking for their specific needs.”
From Is Resistance Futile?: Early Career Game Developers, Generative AI, and Ethical Skepticism (2024) · cites this paper as [3]
In the citing paper
“The domain of Human-centered AI HCAI calls to actively involve people in AI design to ensure systems better reflect the values, preferences and needs of users, and other impacted stakeholders (e.g., [4, 8, 16, 99, 101, 104, 110]).”
From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [4]
In the citing paper
“Alternatively, model fairness can be enforced more directly via optimization techniques. For instance, quantitative fairness metrics can be used as regularization [6] or constraints for the optimization of the model parameters [66].”
From Trustworthy AI: From Principles to Practices (2023) · cites this paper as [6]
15 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
(1996) · Cited by 11,584
Heuristic evaluation of user interfaces
Nielsen, J., & Molich, R. (1990) · Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90 · doi:10.1145/97243.97281 · Cited by 3,533 · cited 1×
“We modeled our study after a heuristic evaluation [31], a common discount usability testing method where evaluators examine an interface for violations of a given set of usability guidelines.”
MethodThe Psychology of Everyday Things
Massaro, D. W., & Norman, D. A. (1990) · The American Journal of Psychology · doi:10.2307/1423268 · Cited by 3,466 · cited 1×
“AI-infused systems 1 can violate established usability guidelines of traditional user interface design (e.g., [31, 32]).”
BackgroundQualitative Research Design: An Interactive Approach
Pitman, M. A. (1998) · Anthropology & Education Quarterly · doi:10.1525/aeq.1998.29.4.499 · Cited by 2,405
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490 · cited 3×
“Horvitz’s formative paper on mixed-initiative systems [17] proposed principles for balancing autonomous actions with direct manipulation constructs, such as supporting user-driven invocation of intelligent services, scoping actions based on inferred goals and confdences, and inferring ideal action in light of costs, b…”
BackgroundInfluentialSelf-citationLuger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176
Finding usability problems through heuristic evaluation
Nielsen, J. (1992) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '92 · doi:10.1145/142750.142834 · Cited by 1,060 · cited 1×
“Nielsen [30] recommends 2-3 evaluators when evaluators have both usability experience and familiarity with the product being tested.”
Background(2003) · Choice Reviews Online · doi:10.5860/choice.40-4648 · Cited by 779
Maxwell, J., & Chmiel, M. (2015) · Oxford Bibliographies Online Datasets · doi:10.1093/obo/9780199756810-0126 · Cited by 767
Voice Interfaces in Everyday Life
Porcheron, M., Fischer, J. E., Reeves, S., & Sharples, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174214 · Cited by 717 · cited 1×
“This scenario has also had a recent resurgence of interest given advances in natural language processing and embedded devices driving the proliferation of conversational agents [26, 29, 35].”
BackgroundThe Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users
Horvitz, E., Breese, J. S., Heckerman, D. E., Hovel, D., & Rommelse, K. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.7385 · Cited by 654 · cited 1×
“For example, researchers have been studying how to efectively interact with intelligent agents for many years (e.g., [18, 33]).”
BackgroundSelf-citationPrinciples of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576 · cited 2×
“For example, a large body of work exists and continues to grow around how to increase transparency or explain the behaviors of AI systems (e.g., [14, 21, 23, 36, 38, 44], to name a few).”
BackgroundIntelligibility and Accountability: Human Considerations in Context-Aware Systems
Bellotti, V., & Edwards, K. (2001) · Human–Computer Interaction · doi:10.1207/s15327051hci16234_05 · Cited by 471
Explanations as Mechanisms for Supporting Algorithmic Transparency
Rader, E., Cotter, K., & Cho, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173677 · Cited by 388 · cited 2×
“For example, a large body of work exists and continues to grow around how to increase transparency or explain the behaviors of AI systems (e.g., [14, 21, 23, 36, 38, 44], to name a few).”
BackgroundA comparison of static, adaptive, and adaptable menus
Findlater, L., & McGrenere, J. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985704 · Cited by 316 · cited 2×
“Similarly, when and how to automatically adapt or personalize interfaces has been studied extensively in a variety of scenarios (e.g., [9, 11–13]).”
BackgroundSteps to take before intelligent user interfaces become real
Höök, K. (2000) · Interacting with Computers · doi:10.1016/s0953-5438(99)00006-5 · Cited by 315 · cited 2×
“Others recommended managing expectations so as not to mislead or frustrate users during interaction with unpredictable adaptive agents [16, 20, 33].”
BackgroundGenderMag: A Method for Evaluating Software's Gender Inclusiveness
Burnett, M., Stumpf, S., Macbeth, J., Makri, S., Beckwith, L., Kwan, I., Peters, A., & Jernigan, W. (2016) · Interacting with Computers · doi:10.1093/iwc/iwv046 · Cited by 293 · cited 1×
“GenderMag[5], a method for identifying gender biases in user interfaces, is one such tool, but further work is needed in this area.”
MethodHow might people interact with agents
Norman, D. A. (1994) · Communications of the ACM · doi:10.1145/176789.176796 · Cited by 287 · cited 2×
“Others recommended managing expectations so as not to mislead or frustrate users during interaction with unpredictable adaptive agents [16, 20, 33].”
BackgroundAn Examination of Questionnaire Evaluation by Expert Reviewers
Olson, K. (2010) · Field Methods · doi:10.1177/1525822x10379795 · Cited by 253 · cited 1×
“Expert reviews have been shown to be efective at identifying problems related to wording and clarity [27, 34, 43].”
BackgroundAssessing demand for intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2009) · Proceedings of the 11th international conference on Ubiquitous computing · doi:10.1145/1620545.1620576 · Cited by 246 · cited 1×
“Similarly, researchers have for decades studied human interaction with intelligent context-aware computing systems including how to design for understandability and control of the underlying sensing systems [3, 23] and how to support ambiguity resolution [10].”
BackgroundPatterns for How Users Overcome Obstacles in Voice User Interfaces
Myers, C., Furqan, A., Nebolsky, J., Caro, K., & Zhu, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173580 · Cited by 213 · cited 1×
“This scenario has also had a recent resurgence of interest given advances in natural language processing and embedded devices driving the proliferation of conversational agents [26, 29, 35].”
BackgroundExploring the design space for adaptive graphical user interfaces
Gajos, K. Z., Czerwinski, M., Tan, D. S., & Weld, D. S. (2006) · Proceedings of the working conference on Advanced visual interfaces - AVI '06 · doi:10.1145/1133265.1133306 · Cited by 198
Self-citationHermawati, S., & Lawson, G. (2016) · Applied Ergonomics · doi:10.1016/j.apergo.2015.11.016 · Cited by 196 · cited 1×
“Moreover, recent work has warned that the lack of rigorous validations of proposed design heuristics in specifc domains makes it difcult to gauge the utility of those heuristics [15].”
BackgroundWhy Do They Refuse to Use My Robot?: Reasons for Non-Use Derived from a Long-Term Home Study
de Graaf, M., Ben Allouch, S., & van Dijk, J. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020236 · Cited by 169
The psychology of everyday things
Shackel, B. (1992) · Acta Psychologica · doi:10.1016/0001-6918(92)90062-i · Cited by 152
Predictability and accuracy in adaptive user interfaces
Gajos, K. Z., Everitt, K., Tan, D. S., Czerwinski, M., & Weld, D. S. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357252 · Cited by 138 · cited 1×
“Similarly, when and how to automatically adapt or personalize interfaces has been studied extensively in a variety of scenarios (e.g., [9, 11–13]).”
BackgroundSelf-citationYang, R., Shin, E., Newman, M. W., & Ackerman, M. S. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2804269 · Cited by 119 · cited 1×
“Recent advances in sensing technologies and the widespread availability of commercial ftness and activity trackers have continued to drive interaction research in these domains [37, 45].”
Are explanations always important?: a study of deployed, low-cost intelligent interactive systems
Bunt, A., Lount, M., & Lauzon, C. (2012) · Proceedings of the 2012 ACM international conference on Intelligent User Interfaces · doi:10.1145/2166966.2166996 · Cited by 106
Distributed mediation of ambiguous context in aware environments
Dey, A., Mankoff, J., Abowd, G., & Carter, S. (2002) · Proceedings of the 15th annual ACM symposium on User interface software and technology · doi:10.1145/571985.572003 · Cited by 102 · cited 1×
“Similarly, researchers have for decades studied human interaction with intelligent context-aware computing systems including how to design for understandability and control of the underlying sensing systems [3, 23] and how to support ambiguity resolution [10].”
BackgroundMaking Sense of Sleep Sensors: How Sleep Sensing Technologies Support and Undermine Sleep Health
Ravichandran, R., Sien, S.-W., Patel, S. N., Kientz, J. A., & Pina, L. R. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025557 · Cited by 95
Expert Evaluation of Diagnostic Instrument for Personal and Organizational Value Congruence
Vveinhardt, J., & Gulbovaitė, E. (2015) · Journal of Business Ethics · doi:10.1007/s10551-014-2527-7 · Cited by 54 · cited 1×
“Expert reviews have been shown to be efective at identifying problems related to wording and clarity [27, 34, 43].”
BackgroundHow Accurately Do Different Evaluation Methods Predict the Reliability of Survey Questions?
Maitland, A., & Presser, S. (2016) · Journal of Survey Statistics and Methodology · doi:10.1093/jssam/smw014 · Cited by 26 · cited 1×
“Expert reviews have been shown to be efective at identifying problems related to wording and clarity [27, 34, 43].”
On the Importance of Spatial Perception for the Design of Adaptive User Interfaces
Deuschel, T., & Scully, T. (2016) · IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO) · doi:10.1109/saso.2016.13 · Cited by 25 · cited 1×
“Similarly, when and how to automatically adapt or personalize interfaces has been studied extensively in a variety of scenarios (e.g., [9, 11–13]).”
BackgroundLee, C.-J., Teevan, J. B., & de la Chica, S. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609588 · Cited by 15 · cited 2×
“Similarly, Lee et al [22] studied automatic changes to search result lists during user interaction and suggested caution in updating those lists to balance stability with presenting new content to users.”
BackgroundSelf-citation
Research works printed but not linked: 8
- Alexa. 2018. Top sites in the United States. Retrieved July, 2018 from https://www.alexa.com/topsites/countries/US
- comScore. 2018. Latest rankings. Retrieved July, 2018 from https: //www.comscore.com/Insights/Rankings
- Defy Media. 2015. Damn You Auto Correct! Retrieved September, 2018 from http://www.damnyouautocorrect.com/
- IDC Corporate USA. 2018. IDC: The premier global market intelligence firm. Retrieved September, 2018 from https://www.idc.com/
- 10.5555/772072.772094 · doi:10.5555/772072.772094
- Josh Lovejoy. 2018. The UX of AI. Google Design. Retrieved September, 2018 from https://design.google/library/ux-ai/
- SimilarWeb. 2018. Top websites ranking. Retrieved July, 2018 from https://www.similarweb.com/top-websites/united-states
- Daniel S Weld and Gagan Bansal. 2018. Intelligible Artificial Intelligence. arXiv preprint arXiv:1803.04263 (2018). · arXiv
Cited by
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
“However, in other application domains, it will be important to follow best practices in human-AI design [5, 107] to understand errors and how they might percolate into the user experience.”
BackgroundGenerative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.03442 · Cited by 2,173
“However, in other application domains, it will be important to follow best practices in human-AI design [5, 107] to understand errors and how they might percolate into the user experience.”
BackgroundAli, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743
“…& Kristensson, 2018) paved the way for Microsoft’s Guidelines for AI-Human Interaction (https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-interaction/), which has 18 guidelines for initial use, normal use, coping with problems, and changes over time (Amershi et al., 2019).”
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
“We also identified gaps between current algorithmic solutions of XAI and what’s needed to deliver satisfying user experiences, in the types of user questions to address and how they are addressed.”
Background
Metadata
- Authors
- Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., Teevan, J., Kikin-Gil, R., & Horvitz, E.
- Affiliations
- Microsoft (United States) (US)
- University of Washington & Microsoft Research, Seattle, WA, USA
- ORCID
- Venue
- Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2019-05-02
- Pages
- 1-13
- OpenAlex
- W2916904544
- Semantic Scholar
- ad3cf68bae32d21f25ac142287d4a556155619d2
- Cited by
- 2,439
- References
- 35
- Citations by year
- 2026: 870 · 2025: 464 · 2024: 281 · 2023: 237 · 2022: 217 · 2021: 208 · 2020: 132 · 2019: 29
- FWCI
- 118.62
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- human-AI interaction, design guidelines, user interface design, human-computer interaction, design principles
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.