Transitioning to Human Interaction with AI Systems: New Challenges and Opportunities for HCI Professionals to Enable Human-Centered AI
The DOI matches its Crossref record; 205 of the 205 references it cites were found in a registry or an index.
Abstract
While AI has benefited humans, it may also harm humans if not appropriately developed. The priority of current HCI work should focus on transiting from conventional human interaction with non-AI computing systems to interaction with AI systems. We conducted a high-level literature review and a holistic analysis of current work in developing AI systems from an HCI perspective. Our review and analysis highlight the new changes introduced by AI technology and the new challenges that HCI professionals face when applying the human-centered AI (HCAI) approach in the development of AI systems. We also identified seven main issues in human interaction with AI systems, which HCI professionals did not encounter when developing non-AI computing systems. To further enable the implementation of the HCAI approach, we identified new HCI opportunities tied to specific HCAI-driven design goals to guide HCI professionals addressing these new issues. Finally, our assessment of current HCI methods shows the limitations of these methods in support of developing HCAI systems. We propose the alternative methods that can help overcome these limitations and effectively help HCI professionals apply the HCAI approach to the development of AI systems. We also offer strategic recommendation for HCI professionals to effectively influence the development of AI systems with the HCAI approach, eventually developing HCAI systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Designing interaction-centric AI (by thoroughly understanding how users interact with the AI system) is essential for creating user-friendly AI-based systems [22,28].”
From Design implications of generative AI systems for visual storytelling for young learners (2023) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 1×
“…and effectively realizes human-AI interactions through user interventions, such as online assessment for crowdsourced human input (Dellermann, Calma, et al., 2019; Mnih et al., 2015), user participation in training, adjustment and testing of algorithms (Acuna et al., 2018; Correia et al., 2019).”
Background"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
An Introduction to MultiAgent Systems
Wooldridge, M. (2002) · Cited by 5,394
Engineering Psychology and Human Performance
Wickens, C. D., Hollands, J. G., Banbury, S., & Parasuraman, R. (2015) · Psychology Press eBooks · doi:10.4324/9781315665177 · Cited by 4,982
A model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1×
“Kaber (2018) auges that the traditional automation taxonomies (e.g., Parasuraman et al., 2000; Sheridan & Verplank, 1978) were not referring to autonomous agents capable of formulating objectives and determining courses of action and make no mention of, for example, self-sufficiency or…”
BackgroundThe Psychology of Human-Computer Interaction
Card, S. K., Moran, T. P., & Newell, A. (2018) · CRC Press eBooks · doi:10.1201/9780203736166 · Cited by 4,103
(2018) · Cited by 3,510
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
The Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014) · Science · doi:10.1126/science.1248506 · Cited by 2,523 · cited 1×
“Many AI professionals are primarily dedicated to studying algorithms, rather than providing useful AI systems to meet user needs, resulting in the failure of many AI systems (Hoffman et al., 2016; Lazer et al., 2014; Lieberman, 2009; Yampolskiy, 2019).”
BackgroundGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 2×
“…their work under a variety of labels, such as human-AI/machine teaming (e.g., Brandt et al., 2018; Brill et al., 2018), human-AI interaction (e.g., Amershi et al., 2019; Yang, Steinfeld, et al., 2020), human-agent interaction (e.g., Prada & Paiva, 2014), human-autonomous system interaction (e.g.,…”
BackgroundMethodBainbridge, L. (1983) · Automatica · doi:10.1016/0005-1098(83)90046-8 · Cited by 2,205 · cited 2×
“The effort to address the classic “ ironies of automation ” issue has existed over 30 years, but is still not completely solved (Bainbridge, 1983; Strauch, 2017).”
BackgroundThe psychology of human-computer interaction
Rogers, Y. (1984) · Applied Ergonomics · doi:10.1016/0003-6870(84)90205-9 · Cited by 1,970
Licklider, J. C. R. (1960) · IRE Transactions on Human Factors in Electronics · doi:10.1109/thfe2.1960.4503259 · Cited by 1,938
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911 · cited 1×
“…a lot of attentions in research and application, and multiple sets of ethical guidelines from various organizations around the world are currently available, large high-tech companies have published internal ethical guidelines for development of AI systems (e.g., Hagendorff, 2020; IEEE, 2019).”
BackgroundThe Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907 · cited 1×
“…a lot of attentions in research and application, and multiple sets of ethical guidelines from various organizations around the world are currently available, large high-tech companies have published internal ethical guidelines for development of AI systems (e.g., Hagendorff, 2020; IEEE, 2019).”
BackgroundPrinciples alone cannot guarantee ethical AI
Mittelstadt, B. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0114-4 · Cited by 1,886
Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743
Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work
Vicente, K. J. (1999) · Cited by 1,686
Artificial Intelligence in Architectural Design
Indonesia, U. K., Harapan, A., Indriani, D., Rizkiya, N. F., & Azbi, R. M. (2021) · International Journal of Design · doi:10.34010/injudes.v1i1.4824 · Cited by 1,633
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 1×
“For instance, classification algorithms for images and text are trained to optimize accuracy on a specific set of human-labeled datasets, the choice/labeling of the dataset and those features represented can substantially influence the behavior (Bolukbasi et al., 2016; Buolamwini & Gebru, 2018).”
International Journal of Design
(2021) · International Journal of Design · doi:10.34010/injudes · Cited by 1,614
The Out-of-the-Loop Performance Problem and Level of Control in Automation
Endsley, M. R., & Kiris, E. O. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779064555 · Cited by 1,406
Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.-F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., et al. (2019) · Nature · doi:10.1038/s41586-019-1138-y · Cited by 1,364
(2013) · Choice Reviews Online · doi:10.5860/choice.50-3307 · Cited by 1,352
Human and Computer Control of Undersea Teleoperators
Sheridan, T. B., & Verplank, W. L. (1978) · doi:10.21236/ada057655 · Cited by 1,227
Introduction to Multi-Agent Systems
Hadzic, M., Wongthongtham, P., Dillon, T., & Chang, E. (2009) · Studies in Computational Intelligence · doi:10.1007/978-3-642-01904-3_2 · Cited by 1,180
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
A strategy for human factors/ergonomics: developing the discipline and profession
Dul, J., Bruder, R., Buckle, P., Carayon, P., Falzon, P., Marras, W. S., Wilson, J. R., & van der Doelen, B. (2012) · Ergonomics · doi:10.1080/00140139.2012.661087 · Cited by 927
From Here to Autonomy: Lessons Learned From Human–Automation Research
Endsley, M. R. (2017) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720816681350 · Cited by 898
Rise of Machine Agency: A Framework for Studying the Psychology of Human–AI Interaction (HAII)
Sundar, S. S. (2020) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmz026 · Cited by 871
How in the World Did We Ever Get into That Mode? Mode Error and Awareness in Supervisory Control
Sarter, N. B., & Woods, D. D. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779049516 · Cited by 849
Reality-based interaction: a framework for post-WIMP interfaces
Jacob, R. J. K., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Solovey, E. T., & Zigelbaum, J. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357089 · Cited by 844
Dellermann, D., Ebel, P., Söllner, M., & Leimeister, J. M. (2019) · Business & Information Systems Engineering · doi:10.1007/s12599-019-00595-2 · Cited by 826
Joint Cognitive Systems: Foundations of Cognitive Systems Engineering
Hollnagel, E., & Woods, D. D. (2005) · Thammasat University Digital Collections · Cited by 787
Project Aura: toward distraction-free pervasive computing
Garlan, D., Siewiorek, D. P., Smailagic, A., & Steenkiste, P. (2002) · IEEE Pervasive Computing · doi:10.1109/mprv.2002.1012334 · Cited by 784
Shneiderman, B. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3419764 · Cited by 768
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
Human–Autonomy Teaming: A Review and Analysis of the Empirical Literature
O’Neill, T., McNeese, N., Barron, A., & Schelble, B. (2022) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720820960865 · Cited by 695
Joint Cognitive Systems: Foundations of Cognitive Systems Engineering
Hollnagel, E., & Woods, D. D. (2005) · doi:10.1201/9781420038194 · Cited by 695
Research Priorities for Robust and Beneficial Artificial Intelligence
Russell, S., Dewey, D., & Tegmark, M. (2015) · AI Magazine · doi:10.1609/aimag.v36i4.2577 · Cited by 637 · cited 1×
“Concerning the potential harm to humans from AI technology in the long run, prominent critics have expressed their concern that AI may create a “ machine world ” and eventually replace humans (Hawking et al., 2015; Russell et al., 2015).”
BackgroundRe-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622
Interdisciplinary research has consistently lower funding success
Bromham, L., Dinnage, R., & Hua, X. (2016) · Nature · doi:10.1038/nature18315 · Cited by 594
Ten challenges for making automation a ‘team player’ in joint human-agent activity
Klein, G., Woods, D. D., Bradshaw, J. M., Hoffman, R. R., & Feltovich, P. J. (2004) · IEEE Intelligent Systems · doi:10.1109/mis.2004.74 · Cited by 590 · cited 2×
“There are debates on whether AI systems can truly work as a teammate with humans (e.g., Klein et al., 2004; Shneiderman, 2020c), we believe that a common HCAI design goal should be shared between both sides to ensure human-controlled AI and human-driven decision-making in human interaction (or…”
BackgroundPrinciples of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576
Meaningful Human Control over Autonomous Systems: A Philosophical Account
Santoni de Sio, F., & van den Hoven, J. (2018) · Frontiers in Robotics and AI · doi:10.3389/frobt.2018.00015 · Cited by 566 · cited 4×
“…design of autonomous systems (de Sio & den Hoven, 2018), so that we can trace back the error data to identify the accountability of system vs. human operators if the autonomous system fails and also be able to use the data for future improvement of the design (de Sio & den Hoven, 2018; Xu, 2021).”
InfluentialHuman Performance Consequences of Stages and Levels of Automation: An Integrated Meta-Analysis
Onnasch, L., Wickens, C. D., Li, H., & Manzey, D. (2014) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720813501549 · Cited by 524
Stephanidis, C., Salvendy, G., Antona, M., Chen, J. Y. C., Dong, J., Duffy, V. G., Fang, X., Fidopiastis, C., Fragomeni, G., Fu, L. P., et al. (2019) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2019.1619259 · Cited by 524 · cited 1×
“The effort to address the classic “ ironies of automation ” issue has existed over 30 years, but is still not completely solved (Bainbridge, 1983; Strauch, 2017).”
BackgroundHuman–Agent Teaming for Multirobot Control: A Review of Human Factors Issues
Chen, J. Y. C., & Barnes, M. J. (2014) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2013.2293535 · Cited by 522
Hawaii international conference on system sciences
Schultz, E. E. (1992) · ACM SIGCHI Bulletin · doi:10.1145/134421.134436 · Cited by 517
2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI)
(2020) · doi:10.1109/isbi45749.2020 · Cited by 472
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5285 · Cited by 466
(2012) · CRC Press eBooks · doi:10.1201/b11963 · Cited by 464
Burns, C. M., & Hajdukiewicz, J. (2017) · CRC Press eBooks · doi:10.1201/9781315272665 · Cited by 463
The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems
Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., & Ebel, P. (2021) · arXiv · doi:10.48550/arxiv.2105.03354 · Cited by 456
Toward human-centered AI: a perspective from human-computer interaction
Xu, W. (2019) · Interactions · doi:10.1145/3328485 · Cited by 454
Self-citationDesigning for Situation Awareness: An Approach to User-Centered Design, Second Edition
Endsley, M. R. (2016) · doi:10.1201/b11371 · Cited by 424
Human-Centered Artificial Intelligence: Three Fresh Ideas
University of Maryland, C. P., & Shneiderman, B. (2020) · AIS Transactions on Human-Computer Interaction · doi:10.17705/1thci.00131 · Cited by 421
Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++
Acuna, D., Ling, H., Kar, A., & Fidler, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00096 · Cited by 414
From ‘automation’ to ‘autonomy’: the importance of trust repair in human–machine interaction
de Visser, E. J., Pak, R., & Shaw, T. H. (2018) · Ergonomics · doi:10.1080/00140139.2018.1457725 · Cited by 398
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · PhilPapers (PhilPapers Foundation) · Cited by 394
UX Design Innovation: Challenges for Working with Machine Learning as a Design Material
Dove, G., Halskov, K., Forlizzi, J., & Zimmerman, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025739 · Cited by 391 · cited 1×
“Research shows that there are a lack of effective methods for designing AI systems and HCI professionals have had difficulty performing the typical HCI activities of conceptualization, rapid prototyping, and testing (Dove et al., 2017; Holmquist, 2017; Yang, Huang, et al., 2020; van Allen, 2018).”
MethodHybrid-augmented intelligence: collaboration and cognition
Zheng, N.-N., Liu, Z.-Y., Ren, P.-J., Ma, Y.-Q., Chen, S.-T., Yu, S.-Y., Xue, J.-R., Chen, B.-D., & Wang, F.-Y. (2017) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700053 · Cited by 387
Viewpoint: Human-in-the-loop Artificial Intelligence
Zanzotto, F. M. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11345 · Cited by 380 · cited 5×
“For instance, when the confidence of the system output is low, humans can intervene by adjusting input, creating a feedback loop to improve the system’s performance (Zheng et al., 2017; Zanzotto, 2019).”
BackgroundInfluentialHuman‐centered artificial intelligence and machine learning
Riedl, M. O. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.117 · Cited by 371
MURPHY, M. (2004) · Cited by 361
A Review of User Interface Design for Interactive Machine Learning
Dudley, J. J., & Kristensson, P. O. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185517 · Cited by 343
The media inequality: Comparing the initial human-human and human-AI social interactions
Mou, Y., & Xu, K. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.02.067 · Cited by 328
Parasuraman, R., & Rizzo, M. (2006) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195177619.001.0001 · Cited by 311
(2018) · Harvard journal of law & technology · Cited by 295
Standardizing Ethical Design for Artificial Intelligence and Autonomous Systems
Bryson, J., & Winfield, A. (2017) · Computer · doi:10.1109/mc.2017.154 · Cited by 293
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Bansal, G., Nushi, B., Kamar, E., Weld, D. S., Lasecki, W. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012429 · Cited by 284
The Future of Human-AI Collaboration: A Taxonomy of Design Knowledge for Hybrid Intelligence Systems
Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., & Ebel, P. (2019) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2019.034 · Cited by 279
Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach
Ehsan, U., & Riedl, M. O. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60117-1_33 · Cited by 274
Computer-mediated communication: Linguistic, social, and cross-cultural perspectives
Whitney, G. (1998) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199807)49:9<859::aid-asi12>3.0.co;2-f · Cited by 273
CHI '86 Conference on Human Factors in Computing Systems
(1985) · Computer · doi:10.1109/mc.1985.1662775 · Cited by 272
(2002) · CRC Press eBooks · doi:10.1201/b12477 · Cited by 260
Designing for Situation Awareness: An Approach to User-Centered Design
McKeown, C. (2013) · Ergonomics · doi:10.1080/00140139.2013.793052 · Cited by 248
Crandall, J. W., Oudah, M., Tennom, Ishowo-Oloko, F., Abdallah, S., Bonnefon, J.-F., Cebrian, M., Shariff, A., Goodrich, M. A., & Rahwan, I. (2018) · Nature Communications · doi:10.1038/s41467-017-02597-8 · Cited by 228
Investigating How Experienced UX Designers Effectively Work with Machine Learning
Yang, Q., Scuito, A., Zimmerman, J., Forlizzi, J., & Steinfeld, A. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196730 · Cited by 226
Miller, T., Howe, P., & Sonenberg, L. (2017) · arXiv · doi:10.48550/arxiv.1712.00547 · Cited by 218
Mueller, S. T., Hoffman, R. R., Clancey, W., Emrey, A., & Klein, G. (2019) · arXiv · doi:10.48550/arxiv.1902.01876 · Cited by 215 · cited 7×
“…it is necessary to overcome the unilateral evaluation methods in some existing studies that only evaluate the performance of AI systems, HCI should promote the evaluation of the AI systems as a human-machine system by adding an end users perspective (Hoffman et al., 2018; Mueller et al., 2019).”
BackgroundInfluentialInsight solutions are correct more often than analytic solutions
Salvi, C., Bricolo, E., Kounios, J., Bowden, E., & Beeman, M. (2016) · Thinking & Reasoning · doi:10.1080/13546783.2016.1141798 · Cited by 213
Human-centered AI: The role of Human-centered Design Research in the development of AI
Auernhammer, J. (2020) · Proceedings of DRS · doi:10.21606/drs.2020.282 · Cited by 201
How do visual explanations foster end users' appropriate trust in machine learning?
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377480 · Cited by 200 · cited 2×
“Research shows that there are a lack of effective methods for designing AI systems and HCI professionals have had difficulty performing the typical HCI activities of conceptualization, rapid prototyping, and testing (Dove et al., 2017; Holmquist, 2017; Yang, Huang, et al., 2020; van Allen, 2018).”
BackgroundMethodRogers, Y., & Marshall, P. (2017) · Synthesis Lectures on Human-Centered Informatics · doi:10.1007/978-3-031-02220-3 · Cited by 193
Farooq, U., & Grudin, J. (2016) · Interactions · doi:10.1145/3001896 · Cited by 175
Using Artificial Intelligence to Augment Human Intelligence
Carter, S., & Nielsen, M. A. (2017) · Distill · doi:10.23915/distill.00009 · Cited by 167
Situation Awareness in Future Autonomous Vehicles: Beware of the Unexpected
Endsley, M. R. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-96071-5_32 · Cited by 165
The Challenges and Opportunities of Human-Centered AI for Trustworthy Robots and Autonomous Systems
He, H., Gray, J. O., Cangelosi, A., Meng, Q., McGinnity, T. M., & Mehnen, J. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3132282 · Cited by 159
When computers take the lead: The automation of leadership
Wesche, J. S., & Sonderegger, A. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.07.027 · Cited by 157
A survey of surveys on the use of visualization for interpreting machine learning models
Chatzimparmpas, A., Martins, R. M., Jusufi, I., & Kerren, A. (2020) · Information Visualization · doi:10.1177/1473871620904671 · Cited by 150
Human Performance Modeling in Aviation
Foyle, D. C., & Hooey, B. L. (2007) · CRC Press eBooks · doi:10.1201/9781420062984 · Cited by 143
Ironies of Automation: Still Unresolved After All These Years
Strauch, B. (2018) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2017.2732506 · Cited by 139
No AI Is an Island: The Case for Teaming Intelligence
Johnson, M., & Vera, A. H. (2019) · AI Magazine · doi:10.1609/aimag.v40i1.2842 · Cited by 134
Intelligence on tap: artificial intelligence as a new design material
Holmquist, L. E. (2017) · Interactions · doi:10.1145/3085571 · Cited by 133
Team situation awareness within the context of human-autonomy teaming
Demir, M., McNeese, N. J., & Cooke, N. J. (2017) · Cognitive Systems Research · doi:10.1016/j.cogsys.2016.11.003 · Cited by 131
A computational cognition model of perception, memory, and judgment
Fu, X., Cai, L., Liu, Y., Jia, J., Chen, W., Zhang, Y., Zhao, G., Liu, Y., & Wu, C. (2014) · Science China Information Sciences · doi:10.1007/s11432-013-4911-9 · Cited by 129
Toward human-centered algorithm design
Baumer, E. P. (2017) · Big Data & Society · doi:10.1177/2053951717718854 · Cited by 125
Overview of the DARPA Augmented Cognition Technical Integration Experiment
St. John, M., Kobus, D. A., Morrison, J. G., & Schmorrow, D. (2004) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc1702_2 · Cited by 124
A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning
Kaluarachchi, T., Reis, A., & Nanayakkara, S. (2021) · Sensors · doi:10.3390/s21072514 · Cited by 124
Some pitfalls in the promises of automated and autonomous vehicles
Hancock, P. A. (2019) · Ergonomics · doi:10.1080/00140139.2018.1498136 · Cited by 122
Three Faces of Human-Computer Interaction
Grudin, J. (2005) · IEEE Annals of the History of Computing · doi:10.1109/mahc.2005.67 · Cited by 118
Human-Centered Autonomous Vehicle Systems: Principles of Effective Shared AutonomyPreprint
Fridman, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.01835 · Cited by 115
Design Lessons From AI’s Two Grand Goals: Human Emulation and Useful Applications
Shneiderman, B. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.2992669 · Cited by 114
IEEE Transactions on Human-Machine Systems
Institute of Electrical and Electronics Engineers (2023) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.6221037 · Cited by 107
Shifting viewpoints: Artificial intelligence and human–computer interaction
Winograd, T. A. (2006) · Artificial Intelligence · doi:10.1016/j.artint.2006.10.011 · Cited by 103
The Emperor of Strong AI Has No Clothes: Limits to Artificial Intelligence
Braga, A., & Logan, R. (2017) · Information · doi:10.3390/info8040156 · Cited by 101
Bakker, S., & Niemantsverdriet, K. (2016) · International Journal of Design · Cited by 98
Common Metrics to Benchmark Human-Machine Teams (HMT): A Review
Damacharla, P., Javaid, A. Y., Gallimore, J. J., & Devabhaktuni, V. K. (2018) · IEEE Access · doi:10.1109/access.2018.2853560 · Cited by 97
Common Metrics to Benchmark Human-Machine Teams (HMT): A Review
Damacharla, P., Javaid, A. Y., Gallimore, J. J., & Devabhaktuni, V. K. (2020) · arXiv · doi:10.48550/arxiv.2008.04855 · Cited by 97
How Society Can Maintain Human-Centric Artificial Intelligence
Bryson, J. J., & Theodorou, A. (2019) · Translational Systems Sciences · doi:10.1007/978-981-13-7725-9_16 · Cited by 94
Focusing on context in human-centered computing
Brézillon, P. (2003) · IEEE Intelligent Systems · doi:10.1109/mis.2003.1200731 · Cited by 91
The design of intelligent socio-technical systems
Jones, A. J. I., Artikis, A., & Pitt, J. V. (2013) · Artificial Intelligence Review · doi:10.1007/s10462-012-9387-2 · Cited by 91
The Effects of Participatory Mode and Task Workload on the Detection of Dynamic System Failures
Wickens, C. D., & Kessel, C. (1979) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1979.4310070 · Cited by 88
Explaining Explanation For “Explainable Ai”
Hoffman, R. R., Klein, G., & Mueller, S. T. (2018) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931218621047 · Cited by 86
Team Coordination and Effectiveness in Human-Autonomy Teaming
Demir, M., Likens, A. D., Cooke, N. J., Amazeen, P. G., & McNeese, N. J. (2019) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2018.2877482 · Cited by 83
AI Ethics – Too Principled to Fail?
Mittelstadt, B. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3391293 · Cited by 82 · cited 2×
“In many cases, ethical design was considered after the technology is developed rather than during development (Mittelstadt, 2019).”
Revolutions and shifting paradigms in human factors & ergonomics
Boff, K. R. (2006) · Applied Ergonomics · doi:10.1016/j.apergo.2006.04.003 · Cited by 81
Kistan, T., Gardi, A., & Sabatini, R. (2018) · Aerospace · doi:10.3390/aerospace5040103 · Cited by 78
Madni, A., & Madni, C. (2018) · Systems · doi:10.3390/systems6040044 · Cited by 77
Psychological foundations of explainability and interpretability in artificial intelligence
Broniatowski, D. A. (2021) · doi:10.6028/nist.ir.8367 · Cited by 76 · cited 2×
“While research on explainable AI is currently underway, some researchers have realized that the ultimate purpose of explainable AI should ensure that target users can understand the output by AI systems (Broniatowski, 2021; Kulesza et al., 2015).”
BackgroundGrand challenges for HCI researchers
Shneiderman, B., Catherine Plaisant, B., Cohen, M. S., Jacobs, S., Elmqvist, N., & Diakopoulos, N. (2016) · interactions · doi:10.1145/2977645 · Cited by 75
Predicting future AI failures from historic examples
Yampolskiy, R. V. (2019) · foresight · doi:10.1108/fs-04-2018-0034 · Cited by 75
A Survey of Human‐Centered Evaluations in Human‐Centered Machine Learning
Sperrle, F., El‐Assady, M., Guo, G., Borgo, R., Chau, D. H., Endert, A., & Keim, D. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14329 · Cited by 75
A state of science on highly automated driving
Navarro, J. (2018) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2018.1439544 · Cited by 73
Xu, W. M. (2020) · interactions · doi:10.1145/3434580 · Cited by 72
Self-citationA conceptual framework of autonomous and automated agents
Kaber, D. B. (2018) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2017.1363314 · Cited by 71
A conceptual model of team dynamical behaviors and performance in human-autonomy teaming
Demir, M., Cooke, N. J., & Amazeen, P. G. (2018) · Cognitive Systems Research · doi:10.1016/j.cogsys.2018.07.029 · Cited by 71
Hybrid-augmented intelligence: collaboration and cognition
Nanning, Zheng, Ziyi, -, L., Pengju, Ren, Yongqiang, Ma, Shitao, Chen, et al. (2017) · Frontiers of Information Technology & Electronic Engineering · Cited by 70
WoZ Way: Enabling Real-time Remote Interaction Prototyping & Observation in On-road Vehicles
Martelaro, N., & Ju, W. G. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3022198.3023271 · Cited by 63
Human-autonomy teaming interface design considerations for multi-unmanned vehicle control
Calhoun, G. L., Ruff, H. A., Behymer, K. J., & Frost, E. (2018) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2017.1315751 · Cited by 62
Yang, Q. (2018) · National Conference on Artificial Intelligence · Cited by 61
Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents
Leibo, J. Z., d’Autume, C. D. M., Zoran, D., Amos, D., Beattie, C., Anderson, K., Castañeda, A. G., Sanchez, M., Green, S., Gruslys, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.08116 · Cited by 54
Where is the Human?: Bridging the Gap Between AI and HCI
Inkpen, K., Chancellor, S., De Choudhury, M., Veale, M., & Baumer, E. P. S. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3299002 · Cited by 54
Explaining Explanation, Part 2: Empirical Foundations
Hoffman, R. R., Mueller, S. T., & Klein, G. (2017) · IEEE Intelligent Systems · doi:10.1109/mis.2017.3121544 · Cited by 53
Coordinating computer-supported cooperative work: A review of research issues and strategies
Kies, J. K., Williges, R. C., & Rosson, M. B. (1998) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199807)49:9<776::aid-asi3>3.0.co;2-d · Cited by 53
Prototyping ways of prototyping AI
van Allen, P. (2018) · Interactions · doi:10.1145/3274566 · Cited by 51
WoZ Way: Enabling Real-time Remote Interaction Prototyping & Observation in On-road Vehicles
Martelaro, N., & Ju, W. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998293 · Cited by 50
Lethal Autonomous Weapons: Meaningful Human Control or Meaningful Human Certification? [Opinion]
Cummings, M. L. (2019) · IEEE Technology and Society Magazine · doi:10.1109/mts.2019.2948438 · Cited by 46
When User Experience Designers Partner with Data Scientists.
Girardin, F., & Lathia, N. (2017) · National Conference on Artificial Intelligence · Cited by 45
Streitz, N. (2007) · Springer eBooks · doi:10.1007/978-3-540-71025-7_1 · Cited by 44
Horvitz, E. (2017) · Science · doi:10.1126/science.aao2466 · Cited by 42 · cited 1×
“As AI technology advances, it is also critical to assess its influence on humans and society on a long-term basis (e.g., a longitudinal perspective), requiring enhanced testing methods to be supported by social scientist and psychologists (Horvitz, 2017; Stephanidis et al., 2019).”
BackgroundUser Interface Goals, AI Opportunities
Lieberman, H. (2009) · AI Magazine · doi:10.1609/aimag.v30i4.2266 · Cited by 41
Trust Across Culture and Context
Klein, H. A., Lin, M., Miller, N., Militello, L. G., Lyons, J. B., & Finkeldey, J. G. (2018) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343418810936 · Cited by 41
How can ergonomics influence design? Moving from research findings to future systems
Dekker, S. W. A., & Nyce, J. M. (2004) · Ergonomics · doi:10.1080/00140130412331290853 · Cited by 40
Closing the AI Knowledge GapPreprint
Epstein, Z., Payne, B. H., Shen, J. H., Dubey, A., Felbo, B., Groh, M., Obradovich, N., Cebrián, M., & Rahwan, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07233 · Cited by 40
Cognitive Orthoses: Toward Human‐Centered AI
Ford, K. M., Hayes, P. J., Glymour, C., & Allen, J. (2015) · AI Magazine · doi:10.1609/aimag.v36i4.2629 · Cited by 39
Neuhauser, L., & Kreps, G. L. (2011) · National Conference on Artificial Intelligence · Cited by 39
Chittajallu, D. R., Dong, B., Tunison, P., Collins, R., Wells, K., Fleshman, J., Sankaranarayanan, G., Schwaitzberg, S., Cavuoto, L., & Enquobahrie, A. (2019) · 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) · doi:10.1109/isbi.2019.8759428 · Cited by 38
Conceptualization of the Human-Machine Symbiosis – A Literature Review
Gerber, A., Derckx, P., Döppner, D. A., & Schoder, D. (2020) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2020.036 · Cited by 38
Introduction to the Special Issue on Human-Centered Machine Learning
Fiebrink, R., & Gillies, M. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3205942 · Cited by 37
Xu, W. Z., Furie, D., Mahabhaleshwar, M., Suresh, B., & Chouhan, H. (2019) · Ergonomics · doi:10.1080/00140139.2019.1588996 · Cited by 36
Self-citationRegulating safety-critical autonomous systems: past, present, and future perspectives
Cummings, M. L., & Britton, D. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-815367-3.00006-2 · Cited by 36
Xu, W. (2007) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872007x249857 · Cited by 35
Self-citationInfluencing Trust for Human–Automation Collaborative Scheduling of Multiple Unmanned Vehicles
Clare, A. S., Cummings, M. L., & Repenning, N. P. (2015) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815587803 · Cited by 35
Brown, B. A. T., Bødker, S., & Höök, K. (2017) · interactions · doi:10.1145/3125387 · Cited by 35
What is design in the context of human-centered computing?
Hoffman, R. R., Roesler, A., & Moon, B. (2004) · IEEE Intelligent Systems · doi:10.1109/mis.2004.36 · Cited by 34
A Human-Autonomy Teaming Approach for a Flight-Following Task
Brandt, S. L., Lachter, J., Russell, R., & Shively, R. J. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-60642-2_2 · Cited by 34
Artificial Intelligence: A Study on Governance, Policies, and Regulations
Wang, W., & Siau, K. L. (2018) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 34
Choudhury, M. D., Lee, M. K., Zhu, H., & Shamma, D. A. (2020) · Human-Computer Interaction · doi:10.1080/07370024.2020.1744146 · Cited by 34
Thoughts on human-computer interaction in the age of artificial intelligence
Feng, T. I. A. N., Junjun, F. A. N., Dai, G., Du, Y., & Zhengjie, L. I. U. (2018) · Scientia Sinica Informationis · doi:10.1360/n112017-00221 · Cited by 33
A Historical and Intellectual Perspective
(1995) · Elsevier eBooks · doi:10.1016/b978-0-08-051574-8.50009-1 · Cited by 32
Human-Autonomy Teaming and Agent Transparency
Chen, J. Y. C., Barnes, M. J., Selkowitz, A. R., Stowers, K., Lakhmani, S. G., & Kasdaglis, N. (2016) · Companion Publication of the 21st International Conference on Intelligent User Interfaces · doi:10.1145/2876456.2879479 · Cited by 30
Long-Term Impacts of Fair Machine Learning
Zhang, X., Khalili, M. M., & Liu, M. (2019) · Ergonomics in Design The Quarterly of Human Factors Applications · doi:10.1177/1064804619884160 · Cited by 30
Enhancing Mitigation in Augmented Cognition
Fuchs, S., Hale, K. S., Stanney, K. M., Juhnke, J., & Schmorrow, D. (2007) · Journal of Cognitive Engineering and Decision Making · doi:10.1518/155534307x255645 · Cited by 29
Effective Team Interaction for Adaptive Training and Situation Awareness in Human-Autonomy Teaming
Demir, M., McNeese, N. J., Johnson, C. J., Gorman, J. C., Grimm, D., & Cooke, N. J. (2019) · IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA) · doi:10.1109/cogsima.2019.8724202 · Cited by 29
Engineering psychology in the era of artificial intelligence
Xu, W. M., & Ge, L. (2020) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2020.01409 · Cited by 27
Self-citationUnderstanding Attitudes Towards Self-Driving Vehicles: Quantitative Analysis of Qualitative Data
Lee, J. D., & Kolodge, K. (2018) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931218621319 · Cited by 25
Viewing Machines as Teammates: A Qualitative Study.
Lyons, J. B., Mahoney, S., Wynne, K. T., & Roebke, M. A. (2018) · National Conference on Artificial Intelligence · Cited by 24
2D Transparency Space—Bring Domain Users and Machine Learning Experts Together
Zhou, J. J., & Chen, F. (2018) · Human-computer interaction series · doi:10.1007/978-3-319-90403-0_1 · Cited by 23
The limits of highly autonomous vehicles: an uncertain future
Endsley, M. R. (2019) · Ergonomics · doi:10.1080/00140139.2019.1563330 · Cited by 23
Team Situation Awareness in Human-Autonomy Teaming: A Systems Level Approach
Grimm, D., Demir, M., Gorman, J. C., & Cooke, N. J. (2018) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931218621034 · Cited by 22
Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++Preprint
Acuna, D., Ling, H., Kar, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09693 · Cited by 22
Human Centered Artificial Intelligence: Weaving UX into Algorithmic Decision Making
Bond, R., Mulvenna, M. D., Finlay, D., Wong, A. M., Koene, A. R., Brisk, R., Boger, J. N., & Adel, T. (2019) · International Conference on Human-Computer Interaction · Cited by 22
Machine Learning and Human Factors: Status, Applications, and Future Directions
Lau, N. K. C., Fridman, L., Borghetti, B. J., & Lee, J. D. (2018) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931218621031 · Cited by 20
Navigating the Advent of Human-Machine Teaming
Brill, J. C., Cummings, M. L., Evans, A. W., Hancock, P. A., Lyons, J. B., & Oden, K. B. (2018) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931218621104 · Cited by 20
Holistic modelling for human-autonomous system interaction
Cummings, M. L., & Clare, A. S. (2015) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2014.1003990 · Cited by 19
Xu, W. Z., & Ge, L. (2018) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2018.01521 · Cited by 19
Self-citationKleppe, M., & Otte, M. (2017) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqx030 · Cited by 18
Salmon, P. M. (2019) · Ergonomics · doi:10.1080/00140139.2018.1563333 · Cited by 18
Spencer, J., Poggi, J., & Gheerawo, R. (2018) · Proceedings of the 4th EAI International Conference on Smart Objects and Technologies for Social Good · doi:10.1145/3284869.3284897 · Cited by 17 · cited 2×
“For examples, these include humanistic design research (Auernhammer, 2020), participatory design (Neuhauser & Kreps, 2011), inclusive design (Spencer et al., 2018), interaction design (Winograd, 2006), human-centered computing (Brezillon, 2003; Ford et al.”
BackgroundSymbiosis between Humans and Artificial Intelligence
Nagao, K. (2019) · Springer eBooks · doi:10.1007/978-981-13-6175-3_6 · Cited by 17
Physical Object Icons Buttons Gesture (PIBG): A New Interaction Paradigm with Pen
Dai, G., & Wang, H. (2005) · Lecture notes in computer science · doi:10.1007/11568421_2 · Cited by 14
Towards a Definition of Symbiotic Relations Between Humans and Machines
Gamberini, L., & Spagnolli, A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-57753-1_1 · Cited by 14
Intelligent autonomous vehicles with an extendable knowledge base under meaningful human control
Beckers, G., Sijs, J., van Diggelen, J., van Dijk, R. V., Bouma, H., Lomme, M., Hommes, R., Hillerstrom, F., van der Waa, J., van Velsen, A., et al. (2019) · Security + Defence · doi:10.1117/12.2533740 · Cited by 14
A psychological model of human-computer cooperation for the era of artificial intelligence
Liu, Y., Wang, Y., Bian, Y., Ren, L., & Xuan, Y. (2018) · Scientia Sinica Informationis · doi:10.1360/n112017-00225 · Cited by 13
Human - System Cooperation in Automated Driving
Biondi, F. N., Álvarez, I., & Jeong, K. (2019) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2018.1561793 · Cited by 13
Human Error Prediction Using Eye Tracking to Improvise Team Cohesion in Human-Machine Teams
Damacharla, P., Javaid, A. Y., & Devabhaktuni, V. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-94391-6_5 · Cited by 11
Improving the Design of Ambient Intelligence Systems: Guidelines Based on a Systematic Review
Oliveira, J. D., Couto, J., Paixão-Côrtes, V. S. M., & Bordini, R. H. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2021.1926114 · Cited by 11
Correia, A., Paredes, H., Schneider, D., Jameel, S., & Fonseca, B. (2019) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2019.8914075 · Cited by 10
Delegation in Human-Machine Teaming: Progress, Challenges and Prospects
van Diggelen, J., Barnhoorn, J., Post, R. A. G., Sijs, J., van der Stap, N., & van der Waa, J. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-68017-6_2 · Cited by 10
Approaching full autonomy in the maritime domain: paradigm choices and Human Factors challenges
van den Broek, J., Schraagen, J. M., Brake, G. M. T., et al. (2017) · University of Twente Research Information · Cited by 9
Ötting, S. K. (2021) · PUB – Publications at Bielefeld University (Bielefeld University) · doi:10.4119/unibi/2953489 · Cited by 9
The myths and costs of autonomous weapon systems
Hoffman, R. R., Cullen, T. M., & Hawley, J. K. (2016) · Bulletin of the Atomic Scientists · doi:10.1080/00963402.2016.1194619 · Cited by 8
Interaction paradigm in intelligent systems
Zhang, X., Lyu, F., & Cheng, S. (2018) · Scientia Sinica Informationis · doi:10.1360/n112017-00217 · Cited by 8
Assessing Workload in Human-Machine Teams from Psychophysiological Data with Sparse Ground Truth
Dearing, D., Novstrup, A., & Goan, T. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-14273-5_2 · Cited by 8
IEEE Transactions on Cognitive and Developmental Systems
(2016) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2016.2522578 · Cited by 8
The HF/E Parade: A Tale of Two Models
Howell, W. C. (2001) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120104500101 · Cited by 7
Using Cognitive Modeling for Adaptive Automation Triggering
Cassenti, D. N., & Veksler, V. D. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-60591-3_34 · Cited by 7
Research works printed but not linked: 17
- Ayaz, H., & Dehais, F. (2018). Neuroergonomics: The Brain at Work and in Everyday Life (edited). Cambridge, Massachusetts: Academic Press.
- Berndt, J.O., Rodermund, S.C., Lorig, F., Timm, I.J. (2017). Modeling User Behavior in Social Media with Complex Agents. HUSO 2017: The Third International Conference on Human and Social Analytics. 19-24, IARIA, 2017 ISBN: 978-1-61208-578-4
- Hawking, S., Musk, E., Wozniak, S. et al., (2015). Autonomous weapons: an open letter from AI & robotics researchers. Future of Life Institute.
- IEEE. (2019). Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems. (First Edition). The Institute of Electrical and Electronics Engineers (IEEE), Incorporated.
- ISO (International Organization for Standardization). (2020). Ergonomics- Ergonomics of Human-System Interaction - Part 810: Robotic, Intelligent and Autonomous Systems (version for review).
- Jun, S. U., Yuming, W. E. I., & Cui, H. U. A. N. G. (2021). An Integrated Analysis Framework of Artificial Intelligence Social Impact Based on Application Scenarios. Science of Science and Management of S. & T. , 42 (05), 3.
- Prada, R., & Paiva, A. (2014). Human-agent interaction: Challenges for bringing humans and agents together. In Proc. of the 3rd Int. Workshop on Human-Agent Interaction Design and Models (HAIDM 2014) at the 13th Int. Conf. on Agent and Multi-Agent Systems (AAMAS 2014) (pp. 110).
- 10.2200/S00764ED1V01Y201703HCI037 · doi:10.2200/s00764ed1v01y201703hci037
- Shively, R. J., Lachter, J., Brandt, S. L., Matessa, M., Battiste, V., & Johnson, W. W. (2017). Why humanautonomy teaming? In International conference on applied human factors and ergonomics (pp. 311). Springer, Cham.
- Wang, F. C, (2019) Intelligent Control , Beijing: China Science and Technology Press, 2019.
- Watson, D. P., & Scheidt, D. H. (2005). Autonomous systems. Johns Hopkins APL technical digest , 26 (4), 368-376.
- Xu, W. (2004). Status and challenges: Human factors in developing modern civil flight decks. Journal of Ergonomics, 10 (4), 53-56.
- Xu, W. (2005). Recent trend of research and applications on human-computer interaction. Journal of Ergonomics, 11 , 37-40.
- Xu, W. (2017). Revisiting user-centered design approach: new challenges and new opportunities. Chinese Journal of Ergonomics , 23 (1), 82-86.
- Xu, W. (2018). User-centered design (III): Methods for user experience and innovative design in the intelligent era. Chinese Journal of Applied Psychology , 25 (1), 3-17.
- Xu, W. (2019b). User-centered design (IV): Human-centered artificial intelligence. Chinese Journal of Applied Psychology, 25 (4), 291-305.
- Xu, W. (2020). User-centered design (V): From automation to the autonomy and autonomous vehicles in the intelligence era. Chinese Journal of Applied Psychology, 26 (2),108-129.
Cited by
Generative AI and ChatGPT: applications, challenges, and AI-human collaboration
Fui-Hoon Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023) · Journal of Information Technology Case and Application Research · doi:10.1080/15228053.2023.2233814 · Cited by 1,363
A survey of human-in-the-loop for machine learning
Wu, X., Xiao, L., Sun, Y., Zhang, J., Ma, T., & He, L. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.014 · Cited by 800
“• Human intervention remains on superficial judgments (such as acceptance/rejection or direction), and exploring more complex feedback is also a critical issue in human-in-the-loop applications [160].”
BackgroundSix Human-Centered Artificial Intelligence Grand Challenges
Ozmen Garibay, O., Winslow, B., Andolina, S., Antona, M., Bodenschatz, A., Coursaris, C., Falco, G., Fiore, S. M., Garibay, I., Grieman, K., et al. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2153320 · Cited by 501
“To effectively address the identified unique issues of AI systems as discussed earlier, over 20 existing methods of HCI, human factors, and other related disciplines from the HCAI perspective have been analyzed (Xu et al., 2022).”
BackgroundSelf-citationHuman-centred learning analytics and AI in education: A systematic literature review
Alfredo, R., Echeverria, V., Jin, Y., Yan, L., Swiecki, Z., Gašević, D., & Martinez-Maldonado, R. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100215 · Cited by 281
“As a participatory practice, HCD involves an iterative process of understanding the context, identifying end-user requirements, and involving stakeholders with lived experiences in the design and evaluation of the system, ideally engaging with stakeholders as equal partners (Xu et al., 2023).”
BackgroundMethodFrom explainable to interactive AI: A literature review on current trends in human-AI interaction
Raees, M., Meijerink, I., Lykourentzou, I., Khan, V.-J., & Papangelis, K. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103301 · Cited by 180
Metadata
- Authors
- Xu, W., Dainoff, M. J., Ge, L., & Gao, Z.
- Affiliations
- Zhejiang University (CN)
- Miami University (US)
- Corresponding author
- Xu, Wei
- Venue
- International Journal of Human–Computer Interaction
- Publisher
- Informa UK Limited
- Published
- 2022-04-06
- Volume
- 39
- Issue
- 3
- Pages
- 494-518
- ISSN
- 1044-7318 (ISSN-L)
- 1532-7590
- OpenAlex
- W3186393076
- Semantic Scholar
- e0e51b697701cca5ee1a46348c73a9eb80ef8094
- Cited by
- 363
- References
- 205
- Citations by year
- 2026: 106 · 2025: 121 · 2024: 85 · 2023: 38 · 2022: 13
- FWCI
- 31.75
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Human-Computer Interaction (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- human-centered AI, AI system development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.