A Multifaceted Vision of the Human-AI Collaboration: A Comprehensive Review
Open accessThe DOI matches its Crossref record; 338 of the 338 references it cites were found in a registry or an index.
Abstract
Human-AI collaboration has evolved into a complex, multidimensional paradigm shaped by research in various domains. Key areas such as human-in-the-loop systems, Interactive Machine Learning (IML), Hybrid Intelligence, and Human-Agent Interaction have significantly contributed to this development. However, these fields often lack cohesion, underscoring the need for a cohesive perspective to advance. This work addresses this gap by integrating insights from diverse aspects of collaboration to present a holistic approach to fostering effective and adaptive interactions between humans and artificial agents. It emphasizes empowering end-users with greater control and involvement in decision-making processes, thereby enhancing both the levels of interactivity and adaptability within intelligent systems. Moving beyond a focus on AI training techniques, this paper presents a broader perspective on incorporating human input into AI decision-making and learning processes, highlighting the importance of flexibility in systems and user engagement. The manuscript proposes a framework encompassing five levels of human integration and examines their relationship with core collaboration aspects, including the system purpose, participant expertise, and system proactivity. By synthesizing current knowledge on human-AI collaboration and outlining essential design principles, this work aims to advance the field and foster interdisciplinary collaboration among researchers, practitioners, and designers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The Human-AI framework offers a comprehensive view of this evolution by emphasizing five collaborative principles: bidirectional learning, explainability, co-validation, shared goals, and mutual augmentation [15].”
From Redefining the Programmer: Human-AI Collaboration, LLMs, and Security in Modern Software Engineering (2025) · cites this paper as [15]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
RETRACTED: Interactive teaching using human-machine interaction for higher education systems
Shang, H., Sivaparthipan, C. B., & ThanjaiVadivel (2022) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2022.107811 · Cited by 60 · cited 1×
“…definition, applications as varied as predictive maintenance [66], the design of smart spaces [67], the analysis of insurance claims [68], sentient analysis [69] have emerged with this collaboration in mind in domains such as healthcare [53], education [70] or cyber-security [71], among others.”
RetractedCortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 4×
“Popular model-free methods include Monte Carlo [123], Temporal Difference Learning [145], Q-learning [146], SARSA [131], and Dyna-Q [147].”
InfluentialGenerative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · cited 1×
“Previous generative models, such as Restricted Boltzmann Machines [161], Deep Belief Networks [162], and Deep Boltzmann Machines [163], had limitations due to their lack of generalization power [164].”
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 1×
“In contrast, since the development of Generative Adversarial Networks (GANs) in 2014 [165], new GAI algorithms have been developed, such as Transformer models [166], Generative Diffusion Models (GDM) [167], Nerf [168], or CLIP [169].”
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · cited 1×
“Consequently, the agent’s goal is to choose an action that will allow the system to maximise the reward obtained for each action over time [125], [126].”
NeRF: representing scenes as neural radiance fields for view synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2022) · Communications of the ACM · doi:10.1145/3503250 · Cited by 6,852
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733 · cited 1×
“A rapidly growing sub-field within reinforcement learning is Deep Reinforcement Learning, which leverages deep neural networks to learn the policy for RL problems [134], [135], [136], [137].”
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733 · cited 1×
“A rapidly growing sub-field within reinforcement learning is Deep Reinforcement Learning, which leverages deep neural networks to learn the policy for RL problems [134], [135], [136], [137].”
Dynamic Programming and Markov Processes.
Freimer, M., & Howard, R. A. (1961) · Journal of the American Statistical Association · doi:10.2307/2282268 · Cited by 3,378
ilastik: interactive machine learning for (bio)image analysis
Berg, S., Kutra, D., Kroeger, T., Straehle, C. N., Kausler, B. X., Haubold, C., Schiegg, M., Ales, J., Beier, T., Rudy, M., et al. (2019) · Nature Methods · doi:10.1038/s41592-019-0582-9 · Cited by 3,368 · cited 1×
“For that, visual exploratory tools [55], [57], [58], [59], data analysis and interactive exploration approaches have been used to improve ML models [60], [61].”
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117 · cited 1×
“…fundamental pillar in the evolution of GAI models was the integration of new techniques into them, such as the capability to learn from human feedback [171], [172], [173] to determine the most appropriate response for a given instruction, improving the model’s reliability and accuracy over time.”
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511 · cited 2×
“The most familiar AIGCs are ChatGPT for text creation [150], [151], [152] and DALL-E for creating unique and high-quality images from textual descriptions [153], [154], [155].”
Zero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417
Bellman, R. (1957) · Indiana University Mathematics Journal · doi:10.1512/iumj.1957.6.56038 · Cited by 2,408 · cited 1×
“However, to use the MDP framework, the current state must be sufficiently representative to not depend on previous states or agent actions [128], [129].”
Information processing in dynamical systems: foundations of harmony theory
Smolensky, P. (1986) · The MIT Press eBooks · Cited by 2,232
Security, privacy and trust in Internet of Things: The road ahead
Sicari, S., Rizzardi, A., Grieco, L. A., & Coen-Porisini, A. (2015) · Computer Networks · doi:10.1016/j.comnet.2014.11.008 · Cited by 2,187 · cited 1×
“To achieve this collaboration, the end-users role must be empowered by giving them an active role and increased control (power decision) over intelligent systems, creating more interactive spaces and services [47].”
Digital divide research, achievements and shortcomings
van Dijk, J. A. G. M. (2006) · Poetics · doi:10.1016/j.poetic.2006.05.004 · Cited by 2,046 · cited 1×
“Additionally, there is the issue of the digital divide, defined as the gap between those with access to computers and the internet and those without [189].”
Seung, H. S., Opper, M., & Sompolinsky, H. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130417 · Cited by 1,651 · cited 1×
“• Query by committee [103]: For this strategy, different models are presented that evaluate the data and decide the following data to be labelled by looking at which one creates the highest disagreement between the different models.”
Temporal difference learning and TD-Gammon
Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618 · cited 1×
“Popular model-free methods include Monte Carlo [123], Temporal Difference Learning [145], Q-learning [146], SARSA [131], and Dyna-Q [147].”
A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development
Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q.-L., & Tang, Y. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123618 · Cited by 1,452 · cited 4×
“For example, ChatGPT has integrated human feedback to enhance performance [151], [174].”
InfluentialReformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435 · cited 1×
“In contrast, since the development of Generative Adversarial Networks (GANs) in 2014 [165], new GAI algorithms have been developed, such as Transformer models [166], Generative Diffusion Models (GDM) [167], Nerf [168], or CLIP [169].”
What is user engagement? A conceptual framework for defining user engagement with technology
O'Brien, H. L., & Toms, E. G. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20801 · Cited by 1,397 · cited 1×
“In this context, user engagement refers to the degree of involvement, interaction, and emotional connection a user has with an AI system or technology [326].”
Generative AI and ChatGPT: applications, challenges, and AI-human collaboration
Fui-Hoon Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023) · Journal of Information Technology Case and Application Research · doi:10.1080/15228053.2023.2233814 · Cited by 1,363 · cited 4×
“AIGC has gained significant popularity owing to its robust capabilities [152].”
InfluentialSelf-citationExamining Science Education in ChatGPT: An Exploratory Study of Generative Artificial Intelligence
Cooper, G. (2023) · Journal of Science Education and Technology · doi:10.1007/s10956-023-10039-y · Cited by 1,263 · cited 1×
“By combining these advancements, models have made significant progress in AIGC tasks and have been adopted in various industries, including health [175], [176], [177], [178], art [179], and education [180].”
A Sequential Algorithm for Training Text Classifiers
Lewis, D. D., & Gale, W. A. (1994) · SIGIR ’94 · doi:10.1007/978-1-4471-2099-5_1 · Cited by 1,208
Cascella, M., Montomoli, J., Bellini, V., & Bignami, E. (2023) · Journal of Medical Systems · doi:10.1007/s10916-023-01925-4 · Cited by 1,120 · cited 1×
“By combining these advancements, models have made significant progress in AIGC tasks and have been adopted in various industries, including health [175], [176], [177], [178], art [179], and education [180].”
A perspective view and survey of meta-learning
Vilalta, R., & Drissi, Y. (2002) · Artificial Intelligence Review · doi:10.1023/a:1019956318069 · Cited by 1,108 · cited 1×
“If you are interested, further exploration can be found in upcoming surveys [115], [116], [117], [118], [119], [120], [121].”
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084 · cited 4×
“Hence, engagement with AI systems is vital for comprehending the user’s role and, consequently, enriching this relationship and enhancing user experience [86].”
InfluentialDeep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · cited 1×
“…fundamental pillar in the evolution of GAI models was the integration of new techniques into them, such as the capability to learn from human feedback [171], [172], [173] to determine the most appropriate response for a given instruction, improving the model’s reliability and accuracy over time.”
Human-in-the-loop machine learning: a state of the art
Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., & Fernández-Leal, Á. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10246-w · Cited by 1,025 · cited 2×
“…of collaboration, especially in pinpointing the specific attributes that characterize collaboration within each field : • Human-in-the-loop (HiTL) : Focuses on a human-centered approach to collaboration by directly involving individuals in AI systems’ decision-making processes [29], [30], [31].”
ChatGPT and other large language models are double-edged swords
Shen, Y., Heacock, L., Elias, J., Hentel, K. D., Reig, B., Shih, G., & Moy, L. (2023) · Radiology · doi:10.1148/radiol.230163 · Cited by 1,008 · cited 1×
“This dependence can lead to inaccuracies, primarily due to deviations and inaccurate information that hinder the ability of GAI to discern the credibility of sources or assign appropriate weights to different information channels [184].”
Pre-trained models: Past, present and future
Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.08.002 · Cited by 994
Shneiderman, B. (2022) · doi:10.1093/oso/9780192845290.001.0001 · Cited by 955
Interactive machine learning for health informatics: when do we need the human-in-the-loop?
Holzinger, A. (2016) · Brain Informatics · doi:10.1007/s40708-016-0042-6 · Cited by 906
Artificial intelligence and the future of global health
Schwalbe, N., & Wahl, B. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30226-9 · Cited by 876 · cited 1×
“Nevertheless, it is recognized that a human-centred design is crucial for creating new technology [77].”
Interacting with educational chatbots: A systematic review
Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11177-3 · Cited by 841 · cited 1×
“These advancements have led to personalized and engaging conversational experiences that can provide on-demand health interventions, among other services [346].”
Automated machine learning: Review of the state-of-the-art and opportunities for healthcare
Waring, J., Lindvall, C., & Umeton, R. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101822 · Cited by 834 · cited 1×
“In such domains, some challenges like the uncertainty of the clinical diagnosis or the lack of quality data, the identification of rare events are hard to solve with fully automated approaches without human intervention [73], [74].”
Progress and prospects of the human–robot collaboration
Ajoudani, A., Zanchettin, A. M., Ivaldi, S., Albu-Schäffer, A., Kosuge, K., & Khatib, O. (2018) · Autonomous Robots · doi:10.1007/s10514-017-9677-2 · Cited by 832 · cited 1×
“The primary purpose of collaboration is to leverage the strengths of each entity to compensate for the limitations of others, enabling achievements beyond what each could accomplish independently [12], [14], [17], [18].”
Dellermann, D., Ebel, P., Söllner, M., & Leimeister, J. M. (2019) · Business & Information Systems Engineering · doi:10.1007/s12599-019-00595-2 · Cited by 826 · cited 1×
“The primary purpose of collaboration is to leverage the strengths of each entity to compensate for the limitations of others, enabling achievements beyond what each could accomplish independently [12], [14], [17], [18].”
A survey of human-in-the-loop for machine learning
Wu, X., Xiao, L., Sun, Y., Zhang, J., Ma, T., & He, L. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.014 · Cited by 800 · cited 1×
“That is to say, IML models are designed to integrate human input into the ML solutions interactively, steering the supervised Learning process of the model in different stages [49].”
Artificial intelligence in the creative industries: a review
Anantrasirichai, N., & Bull, D. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10039-7 · Cited by 793 · cited 1×
“By combining these advancements, models have made significant progress in AIGC tasks and have been adopted in various industries, including health [175], [176], [177], [178], art [179], and education [180].”
Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities
Saeed, W., & Omlin, C. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110273 · Cited by 783
LaMDA: Language Models for Dialog Applications
Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778 · cited 1×
“Notably, LaMDA is a question-answering chatbot focused on two domains: Mount Everest education and music content recommendations [230], [231].”
Dyna, an Integrated Architecture for Learning, Planning, and Reacting
Sutton, R. S. (1991) · ACM SIGART Bulletin · doi:10.1145/122344.122377 · Cited by 777 · cited 1×
“Popular model-free methods include Monte Carlo [123], Temporal Difference Learning [145], Q-learning [146], SARSA [131], and Dyna-Q [147].”
Recent Progress on Generative Adversarial Networks (GANs): A Survey
Pan, Z., Yu, W., Yi, X., Khan, A., Yuan, F., & Zheng, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2905015 · Cited by 701 · cited 1×
“Previous generative models, such as Restricted Boltzmann Machines [161], Deep Belief Networks [162], and Deep Boltzmann Machines [163], had limitations due to their lack of generalization power [164].”
Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review.
Aggarwal, A., Tam, C. C., Wu, D., Li, X., & Qiao, S. (2023) · Journal of medical Internet research · doi:10.2196/40789 · Cited by 683 · cited 1×
“Chatbots,as conversational agents, have evolved significantly in recent years, enhancing their ability to simulate human interactions through text, voice, and visual communication [27], [345].”
Role of Chat GPT in public health
Biswas, S. S. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03172-7 · Cited by 673 · cited 1×
“By combining these advancements, models have made significant progress in AIGC tasks and have been adopted in various industries, including health [175], [176], [177], [178], art [179], and education [180].”
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633 · cited 1×
“In the same line, later, Fails and Olsen introduced a more mature representation of the concept of IML and its comparison against classical ML approaches [51].”
A survey on active learning and human-in-the-loop deep learning for medical image analysis
Budd, S., Robinson, E. C., & Kainz, B. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102062 · Cited by 606 · cited 3×
“Active learning (AL) is a sub-field of machine learning in which the model can query a user or other source of information, called an oracle, to label new data points in order to improve model performance, as shown in Figure 2 [55], [98], [99], [100].”
Learning to summarize from human feedback
Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., & Christiano, P. (2020) · arXiv · doi:10.48550/arxiv.2009.01325 · Cited by 550 · cited 1×
“…fundamental pillar in the evolution of GAI models was the integration of new techniques into them, such as the capability to learn from human feedback [171], [172], [173] to determine the most appropriate response for a given instruction, improving the model’s reliability and accuracy over time.”
Artificial Intelligence (AI) Ethics: Ethics of AI and Ethical AI
Siau, K., & Wang, W. (2020) · Journal of Database Management · doi:10.4018/jdm.2020040105 · Cited by 533 · cited 2×
“Legal, moral, and ethical concerns also loom large, encompassing areas such as copyright infringement in AI-generated artworks [185], cheating and plagiarism in educational institutions, data privacy, security [186], and the malicious use of deepfakes [187].”
Understanding and Improving Teamwork in Organizations: A Scientifically Based Practical Guide
Salas, E., Shuffler, M. L., Thayer, A. L., Bedwell, W. L., & Lazzara, E. H. (2015) · Human Resource Management · doi:10.1002/hrm.21628 · Cited by 528 · cited 1×
“In a formal sense, communication refers to a reciprocal exchange in which team members share messages to coordinate ideas, norms, and strategies to achieve their collective goals [338], [339].”
Cooke, N. J., Gorman, J. C., Myers, C. W., & Duran, J. L. (2013) · Cognitive Science · doi:10.1111/cogs.12009 · Cited by 522 · cited 1×
“In a formal sense, communication refers to a reciprocal exchange in which team members share messages to coordinate ideas, norms, and strategies to achieve their collective goals [338], [339].”
Aung, Y. Y. M., Wong, D. C. S., & Ting, D. S. W. (2021) · British Medical Bulletin · doi:10.1093/bmb/ldab016 · Cited by 511 · cited 1×
“In such domains, some challenges like the uncertainty of the clinical diagnosis or the lack of quality data, the identification of rare events are hard to solve with fully automated approaches without human intervention [73], [74].”
ChatGPT for tourism: applications, benefits and risks
Carvalho, I., & Ivanov, S. (2024) · Tourism Review · doi:10.1108/tr-02-2023-0088 · Cited by 507 · cited 1×
“These algorithms have gained significant popularity in recent years and have revolutionized various fields and applications, such as medicine [44] and tourism [238].”
Reinforcement Learning based Recommender Systems: A Survey
Afsar, M. M., Crump, T., & Far, B. (2023) · ACM Computing Surveys · doi:10.1145/3543846 · Cited by 502 · cited 1×
“The Actor then uses the feedback from the Critic to adjust its policy and make more informed decisions, leading to improved overall performance [132], [133].”
Efficient Learning of Deep Boltzmann Machines
Salakhutdinov, R., & Larochelle, H. (2010) · International Conference on Artificial Intelligence and Statistics · Cited by 501
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500
Q-Learning: Theory and Applications
Clifton, J., & Laber, E. (2020) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031219-041220 · Cited by 492 · cited 1×
“Examples: value interaction algorithm, Q-learning [130] and Sarsa [131].”
Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P. S., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2303.04226 · Cited by 463 · cited 5×
“Therefore, we do not extensively delve into these technical aspects within the scope of this work, but interested readers can find in-depth explorations in forthcoming surveys [19], [20], [21], [170], [181].”
InfluentialIncorporating diversity in active learning with support vector machines
Brinker, K. (2003) · Cited by 458
The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems
Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., & Ebel, P. (2021) · arXiv · doi:10.48550/arxiv.2105.03354 · Cited by 456 · cited 1×
“• Hybrid Intelligence Systems : Examines socio-technical frameworks that synergize human and AI capabilities to address complex problems through collaborative efforts [39], [41], [42].”
ChatGPT in healthcare: A taxonomy and systematic review
Li, J., Dada, A., Puladi, B., Kleesiek, J., & Egger, J. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108013 · Cited by 452
Unleashing the Power of Edge-Cloud Generative AI in Mobile Networks: A Survey of AIGC Services
Xu, M., Du, H., Niyato, D., Kang, J., Xiong, Z., Mao, S., Han, Z., Jamalipour, A., Kim, D. I., Shen, X., et al. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3353265 · Cited by 449 · cited 1×
“Therefore, we do not extensively delve into these technical aspects within the scope of this work, but interested readers can find in-depth explorations in forthcoming surveys [19], [20], [21], [170], [181].”
Why are we averse towards Algorithms? A comprehensive literature Review on Algorithm aversionPreprint
Jussupow, E., Benbasat, I., & Heinzl, A. (2020) · RePEc: Research Papers in Economics · Cited by 440
Designing pixel-oriented visualization techniques: theory and applications
Keim, D. A. (2000) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/2945.841121 · Cited by 425 · cited 1×
“They employed a pixel-oriented technique known as Squarele Segments [288] for intuitive visualization of high-dimensional data, which enhances users’ understanding of the classifications and the data involved.”
Akata, Z., Balliet, D., de Rijke, M., Dignum, F., Dignum, V., Eiben, G., Fokkens, A., Grossi, D., Hindriks, K., Hoos, H., et al. (2020) · Computer · doi:10.1109/mc.2020.2996587 · Cited by 415 · cited 3×
“Thus, through these techniques, the system can incorporate human expertise into account and exploit their strengths, together with ethical, legal, and societal human considerations [14].”
The institutionalization of YouTube: From user-generated content to professionally generated content
Kim, J. (2012) · Media, Culture & Society · doi:10.1177/0163443711427199 · Cited by 410 · cited 1×
“A few years ago, the most common content creation strategies were Professional Generated Content (PGC) [156], [157] and User Generated Content (UGC) [158].”
Köbis, N., & Mossink, L. D. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106553 · Cited by 406
From Show to Tell: A Survey on Deep Learning-Based Image Captioning
Stefanini, M., Cornia, M., Baraldi, L., Cascianelli, S., Fiameni, G., & Cucchiara, R. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3148210 · Cited by 373
Reinforcement Learning Algorithm for Partially Observable Markov Decision Problems
Jaakkola, T., Singh, S. P., & Jordan, M. I. (1994) · Cited by 361
Superior skin cancer classification by the combination of human and artificial intelligence
Hekler, A., Utikal, J. S., Enk, A. H., Hauschild, A., Weichenthal, M., Maron, R. C., Berking, C., Haferkamp, S., Klode, J., Schadendorf, D., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.07.019 · Cited by 354 · cited 1×
“For instance, in the study by Hekler et al., both the expert and the AI operate in parallel, independently classifying images of skin cancer, with their classifications combined to arrive at a final diagnosis based on the accuracy of the AI’s predictions [216].”
A Review of User Interface Design for Interactive Machine Learning
Dudley, J. J., & Kristensson, P. O. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185517 · Cited by 343 · cited 3×
“To facilitate those applications, IML can also involve the design of intelligent user interface (IUI) frameworks that drive this interaction, increasing its accessibility [64].”
InfluentialCutillo, C. M., Sharma, K. R., Foschini, L., Kundu, S., Mackintosh, M., Mandl, K. D., MI in Healthcare Workshop Working Group, Beck, T., Collier, E., Colvis, C., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0254-2 · Cited by 337 · cited 1×
“At the same time, explainability, usability, or transparency are identified areas of improvement for IML systems [84].”
AI-Generated Content (AIGC): A SurveyPreprint
Wu, J., Gan, W., Chen, Z., Wan, S., & Hong, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.06632 · Cited by 333 · cited 9×
“Therefore, we do not extensively delve into these technical aspects within the scope of this work, but interested readers can find in-depth explorations in forthcoming surveys [19], [20], [21], [170], [181].”
InfluentialVanschoren, J. (2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5_2 · Cited by 328
The Future of Human-in-the-Loop Cyber-Physical Systems
Schirner, G., Erdogmus, D., Chowdhury, K., & Padir, T. (2013) · Computer · doi:10.1109/mc.2013.31 · Cited by 327 · cited 1×
“Specifically, this encompasses both the user’s role in creating and defining the model and iterative interactions with learning systems over time [96].”
Use and Adoption Challenges of Wearable Activity Trackers
Shih, P. C., Han, K., Poole, E. S., Rosson, M. B., & Carroll, J. M. (2015) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 316
Smart Environments: Technologies, Protocols, and Applications
(2004) · Wiley eBooks · doi:10.1002/047168659x · Cited by 310
Supporting Human-AI Teams:Transparency, explainability, and situation awareness
Endsley, M. R. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107574 · Cited by 309 · cited 2×
“The concept of explicability and the emphasis on interactive processes stand in contrast to the ‘‘black box’’ nature that AI and machine learning (ML) systems may exhibit when human knowledge is not integrated into their problem-solving processes [212], [315], [316].”
Handbook of Ambient Intelligence and Smart Environments
(2010) · Springer eBooks · doi:10.1007/978-0-387-93808-0 · Cited by 293
Concept Analysis of Interdisciplinary Collaboration
Petri, L. (2010) · Nursing Forum · doi:10.1111/j.1744-6198.2010.00167.x · Cited by 284 · cited 1×
“Generally, collaboration is understood as two or more entities working together, sharing responsibility for solving problems, and actively interacting to achieve a common goal [16].”
Experimental evidence of effective human–AI collaboration in medical decision-making
Reverberi, C., Rigon, T., Solari, A., Hassan, C., Cherubini, P., GI Genius CADx Study Group, Antonelli, G., Awadie, H., Bernhofer, S., Carballal, S., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-18751-2 · Cited by 281 · cited 1×
“A comparable methodology was employed by Reverberi et al., who conducted two distinct experiments to explore this collaboration [235].”
The Future of Human-AI Collaboration: A Taxonomy of Design Knowledge for Hybrid Intelligence Systems
Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., & Ebel, P. (2019) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2019.034 · Cited by 279 · cited 1×
“• Hybrid Intelligence Systems : Examines socio-technical frameworks that synergize human and AI capabilities to address complex problems through collaborative efforts [39], [41], [42].”
Wang, D., Churchill, E., Maes, P., Fan, X., Shneiderman, B., Shi, Y., & Wang, Q. (2020) · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3334480.3381069 · Cited by 271 · cited 1×
“From the human perspective, engagement goes beyond just the functional use of the system; it refers to the user’s active participation and willingness to continue using the system [327].”
"Intelligent Environments: a manifesto"
Augusto, J. C., Callaghan, V., Cook, D., Kameas, A., & Satoh, I. (2013) · Human-centric Computing and Information Sciences · doi:10.1186/2192-1962-3-12 · Cited by 269 · cited 1×
“Building on this foundation, Ambient Intelligence (AmI) and Intelligent Environments (IE) further advanced the principle of user-centric technology, prioritizing interactive and responsive environments [7], [8].”
ModelTracker: Redesigning Performance Analysis Tools for Machine Learning
Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., & Suh, J. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702509 · Cited by 262 · cited 1×
“Additionally, they designed a system where users can build a machine-learning algorithm through a visual interface [283].”
Meta-learning in Reinforcement Learning
Schweighofer, N., & Doya, K. (2003) · Neural Networks · doi:10.1016/s0893-6080(02)00228-9 · Cited by 261 · cited 1×
“If you are interested, further exploration can be found in upcoming surveys [115], [116], [117], [118], [119], [120], [121].”
Deep Reinforcement Learning: An Overview
Mousavi, S. S., Schukat, M., & Howley, E. (2018) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-319-56991-8_32 · Cited by 261
How transparency modulates trust in artificial intelligence
Zerilli, J., Bhatt, U., & Weller, A. (2022) · Patterns · doi:10.1016/j.patter.2022.100455 · Cited by 257 · cited 1×
“Furthermore, Zerilli et al. highlight that a user’s limited understanding of how these devices function, including data collection and measurement calculations, can significantly influence their perception of accuracy [314].”
Learning how to Active Learn: A Deep Reinforcement Learning Approach
Fang, M., Li, Y., & Cohn, T. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1063 · Cited by 252 · cited 1×
“For example, Fang et al. described the active learning scenario as a Markov decision process by considering a state as an unlabeled sample and the action as whether a label is required [113].”
Hybrid collective intelligence in a human–AI society
Peeters, M. M. M., van Diggelen, J., van den Bosch, K., Bronkhorst, A., Neerincx, M. A., Schraagen, J. M., & Raaijmakers, S. (2021) · AI & SOCIETY · doi:10.1007/s00146-020-01005-y · Cited by 229 · cited 2×
“• Collective Hybrid Intelligence : Advocates the notion that true intelligence emerges from the sustained collaboration and co-learning between human and machine counterparts [39].”
Holzinger, A., Plass, M., Kickmeier-Rust, M., Holzinger, K., Crişan, G. C., Pintea, C.-M., & Palade, V. (2019) · Applied Intelligence · doi:10.1007/s10489-018-1361-5 · Cited by 228
EnsembleMatrix: interactive visualization to support machine learning with multiple classifiers
Talbot, J., Lee, B., Kapoor, A., & Tan, D. S. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518895 · Cited by 226
Explainable Reinforcement Learning: A Survey and Comparative Review
Milani, S., Topin, N., Veloso, M., & Fang, F. (2024) · ACM Computing Surveys · doi:10.1145/3616864 · Cited by 224 · cited 1×
“Furthermore, it is crucial for humans to recognize the limitations of AI; otherwise, they may develop unrealistic expectations regarding AI’s capabilities, which can negatively impact their engagement, adaptation, and trust [334], [335].”
Designing transparency for effective Human-AI collaboration
Vössing, M., Kühl, N., Lind, M., & Satzger, G. (2022) · Information Systems Frontiers · doi:10.1007/s10796-022-10284-3 · Cited by 223 · cited 1×
“In summary, the interactive nature of collaborative solutions contributes to building trust in AI systems by increasing user involvement in the decision-making process and enhancing their understanding of how AI functions [321].”
Large-scale Multi-modal Pre-trained Models: A Comprehensive Survey
Wang, X., Chen, G., Qian, G., Gao, P., Wei, X.-Y., Wang, Y., Tian, Y., & Gao, W. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1410-8 · Cited by 223 · cited 2×
“Therefore, we do not extensively delve into these technical aspects within the scope of this work, but interested readers can find in-depth explorations in forthcoming surveys [19], [20], [21], [170], [181].”
GPT4Rec: A Generative Framework for Personalized Recommendation and User Interests InterpretationPreprint
Li, J., Zhang, W., Wang, T., Xiong, G., Lu, A., & Medioni, G. G. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.03879 · Cited by 214 · cited 1×
“Personalization has been extensively explored in the development of recommender systems [302], [303], [304].”
Interactive machine learning: letting users build classifiers
WARE, M., FRANK, E., HOLMES, G., HALL, M., & WITTEN, I. H. (2001) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.2001.0499 · Cited by 207 · cited 1×
“To begin with, the idea of IML was initially approached by Ware et al. [50].”
The impact of educational chatbot on student learning experience
Ait Baha, T., El Hajji, M., Es-Saady, Y., & Fadili, H. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12166-w · Cited by 206 · cited 1×
“Such AIGC algorithms are considered interactive due to the inherent nature of dialogue [244], [245].”
Ambient Intelligence and Smart Environments: A State of the Art
Augusto, J. C., Nakashima, H., & Aghajan, H. K. (2009) · Springer eBooks · doi:10.1007/978-0-387-93808-0_1 · Cited by 205 · cited 1×
“Building on this foundation, Ambient Intelligence (AmI) and Intelligent Environments (IE) further advanced the principle of user-centric technology, prioritizing interactive and responsive environments [7], [8].”
Human-robot mutual adaptation in collaborative tasks: Models and experiments
Nikolaidis, S., Hsu, D., & Srinivasa, S. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917690593 · Cited by 199 · cited 3×
“Many authors postulate that adaptation should occur for both human and AI agents [262].”
Collaborating with Humans without Human DataPreprint
Strouse, D., McKee, K. R., Botvinick, M., Hughes, E., & Everett, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08176 · Cited by 196 · cited 1×
“For example, Strouse et al. explored how to train robots for effective collaboration with human partners without relying on human data [265].”
Active Learning Query Strategies for Classification, Regression, and Clustering: A Survey
Kumar, P., & Gupta, A. (2020) · Journal of Computer Science and Technology · doi:10.1007/s11390-020-9487-4 · Cited by 195 · cited 3×
“Apart from these strategies, which are some of the most widely used, the scientific community has created more to adapt the proposed solutions to the tasks that have arisen [101].”
InfluentialFoundations & Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions
Liang, P. P., Zadeh, A., & Morency, L.-P. (2024) · ACM Computing Surveys · doi:10.1145/3656580 · Cited by 195
Fictitious Self-Play in Extensive-Form Games
Heinrich, J., Silver, D., & DeepMind, G. (2015) · Cited by 193
The role of shared mental models in human-AI teams: a theoretical review
Andrews, R. W., Lilly, J. M., Srivastava, D., & Feigh, K. M. (2023) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2022.2061080 · Cited by 191 · cited 2×
“Technical barriers also emerge from the fundamentally different ways humans and AI process information, which can create obstacles to seamless communication [25], [26].”
Incorporating Diversity and Density in Active Learning for Relevance Feedback
Xu, Z., Akella, R., & Zhang, Y. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-71496-5_24 · Cited by 189 · cited 1×
“• Density-Based [105]: This strategy seeks to label the data that is in an area of high data density as it is believed that areas of high data density have high data representation.”
Deep reinforcement learning in recommender systems: A survey and new perspectives
Chen, X., Yao, L., McAuley, J., Zhou, G., & Wang, X. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110335 · Cited by 187 · cited 1×
“The Actor then uses the feedback from the Critic to adjust its policy and make more informed decisions, leading to improved overall performance [132], [133].”
Watch Me Improve—Algorithm Aversion and Demonstrating the Ability to Learn
Berger, B., Adam, M., Rühr, A., & Benlian, A. (2021) · Business & Information Systems Engineering · doi:10.1007/s12599-020-00678-5 · Cited by 183 · cited 2×
“This exclusive reliance on AI as the sole decision-maker in the healthcare sector can lead to patient aversion [236].”
A Survey of AI-Generated Content (AIGC)
Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P., & Sun, L. (2025) · ACM Computing Surveys · doi:10.1145/3704262 · Cited by 182
AudioLDM 2: Learning Holistic Audio Generation With Self-Supervised Pretraining
Liu, H., Yuan, Y., Liu, X., Mei, X., Kong, Q., Tian, Q., Wang, Y., Wang, W., Wang, Y., & Plumbley, M. D. (2024) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2024.3399607 · Cited by 180 · cited 1×
“AIGCs can also generate audio, which falls into two primary categories: text-to-speech synthesis and voice cloning [251], [252].”
A Survey on Active Learning: State-of-the-Art, Practical Challenges and Research Directions
Tharwat, A., & Schenck, W. (2023) · Mathematics · doi:10.3390/math11040820 · Cited by 175 · cited 2×
“Active learning (AL) is a sub-field of machine learning in which the model can query a user or other source of information, called an oracle, to label new data points in order to improve model performance, as shown in Figure 2 [55], [98], [99], [100].”
Holzinger, A., & Jurišica, I. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-43968-5_1 · Cited by 173 · cited 2×
“For example, human expertise can be utilized to select samples heuristically, reducing the exponential search space and integrating human knowledge into machine intelligence to discover novel, previously unknown insights into data [60], [75].”
Meta-learning in natural and artificial intelligence
Wang, J. X. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.01.002 · Cited by 171
A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIPreprint
Zhang, C., Zhang, C., Zheng, S., Zhang, M., Qamar, M., Bae, S., & Kweon, I. S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.13336 · Cited by 171 · cited 1×
“AIGCs can also generate audio, which falls into two primary categories: text-to-speech synthesis and voice cloning [251], [252].”
Deng, W., Zhao, H., Song, Y., & Xu, J. (2020) · International Journal of Bio-Inspired Computation · doi:10.1504/ijbic.2020.10033314 · Cited by 170 · cited 1×
“One notable example of this integration level is the algorithm proposed by Holzinger et al. They introduced a solution for optimizing the Traveling Salesman Prob-lem [256] using the behavior of the Ant Colony Optimization algorithm [273].”
Maximizing Expected Model Change for Active Learning in Regression
Cai, W., Zhang, Y., & Zhou, J. (2013) · 2013 IEEE 13th International Conference on Data Mining · doi:10.1109/icdm.2013.104 · Cited by 165 · cited 1×
“• Maximizing Expected Model Change [107]: This strategy, based on a stochastic gradient, selects the points that it estimates to generate the maximum change in the current model parameters.”
Scaling up crowd-sourcing to very large datasets: a case for active learning
Mozafari, B., Sarkar, P., Franklin, M., Jordan, M., & Madden, S. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2735471.2735474 · Cited by 165 · cited 1×
“In this vein, Mozafari et al. studied the application of active learning strategies in crowdsourced databases to reduce the number of instances needing labeling by humans, thereby minimizing human effort and enabling the scalability of this approach [224].”
Deep learning in computational mechanics: a review
Herrmann, L., & Kollmannsberger, S. (2024) · Computational Mechanics · doi:10.1007/s00466-023-02434-4 · Cited by 162 · cited 1×
“This approach has gained substantial traction in recent years and is now widely applied across various fields, including computational mechanics [138], chemical industry design processes, astronomy [122], urban water systems [139], and autonomous highway vehicle control [140].”
Engagement in human-agent interaction: An overview
Oertel, C., Castellano, G., Chetouani, M., Nasir, J., Obaid, M., Pelachaud, C., & Peters, C. (2020) · Frontiers in Robotics and AI · doi:10.3389/frobt.2020.00092 · Cited by 160 · cited 1×
“Oertel et al. reviewed previous efforts addressing the relevance of engagement in human-AI interaction from both perspectives [328].”
Humans inherit artificial intelligence biases
Vicente, L., & Matute, H. (2023) · Scientific Reports · doi:10.1038/s41598-023-42384-8 · Cited by 158
Grounding Interactive Machine Learning Tool Design in How Non-Experts Actually Build Models
Yang, Q., Suh, J., Chen, N.-C., & Ramos, G. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196729 · Cited by 156 · cited 4×
“Empirical findings from Yang et al. suggest that incorporating interactive processes can lead to greater confidence among non-experts in learning outcomes, particularly when explainability is prioritized [56].”
InfluentialA review on reinforcement learning-based highway autonomous vehicle control
Irshayyid, A., Chen, J. S., & Xiong, G. (2024) · Green Energy and Intelligent Transportation · doi:10.1016/j.geits.2024.100156 · Cited by 155 · cited 1×
“This approach has gained substantial traction in recent years and is now widely applied across various fields, including computational mechanics [138], chemical industry design processes, astronomy [122], urban water systems [139], and autonomous highway vehicle control [140].”
A survey on model-based reinforcement learning
Luo, F.-M., Xu, T., Lai, H., Chen, X.-H., Zhang, W., & Yu, Y. (2024) · Science China Information Sciences · doi:10.1007/s11432-022-3696-5 · Cited by 148
Game-Theoretic Modeling of Human Adaptation in Human-Robot Collaboration
Nikolaidis, S., Nath, S., Procaccia, A. D., & Srinivasa, S. (2017) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/2909824.3020253 · Cited by 147
Training with Quantization Noise for Extreme Model Compression
Fan, A., Stock, P., Graham, B., Grave, E., Gribonval, R., Jegou, H., & Joulin, A. (2020) · arXiv · doi:10.48550/arxiv.2004.07320 · Cited by 145 · cited 1×
“By leveraging AIGC, users can modify existing images or generate new ones that meet specific requirements based on given prompts [247], [248], all without needing advanced skills or knowledge.”
Mehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 144 · cited 1×
“Understanding the role of trust in engaging humans in collaboration, as well as how trust is influenced during interactions, is crucial for establishing and promoting effective and productive collaborative relationships between humans and AI [308], [309].”
Deep reinforcement learning with experience replay based on SARSA
Zhao, D., Wang, H., Shao, K., & Zhu, Y. (2016) · 2016 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2016.7849837 · Cited by 140 · cited 2×
“Examples: value interaction algorithm, Q-learning [130] and Sarsa [131].”
Rouhani, S. (1985) · Water Resources Research · doi:10.1029/wr021i006p00837 · Cited by 139
A Human-AI Collaborative Approach for Clinical Decision Making on Rehabilitation Assessment
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. B. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445472 · Cited by 137 · cited 1×
“For instance, Lee et al. developed a feature selection system that provides quantitative analysis of patients’ conditions to enhance rehabilitation assessment practices [198].”
Krämer, N. C., von der Pütten, A., & Eimler, S. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-25691-2_9 · Cited by 135 · cited 1×
“Technical barriers also emerge from the fundamentally different ways humans and AI process information, which can create obstacles to seamless communication [25], [26].”
Human-Centred Machine Learning
Gillies, M., Fiebrink, R., Tanaka, A., Garcia, J., Bevilacqua, F., Heloir, A., Nunnari, F., Mackay, W., Amershi, S., Lee, B., et al. (2016) · Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/2851581.2856492 · Cited by 134 · cited 1×
“Taking as guidelines the previous statements, IML solutions should also be a driver to promote this human-AI collaboration in different phases of the model design, creating long-lasting interactions to make ML algorithms more useful, accessible and usable for every particular individual [65].”
Zhang, R., Duan, W., Flathmann, C., McNeese, N., Freeman, G., & Williams, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610072 · Cited by 134 · cited 1×
“The more effectively agents comprehend each other’s capabilities, limitations, inputs, outputs, and context, the more efficiently they can collaborate to achieve their shared objectives [331], [332].”
Toward harnessing user feedback for machine learning
Stumpf, S., Rajaram, V., Li, L., Burnett, M., Dietterich, T., Sullivan, E., Drummond, R., & Herlocker, J. (2007) · Proceedings of the 12th international conference on Intelligent user interfaces · doi:10.1145/1216295.1216316 · Cited by 132 · cited 1×
“Thus, as relevant as it is to understand how to build better models, it is to evaluate the human-centred perspective of ML [78] and how to frame this interaction [79].”
Human Robot Collaboration in Industry 4.0: a literature review
Baratta, A., Cimino, A., Gnoni, M. G., & Longo, F. (2023) · Procedia Computer Science · doi:10.1016/j.procs.2022.12.389 · Cited by 132 · cited 1×
“• Human-Robot Collaboration : Centers on developing robots designed to work collaboratively with humans in physical spaces to achieve common objectives [33].”
AI-teaming: Redefining collaboration in the digital era
Schmutz, J. B., Outland, N., Kerstan, S., Georganta, E., & Ulfert, A.-S. (2024) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2024.101837 · Cited by 132 · cited 1×
“• Human-AI team: Suggests that humans and AI function not as separate entities but as integrated parts of a cohesive team, working symbiotically towards shared goals [45], [46].”
A Survey on Interactive Reinforcement Learning: Design Principles and Open Challenges
Cruz, C. A., & Igarashi, T. (2020) · Designing Interactive Systems Conference · doi:10.1145/3357236.3395525 · Cited by 131 · cited 1×
“…to the application of this interaction, Active learning (AL) [30] and Reinforcement Learning (RL), also named Interactive reinforcement learning [97], are the most popular interactive learning approaches devised to facilitate performance improvement in ML models with the help of human knowledge.”
AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head
Huang, R., Li, M., Yang, D., Shi, J., Chang, X., Ye, Z., Wu, Y., Hong, Z., Huang, J., Liu, J., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i21.30570 · Cited by 131
AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head
Huang, R., Li, M., Yang, D., Shi, J., Chang, X., Ye, Z., Wu, Y., Hong, Z., Huang, J., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.12995 · Cited by 131 · cited 1×
“Another innovative application, AudioGPT, created by Huang et al., excels in generating diverse audio content, including speech, music, sound effects, and talking head tasks [218].”
Training a Robot via Human Feedback: A Case Study
Knox, W. B., Stone, P., & Breazeal, C. (2013) · Lecture notes in computer science · doi:10.1007/978-3-319-02675-6_46 · Cited by 130
A review of intelligent music generation systems
Wang, L., Zhao, Z., Liu, H., Pang, J., Qin, Y., & Wu, Q. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09418-2 · Cited by 123 · cited 1×
“Artificial Intelligence Generated Content (AIGC) creates high-quality and quick content, such as images, text [148], music [149], 3D models, and natural language, utilizing Generated Artificial Intelligence (GAI) algorithms to meet users’ requirements.”
Musick, G., O’Neill, T., Schelble, B. G., McNeese, N. J., & Henke, J. B. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106852 · Cited by 122 · cited 1×
“The facilitator utilized the ‘‘Wizard of Oz’’ (WoZ) technique [213], a common method for comparing perceptions between human-human and human-agent teams.”
The limitations and ethical considerations of ChatGPT
Hua, S., Jin, S., & Jiang, S. (2024) · Data Intelligence · doi:10.1162/dint_a_00243 · Cited by 120 · cited 1×
“For example, ChatGPT utilizes prior user input to generate more customized responses [151], [174], [300], [301].”
Yang, R., Shin, E., Newman, M. W., & Ackerman, M. S. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2804269 · Cited by 119 · cited 2×
“…emphasizes the need for considering each person’s unique circumstances and data when creating intelligent systems dedicated to monitoring users’ activities and behaviors, as the accuracy of the same intelligent system is perceived very differently by each user depending on their expectations [94].”
Recent research advances on interactive machine learning
Jiang, L., Liu, S., & Chen, C. (2018) · Journal of Visualization · doi:10.1007/s12650-018-0531-1 · Cited by 119 · cited 1×
“Considering the relevant role that humans may take in the ML process and the benefit of this collaboration, Jiang et al. [62] presented a task-oriented taxonomy with nine categories to determine the most prominent application fields of IML.”
Exploring the future of nursing: Insights from the ChatGPT model
Gunawan, J. (2023) · Belitung Nursing Journal · doi:10.33546/bnj.2551 · Cited by 117
A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications
Maadi, M., Akbarzadeh Khorshidi, H., & Aickelin, U. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18042121 · Cited by 116 · cited 1×
“This involves those approaches that take advantage of human agents (i.e., domain experts) to create improved models and solutions that can optimize their learning behaviour based on human knowledge and experience [53], [54], [55].”
Defining Active Learning: A Restricted Systemic Review
Doolittle, P., Wojdak, K., & Walters, A. (2023) · Teaching and Learning Inquiry · doi:10.20343/teachlearninqu.11.25 · Cited by 115 · cited 1×
“Active learning (AL) is a sub-field of machine learning in which the model can query a user or other source of information, called an oracle, to label new data points in order to improve model performance, as shown in Figure 2 [55], [98], [99], [100].”
University of New South Wales, Carter, L., Liu, D., University of New South Wales, Cantrell, C., & Virginia Commonwealth University (2020) · AIS Transactions on Human-Computer Interaction · doi:10.17705/1thci.00138 · Cited by 111 · cited 1×
“Emerging technologies like generative AI may inadvertently exacerbate this societal divide [190], [191].”
Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models
Dhillon, P. S., Molaei, S., Li, J., Golub, M., Zheng, S., & Robert, L. P. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642134 · Cited by 111 · cited 1×
“Although these methods have recently gained traction and transformed the way many tasks are performed, particularly in education [239], [240], [241], they also hold potential influence over the academic peer review process [242].”
Deep Reinforcement Active Learning for Human-in-the-Loop Person Re-Identification
Liu, Z., Wang, J., Gong, S., Tao, D., & Lu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00622 · Cited by 109
Toward Trustworthy Artificial Intelligence (TAI) in the Context of Explainability and Robustness
chander, B., John, C., Warrier, L., & Gopalakrishnan, K. (2025) · ACM Computing Surveys · doi:10.1145/3675392 · Cited by 108 · cited 1×
“The concept of explicability and the emphasis on interactive processes stand in contrast to the ‘‘black box’’ nature that AI and machine learning (ML) systems may exhibit when human knowledge is not integrated into their problem-solving processes [212], [315], [316].”
Human-Computer Interaction System: A Survey of Talking-Head Generation
Zhen, R., Song, W., He, Q., Cao, J., Shi, L., & Luo, J. (2023) · Electronics · doi:10.3390/electronics12010218 · Cited by 106 · cited 1×
“• Human-Computer Interaction : Seeks to optimize the design of intuitive and efficient interaction methods between users and autonomous systems [37], [38].”
Leveraging explanations in interactive machine learning: An overview
Teso, S., Alkan, Ö., Stammer, W., & Daly, E. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1066049 · Cited by 106
Human-machine symbiosis: A multivariate perspective for physically coupled human-machine systems
Inga, J., Rueß, M., Robens, J. H., Nelius, T., Rothfuß, S., Kille, S., Dahlinger, P., Lindenmann, A., Thomaschke, R., Neumann, G. M., et al. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102926 · Cited by 103
Human-Robot Mutual Adaptation in Shared Autonomy
Nikolaidis, S., Zhu, Y. X., Hsu, D., & Srinivasa, S. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020252 · Cited by 101 · cited 3×
“In this process, it is essential for the system to maintain human trust without diminishing it [263], [282] or exceeding it beyond reasonable limits throughout the collaboration.”
UX Research on Conversational Human-AI Interaction: A Literature Review of the ACM Digital Library
Zheng, Q., Tang, Y., Liu, Y., Liu, W., & Huang, Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501855 · Cited by 100 · cited 1×
“Those efforts aim to support the iterative design, development, and dissemination processes of learning systems oriented toward conversational AI agents [330] or to define a human-centered thinking approach to applying IML methods [82].”
Deep reinforcement learning for process design: Review and perspective
Gao, Q., & Schweidtmann, A. M. (2024) · Current Opinion in Chemical Engineering · doi:10.1016/j.coche.2024.101012 · Cited by 100
Zhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99 · cited 1×
“Another health-related example is provided by Zhang et al., who developed a method for diagnosing sepsis [233].”
Pollefeys, M., & Gool, L. V. (2002) · Communications of the ACM · doi:10.1145/514236.514263 · Cited by 96 · cited 1×
“Additionally, some applications can translate 2D images into 3D models [225].”
Machine Learning for Health Informatics: State-of-the-Art and Future Challenges
(2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-50478-0 · Cited by 94 · cited 1×
“One very illustrative example of this interactive paradigm for incorporating such non-experts’ knowledge in ML models is healthcare [72].”
Reinforcing User Retention in a Billion Scale Short Video Recommender System
Cai, Q., Liu, S., Wang, X., Zuo, T., Xie, W., Yang, B., Zheng, D., Jiang, P., & Gai, K. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543873.3584640 · Cited by 94 · cited 1×
“Another example is the system created by Cai et al., which employs reinforcement learning (RL) for video recommendations [193].”
Too Many Cooks: Bayesian Inference for Coordinating Multi‐Agent Collaboration
Wu, S. A., Wang, R. E., Evans, J. A., Tenenbaum, J. B., Parkes, D. C., & Kleiman‐Weiner, M. (2021) · Topics in Cognitive Science · doi:10.1111/tops.12525 · Cited by 92 · cited 1×
“The Overcooked game has been used for various research purposes, including studying human collaboration across different task loads in human-AI environments [278], [279].”
Xu, W. M., & Gao, Z. (2024) · interactions · doi:10.1145/3635116 · Cited by 91 · cited 1×
“• Human-AI team: Suggests that humans and AI function not as separate entities but as integrated parts of a cohesive team, working symbiotically towards shared goals [45], [46].”
“All around me are synthetic faces”: The mad world of AI-generated media
Whittaker, L., Kietzmann, T. C., Kietzmann, J., & Dabirian, A. (2020) · IT Professional · doi:10.1109/mitp.2020.2985492 · Cited by 90 · cited 2×
“Legal, moral, and ethical concerns also loom large, encompassing areas such as copyright infringement in AI-generated artworks [185], cheating and plagiarism in educational institutions, data privacy, security [186], and the malicious use of deepfakes [187].”
Human–agent collaboration for disaster response
Ramchurn, S. D., Wu, F., Jiang, W., Fischer, J. E., Reece, S., Roberts, S., Rodden, T., Greenhalgh, C., & Jennings, N. R. (2016) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-015-9286-4 · Cited by 89 · cited 3×
“For example, Ramchurn et al. created a flexible human-AI interaction system for generating task allocation plans in emergent scenarios [207], [211].”
Combiner: Full Attention Transformer with Sparse Computation CostPreprint
Ren, H., Dai, H., Dai, Z., Yang, M., Leskovec, J., Schuurmans, D., & Dai, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.05768 · Cited by 88
Internet of Intelligent Things (IoT) – An Overview
Salama, R., Al-Turjman, F. M., Aeri, M., & Yadav, S. P. (2023) · International Conference on Computational Intelligence, Communication Technology and Networking (CICTN) · doi:10.1109/cictn57981.2023.10141157 · Cited by 87 · cited 1×
“In more recent developments, the concept of the Internet of People (IoP) has shifted its focus towards recognizing users as active contributors within technological ecosystems, rather than passive recipients of data-driven services [2], [9].”
Rezwana, J., & Maher, M. L. (2022) · Creativity and Cognition · doi:10.1145/3527927.3532789 · Cited by 85 · cited 1×
“Therefore, in the context of human-AI collaboration, user engagement also encompasses the level of communication and transparency perceived from the AI system [329].”
The Role of Adaptation in Collective Human–AI Teaming
Zhao, M., Simmons, R. G., & Admoni, H. (2022) · Topics in Cognitive Science · doi:10.1111/tops.12633 · Cited by 84 · cited 1×
“Consequently, adaptability encompasses communication, perception, and an interactive process [290].”
Interactive video stylization using few-shot patch-based training
Texler, O., Futschik, D., Kučera, M., Jamriška, O., Sochorová, Š., Chai, M., Tulyakov, S., & Sýkora, D. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392453 · Cited by 83 · cited 1×
“AIGCs operate similarly in video generation, processing each frame individually and utilizing AI algorithms to generate video clips, which can be employed to create trailers and promotional videos [223], [229].”
Text-to-image Diffusion Models in Generative AI: A Survey
Zhang, C., Zhang, C., Zhang, M., Kweon, I. S., & Kim, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.07909 · Cited by 82 · cited 1×
“Further examples can be found in the survey conducted by Zhang et al. [250].”
The Future of Disaster Response: Humans Working with Multiagent Teams using DEFACTO.
Schurr, N., Marecki, J., Tambe, M., Scerri, P., Kasinadhuni, N., & Lewis, J. P. (2005) · National Conference on Artificial Intelligence · Cited by 81
Closing the Loop: User-Centered Design and Evaluation of a Human-in-the-Loop Topic Modeling System
Smith, A., Kumar, V., Boyd-Graber, J., Seppi, K., & Findlater, L. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172965 · Cited by 79
Riveiro, M., & Thill, S. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103507 · Cited by 79 · cited 1×
“Furthermore, it is crucial for humans to recognize the limitations of AI; otherwise, they may develop unrealistic expectations regarding AI’s capabilities, which can negatively impact their engagement, adaptation, and trust [334], [335].”
HOLZINGER, A., PLASS, M., HOLZINGER, K., CERASELA CRIS¸AN, G., PINTEA, C.-M., & PALADE, V. (2019) · Creative Mathematics and Informatics · doi:10.37193/cmi.2019.02.04 · Cited by 77
Li, H., Wang, Y., & Qu, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642726 · Cited by 77 · cited 1×
“The progression of technology has significantly influenced the concept of collaboration across multiple disciplines over the years [10], [11].”
Bozkurt, A., & Sharma, R. C. (2023) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7755272 · Cited by 77
WavJourney: Compositional Audio Creation With Large Language Models
Liu, X., Zhu, Z., Liu, H., Yuan, Y., Huang, Q., Cui, M., Liang, J., Cao, Y. P., Kong, Q., Plumbley, M. D., et al. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3574867 · Cited by 77
A Survey of Human‐Centered Evaluations in Human‐Centered Machine Learning
Sperrle, F., El‐Assady, M., Guo, G., Borgo, R., Chau, D. H., Endert, A., & Keim, D. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14329 · Cited by 75 · cited 1×
“Thus, as relevant as it is to understand how to build better models, it is to evaluate the human-centred perspective of ML [78] and how to frame this interaction [79].”
Honeycutt, D., Nourani, M., & Ragan, E. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7464 · Cited by 72
Human-Computer Interaction: The Agency Perspective
(2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-25691-2 · Cited by 72
Older Adults’ Use of and Attitudes toward Activity Monitoring Technologies
Fausset, C. B., Mitzner, T. L., Price, C. E., Jones, B. D., Fain, B. W., & Rogers, W. A. (2013) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213571374 · Cited by 70 · cited 1×
“When HAR solutions fail or can not classify users’ behaviors, users may become frustrated, leading to future disengagement or discomfort with AI models and intelligent solutions [92], [93].”
Ostheimer, J., Chowdhury, S., & Iqbal, S. (2021) · Technology in Society · doi:10.1016/j.techsoc.2021.101647 · Cited by 69 · cited 1×
“Understanding the role of trust in engaging humans in collaboration, as well as how trust is influenced during interactions, is crucial for establishing and promoting effective and productive collaborative relationships between humans and AI [308], [309].”
On Natural Language User Profiles for Transparent and Scrutable Recommendation
Radlinski, F., Balog, K., Díaz, F., Dixon, L., & Wedin, B. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531873 · Cited by 69 · cited 1×
“Personalization has been extensively explored in the development of recommender systems [302], [303], [304].”
Definitions of engagement in human-agent interaction
Glas, N., & Pelachaud, C. (2015) · 2015 International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii.2015.7344688 · Cited by 68 · cited 1×
“These concepts of involvement, interest, and attention relate to the idea of engagement, which is crucial in human-agent interaction as it is usually a prerequisite for the objective of the interaction [325].”
Wang, P., & Ding, H. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103732 · Cited by 68 · cited 1×
“This lack of transparency has significantly influenced AI development by increasing human aversion and diminishing user trust [317], [318].”
The Role of Explainability in Collaborative Human-AI Disinformation Detection
Schmitt, V., Villa-Arenas, L. F., Feldhus, N., Meyer, J., Spang, R. P., & Möller, S. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659031 · Cited by 68 · cited 1×
“The concept of explicability and the emphasis on interactive processes stand in contrast to the ‘‘black box’’ nature that AI and machine learning (ML) systems may exhibit when human knowledge is not integrated into their problem-solving processes [212], [315], [316].”
Second Chance for a First Impression? Trust Development in Intelligent System Interaction
Tolmeijer, S., Gadiraju, U., Ghantasala, R., Gupta, A., & Bernstein, A. (2021) · Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3450613.3456817 · Cited by 67 · cited 1×
“Additionally, in the interactive collaboration process, initial interactions play a critical role, as demonstrated by Tolmeijer et al. [310].”
Artificial Intelligence-Based Tools in Research Writing
Adams, D., & Chuah, K.-M. (2022) · Artificial Intelligence in Higher Education · doi:10.1201/9781003184157-9 · Cited by 66
Bio-inspired robotic impedance adaptation for human-robot collaborative tasks
Zeng, C., Yang, C., & Chen, Z. (2020) · Science China Information Sciences · doi:10.1007/s11432-019-2748-x · Cited by 64 · cited 1×
“Although several studies have concentrated on how robots should adapt during collaboration [296], a new paradigm is emerging that seeks to understand how human adaptation evolves through interactive collaboration [293].”
Woodward, M. P., & Finn, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.06559 · Cited by 63 · cited 1×
“Woodward et al. used deep reinforcement learning with Long Short-Term Memory (LSTM) to design a function that determines if a data point label needs to be queried for stream-based active learning [114].”
The Ethical Implications of DALL-E: Opportunities and Challenges
Zhou, K.-Q., & Nabus, H. (2023) · Mesopotamian Journal of Computer Science · doi:10.58496/mjcsc/2023/003 · Cited by 61 · cited 1×
“The most familiar AIGCs are ChatGPT for text creation [150], [151], [152] and DALL-E for creating unique and high-quality images from textual descriptions [153], [154], [155].”
Deep reinforcement learning challenges and opportunities for urban water systems.
Negm, A., Ma, X.-D., & Aggidis, G. (2024) · Water Research · doi:10.1016/j.watres.2024.121145 · Cited by 59 · cited 2×
“However, it also faces significant challenges, such as instability during training and difficulties in accurately defining the environment [139].”
Artificial intelligence generated content (AIGC) in medicine: A narrative review
Shao, L., Chen, B., Zhang, Z., Zhang, Z., & Chen, X. (2024) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2024073 · Cited by 58 · cited 2×
“• Artificial Intelligence Generative Content (AIGC) : Involves AI systems that generate creative outputs like text, images, or music in a process that promotes collaborative learning and user involvement [19], [20], [43], [44].”
A collaborative model of feedback in human-computer interaction
Pérez-Quiñones, M. A., & Sibert, J. L. (1996) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/238386.238535 · Cited by 57
Research works printed but not linked: 23
- Miletić, 2021, Human-artificial intelligence symbiosis: The possibility of moral augmentation
- Holzinger, On knowledge discovery and interactive intelligent visualization of biomedical data, 5
- Tegen, 2022, The Effects of Reluctant and Fallible Users in Interactive Online Machine Learning, 209
- Konyushkova, Learning active learning from data, 30, 4225
- Saadallah, Interpretable meta-active learning for regression ensemble learning, 1
- Ernst, 2024, Introduction to Reinforcement Learning, 111
- Winder, 2020, Reinforcement Learning
- Russell, 2010, Artificial Intelligence a Modern Approach
- Ng, PEGASUS: A policy search method for large MDPs and POMDPs, 406
- Schulman, 2022, ChatGPT: Optimizing language models for dialogue, OpenAI blog, 2
- Lehtinen, Noise2Noise: Learning image restoration without clean data, 4620
- Zhang, The ethics of AI-generated maps: DALL-E 2 and AI’s implications for cartography (short paper)
- Zakershahrak, 2021, Towards human-machine symbiosis: Design for effective AI facilitation
- Yang, AI as an active writer: Interaction strategies with generated text in human-AI collaborative fiction writing, 10, 1
- Cao, Enhancing human-AI collaboration through logic-guided reasoning
- Gómez-Carmona, 2021, Promoting the perception of emerging technologies in work environments through edge computing and hybrid intelligence
- 10.1504/IJBIC.2020.111267 · doi:10.1504/ijbic.2020.111267 · Paper page
- Song, V-MPO: On-policy maximum a posteriori policy optimization for discrete and continuous control
- Wiethof, Toward a hybrid intelligence system in customer service: Collaborative learning of human and AI
- Nikolaidis, 2017, Mathematical models of adaptation in human-robot collaboration
- Paripati, 2024, AI algorithms for personalization: Recommender systems, predictive analytics, and beyond, Darpan Int. Res. Anal., 12, 51
- Scharowski, 2024, Trust in artificial intelligence: Understanding and calibrating trust and distrust in the human-AI interaction
- Van Den Bosch, 2018, Human-AI Cooperation to Benefit Military Decision Making
Cited by
Zhang, R., Liu, G., Liu, Y., Zhao, C., Wang, J., Xu, Y., Niyato, D. T., Kang, J., Li, Y., Mao, S., et al. (2026) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2026.3651702 · Cited by 148
“E.g., tokenization, stop-word removal, NER , VAD, MFCC feature extraction.”
Olawade, D. B., Plabon, S. B., Ojo, A., Ogunbona, M. A., Makanjuola, B. D., & Olasilola, R. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106362 · Cited by 105
Karim, M. M., Khan, S., Van, D. H., Liu, X. L., Wang, C., & Qu, Q. (2025) · Future Internet · doi:10.3390/fi17080353 · Cited by 50
Exploring pre-service teachers’ generative AI readiness and behavioral intentions: A pilot study
Bui, P., Korhonen, T., Kontkanen, S., Karme, S., Piispa-Hakala, S., & Veermans, M. (2025) · LUMAT International Journal on Math Science and Technology Education · doi:10.31129/lumat.13.1.2755 · Cited by 32
Human-Controllable AI: Meaningful Human Control
Liu, C., & Xu, W. (2025) · Springer eBooks · doi:10.1007/978-981-97-8440-0_58-1 · Cited by 26
Metadata
- Authors
- Puerta-Beldarrain, M., Gómez-Carmona, O., Sánchez-Corcuera, R., Casado-Mansilla, D., López-de-Ipiña, D., & Chen, L.
- Affiliations
- Universidad de Deusto (ES)
- Deustotech, University of Deusto, Bilbao, Spain
- Faculty of Engineering, University of Deusto, Bilbao, Spain
- Dalian University of Technology (CN)
- ORCID
- Venue
- IEEE Access
- Published
- 2025
- Volume
- 13
- Pages
- 29375-29405
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4406947263
- Cited by
- 73
- References
- 329
- Citations by year
- 2026: 50 · 2025: 23
- FWCI
- 80.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DeustoTeka (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Basque Government, which recognizes Human-centric Computing for Smart Sustainable Communities and Environments (IT1582-22)
- excellent research group under the Basque university system. Besides, we acknowledge the Ministerio de Ciencia e Innovación for Internet of People (PID2020-119682RB-I00)
- European Commission through the Wearables and drones for City Socio-Environmental Observations and Behavioral Change (101037648)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Keywords
- human-AI collaboration, human-in-the-loop systems, interactive machine learning, hybrid intelligence, human-agent interaction, end-user control, user engagement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.