Applications, Challenges, and Future Directions of Human-in-the-Loop Learning
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
Machine learning (ML) has become a popular technique for various automation tasks in the era of Industry 4.0, such as the analysis and synthesis of visual data such as images and videos, natural language and speech, financial data, and biomedical applications. However, ML-based automation techniques are facing difficulties like decision-making, thus incorporating user expertise into the system might be advantageous. The goal of adding human domain expertise with ML-based automation is to provide more accurate prediction models. Human-in-the-loop (HITL) systems that integrate human expertise with ML algorithms are becoming more and more common in various industries. However, there are a number of methodological, technical, and ethical difficulties with the development and application of HITL systems. This paper aims to explore the methodologies, challenges, and opportunities associated with HITL systems implementations.We also discuss a number of issues that must be resolved for HITL systems to be effective, including data quality, bias, and user engagement. Besides, we also explored several approaches that can be utilized to enhance the performance of HITL systems, such as active learning (AL), iterative ML, and reinforcement learning, as well as the current state of the art in HITL systems.We also selectively highlighted the advantages of HITL systems, such as their potential to increase decision-making process accountability and transparency by utilizing human experience to improve ML decision-making capability. The paper will be very useful for researchers, practitioners, and policymakers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
DeepWalk: online learning of social representations
Perozzi, B., Al-Rfou, R., & Skiena, S. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623732 · Cited by 10,123
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Self-citationEvaluating the quality of medical care
DONABEDIAN, A. (2005) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2005.00397.x · Cited by 6,102 · cited 1×
“These criteria should be defined in collaboration with domain experts and should be based on measurable outcomes [165].”
BackgroundSimple online and realtime tracking with a deep association metric
Wojke, N., Bewley, A., & Paulus, D. (2017) · 2017 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2017.8296962 · Cited by 5,286
Image segmentation using deep learning: A survey
Minaee, S., Boykov, Y. Y., Porikli, F., Plaza, A. J., Kehtarnavaz, N., & Terzopoulos, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059968 · Cited by 3,796
Object Detection in 20 Years: A Survey
Zou, Z., Chen, K., Shi, Z., Guo, Y., & Ye, J. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3238524 · Cited by 3,689 · cited 1×
“In recent times, it has garnered substantial attention [71].”
BackgroundImage Inpainting for Irregular Holes Using Partial Convolutions
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_6 · Cited by 2,419
European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 1×
“4, which can be critical in domains where trust, accountability, and human oversight are important, such as medicine, law, and defense [51].”
BackgroundA survey on deep learning and its applications
Dong, S., Wang, P., & Abbas, K. (2021) · Computer Science Review · doi:10.1016/j.cosrev.2021.100379 · Cited by 1,800
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2017) · arXiv · doi:10.48550/arxiv.1711.11279 · Cited by 1,240
A Survey of Deep Active Learning
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Gupta, B. B., Chen, X., & Wang, X. (2022) · ACM Computing Surveys · doi:10.1145/3472291 · Cited by 1,169
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · cited 1×
“Reinforcement learning (RL) is a type of ML that enables an agent to learn from its own interactions with an environment by receiving feedback in the form of rewards or penalties [42], [43], [44], [45].”
BackgroundInteractive machine learning for health informatics: when do we need the human-in-the-loop?
Holzinger, A. (2016) · Brain Informatics · doi:10.1007/s40708-016-0042-6 · Cited by 906
A survey of human-in-the-loop for machine learning
Wu, X., Xiao, L., Sun, Y., Zhang, J., Ma, T., & He, L. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.014 · Cited by 800 · cited 1×
“The paper discusses meth-ods to improve model performance through data processing, intervention-based model training, and the configuration of system-independent ‘‘human-in-the-loop’’ setups [23], [24], [25], [26], [27].”
BackgroundLearning Loss for Active Learning
Yoo, D., & Kweon, I. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00018 · Cited by 740
The Power of Ensembles for Active Learning in Image Classification
Beluch, W. H., Genewein, T., Nurnberger, A., & Kohler, J. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00976 · Cited by 655
Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614 · cited 1×
“Beede et al. [104], conducted a human-centered study of a DL system used in clinics for the detection of diabetic eye disease.”
BackgroundCai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594
Learning to summarize from human feedback
Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., & Christiano, P. (2020) · arXiv · doi:10.48550/arxiv.2009.01325 · Cited by 550
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5285 · Cited by 466
Fine-Tuning Language Models from Human Preferences
Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., & Irving, G. (2019) · arXiv · doi:10.48550/arxiv.1909.08593 · Cited by 413
Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377 · cited 4×
“This could involve developing new approaches for designing fair and transparent ML models or investigating the legal and social implications of using ML in decision-making contexts [100].”
BackgroundMethodInfluentialCharacterizing the influence of domain expertise on web search behavior
White, R. W., Dumais, S. T., & Teevan, J. (2009) · Proceedings of the Second ACM International Conference on Web Search and Data Mining · doi:10.1145/1498759.1498819 · Cited by 312
A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
De-Arteaga, M., Fogliato, R., & Chouldechova, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376638 · Cited by 221 · cited 1×
“One of the most significant challenges of HITL is the involvement of humans in the decision-making process [27], [144], [145], [146], [147].”
BackgroundHuman factors in model interpretability: Industry practices, challenges, and needs
Hong, S. R., Hullman, J., & Bertini, E. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392878 · Cited by 219 · cited 1×
“Model interpretability refers to the ability to understand how a machine-learning model arrives at a decision [156], [157].”
BackgroundWu, J., Huang, Z., Hu, Z., & Lv, C. (2023) · Engineering · doi:10.1016/j.eng.2022.05.017 · Cited by 205 · cited 1×
“Wu et al. [61] present an innovative approach involving a HITL deep reinforcement learning (DRL) framework.”
BackgroundBanking on AI: mandating a proactive approach to AI regulation in the financial sector
Truby, J., Brown, R., & Dahdal, A. (2020) · Law and Financial Markets Review · doi:10.1080/17521440.2020.1760454 · Cited by 175 · cited 1×
“Truby et al. [143] suggested that to promote a sustainable future in AI innovation within the financial sector, it’s crucial to adopt a proactive regulatory approach that emphasizes preventive measures rather than reactive ones.”
Learning from Dialogue after Deployment: Feed Yourself, Chatbot!
Hancock, B., Bordes, A., Mazare, P.-E., & Weston, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1358 · Cited by 161
Weak Supervision for Fake News Detection via Reinforcement Learning
Wang, Y., Yang, W., Ma, F., Xu, J., Zhong, B., Deng, Q., & Gao, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5389 · Cited by 157
Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure
Nushi, B., Kamar, E., & Horvitz, E. (2018) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v6i1.13337 · Cited by 126 · cited 1×
“However, these metrics offer limited insight into the rationale behind erroneous predictions [118].”
BackgroundRams, hounds and white boxes: investigating human–AI collaboration protocols in medical diagnosis
Cabitza, F., Campagner, A., Ronzio, L., Cameli, M., Mandoli, G. E., Pastore, M. C., Sconfienza, L. M., Folgado, D., Barandas, M., & Gamboa, H. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102506 · Cited by 117 · cited 1×
“Cabitza et al. [109] scrutinize human-AI collaboration protocols, a design construct focused on evaluating how humans and AI harmoniously collaborate in cognitive tasks.”
BackgroundOpen Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15217 · Cited by 101 · cited 1×
“Reinforcement learning (RL) is a type of ML that enables an agent to learn from its own interactions with an environment by receiving feedback in the form of rewards or penalties [42], [43], [44], [45].”
BackgroundDialogue Learning With Human-In-The-Loop
Li, J., Miller, A. H., Chopra, S., Ranzato, M., & Weston, J. (2016) · arXiv · doi:10.48550/arxiv.1611.09823 · Cited by 90
Huang, Z., Sheng, Z., Ma, C., & Chen, S. (2024) · Communications in Transportation Research · doi:10.1016/j.commtr.2024.100127 · Cited by 88 · cited 1×
“Human as AI mentor-based deep reinforcement learning (HAIM-DRL) [63] framework for mixed traffic platoons shows promise for advancing safe and efficient autonomous driving by incorporating human expertise into the learning process.”
BackgroundHuman-centred mechanism design with Democratic AI
Koster, R., Balaguer, J., Tacchetti, A., Weinstein, A., Zhu, T., Hauser, O., Williams, D., Campbell-Gillingham, L., Thacker, P., Botvinick, M., et al. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01383-x · Cited by 84 · cited 1×
“Future research could explore these issues in greater depth, as well as investigate the potential benefits and risks of using AI systems in place of human decision-makers [200].”
BackgroundConstructing Ethical AI Based on the “Human-in-the-Loop” System
Chen, X., Wang, X., & Qu, Y. (2023) · Systems · doi:10.3390/systems11110548 · Cited by 45 · cited 1×
“Restrictions apply. human and machine intelligence throughout the entire ethical AI design process [166].”
BackgroundA Survey of Reinforcement Learning from Human Feedback
Kaufmann, T., Weng, P., Bengs, V., & Hüllermeier, E. (2023) · arXiv · doi:10.48550/arxiv.2312.14925 · Cited by 43
Models of human preference for learning reward functions
Knox, W. B., Hatgis-Kessell, S., Booth, S., Niekum, S., Stone, P., & Allievi, A. (2022) · arXiv · doi:10.48550/arxiv.2206.02231 · Cited by 8 · cited 1×
“Knox et al. [50] introduce a new ‘‘regret preference modeling’’ approach for learning reward functions in reinforcement learning from human feedback.”
BackgroundNano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control
Fan, X., Lyu, Y., Liang, P. P., Salakhutdinov, R., & Morency, L.-P. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.758 · Cited by 3
Cited by
Mienye, I. D., Obaido, G., Jere, N., Mienye, E., Aruleba, K., Emmanuel, I. D., & Ogbuokiri, B. (2024) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2024.101587 · Cited by 212
Chow, J. C. L., & Li, K. (2025) · Information · doi:10.3390/info16070549 · Cited by 84
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 105
“Recent advances have highlighted the potential of human-in-the-loop trust modeling [337] and ethical instruction tuning [19] to improve the controllability and societal acceptability of robot behavior.”
VeriPlan: Integrating Formal Verification and LLMs into End-User Planning
Lee, C. P., Porfirio, D., Wang, X. J., Zhao, K. C., & Mutlu, B. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714113 · Cited by 21
“Existing research shows that user input on preferences and domain knowledge during initial interactions is important to effectively guide system behavior [46].”
Designing an AI Chatbot for Team-Based Diabetes Care: An Iterative Human-in-the-Loop Approach
Tseng, Y.-C., Chen, S., Mah, K.-H., & Chen, Y.-C. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-93736-1_17 · Cited by 5
Metadata
- Authors
- Kumar, S., Datta, S., Singh, V., Datta, D., Kumar Singh, S., & Sharma, R.
- Venue
- IEEE Access
- Published
- 2024
- Volume
- 12
- Pages
- 75735-75760
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4396941516
- Cited by
- 168
- References
- 44
- FWCI
- 25.00
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.