Constructing Ethical AI Based on the “Human-in-the-Loop” System
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
The Human-in-the-Loop (HITL) system was first proposed by Robert Monarch, a machine learning expert. It adopted a “hybrid” strategy combining human intelligence and machine intelligence, aiming to improve the accuracy of machine learning models and assist human learning. At present, there have been a number ethical design attempts based on the HITL system, and some progress has been made in the ethical choices of disaster rescue robots and nursing robots. However, there is no analysis of why the HITL system can serve as an effective path in constructing ethical AI and how it can implement the efficiency of AI in ethical scenarios. This paper draws on the feasibility of the HITL system and analyzes how ethical AIs are possible when using the HITL system. We advocate for its application to the entire process of ethical AI design.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Chen et al. developed methodologies for creating ethical AI systems based on human-in-the-loop approaches, arguing that value alignment requires ongoing human involvement, not just initial goal specification [69].”
From Human-in-the-Loop Artificial Intelligence: A Systematic Review of Concepts, Methods, and Applications (2026) · cites this paper as [69]
The cited source says what the claim attributes to it.
In the citing paper
“Their model recognizes that human values are complex, situation-dependent, and sometimes internally contradictory, requiring means of human judgment to control system behavior when algorithmic methods are insufficient.”
From Human-in-the-Loop Artificial Intelligence: A Systematic Review of Concepts, Methods, and Applications (2026) · cites this paper as [69]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ethical Decision Making by Individuals in Organizations: An Issue-Contingent Model
Jones, T. M. (1991) · The Academy of Management Review · doi:10.2307/258867 · Cited by 4,135
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., & Rahwan, I. (2018) · Nature · doi:10.1038/s41586-018-0637-6 · Cited by 1,859
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842
Unsupervised Data Augmentation for Consistency Training
Xie, Q., Dai, Z., Hovy, E., Luong, M.-T., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1904.12848 · Cited by 1,517
HiDDeN: Hiding Data With Deep Networks
Zhu, J., Kaplan, R., Johnson, J., & Fei-Fei, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_40 · Cited by 1,129
Greeno, J. G. (1994) · Psychological Review · doi:10.1037//0033-295x.101.2.336 · Cited by 812
A survey of human-in-the-loop for machine learning
Wu, X., Xiao, L., Sun, Y., Zhang, J., Ma, T., & He, L. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.014 · Cited by 800
Wallach, W., & Allen, C. (2009) · Oxford University Press · doi:10.1093/acprof:oso/9780195374049.001.0001 · Cited by 711
Floridi, L. (2013) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199641321.001.0001 · Cited by 651
A survey on active learning and human-in-the-loop deep learning for medical image analysis
Budd, S., Robinson, E. C., & Kainz, B. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102062 · Cited by 606
Research works printed but not linked: 7
- 6. Cai, L.; Wang, S.; Liu, j.; Zhu, Y. Review of Data Annotation Research. J. Softw. 2020, 31, 302-320. [CrossRef]
- 7. Liu, J. Human-in-the-Loop Ethical AI for Care Robots and Confucian Virtue Ethics. In International Conference on Social Robotics; Springer Nature: Cham, Switzerland, 2022.
- 10. Monarch, R.M. Human-In-the-Loop Machine Learning: Active Learning and Annotation for Human-Centered AI; Simon and Schuster: New York, NY, USA, 2021; pp. 31-63.
- 11. Narayanan, S.; Rothrock, L. Human-In-the-Loop Simulations: Methods and Practice; Springer: London, UK; Dordrecht, The Netherlands; Heidelberg, Germany; New York, NY, USA, 2011; p. 16.
- 12. Jo, A. The promise and peril of generative AI. Nature 2023, 614, 214-216.
- 14. Thiam, P.; Meudt, S.; Schwenker, F.; Palm, G. Active learning for speech event detection in HCI. In Artificial Neural Networks in Pattern Recognition: 7th IAPR TC3 Workshop, ANNPR 2016, Ulm, Germany, September 28-30, 2016, Proceedings 7; Springer International Publishing: Cham, Switzerland, 2016.
- 21. Chai, C.; Li, G. Human-in-the-loop Techniques in Machine Learning. IEEE Data Eng. Bull. 2020, 43, 37-52.
Cited by
Applications, Challenges, and Future Directions of Human-in-the-Loop Learning
Kumar, S., Datta, S., Singh, V., Datta, D., Kumar Singh, S., & Sharma, R. (2024) · IEEE Access · doi:10.1109/access.2024.3401547 · Cited by 168
“Restrictions apply. human and machine intelligence throughout the entire ethical AI design process [166].”
BackgroundLazaros, K., Vrahatis, A. G., & Kotsiantis, S. (2026) · Entropy · doi:10.3390/e28040377 · Cited by 50
Empowering Student Learning Through Artificial Intelligence: A Bibliometric Analysis
Yun, G., Lee, K. M., & Choi, H. H. (2024) · Journal of Educational Computing Research · doi:10.1177/07356331241278636 · Cited by 36
“Another focus of study under this cluster lies in evaluating the effectiveness and competitiveness of the prediction or evaluation models (Jamil & Hameed, 2023; Ni et al., 2023; Rivas-Posada & Chacon-Murguia, 2024; Wang, 2023).”
InfluentialJenkins, D. M., & Khanna, G. (2025) · Journal of Leadership Studies · doi:10.1002/jls.70004 · Cited by 37
Nolin-Lapalme, A., Corbin, D., Tastet, O., Avram, R., & Hussin, J. (2024) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2024.04.026 · Cited by 21
Ethical considerations in AI-based user profiling for knowledge management: A critical review
Njiru, D. K., Mugo, D. M., & Musyoka, F. M. (2025) · Telematics and Informatics Reports · doi:10.1016/j.teler.2025.100205 · Cited by 20
Challenges and opportunities in using interpretable AI to develop relationship interventions
Puhlman, D. J., & Chen, C. (2025) · Family Relations · doi:10.1111/fare.13172 · Cited by 10
Behavioral Impacts of AI Reliance in Diagnostics: Balancing Automation with Skill Retention
Sunday, O. (2025) · Epidemiology and Health Data Insights · doi:10.63946/ehdi/16894 · Cited by 10
“The model emphasizes the interaction be-tween AI analysis, human interpretation, and continuous feedback, thereby ensuring the retention of skills and the accuracy of the analysis (40).”
Towards ethical evolution: responsible autonomy of artificial intelligence across generations
Terziyan, V. Y., Tiihonen, T., Shukla, A. K., Gryshko, S., Golovianko, M., Terziyan, O., & Vitko, O. (2025) · AI and Ethics · doi:10.1007/s43681-025-00759-9 · Cited by 19
Human digital twin for optimizing labor productivity in construction 5.0
Ohueri, C. C. (2026) · Automation in Construction · doi:10.1016/j.autcon.2026.106800 · Cited by 8
Liu, W., He, R., Liu, Y., Jiang, S., Fang, J., Zhu, X., Wu, L., & Lu, F. (2025) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2025.124458 · Cited by 4
Human–AI Collaboration in Creativity, Design, and Innovation
Putri, A. A., & Sekti, B. A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7153-5.ch009 · Cited by 3
Jiang, L., Wu, W., & Hao, W. (2026) · Sustainability · doi:10.3390/su18052380 · Cited by 3
Advances in Closed-Loop Artificial Intelligence for Healthcare
Das, D., Adams, S. D., Corva, D. M., Bucknall, T. K., & Kouzani, A. Z. (2026) · Electronics · doi:10.3390/electronics15071396 · Cited by 3
AI-Powered Surveillance vs. Privacy Rights: Striking the Right Balance
Bodimani, M. (2025) · International Journal For Multidisciplinary Research · doi:10.36948/ijfmr.2025.v07i02.42672 · Cited by 3
Ensuring AI Safety in Autonomous Vehicles: A Framework Based on ISO PAS 8800
Thomas, J. (2025) · doi:10.38124/ijisrt/25apr1584 · Cited by 1
“HITL reinforces public assurance that human responsibility has not been delegated entirely to automated agents [74].”
InfluentialThe Integration of Ethical and Trustworthy AI in Industrial Engineering: Practical Approaches
Salvatore, S. D., Yahyaoui, O. D., Marchi, M. D., & Rauch, E. (2025) · International Conference on Smart Management in Industrial and Logistics Engineering (SMILE) · doi:10.3390/engproc2025097042 · Cited by 2
Kelters, B. (2026) · AI & Society · doi:10.1007/s00146-026-03055-0 · Cited by 1
“It also isn’t clear that unacceptable SAD outcomes can be prevented by limiting AI empowerment through imposing ‘human-in-the-loop’ architectures (Chen et al. 2023) on AI systems, thereby giving humans’ power over specific decisions rather than entire systems.”
Rifaat, S., AlNassar, A., AlQuraishi, A., Alqahtani, N. A., Wafai, T., Farooqi, F. A., Gaffar, B. O., & Taymour, N. (2026) · Frontiers in Oral Health · doi:10.3389/froh.2026.1794097 · Cited by 2
“This finding aligns with the “human-in-the-loop” approach advocated by many AI ethicists, where AI serves as a decision support tool rather than an autonomous decision-maker (28).”
Soltani, A., Bafarasat, A. Z., Bibri, S. E., Ramezani, L., & Qadikolaei, M. R. (2026) · Progress in Planning · doi:10.1016/j.progress.2026.101043 · Cited by 1
Exploring a Human-in-the-Loop Framework for Adaptive Sign Language Translation in Deaf Education
Darmawan, I. D. M. B. A., Linawati, L., Sukadarmika, G., Wirastuti, N. M. A. E. D., Pulungan, R., & Ardiana, D. P. Y. (2025) · International Conference on Engineering of Modern Electric Systems · doi:10.1109/emes65692.2025.11045598
Zalando's Use of AI to Create a Personalized Shopping Experience
Mahdi, M. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-6582-4.ch006
Human-in-the-Loop Reinforcement Learning for Ethical and Compliant Risk Decisions
Swidan, A. (2025) · International Journal of Scientific Research in Science Engineering and Technology · doi:10.32628/ijsrset2513857 · Cited by 4
Accounting Reporting 5.0: Technological Advancements and Educational Challenges
Tavares, M. C., Machado, A. C., & Pinto, M. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-10728-2_13 · Cited by 1
Augmented Creativity Uniting Human Intuition and Artificial Intelligence in Design
Shome, S., Bhaumik, P., & Das, S. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7153-5.ch003
Qi, R., Papyshev, G., Zhang, J., Tsai, K. S., Chan, A., & Hsiao, J. H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6251014
Ebo, T. O., Osunmakinde, A., Asaolu, A. J., Ebo, D. M., Egbon, E., & Olawade, D. B. (2026) · AI & Society · doi:10.1007/s00146-026-02930-0 · Cited by 1
Ramdani, M., Ulumuddin, A. M., & Sekti, B. A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5525-2.ch002
Ethical and legal consideration associated with AI use
Priyadarshini, A., Dahiya, D., Baby, Rani, R., & Choudhary, R. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-34423-7.00009-2
Leveraging AI Technologies for Enhanced Project Management in Industry 4.0 Construction
Qazi, K. (2026) · Global Conference on Wireless and Optical Technologies (GCWOT) · doi:10.1109/gcwot69191.2026.11499661
“This integration of statistical insights, visual interpretations, and thematic relationships reveals key themes and their interconnections [17].”
Why educated users still click: a review on phishing and the limits of human judgment
Gagniuc, P. A. (2026) · Computer Science Review · doi:10.1016/j.cosrev.2026.101025 · Cited by 1
Güngör, A., & Güngör, B. Ş. (2026) · İçtimaiyat · doi:10.33709/ictimaiyat.1820836 · Cited by 1
Human-in-the-Loop Framework for Hallucination Mitigation: A Controlled Comparison of GPT and LLaMA
Kai, J. N. J., Syahidi, A. A., & Kiong, K. B. (2026) · International Conference on Business and Industrial Research (ICBIR) · doi:10.1109/icbir69932.2026.11584794
“In contrast, many practical systems can only support lightweight Human-in-the-Loop (HITL) feedback, such as binary correctness judgments or short corrective annotations provided opportunistically during use [20], [21].”
Application of Artificial Intelligence for Sustainable Biomedical Waste Management
Joshi, N., Chauhan, D., & Kumar, S. N. K. S. N. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-27563-9_12
If AI Can Do It, It Should? Task Delegation Preferences of Future Professionals
Bangerl, M. M., Disch, L., Isailovic, E., & Pammer‐Schindler, V. (2026) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3805029.3818284
“At the same time, many professional tasks require contextual judgment, accountability, and opportunities for humans to intervene [6].”
PromoCraft+: a human-in-the-loop framework for auditing AI-generated insurance promotions
Saxena, A. (2026) · Marketing Intelligence & Planning · doi:10.1108/mip-03-2025-0273
Singh, A., Pandey, S., & Sharma, A. (2026) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-80686-813-120261014
Somarajpalli, B. C. (2026) · Universal library of engineering technology. · doi:10.70315/uloap.ulete.2026.0303003
“Scoped, high-value AI with human oversight wins in serious enterprises, and autonomy can expand from that base once metrics and guardrails are in place, since the case for keeping human judgment inside the loop rests on accountability and on the of automated decisions (Chen et al., 2023).”
de Wet, M., & Bohnsack, R. (2026) · Business Strategy and the Environment · doi:10.1002/bse.71395
Baylor Environmental AI Research System (BEARS): An Agentic AI Project to Combat Climate Change
Orojo, A., Khanal, B., El-Mahmoud, E., Yu, J., Rashid, M. B., Quansah, P. K., Tella, S. M., Ding, T. K., Fang, X., Rivas, P., et al. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-22190-2_19
Tian, J., Shen, X., Li, N., Wu, C., Hu, Y., Teng, M., & Tan, M. (2026) · Nurse Education Today · doi:10.1016/j.nedt.2026.107363
Pap, L. (2026) · International Symposium Engineering Management and Competitiveness (EMC) · doi:10.5937/emc26151p
Digital Saturation and Psychological Evolution: Interdisciplinary Insights Into the AI Generation
Priya, S., & Santhoshkumar, S. P. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9109-0.ch002
Detherage, R., & Connelly, S. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1841079
Metadata
- Authors
- Chen, X., Wang, X., & Qu, Y.
- Affiliations
- Xi'an Jiaotong University (CN)
- Corresponding author
- Chen, Ximeng
- Venue
- Systems
- Publisher
- MDPI AG
- Published
- 2023-11-13
- Volume
- 11
- Issue
- 11
- Pages
- 548
- ISSN
- 2079-8954 (ISSN-L)
- OpenAlex
- W4388628811
- Cited by
- 45
- References
- 25
- Citations by year
- 2026: 30 · 2025: 12 · 2024: 3
- FWCI
- 11.26
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Systems (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Soft Science Research Plan of Shaanxi Province (2023-CX-RKX-140, SK2022047)
- 2022 Basic Scientific Research Project of Xi’an Jiaotong University (2023-CX-RKX-140, SK2022047)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- human-in-the-loop, ethical AI, AI ethics, human-machine collaboration, disaster rescue robots, nursing robots
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.