The DOI matches its Crossref record.
Human-in-the-Loop AI: Rethinking Automation Ethics in Decision-Sensitive Domains Case Study of the Education, IT and Non-for-Profit sectors
Open accessAbstract
This study develops and applies the Human-in-the-Loop (HITL) Ethical Assessment Framework (EHAF) to examine the ethical sufficiency of HITL artificial intelligence (AI) across education, information technology (IT), and non-profit sectors. The research objective was to evaluate how effectively HITL practices safeguard human values in decision-sensitive contexts and to identify sector-specific challenges that may compromise ethical adequacy. Adopting a qualitative thematic approach, we analyzed survey responses from professionals in the three sectors. Responses were coded against the four diagnostic dimensions of EHAF Impact Severity, Contextual Ambiguity, Human Agency, and Transparency & Auditing while also allowing for the identification of emergent themes. Retroductive reasoning was used to move beyond surface patterns to uncover generative mechanisms shaping HITL practices. Findings demonstrate sectoral variation in how HITL systems are operationalized and valued. In education, ethical sufficiency is closely tied to human oversight given the high stakes of student outcomes and the importance of cultural contextualization. In the non-profit sector, transparency and auditing dominate due to donor accountability pressures and reporting requirements. IT organizations, by contrast, privilege efficiency and scalability, but often provide weaker safeguards for human agency and oversight. Across all sectors, emergent themes such as training, trust, infrastructure readiness, and donor influence were found to condition HITL adequacy. Generative mechanisms identified include institutional role ambiguity, donor pressure, cultural misalignment, and capacity constraints. The study concludes by proposing an extension to EHAF that incorporates a fifth dimension, Capacity and Governance Context to better capture systemic and institutional influences. Conceptually, the paper refines the assessment of HITL ethics, while practically offering sector-specific recommendations to strengthen oversight, accountability, and trust in AI-enabled decision-making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A coherent HITL bias-mitigation strategy therefore requires dual-level evaluation: case-level review quality and outcome-level disparity monitoring over time, with escalation triggers when the two diverge [68].”
From Human-in-the-Loop Artificial Intelligence: A Systematic Review of Concepts, Methods, and Applications (2026) · cites this paper as [68]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Relying solely on human oversight to review decisions made on a case-by-case basis may not be sufficient to point to biases that only become apparent when outcomes are reviewed on a wide range of decisions.”
From Human-in-the-Loop Artificial Intelligence: A Systematic Review of Concepts, Methods, and Applications (2026) · cites this paper as [68]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The evaluation of HITL systems requires a coherent framework that treats performance as a joint property of humans, AI models, interfaces, and organizational context rather than as model accuracy alone.”
From Human-in-the-Loop Artificial Intelligence: A Systematic Review of Concepts, Methods, and Applications (2026) · cites this paper as [68]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“They found that while biases are small individually, their aggregated effect can be substantial [68].”
From Human-in-the-Loop Artificial Intelligence: A Systematic Review of Concepts, Methods, and Applications (2026) · cites this paper as [68]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Still being checked.
Research works printed but not linked: 14
- [1]1. Berente, N., et al., Managing Artificial Intelligence. MIS Quarterly, 2021. 45(3): p. 1433-1450.
- [2]2. Shrestha, Y.R., S.M. Ben-Menahem, and G. von Krogh, Organizational Decision-Making Structures in the Age of Artificial Intelligence. California Management Review, 2019. 61(4): p. 66-83.
- [3]3. Wamba, S.F., Akter, S., & Guthrie, C., Making big data analytics perform: the mediating effect of big data analytics dependent organizational agility. . Systèmes d'information & management, , 2020. 25(2), 7-31.
- [4]4. Lyytinen, K., J.V. Nickerson, and J.L. King, Metahuman systems = humans + machines that learn. Journal of Information Technology, 2020. 36(4): p. 427-445.
- [5]5. Jarvenpaa, S.L., Sourcing data for data-driven applications: foundational questions. In Research handbook on artificial intelligence and decision making in organizations. Edward Elgar Publishing., 2024: p. (pp. 17-37). .
- [6]6. Owoc, M.L., Sawicka, A., & Weichbroth, P. , Artificial intelligence technologies in education: benefits, challenges and strategies of implementation. In IFIP international workshop on artificial intelligence for knowledge management. 2019: p. (pp. 37-58).
- [8]8. Amershi, S., et al., Guidelines for Human-AI Interaction, in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 2019. p. 1-13.
- [9]9. Goddard, K., Roudsari, A., & Wyatt, J. C. (2012). , Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association, , 2012. 19(1), : p. 121-127.
- [10]10. Abdel-Karim, B., et al., How AI-Based Systems Can Induce Reflections: The Case of AI-Augmented Diagnostic Work. MIS Quarterly, 2023. 47(4): p. 1395-1424.
- [11]11. Teodorescu, M.H.M., et al., Failures of Fairness in Automation Require a Deeper Understanding of Human-ML Augmentation. MIS Quarterly, 2021. 45(3): p. 1483-1500.
- [12]12. Yin, R.K., Case study research and applications. Thousand Oaks, CA: Sage., 2018. (Vol. 6). .
- [17]17. Doshi-Velez, F., & Kim, B., Towards a rigorous science of interpretable machine learning. . arXiv preprint arXiv:, 2017. 1702.08608.
- [18]18. Jiang, N., Liu, X., Liu, H., Lim, E. T. K., Tan, C. W., & Gu, J., Beyond AI-powered context-aware services: the role of human-AI collaboration. Industrial Management & Data Systems, , 2023. 123(11), 2771-2802.
- [20]20. Rahwan, I., Society-in-the-loop: programming the algorithmic social contract. . Ethics and information technology, , 2018. 20(1), 5-14.
Cited by
Sorted by citations
Lazaros, K., Vrahatis, A. G., & Kotsiantis, S. (2026) · Entropy · doi:10.3390/e28040377 · Cited by 50
Ravikumar, R. N., Turayevich, J. I., Jumanazarov, J., Kholmatova, A., Aarthi, S., & Nazirjon, R. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7966-1.ch001
Blockchain-Enabled AIoT Systems for Pollution Source Identification and Environmental Monitoring
Jani, S., Garg, A. G. A., Raghuwanshi, S., & Ul‐Durar, S. (2026) · Studies in systems, decision and control · doi:10.1007/978-3-032-32770-3_12
Metadata
- Authors
- Joseph, N. S., & Yakubu, M. N.
- Affiliations
- Arden University (GB)
- Venue
- International Journal of Research and Innovation in Applied Science
- Publisher
- RSIS International
- Published
- 2025-10-30
- Volume
- 10
- Issue
- 10
- Pages
- 361-372
- ISSN
- 2454-6194 (ISSN-L)
- OpenAlex
- W4415741572
- Cited by
- 3
- References
- 0
- Citations by year
- 2026: 3
- FWCI
- 3.10
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Journal of Research and Innovation in Applied Science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- human-in-the-loop AI, AI ethics, ethics of automation, human oversight, human agency, education sector, IT industry, third sector, donor accountability, trust in AI, AI decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
