Cited by 382Ethics and Social Impacts of AILaw, AI, and Intellectual PropertyAI and HR Technologies

Viewpoint: Human-in-the-loop Artificial Intelligence

Open access
Zanzotto, Fabio Massimo·2019·Journal of Artificial Intelligence Research

Abstract

Little by little, newspapers are revealing the bright future that Artificial Intelligence (AI) is building. Intelligent machines will help everywhere. However, this bright future may have a possible dark side: a dramatic job market contraction before its unpredictable transformation. Hence, in a near future, large numbers of job seekers may need financial support while catching up with these novel unpredictable jobs. This possible job market crisis has an antidote inside. In fact, the rise of AI is sustained by the biggest knowledge theft of the recent years. Many learning AI machines are extracting knowledge from unaware skilled or unskilled workers by analyzing their interactions. By passionately doing their jobs, many of these workers are shooting themselves in the feet. In this paper, we propose Human-in-the-loop Artificial Intelligence (HitAI) as a fairer paradigm for AI systems. Recognizing that any AI system has humans in the loop, HitAI will reward these aware and unaware knowledge producers with a different scheme: decisions of AI systems generating revenues will repay the legitimate owners of the knowledge used for taking those decisions. As modern Merry Men, HitAI researchers should fight for a fairer Robin Hood Artificial Intelligence that gives back what it steals. This article is part of the special track on AI and Society.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1613/jair.1.11345

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The standard response to new issues of legitimacy and control in ADM has been to reinsert human agency 'in-the-loop'.

    “The standard response to new issues of legitimacy and control in ADM has been to reinsert human agency 'in-­ the-­ loop' (Enarsson et  al.  2022; Zanzotto  2019).”

    From Algorithmic Governance: Experimental Evidence on Citizens' and Public Administrators' Legitimacy Perceptions of Automated Decision‐Making (2026) · cites this paper as Zanzotto 2019

    Section 2.2 describes leaving the final decision to human professionals as one form of human-in-the-loop AI. It says this form is confined to specialized workers; the Introduction instead proposes a novel approach focused on crediting knowledge producers. The paper does not establish that reinserting human decision-makers is the standard response to legitimacy and control issues in automated decision-making. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • Another advantage of using CBR is its interactive nature, which keeps human experts in the loop, who therefore remain responsible for the decision based on the results of the CBR application.

    “Another advantage of using CBR is its interactive nature, which keeps human experts in the loop [24], who therefore remain responsible for the decision based on the results of the CBR application.”

    From Case-Based Data Quality Management for IoT Logs: A Case Study Focusing on Detection of Data Quality Issues (2025) · cites this paper as [24]

    Section 2.2 says explainable AI can keep specialized professionals in the loop by giving them the last word when they use machines as advisers. It does not identify case-based reasoning (CBR) or establish that CBR’s interactive nature produces this arrangement; no page number was provided in our copy. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 15

  1. Aha, D. W., Darrell, T., Pazzani, M., Reid, D., Sammut, C., & Stone, P. (Eds.). (2017). Proceedings of the IJCAI Workshop on Explainable Artificial Intelligence (XAI).
  2. Beccaceci, R., Fallucchi, F., Giannone, C., Spagnoulo, F., & Zanzotto, F. M. (2009). Ed- ucation with '"Living Artworks" in Museums. In Proceedings of the International Conference on Computer Supported Education (CSEDU), pp. 346-349.
  3. Briot, J.-P., Hadjeres, G., & Pachet, F. (2017). Deep Learning Techniques for Music Generation-A Survey. arXiv preprint arXiv:1709.01620. · arXiv
  4. Camp, J. (2004). Digital Identity. IEEE Technology and society Magazine, 23 (3), 34-41.
  5. European Parliament and European Council (2016). Regulation (EU) 2016/679 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union, L119, 1-88.
  6. Ferroni, P., Zanzotto, F. M., Scarpato, N., Riondino, S., Nanni, U., Roselli, M., & Guadagni, F. (2017). Risk Assessment for Venous Thromboembolism in Chemotherapy-Treated Ambulatory Cancer Patients: A Machine Learning Approach. Medical Decision Mak- ing, 37 (2), 234-242.
  7. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative Adversarial Nets. In Advances in Neural Infor- mation Processing Systems, pp. 2672-2680.
  8. Haykin, S. (1998). Neural Networks: A Comprehensive Foundation (2nd edition). Prentice Hall PTR, Upper Saddle River, NJ, USA.
  9. Kennedy, J. F. (1961). Inaugural Address of the President of United States.. Washington, D.C. Kerly, A., Hall, P., & Bull, S. (2007). Bringing Chatbots into Education: Towards Natural Language Negotiation of Open Learner Modelsmodels. Knowledge-Based Systems, 20 (2), 177-185.
  10. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep Learning. Nature, 521 (7553), 436-444.
  11. Plate, T. A. (1995). Holographic Reduced Representations. IEEE Transactions on Neural Networks, 6 (3), 623-641.
  12. Morgan Kaufmann, San Mateo.
  13. Ulrich, I., Mondada, F., & Nicoud, J.-D. (1997). Autonomous Vacuum Cleaner. Robotics and Autonomous Systems, 19 (3-4), 233-245.
  14. Vinyals, O., Kaiser, L. u., Koo, T., Petrov, S., Sutskever, I., & Hinton, G. (2015a). Grammar as a Foreign Language. In Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M., & Garnett, R. (Eds.), Advances in Neural Information Processing Systems 28, Vol. 2, pp. 2773-2781.
  15. Wallace, R. S. (2009). The Anatomy of A.L.I.C.E., pp. 181-210. Springer Netherlands, Dordrecht.

Cited by

Sorted by citations

Authors
Zanzotto, F. M.
Affiliations
University of Rome Tor Vergata (IT)
Corresponding author
Zanzotto, Fabio Massimo
Venue
Journal of Artificial Intelligence Research
Published
2019-02-10
Volume
64
Pages
243-252
ISSN
1076-9757 (ISSN-L)
OpenAlex
W2766005220
Cited by
380
References
29
Citations by year
2026: 53 · 2025: 60 · 2024: 68 · 2023: 59 · 2022: 55 · 2021: 57 · 2020: 19 · 2019: 9
FWCI
61.18
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Artificial Intelligence Research (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • Cineca Institutional Research Information System (Tor Vergata University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
  • Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
Keywords
knowledge ownership, algorithmic fairness, AI and society

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review