Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 33

The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.

Beyond human-in-the-loop: Sensemaking between artificial intelligence and human intelligence collaboration

Hao, Xinyue, Demir, Emrah, Eyers, Daniel · 2025 · Sustainable Futures

Abstract

In contemporary operational environments, decision-making is increasingly shaped by the interaction between intuitive, fast-acting System 1 processes and slow, analytical System 2 reasoning. Human intelligence (HI) navigates fluidly between these cognitive modes, enabling adaptive responses to both structured and ambiguous situations. In parallel, artificial intelligence (AI) has rapidly evolved to support tasks typically associated with System 2 reasoning, such as optimization, forecasting, and rule-based analysis, with speed and precision that in certain structured contexts can exceed human capabilities. To investigate how AI and HI collaborate in practice, we conducted 28 in-depth interviews across 9 leading firms recognized as benchmarks in AI adoption within operations and supply chain management (OSCM). These interviews targeted key HI agents, operations managers, data scientists, and algorithm engineers, and were situated within carefully selected, AI-rich scenarios. Using a sensemaking framework and cognitive mapping methodology, we explored how HI interpret and interact with AI across pre-development, deployment, and post-development phases. Our findings reveal that collaboration is a dynamic and co-constitutive process of institutional co-production, structured by epistemic asymmetry, symbolic accountability, and infrastructural interdependence. While AI contributes speed, scale, and pattern recognition in routine, structured environments, human actors provide ethical oversight, contextual judgment, and strategic interpretation, particularly vital in uncertain or ethically charged contexts. Moving beyond static models such as “human-in-the-loop” or “AI-assistance,” this study offers a novel framework that conceptualizes AI and HI collaboration as a sociotechnical system. Theoretically, it bridges fragmented literatures in AI, cognitive science, and institutional theory. Practically, it offers actionable insights for designing collaborative infrastructures that are both ethically aligned and organizationally resilient. As AI ecosystems grow more complex and decentralized, our findings highlight the need for reflexive governance mechanisms to support adaptive, interpretable, and accountable human–machine decision-making.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.sftr.2025.101152

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 92

  1. Abaku, 2024, Theoretical approaches to AI in supply chain optimization: Pathways to efficiency and resilience, Int. J. Sci. Technol. Res. Arch., 6, 092, 10.53771/ijstra.2024.6.1.0033 · DOI
  2. Abubakar, 2019, Applying artificial intelligence technique to predict knowledge hiding behavior, Int. J. Inf. Manag., 49, 45
  3. Adam, 2020, Intuition in decision making-Risk and opportunity, J. Decis. Syst., 29, 98, 10.1080/12460125.2020.1848375 · DOI
  4. Akter, 2022, Algorithmic bias in machine learning-based marketing models, J. Bus. Res., 144, 201, 10.1016/j.jbusres.2022.01.083 · DOI · Paper page
  5. Arcila, 2024, AI liability in Europe: How does it complement risk regulation and deal with the problem of human oversight?, Comput. Law Secur. Rev., 54
  6. Bahmer, 2023, Modern artificial neural networks: Is evolution cleverer?, Neural Comput., 35, 763, 10.1162/neco_a_01575 · DOI
  7. Basel, 2013, Rationality and dual process models of reasoning in managerial cognition and decision making, Eur. Manag. J., 31, 745, 10.1016/j.emj.2013.07.004 · DOI
  8. Bassiouni, 2024, Deep learning approaches to identify order status in a complex supply chain, Expert Syst. Appl., 250, 10.1016/j.eswa.2024.123947 · DOI
  9. Bhandari, 2021, Identifying and controlling biases in expert-opinion research: Guidelines for variations of Delphi, nominal group technique, and focus groups, J. Manag. Eng., 37
  10. Bhatti, 2024, Exploring data-driven innovation: what’s missing in the relationship between big data analytics capabilities and supply chain innovation?, Ann. Oper. Res., 333, 799, 10.1007/s10479-022-04772-7 · DOI
  11. Bircan, 2025, Unmasking inequalities of the code: Disentangling the nexus of AI and inequality, Technol. Forecast. Soc. Change, 211, 10.1016/j.techfore.2024.123925 · DOI
  12. Bishop, 2021, Artificial intelligence is stupid and causal reasoning will not fix it, Front. Psychol., 11, 10.3389/fpsyg.2020.513474 · DOI · Paper page
  13. Boldt, 2022, Towards a unified list of ethical principles for emerging technologies. An analysis of four European reports on molecular biotechnology and artificial intelligence, Sustain. Futures, 4, 10.1016/j.sftr.2022.100086 · DOI
  14. Booyse, 2024, Barriers to adopting automated organisational decision-making through the use of artificial intelligence, Manag. Res. Rev., 47, 64, 10.1108/MRR-09-2021-0701 · DOI
  15. Borisov, 2022, Deep neural networks and tabular data: A survey, IEEE trans. neural netw. learn. syst.
  16. Calegari, 2020, Logic-based technologies for intelligent systems: State of the art and perspectives, Information, 11, 167, 10.3390/info11030167 · DOI
  17. Calzada, 2024, Artificial intelligence for social innovation: beyond the noise of algorithms and datafication, Sustainability, 16, 8638, 10.3390/su16198638 · DOI
  18. Cannarsa, 2021, Ethics guidelines for trustworthy AI, Camb. Handb. Lawyering digit. Age, 283, 10.1017/9781108936040.022 · DOI · Paper page
  19. Cannas, 2024, Artificial intelligence in supply chain and operations management: a multiple case study research, Int. J. Prod. Res., 62, 3333, 10.1080/00207543.2023.2232050 · DOI · Paper page
  20. Chase, 1998, Visions of rationality, Trends Cogn. Sci., 2, 206, 10.1016/S1364-6613(98)01179-6 · DOI · Paper page
  21. Chin, 2025, The masking effect of entrepreneurship on the link of university-firm technology transfers to inclusive growth: a Yin-Yang dialectical systems view, J. Technol. Transf., 1
  22. Conlisk, 1996, Why bounded rationality?, J. econ. lit., 34, 669
  23. Cossette-Lefebvre, 2023, AI’s fairness problem: understanding wrongful discrimination in the context of automated decision-making, AI Ethics, 3, 1255, 10.1007/s43681-022-00233-w · DOI
  24. Crivellaros, 2025, Co-creating conditions for social justice in digital societies: modes of resistance in HCI collaborative endeavors and evolving socio-technical landscapes, ACM Trans. Comput.-Hum. Interact., 10.1145/3711840 · DOI
  25. Cybulski, 2021, Data science in organizations: Conceptualizing its breakthroughs and blind spots, J. Inf. Technol., 36, 154, 10.1177/0268396220988539 · DOI
  26. Dawes, 1988, Ration. choice uncertain world
  27. de Wilde, 2018, The anchoring-bias in groups, J. Exp. Soc. Psychol., 76, 116, 10.1016/j.jesp.2018.02.001 · DOI
  28. Diamond, 2013, Executive functions, Annu. Rev. Psychol., 64, 135, 10.1146/annurev-psych-113011-143750 · DOI · Paper page
  29. Dou, 2025, Towards energy transition: Accessing the significance of artificial intelligence in ESG performance, Energy Econ., 10.1016/j.eneco.2025.108515 · DOI
  30. Ebermann, 2023, Explainable AI: The effect of contradictory decisions and explanations on users’ acceptance of AI systems, Int. J. Hum.–Comput. Interact., 39, 1807, 10.1080/10447318.2022.2126812 · DOI
  31. Eden, 2004, Cognitive mapping expert views for policy analysis in the public sector, Eur. J. Oper. Res., 152, 615, 10.1016/S0377-2217(03)00061-4 · DOI · Paper page
  32. Evans, 2008, Dual-processing accounts of reasoning, judgment, and social cognition, Annu. Rev. Psychol, 59, 255, 10.1146/annurev.psych.59.103006.093629 · DOI · Paper page
  33. Fareed, 2024, Role and applications of advanced digital technologies in achieving sustainability in multimodal logistics operations: A systematic literature review, Sustain. Futures, 10.1016/j.sftr.2024.100278 · DOI
  34. Fazelpour, 2021, Algorithmic bias: Senses, sources, solutions, Philos. Compass, 16, 10.1111/phc3.12760 · DOI · Paper page
  35. Felin, 2024, Theory is all you need: AI, human cognition, and decision making, Hum. Cogn. Decis. Mak. (Febr. 23 2024)
  36. Ferretti, 2016, From stakeholders analysis to cognitive mapping and Multi-Attribute Value Theory: An integrated approach for policy support, Eur. J. Oper. Res., 253, 524, 10.1016/j.ejor.2016.02.054 · DOI
  37. Fischer, 2021, In-the-loop or on-the-loop? Interactional arrangements to support team coordination with a planning agent", Concurr. Comput.: Pract. Exp., 33, 10.1002/cpe.4082 · DOI
  38. Frankish, 2010, Dual-process and dual-system theories of reasoning, Philos. Compass, 5, 914, 10.1111/j.1747-9991.2010.00330.x · DOI · Paper page
  39. Glöckner, 2008, Modeling option and strategy choices with connectionist networks: Towards an integrative model of automatic and deliberate decision making, Judgm. Decis. mak., 3, 215, 10.1017/S1930297500002424 · DOI · Paper page
  40. Gómez-Carmona, 2024, Human-in-the-loop machine learning: Reconceptualizing the role of the user in interactive approaches, Internet Things, 25, 10.1016/j.iot.2023.101048 · DOI
  41. Gonzalez, 2024, Building human-like artificial agents: A general cognitive algorithm for emulating human decision-making in dynamic environments, Perspect. Psychol. Sci., 19, 860, 10.1177/17456916231196766 · DOI
  42. Gupta, 2022, Artificial intelligence for decision support systems in the field of operations research: review and future scope of research, Ann. Oper. Res., 308, 215, 10.1007/s10479-020-03856-6 · DOI · Paper page
  43. Hao, 2024, Exploring collaborative decision-making: A quasi-experimental study of human and Generative AI interaction, Technol. Soc., 78, 10.1016/j.techsoc.2024.102662 · DOI
  44. Hao, 2025, Critical success and failure factors in the AI lifecycle: a knowledge graph-based ontological study, J. Model. Manag., 10.1108/JM2-06-2024-0204 · DOI
  45. Hao, 2025, When customers know it’s AI: Experimental comparison of human and LLM-Based Communication in service recovery, J. Mark. Commun., 1, 10.1080/13527266.2025.2540376 · DOI
  46. Hao, 2021, The Effect of Classroom Environment on Satisfaction and Performance: Towards IoT-Sustainable Space, 443, 10.24928/2021/0145 · doi:10.24928/2021/0145
  47. Herath Pathirannehelage, 2025, Design principles for artificial intelligence-augmented decision making: An action design research study, Eur. J. Inf. Syst., 34, 207, 10.1080/0960085X.2024.2330402 · DOI
  48. Hodgkinson, 2009, Intuition in organizations: Implications for strategic management, Long range plan., 42, 277, 10.1016/j.lrp.2009.05.003 · DOI · Paper page
  49. Holzinger, 2017, glass-box interact. mach. learn. approach solving NP-hard probl. hum.-in-the-loop
  50. Huettel, 2005, Decisions under uncertainty: probabilistic context influences activation of prefrontal and parietal cortices, J. Neurosci., 25, 3304, 10.1523/JNEUROSCI.5070-04.2005 · DOI · Paper page
  51. Huq, 2016, Supply chain configuration conundrum: how does the pharmaceutical industry mitigate disturbance factors?, Prod. Plan. Control, 27, 1206
  52. Hutchinson, 2005, Simple heuristics and rules of thumb: Where psychologists and behavioural biologists might meet, Behav. process., 69, 97, 10.1016/j.beproc.2005.02.019 · DOI · Paper page
  53. Jackson, 2021, Artificial intelligence & algorithmic bias: the issues with technology reflecting history & humans, J. Bus, Tech, L, 16, 299
  54. Jackson, 1988, Discerning threats and opportunities, Adm. sci. q., 370, 10.2307/2392714 · DOI · Paper page
  55. Johnson, 2023, Automating the OODA loop in the age of intelligent machines: reaffirming the role of humans in command-and-control decision-making in the digital age, Def. Stud., 23, 43, 10.1080/14702436.2022.2102486 · DOI
  56. Kaufmann, 2010, Debiasing the supplier selection decision: a taxonomy and conceptualization, Int. J. Phys. Distrib. Logist. Manag., 40, 792, 10.1108/09600031011093214 · DOI
  57. Kok, 2013, Prior expectations bias sensory representations in visual cortex, J. Neurosci., 33, 16275, 10.1523/JNEUROSCI.0742-13.2013 · DOI · Paper page
  58. Krause, 2024, Managing human-AI collaborations within industry 5.0 scenarios via knowledge graphs: key challenges and lessons learned, Front. Artif. Intell., 7, 10.3389/frai.2024.1247712 · DOI
  59. Krijestorac, 2021, Decisions under the illusion of objectivity: Digital embeddedness and B2B purchasing, Prod. Oper. Manag., 30, 2232, 10.1111/poms.13363 · DOI
  60. Kruglanski, 2018, Intuitive and deliberate judgments are based on common principles, motiv. mind, 104, 10.4324/9781315175867-4 · DOI
  61. Kuru, 2021, Conceptualisation of human-on-the-loop haptic teleoperation with fully autonomous self-driving vehicles in the urban environment, IEEE Open J. Intell. Transp. Syst., 2, 448, 10.1109/OJITS.2021.3132725 · DOI
  62. León, 2021, Accountability increases resource sharing: Effects of accountability on human and AI system performance, Int. J. Hum.–Comput. Interact., 37, 434, 10.1080/10447318.2020.1824695 · DOI
  63. Li, 2025, Why is ai not a panacea for data workers? an interview study on human-ai collaboration in data storytelling", IEEE Trans. Vis. Comput. Graph.
  64. Li, 2024, Tackling cooperative incompatibility for zero-shot human-ai coordination, J. Artif. Intell. Res., 80, 1139, 10.1613/jair.1.15884 · DOI
  65. Lomborg, 2023, Automated decision-making: toward a people-centred approach, Sociol. Compass, 17, 10.1111/soc4.13097 · DOI
  66. Mehrabi, 2021, A survey on bias and fairness in machine learning, ACM comput. surv. (CSUR), 54, 1, 10.1145/3457607 · DOI · Paper page
  67. Mensah, 2023, Artificial intelligence and ethics: a comprehensive review of bias mitigation, transparency, and accountability in AI Systems, Prepr. Novemb., 10
  68. Meub, 2018, Are groups ‘less behavioral’? The case of anchoring", Theory Decis., 85, 117, 10.1007/s11238-017-9608-x · DOI
  69. Min, 2010, Artificial intelligence in supply chain management: theory and applications, Int. J. Logist.: Res. Appl., 13, 13, 10.1080/13675560902736537 · DOI · Paper page
  70. Mosqueira-Rey, 2023, Human-in-the-loop machine learning: a state of the art, Artif. Intell. Rev., 56, 3005, 10.1007/s10462-022-10246-w · DOI · Paper page
  71. Murphy, 2008, The nominating process for corporate boards of directors: A decision-making analysis, Berkeley Bus. LJ, 5, 131
  72. Norman, 1968, Toward a theory of memory and attention, Psychol. rev., 75, 522, 10.1037/h0026699 · DOI · Paper page
  73. Primrose, 2017, The subjectification of'homo economicus' in behavioural economics, J. Aust. Polit. Econ., 88
  74. Runco, 2023, AI can only produce artificial creativity, J. Creat., 33
  75. Saba, 2024, The impact of artificial intelligence (AI) investment on human well-being in G-7 countries: Does the moderating role of governance matter?, Sustain. futures, 7, 10.1016/j.sftr.2024.100156 · DOI
  76. Sadler-Smith, 2004, The intuitive executive: Understanding and applying ‘gut feel’in decision-making, Acad. Manag. Perspect., 18, 76, 10.5465/ame.2004.15268692 · DOI · Paper page
  77. Schoppek, 2023, A dual processing approach to complex problem solving, J. Dyn. Decis. Mak., 9
  78. Semeraro, 2023, Human–robot collaboration and machine learning: A systematic review of recent research", Robot. Comput.-Integr. Manuf., 79, 10.1016/j.rcim.2022.102432 · DOI · Paper page
  79. Shahbazi, 2023, Representation bias in data: A survey on identification and resolution techniques, ACM Comput. Surv., 55, 1, 10.1145/3588433 · DOI
  80. Sharma, 2020, Artificial intelligence and effective governance: A review, critique and research agenda, Sustain. Futures, 2, 10.1016/j.sftr.2019.100004 · DOI · Paper page
  81. Shepherd, 2024, Going with the gut: Exploring top management team intuition in strategic decision-making, J. Bus. Res., 181
  82. Simon, 1990, Invariants of human behavior, Annu. rev. psychol., 41, 1, 10.1146/annurev.ps.41.020190.000245 · DOI · Paper page
  83. Slack, 2021, Reliable post hoc explanations: Modeling uncertainty in explainability, Adv. neural inf. process. syst., 34, 9391
  84. Sloman, 1996, The empirical case for two systems of reasoning, Psychol. bull., 119, 3, 10.1037/0033-2909.119.1.3 · DOI · Paper page
  85. Surden, 2018, Artificial intelligence and law: An overview, Ga, St. UL Rev, 35, 1305
  86. Sutton, 2013, Focus group methods: Using interactive and nominal groups to explore emerging technology-driven phenomena in accounting and information systems, Int. J. Account. Inf. Syst., 14, 81, 10.1016/j.accinf.2011.10.001 · DOI
  87. Thiebes, 2021, Trustworthy artificial intelligence, Electron. Mark., 31, 447, 10.1007/s12525-020-00441-4 · DOI · Paper page
  88. Van Der Vlist, 2024, Big AI: Cloud infrastructure dependence and the industrialisation of artificial intelligence, Big Data Soc., 11, 10.1177/20539517241232630 · DOI · Paper page
  89. Wang, 2024, "Friend or foe? Teaming between artificial intelligence and workers with variation in experience, Manag. Sci., 70, 5753
  90. Wang, 2024, Enhancing the sustainability and robustness of critical material supply in electrical vehicle market: an AI-powered supplier selection approach, Ann. Oper. Res., 342, 921, 10.1007/s10479-023-05698-4 · DOI
  91. Wood, 2014, Opportunity evaluation as rule-based decision making, J. manag. stud., 51, 573, 10.1111/joms.12018 · DOI · Paper page
  92. Zhang, 2024, Corporate social responsibility as the pathway towards sustainability: a state-of-the-art review in Asia economics, Discov. Sustain., 5, 348, 10.1007/s43621-024-00577-9 · DOI

Metadata

Authors
Hao, X., Demir, E., & Eyers, D.
Venue
Sustainable Futures
Publisher
Elsevier BV
Published
2025-12
Volume
10
Pages
101152
ISSN
2666-1888
Cited by
33
References
27
Language
en
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.