Sociotechnical Envelopment of Artificial Intelligence: An Approach to Organizational Deployment of Inscrutable Artificial Intelligence Systems
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
The paper presents an approach for implementing inscrutable (i.e., nonexplainable) artificial intelligence (AI) such as neural networks in an accountable and safe manner in organizational settings. Drawing on an exploratory case study and the recently proposed concept of envelopment, it describes a case of an organization successfully “enveloping” its AI solutions to balance the performance benefits of flexible AI models with the risks that inscrutable models can entail. The authors present several envelopment methods—establishing clear boundaries within which the AI is to interact with its surroundings, choosing and curating the training data well, and appropriately managing input and output sources—alongside their influence on the choice of AI models within the organization. This work makes two key contributions: It introduces the concept of sociotechnical envelopment by demonstrating the ways in which an organization’s successful AI envelopment depends on the interaction of social and technical factors, thus extending the literature’s focus beyond mere technical issues. Secondly, the empirical examples illustrate how operationalizing a sociotechnical envelopment enables an organization to manage the trade-off between low explainability and high performance presented by inscrutable models. These contributions pave the way for more responsible, accountable AI implementations in organizations, whereby humans can gain better control of even inscrutable machine-learning models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Seeking Qualitative Rigor in Inductive Research: Notes on the Gioia Methodology
Gioia, D. A., Corley, K. G., & Hamilton, A. L. (2013) · Organizational Research Methods · doi:10.1177/1094428112452151 · Cited by 13,227
Constructing Grounded Theory. A Practical Guide Through Qualitative Analysis
(2006) · QMiP Bulletin · doi:10.53841/bpsqmip.2006.1.2.36 · Cited by 11,828
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“This assertion stems from the so-called Rashomon set argument (Rudin, 2019), under which there is a good chance that at least one of the acceptable models is interpretable yet still accurate.”
BackgroundLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems1
Klein, H. K., & Myers, M. D. (1999) · MIS Quarterly · doi:10.2307/249410 · Cited by 5,263
The Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
Digital Twin in manufacturing: A categorical literature review and classification
Kritzinger, W., Karner, M., Traar, G., Henjes, J., & Sihn, W. (2018) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2018.08.474 · Cited by 3,612 · cited 1×
“approach that simplifies envelopment is to strive for “gray-box models,” as exemplified by the creation of “digital twins” that can simulate real, physical processes (see El Saddik, 2018; Kritzinger et al., 2018).”
BackgroundInternational evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287 · cited 1×
“analysis and decision-making situations (McKinney et al., 2020), their operation logic differs dramatically from humans’ ways of approaching similar problems.”
BackgroundHow the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 3×
“Inscrutability takes many forms, linked to such elements as intentional corporate or state secrecy, technical illiteracy, and innate characteristics of ML models (Burrell, 2016).”
Background
Cited by
Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2024) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00834-7 · Cited by 2,267
“behavior of humans and AI systems to enable effective and efficient use in AI applications on the one hand and, on the other hand, to ensure envelopment of AI agency and reach (Asatiani et al., 2021).”
Managing artificial intelligence
Berente, N., Gu, B., Recker, J., & Santhanam, R. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16274 · Cited by 898
Keding, C., & Meissner, P. (2021) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.120970 · Cited by 168
How to explain AI systems to end users: a systematic literature review and research agenda
Laato, S., Tiainen, M., Najmul Islam, A. K. M., & Mäntymäki, M. (2022) · Internet Research · doi:10.1108/intr-08-2021-0600 · Cited by 155
Designing a feature selection method based on explainable artificial intelligence
Zacharias, J., von Zahn, M., Chen, J., & Hinz, O. (2022) · Electronic Markets · doi:10.1007/s12525-022-00608-1 · Cited by 109
“Such metrics steer the development process and facilitate the objective comparison of competing classifiers (Asatiani et al., 2021).”
BackgroundOladoyinbo, T. O., Olabanji, S. O., Olaniyi, O. O., Adebiyi, O. O., Okunleye, O. J., & Alao, A. I. (2024) · Asian Journal of Advanced Research and Reports · doi:10.9734/ajarr/2024/v18i2601 · Cited by 105
International Business, digital technologies and sustainable development: Connecting the dots
Ciulli, F., & Kolk, A. (2023) · Journal of World Business · doi:10.1016/j.jwb.2023.101445 · Cited by 86
The Vicious Circles of Skill Erosion: A Case Study of Cognitive Automation
Rinta-Kahila, T., The University of Queensland Business School, Penttinen, E., Aalto University School of Business, Salovaara, A., Aalto University / University of Helsinki, Soliman, W., University of Agder, N. /. U. O. J., Ruissalo, J., & Aalto University School of Business (2023) · Journal of the Association for Information Systems · doi:10.17705/1jais.00829 · Cited by 86
“It also accounts for the influence of AI explanations on this balancing act, which has become an increasingly important consideration due to the black-boxed nature of advanced AI systems (Asatiani et al., 2021).”
BackgroundSelf-citationEffective human–AI work design for collaborative decision-making
Jain, R., Garg, N., & Khera, S. N. (2023) · Kybernetes · doi:10.1108/k-04-2022-0548 · Cited by 85
Adam, M., Diebel, C., Goutier, M., & Benlian, A. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114193 · Cited by 62
Yang, J., Amrollahi, A., & Marrone, M. (2024) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2024.101864 · Cited by 59
Extending a Human-AI Collaboration Framework with Dynamism and Sociality
Muller, M., & Weisz, J. (2022) · 2022 Symposium on Human-Computer Interaction for Work · doi:10.1145/3533406.3533407 · Cited by 37
Exploring citizens’ stances on AI in public services: A social contract perspective
Schmager, S., Grøder, C. H., Parmiggiani, E., Pappas, I., & Vassilakopoulou, P. (2024) · Data & Policy · doi:10.1017/dap.2024.13 · Cited by 35
“It responds to calls for research on citizens ’ attitudes toward AI (Wirtz et al., 2019; Asatiani et al., 2021; Saura et al., 2022) drawing from rich empirical data collected using a combination of closedand openquestions and observations of participants ’ interactions with a prototype.”
Background
Metadata
- Authors
- Asatiani, A., University of Gothenburg, S., Malo, P., Aalto University School of Business, F., Nagbøl, P. R., IT University of Copenhagen, D., Penttinen, E., Aalto University School of Business, F., Rinta-Kahila, T., The University of Queensland, A., Salovaara, A., & Aalto University School of Arts, D. A. A.
- Venue
- Journal of the Association for Information Systems
- Publisher
- Association for Information Systems
- Published
- 2021
- Volume
- 22
- Issue
- 2
- Pages
- 325-352
- ISSN
- 1536-9323 (ISSN-L)
- OpenAlex
- W3142698583
- Cited by
- 173
- References
- 33
- FWCI
- 8.91
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.