Coordinating Human and Machine Learning for Effective Organizational Learning
The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
With the rise of machine learning (ML), humans are no longer the only ones capable of learning and contributing to an organization’s stock of knowledge. We study how organizations can coordinate human learning and ML in order to learn effectively as a whole. Based on a series of agent-based simulations, we find that, first, ML can reduce an organization’s demand for human explorative learning that is aimed at uncovering new ideas; second, adjustments to ML systems made by humans are largely beneficial, but this effect can diminish or even become harmful under certain conditions; and third, reliance on knowledge created by ML systems can facilitate organizational learning in turbulent environments, but this requires significant investments in the initial setup of these systems as well as adequately coordinating them with humans. These insights contribute to rethinking organizational learning in the presence of ML and can aid organizations in reallocating scarce resources to facilitate organizational learning in practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Exploration and Exploitation in Organizational Learning
March, J. G. (1991) · Organization Science · doi:10.1287/orsc.2.1.71 · Cited by 21,942
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Innovation and Learning: The Two Faces of R & D
Cohen, W. M., & Levinthal, D. A. (1989) · The Economic Journal · doi:10.2307/2233763 · Cited by 7,530
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Beyond Accuracy: What Data Quality Means to Data Consumers
Wang, R. Y., & Strong, D. M. (1996) · Journal of Management Information Systems · doi:10.1080/07421222.1996.11518099 · Cited by 4,672
Reconceptualizing Organizational Routines as a Source of Flexibility and Change
Feldman, M. S., & Pentland, B. T. (2003) · Administrative Science Quarterly · doi:10.2307/3556620 · Cited by 4,020
A few useful things to know about machine learning
Domingos, P. (2012) · Communications of the ACM · doi:10.1145/2347736.2347755 · Cited by 3,326
Applications of machine learning in drug discovery and development
Vamathevan, J., Clark, D., Czodrowski, P., Dunham, I., Ferran, E., Lee, G., Li, B., Madabhushi, A., Shah, P., Spitzer, M., et al. (2019) · Nature Reviews Drug Discovery · doi:10.1038/s41573-019-0024-5 · Cited by 3,228
What Firms Do? Coordination, Identity, and Learning
Kogut, B., & Zander, U. (1996) · Organization Science · doi:10.1287/orsc.7.5.502 · Cited by 3,078
Artificial Intelligence and Management: The Automation–Augmentation Paradox
Raisch, S., & Krakowski, S. (2021) · Academy of Management Review · doi:10.5465/amr.2018.0072 · Cited by 2,755
Rokach, L. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-009-9124-7 · Cited by 2,657
Organizational Ambidexterity: Balancing Exploitation and Exploration for Sustained Performance
Raisch, S., Birkinshaw, J., Probst, G., & Tushman, M. L. (2009) · Organization Science · doi:10.1287/orsc.1090.0428 · Cited by 2,224
Ananny, M., & Crawford, K. (2018) · New Media & Society · doi:10.1177/1461444816676645 · Cited by 1,836
Drug discovery with explainable artificial intelligence
Jiménez-Luna, J., Grisoni, F., & Schneider, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00236-4 · Cited by 1,063
Rethinking drug design in the artificial intelligence era
Schneider, P., Walters, W. P., Plowright, A. T., Sieroka, N., Listgarten, J., Goodnow, R. A., Fisher, J., Jansen, J. M., Duca, J. S., Rush, T. S., et al. (2020) · Nature Reviews Drug Discovery · doi:10.1038/s41573-019-0050-3 · Cited by 894
Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766
Accountability in algorithmic decision making
Diakopoulos, N. (2016) · Communications of the ACM · doi:10.1145/2844110 · Cited by 763
Brock, J. K.-U., & von Wangenheim, F. (2019) · California Management Review · doi:10.1177/1536504219865226 · Cited by 622
Comprehensible credit scoring models using rule extraction from support vector machines
Martens, D., Baesens, B., Van Gestel, T., & Vanthienen, J. (2007) · European Journal of Operational Research · doi:10.1016/j.ejor.2006.04.051 · Cited by 448
Information technology and organizational learning: a review and assessment of research
Robey, D., Boudreau, M.-C., & Rose, G. M. (2000) · Accounting, Management and Information Technologies · doi:10.1016/s0959-8022(99)00017-x · Cited by 377
Augmenting the algorithm: Emerging human-in-the-loop work configurations
Grønsund, T., & Aanestad, M. (2020) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2020.101614 · Cited by 364
Brunswicker, S., Almirall, E., & Majchrzak, A. (2019) · MIS Quarterly · doi:10.25300/misq/2019/13561 · Cited by 59
Cited by
Managing artificial intelligence
Berente, N., Gu, B., Recker, J., & Santhanam, R. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16274 · Cited by 898
Understanding and shaping the future of work with self-determination theory
Gagné, M., Parker, S. K., Griffin, M. A., Dunlop, P. D., Knight, C., Klonek, F. E., & Parent-Rocheleau, X. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-022-00056-w · Cited by 434
Yang, J., Amrollahi, A., & Marrone, M. (2024) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2024.101864 · Cited by 59
Artificial Intelligence and Groups: Effects of Attitudes and Discretion on Collaboration
Bezrukova, K., Griffith, T. L., Spell, C., Rice, V., & Yang, H. E. (2023) · Group & Organization Management · doi:10.1177/10596011231160574 · Cited by 51
“We acknowledge a special role for humans “ in the loop, ” given the possibility of grave errors at this stage in AI development (A. Gupta et al., 2021) and the value of deep human domain expertise (Sturm et al., 2021).”
Background
Metadata
- Authors
- Sturm, T., Gerlach, J. P., Pumplun, L., Mesbah, N., Peters, F., Tauchert, C., Nan, N., & Buxmann, P.
- Venue
- MIS Quarterly
- Publisher
- MIS Quarterly
- Published
- 2021-09-01
- Volume
- 45
- Issue
- 3
- Pages
- 1581-1602
- ISSN
- 0276-7783
- 2162-9730
- OpenAlex
- W3200943004
- Cited by
- 144
- References
- 24
- FWCI
- 18.07
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.