Decision authority and the returns to algorithms
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Research Summary: We evaluate a pilot in an Inspections Department to explore the returns to a pair of algorithms that varied in their sophistication. We find that both algorithms provided substantial prediction gains, suggesting that even simple data may be helpful. However, these gains did not result in improved decisions. Inspectors often used their decision authority to override algorithmic recommendations, partly to consider other organizational objectives without improving outcomes. Interviews with 55 departments find that while some ran pilots seeking to prioritize inspections using data, all provided considerable decision authority to inspectors. These findings suggest that for algorithms to improve managerial decisions, organizations must consider both the returns to algorithms in the context and how decision authority is managed. Managerial Summary: We evaluate a pilot in an Inspections Department to explore the returns to algorithms on decisions. We find that the greatest gains in this context come from integrating data into the decision process in the form of simple heuristics, rather than from increasing algorithmic sophistication or additional data. We also find that these improvements in prediction do not fully translate into improved decisions. Decision‐makers were less likely to follow data‐driven recommendations, partly in consideration of other organizational objectives, but without substantially improving on them overall. These findings suggest that organizations should consider the returns to technical sophistication in each context, and that the design and management of decision authority can be a key choice that impacts the value organizations can capture from using predictive analytics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gigerenzer, G., & Gaissmaier, W. (2011) · Annual Review of Psychology · doi:10.1146/annurev-psych-120709-145346 · Cited by 4,037
Reasoning the fast and frugal way: Models of bounded rationality.
Gigerenzer, G., & Goldstein, D. G. (1996) · Psychological Review · doi:10.1037/0033-295x.103.4.650 · Cited by 3,504
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Artificial Intelligence and Management: The Automation–Augmentation Paradox
Raisch, S., & Krakowski, S. (2021) · Academy of Management Review · doi:10.5465/amr.2018.0072 · Cited by 2,755
“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
The robust beauty of improper linear models in decision making.
Dawes, R. M. (1979) · American Psychologist · doi:10.1037/0003-066x.34.7.571 · Cited by 1,966
Homo Heuristicus: Why Biased Minds Make Better Inferences
Gigerenzer, G., & Brighton, H. (2009) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2008.01006.x · Cited by 1,801
Grove, W. M., & Meehl, P. E. (1996) · Psychology, Public Policy, and Law · doi:10.1037/1076-8971.2.2.293 · Cited by 1,457
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Americans Do IT Better: US Multinationals and the Productivity Miracle
Bloom, N., Sadun, R., & Reenen, J. V. (2012) · American Economic Review · doi:10.1257/aer.102.1.167 · Cited by 1,028
Artificial Intelligence: The Ambiguous Labor Market Impact of Automating Prediction
Agrawal, A., Gans, J. S., & Goldfarb, A. (2019) · Journal of Economic Perspectives · doi:10.1257/jep.33.2.31 · Cited by 685
Felten, E., Raj, M., & Seamans, R. (2021) · Strategic Management Journal · doi:10.1002/smj.3286 · Cited by 574
Tong, S., Jia, N., Luo, X., & Fang, Z. (2021) · Strategic Management Journal · doi:10.1002/smj.3322 · Cited by 531
Making sense of recommendations
Yeomans, M., Shah, A., Mullainathan, S., & Kleinberg, J. (2019) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2118 · Cited by 459
Machine learning and human capital complementarities: Experimental evidence on bias mitigation
Choudhury, P., Starr, E., & Agarwal, R. (2020) · Strategic Management Journal · doi:10.1002/smj.3152 · Cited by 293
Hoffman, M., Kahn, L. B., & Li, D. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx042 · Cited by 261
An impact assessment of machine learning risk forecasts on parole board decisions and recidivism
Berk, R. (2017) · Journal of Experimental Criminology · doi:10.1007/s11292-017-9286-2 · Cited by 169
Human–AI collaborative decision-making as an organization design problem
Puranam, P. (2021) · Journal of Organization Design · doi:10.1007/s41469-021-00095-2 · Cited by 162
Cited by
Dell'Acqua, F., McFowland III, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4573321 · Cited by 794
The Crowdless Future? Generative AI and Creative Problem-Solving
Boussioux, L., Lane, J. N., Zhang, M., Jacimovic, V., & Lakhani, K. R. (2024) · Organization Science · doi:10.1287/orsc.2023.18430 · Cited by 329
Dell’Acqua, F., McFowland, E., Mollick, E., Lifshitz, H., Kellogg, K. C., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2026) · Organization Science · doi:10.1287/orsc.2025.21838 · Cited by 1,189
Theory Is All You Need: AI, Human Cognition, and Causal Reasoning
(2024) · doi:10.1287/stsc.2024.0189 · Cited by 151
Human-centered artificial intelligence: a field experiment
Krakowski, S., Haftor, D., Luger, J., Pashkevich, N., & Raisch, S. (2026) · Management Science · doi:10.1287/mnsc.2022.03849 · Cited by 53
Metadata
- Authors
- Kim, H., Glaeser, E. L., Hillis, A., Kominers, S. D., & Luca, M.
- Venue
- Strategic Management Journal
- Publisher
- Wiley
- Published
- 2023-12-05
- Volume
- 45
- Issue
- 4
- Pages
- 619-648
- ISSN
- 0143-2095 (ISSN-L)
- 1097-0266
- OpenAlex
- W4389396390
- Cited by
- 57
- References
- 19
- FWCI
- 15.64
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.