Bias and Productivity in Humans and Machines
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Where should better learning technology (such as machine learning or AI) improve decisions? I develop a model of decision-making in which better learning technology is complementary with experimentation. Noisy, inconsistent decision-making introduces quasi-experimental variation into training datasets, which complements learning. The model makes heterogeneous predictions about when machine learning algorithms can improve human biases. These algorithms can remove human biases exhibited in historical training data, but only if the human training decisions are sufficiently noisy; otherwise, the algorithms will codify or exacerbate existing biases. Algorithms need only a small amount of noise to correct biases that cause large productivity distortions. As the amount of noise increases, the machine learning can correct both large and increasingly small productivity distortions. The theoretical conditions necessary to completely eliminate bias are extreme and unlikely to appear in real datasets. The model provides theoretical microfoundations for why learning from biased historical datasets may lead to a decrease (if not a full elimination) of bias, as has been documented in several empirical settings. The model makes heterogeneous predictions about the use of human expertise in machine learning. Expert-labeled training datasets may be suboptimal if experts are insufficiently noisy, as prior research suggests. I discuss implications for regulation, labor markets, and business strategy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The future of employment: How susceptible are jobs to computerisation?
Frey, C. B., & Osborne, M. A. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.08.019 · Cited by 8,727
Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
Wager, S., & Athey, S. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1319839 · Cited by 2,917
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943
Inherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Discrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783
What Can Machines Learn and What Does It Mean for Occupations and the Economy?
Brynjolfsson, E., Mitchell, T., & Rock, D. (2018) · AEA Papers and Proceedings · doi:10.1257/pandp.20181019 · Cited by 695
Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination
Datta, A., Tschantz, M. C., & Datta, A. (2015) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0007 · Cited by 676
Cited by
Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y., & Ghassemi, M. (2021) · Nature Medicine · doi:10.1038/s41591-021-01595-0 · Cited by 928
Felten, E., Raj, M., & Seamans, R. (2021) · Strategic Management Journal · doi:10.1002/smj.3286 · Cited by 574
What Is Different About Digital Strategy? From Quantitative to Qualitative Change
Adner, R., Puranam, P., & Zhu, F. (2019) · Strategy Science · doi:10.1287/stsc.2019.0099 · Cited by 328
When does AI pay off? AI-adoption intensity, complementary investments, and R&D strategy
Lee, Y. S., Kim, T., Choi, S., & Kim, W. (2022) · Technovation · doi:10.1016/j.technovation.2022.102590 · Cited by 141
OK computer: Worker perceptions of algorithmic recruitment
Fumagalli, E., Rezaei, S., & Salomons, A. (2022) · Research Policy · doi:10.1016/j.respol.2021.104420 · Cited by 46
Metadata
- Authors
- Cowgill, B.
- Publisher
- W.E. Upjohn Institute
- Published
- 2019-08-06
- OpenAlex
- W2965950758
- Cited by
- 46
- References
- 16
- Type
- report
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.