A Hierarchy of Limitations in Machine Learning
Open accessThe DOI matches its OpenAlex record; 65 of the 65 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
Power-Law Distributions in Empirical Data
Clauset, A., Shalizi, C. R., & Newman, M. E. J. (2009) · SIAM Review · doi:10.1137/070710111 · Cited by 8,019
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“A recent result by Obermeyer et al. (2019) can be interpreted in terms of constructs: they detail a machine learning model used by a major healthcare company that de-prioritized care for black patients versus equivalently sick non-black patients.”
BackgroundLipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages
McNemar, Q. (1947) · Psychometrika · doi:10.1007/bf02295996 · Cited by 4,754
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340
Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
Dietterich, T. G. (1998) · Neural Computation · doi:10.1162/089976698300017197 · Cited by 3,840
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · arXiv · doi:10.48550/arxiv.1906.02243 · Cited by 3,735
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684 · cited 2×
“, 2019) but better results come at the cost of potentially massive carbon emissions for the energy the computation requires (Strubell et al., 2019).”
BackgroundA survey of cross-validation procedures for model selection
Arlot, S., & Celisse, A. (2010) · Statistics Surveys · doi:10.1214/09-ss054 · Cited by 3,522
A few useful things to know about machine learning
Domingos, P. (2012) · Communications of the ACM · doi:10.1145/2347736.2347755 · Cited by 3,326
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
Wager, S., & Athey, S. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1319839 · Cited by 2,917
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
The Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014) · Science · doi:10.1126/science.1248506 · Cited by 2,523 · cited 1×
“Lazer et al. (2014) phrase this as a problem of overfitting, and indeed we can think of models that fit to anything other than an underlying causal process as “overfitting,” but it is more strongly a lesson about causality (Harford, 2014).”
BackgroundRobust physical-world attacks on deep learning visual classification
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., & Song, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00175 · Cited by 2,173
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119 · cited 1×
“36Confusion around this is shown in a recent exchange of Christodoulou et al. (2019); Bian et al. (2019); Van Calster et al. (2019).”
ResultMachine Learning: An Applied Econometric Approach
Mullainathan, S., & Spiess, J. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.87 · Cited by 1,989
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631
Statistics versus machine learning
Bzdok, D., Altman, N., & Krzywinski, M. (2018) · Nature Methods · doi:10.1038/nmeth.4642 · Cited by 1,508 · cited 1×
“47Similar papers, arguing for the neglected importance of prediction, are recently appearing for many other fields as well such as Yarkoni and Westfall (2017) for psychology and Bzdok et al. (2018) for biology.”
BackgroundMaking Things Happen: A Theory of Causal Explanation
Woodward, J. (2004) · doi:10.1093/0195155270.001.0001 · Cited by 1,302
Key Concepts in Social Research
Payne, G., & Payne, J. (2004) · doi:10.4135/9781849209397 · Cited by 975
The Statistical Crisis in Science
Gelman, A., & Loken, E. (2014) · American Scientist · doi:10.1511/2014.111.460 · Cited by 963
Assessing the impact of planned social change
Campbell, D. T. (1979) · Evaluation and Program Planning · doi:10.1016/0149-7189(79)90048-x · Cited by 957
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
McCoy, R. T., Pavlick, E., & Linzen, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1334 · Cited by 944
Gelman, A., Fagan, J., & Kiss, A. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001040 · Cited by 873 · cited 1×
“For example, why should quantitative evidence (e.g., Gelman et al., 2007), rather than people’s lived experience, be the arbiter of whether or not policing is biased?”
BackgroundDeep Learning: A Critical Appraisal
Marcus, G. (2018) · arXiv · doi:10.48550/arxiv.1801.00631 · Cited by 853
Tufekci, Z. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14517 · Cited by 780
Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766 · cited 4×
“Consider also Letham et al. (2015), who present interpretable decision rules for classification using the Titanic dataset as an example. They write, “For example, we predict that a passenger is less likely to survive than not because he or she was in the 3rd class” [emphasis original]. The “because” refers here to the…”
BackgroundMethodInfluentialSocial Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763 · cited 1×
“The decision of what data to bring to a particular problem supersedes all other limitations and questions (Mallows, 1998), but data has also been covered extensively, most recently around social media data in comprehensive review by Olteanu et al. (2019).”
BackgroundSocial Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016) · doi:10.2139/ssrn.2886526 · Cited by 763
Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning
Raschka, S. (2018) · arXiv · doi:10.48550/arxiv.1811.12808 · Cited by 735
Transcending General Linear Reality
Abbott, A. (1988) · Sociological Theory · doi:10.2307/202114 · Cited by 716
Bickel, P. J., Hammel, E. A., & O'Connell, J. W. (1975) · Science · doi:10.1126/science.187.4175.398 · Cited by 659
Kleinberg, J., Ludwig, J., Mullainathan, S., & Obermeyer, Z. (2015) · American Economic Review · doi:10.1257/aer.p20151023 · Cited by 584 · cited 8×
“But for pre-testing the design of a system for detection/classification by a given schema (especially if developed by or with relevant stakeholders), representing an arguably enormous space of use cases neglected prior to the development of machine learning (Breiman, 2001; Mullainathan and Spiess, 2017), machine learn…”
BackgroundInfluentialClassification Situations: Life- chances in the neoliberal era
Fourcade, M., & Healy, K. (2013) · Accounting, Organizations and Society · doi:10.1016/j.aos.2013.11.002 · Cited by 544
The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition
Keyes, O. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274357 · Cited by 505
Interference Between Units in Randomized Experiments
Rosenbaum, P. R. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001112 · Cited by 371
Seawright, J. (2016) · Cambridge University Press · doi:10.1017/cbo9781316160831 · Cited by 338
Artificial intelligence meets natural stupidity
McDermott, D. (1976) · ACM SIGART Bulletin · doi:10.1145/1045339.1045340 · Cited by 323
Breaking NLI Systems with Sentences that Require Simple Lexical Inferences
Glockner, M., Shwartz, V., & Goldberg, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2103 · Cited by 319 · cited 1×
“Word embeddings, a representation derived from “unsupervised” learning, work fascinatingly well for many language-based prediction tasks; however, it is actually a supervised learning task of imputing a held-out word, and beyond its impressive successes, word embeddings have key limitations for what might be said to b…”
BackgroundMeng, X.-L. (2018) · The Annals of Applied Statistics · doi:10.1214/18-aoas1161sf · Cited by 316
The reusable holdout: Preserving validity in adaptive data analysis
Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., & Roth, A. (2015) · Science · doi:10.1126/science.aaa9375 · Cited by 307
Causal discovery and inference: concepts and recent methodological advances
Spirtes, P., & Zhang, K. (2016) · Applied Informatics · doi:10.1186/s40535-016-0018-x · Cited by 304
Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments
Green, B., & Chen, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287563 · Cited by 275
Lipton, Z. C., & Steinhardt, J. (2019) · Queue · doi:10.1145/3317287.3328534 · Cited by 231
Bailey, D. H., Borwein, J. M., López de Prado, M., & Zhu, Q. J. (2014) · Notices of the American Mathematical Society · doi:10.1090/noti1105 · Cited by 221
Consistent cross-validatory model-selection for dependent data: hv-block cross-validation
Racine, J. (2000) · Journal of Econometrics · doi:10.1016/s0304-4076(00)00030-0 · Cited by 205
"The human body is a black box": supporting clinical decision-making with deep learning
Sendak, M., Elish, M. C., Gao, M., Futoma, J., Ratliff, W., Nichols, M., Bedoya, A., Balu, S., & O'Brien, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372827 · Cited by 196
(2019) · doi:10.1145/3351095.3372862 · Cited by 167
Toward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153 · cited 1×
“But the biggest point of implementation is not in software or even computational infrastructure; it is in institutions and power dynamics, for which nontechnical parts of interventions supersede the importance of technical parts (Katell et al., 2020).”
BackgroundWagstaff, K. (2012) · arXiv · doi:10.48550/arxiv.1206.4656 · Cited by 148
The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence
Watson, D. (2019) · Minds and Machines · doi:10.1007/s11023-019-09506-6 · Cited by 148
Bellotti, E. (2014) · doi:10.4324/9780203837122 · Cited by 140
When Genius Errs: R. A. Fisher and the Lung Cancer Controversy
Stolley, P. D. (1991) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a115904 · Cited by 119
Let's (not) stick together: pairwise similarity biases cross-validation in activity recognition
Hammerla, N. Y., & Plötz, T. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2807551 · Cited by 91
Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment
Barabas, C., Dinakar, K., Ito, J., Virza, M., & Zittrain, J. (2017) · arXiv · doi:10.48550/arxiv.1712.08238 · Cited by 87
What's sex got to do with machine learning?
Hu, L., & Kohler-Hausmann, I. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375674 · Cited by 74
Identifying platform effects in social media data
Malik, M., & Pfeffer, J. (2016) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v10i1.14756 · Cited by 45
Self-citationAlgorithmic Risk Assessment in the Hands of Humans
Stevenson, M., & Doleac, J. L. (2020) · doi:10.2139/ssrn.3513695 · Cited by 42
Patton, D., Blandfort, P., Frey, W., Gaskell, M., & Karaman, S. (2019) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2019.260 · Cited by 23
The Dangers of Counterfactual Causal Thinking about Detecting Racial Discrimination
Kohler-Hausmann, I. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3050650 · Cited by 9
'Improving ratings': audit in the British university system
Strathern, M. (1997) · European Review · doi:10.1002/(sici)1234-981x(199707)5:3<305::aid-euro184>3.0.co;2-4
Cited by
Leakage and the reproducibility crisis in machine-learning-based science
Kapoor, S., & Narayanan, A. (2023) · Patterns · doi:10.1016/j.patter.2023.100804 · Cited by 1,387
“, without any assumptions about independence in the data) is a hard problem, because we might not know the underlying dependency structure of the task in many cases.(60) [L3.”
BackgroundTowards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548
“Without direct measures for these highly complex considerations, AI development teams use proxies, which can create many risks [150].”
BackgroundDiscriminating Data: Correlation, Neighborhoods, and the New Politics of Recognition
Chun, W. H. K., & Barnett, A. (2021) · The MIT Press eBooks · doi:10.7551/mitpress/14050.001.0001 · Cited by 429
Rethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches
Weinberg, L. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13196 · Cited by 127
Metadata
- Authors
- Malik, M. M.
- Venue
- RePEc: Research Papers in Economics
- Publisher
- Federal Reserve Bank of St. Louis
- Published
- 2020-02-12
- arXiv
- 2002.05193
- OpenAlex
- W3006027555
- Cited by
- 51
- References
- 65
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.