The Values Encoded in Machine Learning Research
Open accessThe DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
Machine learning currently exerts an outsized influence on the world, increasingly affecting institutional practices and impacted communities. It is therefore critical that we question vague conceptions of the field as value-neutral or universally beneficial, and investigate what specific values the field is advancing. In this paper, we first introduce a method and annotation scheme for studying the values encoded in documents such as research papers. Applying the scheme, we analyze 100 highly cited machine learning papers published at premier machine learning conferences, ICML and NeurIPS. We annotate key features of papers which reveal their values: their justification for their choice of project, which attributes of their project they uplift, their consideration of potential negative consequences, and their institutional affiliations and funding sources. We find that few of the papers justify how their project connects to a societal need (15%) and far fewer discuss negative potential (1%). Through line-by-line content analysis, we identify 59 values that are uplifted in ML research, and, of these, we find that the papers most frequently justify and assess themselves based on Performance, Generalization, Quantitative evidence, Efficiency, Building on past work, and Novelty. We present extensive textual evidence and identify key themes in the definitions and operationalization of these values. Notably, we find systematic textual evidence that these top values are being defined and applied with assumptions and implications generally supporting the centralization of power. Finally, we find increasingly close ties between these highly cited papers and tech companies and elite universities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In a study on the values encoded in the ML research literature, Birhane et al. identified a number of ethical values [45], from which the papers would draw motivation.”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [45]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we start from the premise that these AI writing tools are both flawed and fallible, due to their algorithmic biases and technical shortcomings [9], but that they also have powerful affordances, including the ability to assist the large numbers of scientists and engineers around the world who need to publish and present their work in English but are not native speakers of the language.”
From Incorporating generative AI into a writing-intensive undergraduate course without off-loading learning (2025) · cites this paper as [9]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Benchmarks set the agenda and orient progress: we should aspire for holistic, pluralistic, and democratic benchmarks.3”
From Holistic Evaluation of Language Models (2023) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“In a total of 100 highly cited papers, the study found that performance was clearly the most frequent value and pointed out that performance is not a neutral term but comes with ethical implications.”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [45]
The cited source says what the claim attributes to it.
In the citing paper
“They note that most of these focus on endogenous notions of technical performance and novelty, and leave out considerations of broader context and impact that are necessary to shaping a governance effort.”
From Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022) · cites this paper as [18]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Three Approaches to Qualitative Content Analysis
Hsieh, H.-F., & Shannon, S. E. (2005) · Qualitative Health Research · doi:10.1177/1049732305276687 · Cited by 44,601
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,312
Qualitative evaluation and research methods
Ellis, D. (1990) · International Journal of Information Management · doi:10.1016/0268-4012(90)90041-p · Cited by 17,943
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Content Analysis: An Introduction to Its Methodology
Krippendorff, K. (2019) · doi:10.4135/9781071878781 · Cited by 15,448
(2005) · PubMed · Cited by 7,011
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Sorting Things Out: Classification and Its Consequences
(2000) · The Journal of Academic Librarianship · doi:10.1016/s0099-1333(00)00155-5 · Cited by 5,210
How to plan and perform a qualitative study using content analysis
Bengtsson, M. (2016) · NursingPlus Open · doi:10.1016/j.npls.2016.01.001 · Cited by 4,616 · cited 3×
“, LWPR [8], standard GPR [1], sparse online Gaussian process regression (OGP) [5] and υ-support vector regression (υ-SVR) [11],”
BackgroundMethodRigor in Qualitative Research: The Assessment of Trustworthiness
Krefting, L. (1991) · American Journal of Occupational Therapy · doi:10.5014/ajot.45.3.214 · Cited by 4,174 · cited 1×
“We adhere to a number of best practices to establish reliability: We practice prolonged engagement, conducting long-term orientation to and analysis of data over more than a year (in lieu of short-term analysis that is dominated by preconceptions) [40]; We triangulate across researchers (six researchers) and points in…”
MethodEnhancing the quality and credibility of qualitative analysis.
(1999) · PubMed Central · Cited by 4,133
The Sociology of Science: Theoretical and Empirical Investigations
Kaplan, N., Merton, R. K., & Storer, N. W. (1974) · Technology and Culture · doi:10.2307/3102264 · Cited by 4,005
The Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
(2018) · Cited by 3,510
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
(2016) · Cited by 2,334
Winner, L. (2017) · Computer Ethics · doi:10.4324/9781315259697-21 · Cited by 2,317
Merz, S. (2020) · Ethnic and Racial Studies · doi:10.1080/01419870.2020.1715454 · Cited by 1,870
An overview of the qualitative descriptive design within nursing research
Doyle, L., McCabe, C., Keogh, B., Brady, A., & McCann, M. (2020) · Journal of Research in Nursing · doi:10.1177/1744987119880234 · Cited by 1,855
Sociological Methods: A Sourcebook.
Thiessen, V., & Denzin, N. K. (1970) · Social Forces · doi:10.2307/2576535 · Cited by 1,839
Issues of validity and reliability in qualitative research
Noble, H., & Smith, J. (2015) · Evidence Based Nursing · doi:10.1136/eb-2015-102054 · Cited by 1,758
A Unified Framework of Five Principles for AI in Society
Floridi, L., & Cowls, J. (2019) · Harvard Data Science Review · doi:10.1162/99608f92.8cd550d1 · Cited by 1,631
Theoretically principled trade-off between robustness and accuracy
Zhang, H., Yu, Y., Jiao, J., Xing, E. P., Ghaoui, L. E., & Jordan, M. I. (2019) · arXiv · doi:10.48550/arxiv.1901.08573 · Cited by 1,560
Autonomous Technology: Technics-out-of-Control as a Theme in Political Thought
Carpenter, S. R., & Winner, L. (1978) · Technology and Culture · doi:10.2307/3103332 · Cited by 1,531
Review of “Race After Technology: Abolitionist Tools for the New Jim Code”
Jipguep-Akhtar, M. (2020) · Social Forces · doi:10.1093/sf/soz162 · Cited by 1,473
THE SOCIOLOGY OF SCIENCE:: Theoretical and Empirical Investigations
Merton, R. Κ., & Storer, N. W. (2017) · Cited by 1,462
Simplifying Graph Convolutional Networks
Wu, F., Zhang, T., Junior, A. H. D. S., Fifty, C., Yu, T., & Weinberger, K. Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07153 · Cited by 1,165
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
Mohamed, S., Png, M.-T., & Isaac, W. (2020) · Philosophy & Technology · doi:10.1007/s13347-020-00405-8 · Cited by 912
Feature hashing for large scale multitask learning
Weinberger, K., Dasgupta, A., Langford, J., Smola, A., & Attenberg, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553516 · Cited by 906
Hierarchical Graph Representation Learning with Differentiable Pooling
Ying, R., You, J., Morris, C., Ren, X., Hamilton, W. L., & Leskovec, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08804 · Cited by 798
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777
Computer Power and Human Reason: From Judgement to Calculation.
Crowther, J. F., & Weizenbaum, J. (1977) · Contemporary Sociology · doi:10.2307/2062802 · Cited by 722
Why Science Is Not Necessarily Self-Correcting
Ioannidis, J. P. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612464056 · Cited by 563 · cited 1×
“However, this is not always the case [30] and attention to the ways novelty and building on past work are defined and implemented reveals two key social commitments.”
BackgroundReplicated Softmax: an Undirected Topic Model
Hinton, G. E., & Salakhutdinov, R. (2009) · neural information processing systems · Cited by 538
Adversarially Robust Generalization Requires More Data
Schmidt, L., Santurkar, S., Tsipras, D., Talwar, K., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.11285 · Cited by 476
The Menlo Report: Ethical Principles Guiding Information and Communication Technology Research
Kenneally, E., & Dittrich, D. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2445102 · Cited by 363
Construction of the Literature Graph in Semantic Scholar
Ammar, W., Groeneveld, D., Bhagavatula, C., Beltagy, I., Crawford, M., Downey, D., Dunkelberger, J., Elgohary, A., Feldman, S., Ha, V., et al. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers) · doi:10.18653/v1/n18-3011 · Cited by 358
Algorithmic Colonization of Africa
Birhane, A. (2020) · SCRIPTed: A Journal of Law, Technology & Society · doi:10.2966/scrip.170220.389 · Cited by 349 · cited 2×
“Works such as [11, 46] critique AI, ML, and data using a decolonial lens, noting how these technologies replicate colonial power relationships and values, and propose decolonial values and methods.”
BackgroundMethodSelf-citationCognitive and Non-Cognitive Values in Science: Rethinking the Dichotomy
Longino, H. E. (1996) · Feminism, Science, and the Philosophy of Science · doi:10.1007/978-94-009-1742-2_3 · Cited by 348
Feminism, Science, and the Philosophy of Science
(1996) · Springer eBooks · doi:10.1007/978-94-009-1742-2 · Cited by 311
NAS-bench-101: Towards reproducible neural architecture search
Ying, C., Klein, A., Real, E., Christiansen, E., Murphy, K., & Hutter, F. (2019) · arXiv · doi:10.48550/arxiv.1902.09635 · Cited by 297
McMullin, E. (1982) · PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association · doi:10.1086/psaprocbienmeetp.1982.2.192409 · Cited by 286
Large image datasets: A pyrrhic win for computer vision?
Birhane, A., & Prabhu, V. U. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00158 · Cited by 257 · cited 3×
“[36] Vinay Uday Prabhu and Abeba Birhane.”
BackgroundSelf-citationVinkers, C. H., Tijdink, J. K., & Otte, W. M. (2015) · BMJ · doi:10.1136/bmj.h6467 · Cited by 237
The Moral Character of Cryptographic WorkPreprint
Rogaway, P. (2016) · IACR Cryptology ePrint Archive · Cited by 231
Lipton, Z. C., & Steinhardt, J. (2019) · Queue · doi:10.1145/3317287.3328534 · Cited by 231
Hamidi, F., Scheuerman, M. K., & Branham, S. M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173582 · Cited by 222
Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Verma, S. (2019) · Vikalpa: The Journal for Decision Makers · doi:10.1177/0256090919853933 · Cited by 219
Don’t ask if artificial intelligence is good or fair, ask how it shifts power
Kalluri, P. (2020) · Nature · doi:10.1038/d41586-020-02003-2 · Cited by 213
Self-citationPerformative PredictionPreprint
Perdomo, J. C., Zrnic, T., Mendler-Dünner, C., & Hardt, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06673 · Cited by 172
Proceedings of the Conference on Fairness, Accountability, and Transparency
(2019) · doi:10.1145/3287560 · Cited by 159
Wagstaff, K. (2012) · arXiv · doi:10.48550/arxiv.1206.4656 · Cited by 148
Utility is in the Eye of the User: A Critique of NLP Leaderboards
Ethayarajh, K., & Jurafsky, D. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.393 · Cited by 147
Design Justice, A.I., and Escape from the Matrix of Domination
Costanza-Chock, S. (2018) · Journal of Design and Science · doi:10.21428/96c8d426 · Cited by 138
On the Impact of the Computer on Society: How does one insult a machine?
Weizenbaum, J. (1972) · Science · doi:10.1126/science.176.4035.609 · Cited by 137
How novelty in knowledge earns recognition: The role of consistent identities
Trapido, D. (2015) · Research Policy · doi:10.1016/j.respol.2015.05.007 · Cited by 107
Against Scale: Provocations and Resistances to Scale ThinkingPreprint
Hanna, A., & Park, T. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08850 · Cited by 106
Ahmed, N., & Wahed, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.15581 · Cited by 105
The Grey Hoodie Project: Big Tobacco, Big Tech, and the Threat on Academic Integrity
Abdalla, M., & Abdalla, M. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462563 · Cited by 99
Large image datasets: A pyrrhic win for computer vision?
Prabhu, V. U., & Birhane, A. (2020) · arXiv · doi:10.48550/arxiv.2006.16923 · Cited by 91
Self-citationA Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. (2021) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3465416.3483305 · Cited by 82
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2019) · arXiv · doi:10.48550/arxiv.1901.10002 · Cited by 82 · cited 1×
“Although a plethora of work exists on sources of harm that can arise in relation to ML research [7, 15, 25, 28, 60], we observe that these discussions are ignored in these influential conference publications.”
BackgroundInter-Coder Agreement in One-to-Many Classification: Fuzzy Kappa
Kirilenko, A. P., & Stepchenkova, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0149787 · Cited by 78
Unlabeled Data Improves Adversarial Robustness
Carmon, Y., Raghunathan, A., Schmidt, L., Liang, P., & Duchi, J. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13736 · Cited by 65
Algorithms of Oppression: How Search Engines Reinforce Racism
Hankwitz, M. (2023) · Leonardo · doi:10.1162/leon_r_02331 · Cited by 59
The Essential Tension. Selected Studies in Scientific Tradition and Change.
Hacking, I., & Kuhn, T. S. (1979) · History and Theory · doi:10.2307/2504757 · Cited by 58
Fairness in representation: quantifying stereotyping as a representational harm
Abbasi, M., Friedler, S. A., Scheidegger, C., & Venkatasubramanian, S. (2019) · Proceedings of the 2019 SIAM International Conference on Data Mining · doi:10.1137/1.9781611975673.90 · Cited by 44 · cited 3×
“, LWPR [8], standard GPR [1], sparse online Gaussian process regression (OGP) [5] and υ-support vector regression (υ-SVR) [11], respectively.”
BackgroundMethodValue-laden disciplinary shifts in machine learning
Dotan, R., & Milli, S. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3373157 · Cited by 37
Self-citationUnpacking the Expressed Consequences of AI Research in Broader Impact Statements
Nanayakkara, P., Hullman, J., & Diakopoulos, N. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462608 · Cited by 32
Like a Researcher Stating Broader Impact For the Very First TimePreprint
Abuhamad, G., & Rheault, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13032 · Cited by 18
Research works printed but not linked: 10
- Samy Bengio and Deborah Raji. 2021. A Retrospective on the NeurIPS 2021 Ethics Review Process. https://blog.neurips.cc/2021/12/03/a-retrospective-on-the-neurips-2021-ethics-review-process/
- Bruce L. Berg and Howard Lune. 2017. Qualitative research methods for the social sciences (ninth edition ed.). Pearson.
- Kate Crawford. 2017. The Trouble with Bias. (2017). NeurIPS Keynote.
- Catherine D’Ignazio and Lauren F Klein. 2020. Data Feminism. MIT Press.
- Barney G. Glaser and Anselm L. Strauss. 1999. The discovery of grounded theory: strategies for grounded research. Aldine de Gruyter.
- Ben Green. 2019. ‘Good’ isn’t Good Enough. In NeurIPS Joint Workshop on AI for Social Good.
- Maurice G Kendall and B Babington Smith. 1939. The problem of m rankings. The annals of mathematical statistics 10, 3 (1939), 275–287.
- Yvonna S. Lincoln and Egon G. Guba. 2006. Naturalistic inquiry. Sage Publ.
- Frank Pasquale. 2015. The black box society. Harvard University Press.
- Anna Rogers. 2019. Peer review in NLP: reject-if-not-SOTA. Hacking Semantics blog(2019). https://hackingsemantics.xyz/2020/reviewing-models/#everything-wrong-with-reject-if-not-sota
Cited by
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Liang, W., Zhang, Y., Cao, H., Wang, B., Ding, D., Yang, X., Vodrahalli, K., He, S., Smith, D., Yin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01783 · Cited by 633
British Journal of Social Psychology
(2010) · British Journal of Social Psychology · doi:10.1111/(issn)2044-8309 · Cited by 558
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548
“The Test and Evaluation stage is continuous throughout the entire AI Development Lifecycle.”
MethodInfluentialHolistic Evaluation of Language Models
Bommasani, R., Liang, P., & Lee, T. (2023) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15007 · Cited by 480
Metadata
- Authors
- Birhane, A., Kalluri, P., Card, D., Agnew, W., Dotan, R., & Bao, M.
- Affiliations
- University College Dublin (IE)
- Stanford University (US)
- University of Michigan (US)
- University of Washington (US)
- University of Pittsburgh (US)
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 173-184
- OpenAlex
- W3174220540
- Cited by
- 317
- References
- 69
- Citations by year
- 2026: 72 · 2025: 66 · 2024: 63 · 2023: 70 · 2022: 36
- FWCI
- 58.38
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-nc-sa)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Science Foundation Ireland (13/RC/2094-2)
- National Defense Science and Engineering Graduate
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- value neutrality, research ethics, research funding sources, institutional affiliation, societal impact of AI, novelty, centralization of power, NeurIPS
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.