Engaging the ethics of data science in practice
Open accessAbstract
Seeking more common ground between data scientists and their critics.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
2 of 7 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured
- Findingown data§ Toward a Constructive Collaboration
The fallibility, incompleteness, and insufficiency of data scientists' informal networks is often frustrating for data scientists themselves.
“and that this is often frustrating for data scientists themselves.”
Quote in the paperSubstantial claim - Findingreasoning§ Ethics in Practice
Practitioners face difficult trade-offs even when attempting to address ethical issues more explicitly.
“Even when attempting to address ethical issues more explicitly, practitioners face difficult trade-offs.”
Quote in the paperSubstantial claim - Findingown data§ Abstract
Data scientists expressed frustration that critics failed to account for the careful thinking and critical reflection that data scientists already do as part of their everyday work.
“they also expressed frustration that critics failed to account for the careful thinking and critical reflection that data scientists already do as part of their everyday work.”
Quote in the paperSubstantial claim - Interpretationreasoning§ Ethics in Practice
Broad critiques of data science practices cannot account for the diversity of practices, concerns, or efforts among data scientists.
“Broad critiques of data science practices cannot account for the diversity of practices, concerns, or efforts among data scientists.”
Quote in the paperSubstantial claim - Interpretationown data§ Toward a Constructive Collaboration
Even if data scientists take care in their work and seek to engage critics, they may not be well prepared to consider the full range of ethical issues that such work raises.
“Of course, even if data scientists take care in their work and seek to engage critics, they may not be well prepared to consider the full range of ethical issues that such work raises.”
Quote in the paperSubstantial claim
- May combine two claims. 10
- Classified as probably not a substantial claim. 28
May combine two claims.
Premisecitation§ Members at Large.Satisfying all three fairness expectations (equal accuracy, equal false positive rates, equal false negative rates) at the same time would be impossible in most situations; meeting two will mean failing to comply with the third.
“recent work has established that satisfying all three at the same time would be impossible in most situations; meeting two will mean failing to comply with the third.4,7”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [4] 4. Corbett-Davies, S. et al. A computer program used for bail and sentencing decisions was labeled biased against blacks. It's actually not that clear. Washington Post (Oct. 17, 2016); https://www.washingtonpost. com/news/monkey-cage/wp/2016/10/17/can-an- algorithm-be-racist-our-analysis-is-more-cautious- than-propublicas/
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [7] 7. Kleinberg, J., Mullainathan, S. and Raghavan, S. Inherent Trade-Offs in the Fair Determination of Risk Scores. Arxiv.org. 2016; https://arxiv.org/ abs/1609.05807 source
The characterization theorem (Section 1.2, p. 5) supports the central trade-off: all three conditions can hold together only with perfect prediction or equal base rates. But the paper defines its conditions as calibration within groups and balance in average scores for negative and positive cases (Section 1.1, p. 4), not equal accuracy and equal false-positive and false-negative rates. It calls the latter two conditions generalizations of equal error rates (Introduction, p. 2). (LLM, conf=0.88)
Experimental — accuracy not yet measured
May combine two claims.
Premisecitation§ Ethics in PracticeOther researchers concerned about gender biases in data have attempted to build technical interventions to address them, but such an approach requires trading off privacy in order to construct a viable fairness remedy.
“Other researchers who are concerned about gender biases in data have attempted to build technical interventions to address them, 5 but such an approach requires trading off privacy in order to construct a viable fairness remedy, a decision that presents its own challenges.4”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [5] 5. Feldman, M. et al. Certifying and removing disparate impact. In Proceedings of the 21 st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, (2015), 259-268. source
- cites [4] 4. Corbett-Davies, S. et al. A computer program used for bail and sentencing decisions was labeled biased against blacks. It's actually not that clear. Washington Post (Oct. 17, 2016); https://www.washingtonpost. com/news/monkey-cage/wp/2016/10/17/can-an- algorithm-be-racist-our-analysis-is-more-cautious- than-propublicas/
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Findingown data§ AbstractCommon criticisms of data science tend to lack technical specificity or rest on faulty understandings of the relevant techniques.
“While practitioners were quick to point out that common criticisms of data science tend to lack technical specificity or rest on faulty understandings of the relevant techniques”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Findingreasoning§ Toward a Constructive CollaborationPractitioners want their analysis to be accurate and are deeply troubled by the limits of tests of validity, the problems with reproducibility, and the shortcomings of their methods.
“Practitioners want their analysis to be accurate and they are deeply troubled by the limits of tests of validity, the problems with reproducibility, and the shortcomings of their methods.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Findingreasoning§ Toward a Constructive CollaborationBoth data scientists and critics see a need for nuances within the field.
“Both see a need for nuances within the field.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingnone given§ Ethics in PracticeThe data scientists encountered in the study are often sympathetic to the sentiment behind the critiques they hear, but feel maligned and misunderstood, unacknowledged for their efforts, and frustrated by vague recommendations that are not actionable.
“All too often, the data scientists we have encountered are quite sympathetic to the sentiment behind the critiques they hear, but feel maligned and misunderstood, unacknowledged for their efforts, and frustrated by vague recommendations that are not actionable.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown data§ Toward a Constructive CollaborationThe informal networks that data scientists rely on are fallible, incomplete, and insufficient.
“Our research suggests the informal networks that data scientists rely on are fallible, incomplete, and insufficient”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone given§ Ethics in PracticeWhen pressed about ethics directly, many data scientists say it is not their area, even though they draw on a wide range of values to work through difficult tensions.
“Pressed about”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Findingnone given§ Ethics in PracticeMany data scientists do not use the language of ethics to talk about their practices.
“Yet, many data scientists do not use the language of ethics to talk about these practices.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Findingnone given§ Ethics in PracticeData scientists engage in countless acts of implicit ethical deliberation as part of their everyday work.
“Data scientists engage in countless acts of implicit ethical deliberation while viewpoints dealt with independently or on top of technical practice-without noticing that ethical deliberation is embedded in the everyday work of data scientists.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Findingnone given§ Toward a Constructive CollaborationMany data scientists feel as though critics only talk at them.
“Many data scientists feel as though critics only talk at them.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingown data§ AbstractThese data scientists felt that easy criticisms overlooked the kinds of routine deliberative activities that outsiders seem to have in mind when they talk about ethics.
“Instead, these data scientists felt that easy criticisms overlooked the kinds of routine deliberative activities that outsiders seem to have in mind when they talk about ethics.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Findingnone given§ Toward a Constructive CollaborationMany data scientists are deeply disturbed by those entering the field without rigorous training and by those playing into the hype by promising analyses that are not technically or socially responsible.
“Many data scientists are also deeply disturbed by those who are coming into the field without rigorous training and those who are playing into the hype by promising analyses that are not technically or socially responsible.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationnone given§ Members at Large.Fuzzy values will be difficult to resolve computationally.
“that fuzzy values will be difficult to resolve computationally”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning§ Toward a Constructive CollaborationUniversalizing critiques may undermine critics' opportunities to work with data scientists to meaningfully address some of the most urgent problems.
“Unfortunately, universalizing critiques may undermine critics' opportunities to work with data scientists to meaningfully address some of the most urgent problems.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ Members at Large.Favoring certain fairness properties over others could just as well have reflected a difference in values, rather than a failure to recognize the values at stake.
“Favoring certain fairness properties over others could just as well have reflected a difference in values, rather than a failure to recognize the values at stake.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ Toward a Constructive CollaborationThe gaps between data scientists and critics are wide.
“The gaps between data scientists and critics are wide”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Premisecitation§ Ethics in PracticeBlack defendants in Broward County, FL, who did not reoffend were twice as likely to be mislabeled as posing a high risk of recidivism than white defendants.
“black defendants in Broward County, FL, who did not reoffend were twice as likely to be mislabeled as posing a high risk of recidivism than white defendants.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [2] 2. Angwin, J. et al. Machine bias. ProPublica. (May 23, 2016); https://www.propublica.org/article/machine- bias-risk-assessments-in-criminal-sentencing
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Interpretationreasoning§ Toward a Constructive CollaborationThe work of data scientists is in many ways a qualitative practice.
“In many ways, the work of data scientists is a qualitative practice: they are called upon to parse an amorphous problem, wrangle a messy collection of data, and make it amenable to systematic analysis.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning§ Toward a Constructive CollaborationData scientists often need help from those who are willing to take time to understand what they are doing and the challenges of their practice.
“but they often need help from those who are willing to take time to understand what they are doing and the challenges of their practice.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Interpretationreasoning§ Members at Large.This use of data science has prompted a vigorous debate.
“One thing is for certain: this use of data science has prompted a vigorous debate”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Premisecitation§ AbstractThere is a social-technical gap between what we know we must support socially and what we can support technically.
“Mark Ackerman reframed this as a social-technical gap between”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [1] 1. Ackerman, M.S. The intellectual challenge of CSCW: The gap between social requirements and technical feasibility. Human-Computer Interaction 15, (2-3), 2000, 179-203. source
Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationreasoning§ Toward a Constructive CollaborationMaking the deliberation that is already a crucial part of data science visible will reveal far more common ground between data scientists and their critics and provide a meaningful foundation from which to articulate shared values.
“Doing so will reveal far more common ground between data scientists and their critics and provide a meaningful foundation from which to articulate shared values.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning§ Toward a Constructive CollaborationTechnical actors are often far more sophisticated than critics at understanding the limits of their analysis.
“Technical actors are often far more sophisticated than critics at under-standing the limits of their analysis.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationnone given§ Toward a Constructive CollaborationThe critical writing on data science has taken the paradoxical position of insisting that normative issues pervade all work with data while leaving unaddressed the issue of data scientists' ethical agency.
“The critical writing on data science has taken the paradoxical position of insisting that normative issues pervade all work with data while leaving unaddressed the issue of data scientists' ethical agency.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ Members at Large.Existing ethical frameworks may not deliver clear answers to data science challenges.
“and that existing ethical frameworks may not deliver clear answers to data science challenges.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Premisecitation§ Ethics in PracticeData science is being deployed by powerful organizations to achieve goals that can magnify inequality and undermine democratic decision-making.
“data science is being deployed by powerful organizations to achieve goals that can magnify inequality and undermine democratic decision-making.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [8] 8. O'Neil, C. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown, 2016.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ Ethics in PracticeCertain problems may stem from genuine value conflicts, not simply a lack of attention to the values at stake.
“Unfortunately, certain problems may stem from genuine value conflicts, not simply a lack of attention to the values at stake.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationreasoning§ Ethics in PracticeBroad critiques of data science often presume ignorance or corrupt intentions among data scientists.
“Instead, they often presume ignorance or corrupt intentions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning§ Toward a Constructive CollaborationCritique divorced from practice only increases the gaps between data scientists and critics.
“but critique divorced from practice only increases them.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ Members at Large.Even if Northpointe had been more sensitive to disparities in the false positive and false negative rates, the appropriate way to handle such a situation may not have been obvious.
“Even if Northpointe had been more sensitive to disparities in the false positive and false negative rates, the appropriate way to handle such a situation may not have been obvious.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ Members at Large.Our normative commitments are not well articulated.
“making clear that our normative commitments are not well articulated”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning§ Toward a Constructive CollaborationData scientists, as the ones closest to the work, are often the best positioned to address ethical concerns.
“Data scientists, as the ones closest to the work, are often the best positioned to address ethical concerns”
Quote in the paperNot a substantial claim
Cites
(2016) · Cited by 2,334 · see citing works
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · see citing works
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · see citing works
Inherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941 · see citing works
The Intellectual Challenge of CSCW: The Gap Between Social Requirements and Technical Feasibility
Ackerman, M. S. (2000) · Human–Computer Interaction · doi:10.1207/s15327051hci1523_5 · Cited by 892 · see citing works
Artificial experts: social knowledge and intelligent machines
(1991) · Choice Reviews Online · doi:10.5860/choice.28-6285 · Cited by 313 · see citing works
Studying those who study us: an anthropologist in the world of artificial intelligence
(2002) · Choice Reviews Online · doi:10.5860/choice.39-4050 · Cited by 269 · see citing works
Žliobaitė, I., & Custers, B. (2016) · Artificial Intelligence and Law · doi:10.1007/s10506-016-9182-5 · Cited by 125 · see citing works
Certifying and removing disparate impactPreprint
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.3756 · Cited by 21 · see citing works
Artificial experts: Social knowledge and intelligent machines
Sharples, M. (1992) · Research Policy · doi:10.1016/0048-7333(92)90040-b · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Angwin , J. et al. Machine bias . ProPublica. (May 23 , 2016 ); https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing Angwin, J. et al. Machine bias. ProPublica. (May 23, 2016); https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Cited by
Sorted by citations
Industry 4.0 and Health: Internet of Things, Big Data, and Cloud Computing for Healthcare 4.0
Aceto, G., Persico, V., & Pescapè, A. (2020) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2020.100129 · Cited by 916 · see citing works
“Big Data • new insights and actionable information from new data sources [140] • natural transformation of descriptive research into predictive and prescriptive one [141] • extreme heterogeneity [142] • opacity of analytics [143, 144]”
BackgroundKatzenbach, C., & Ulbricht, L. (2019) · Internet Policy Review · doi:10.14763/2019.4.1424 · Cited by 359 · see citing works
Problem Formulation and Fairness
Passi, S., & Barocas, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287567 · Cited by 283 · see citing works
Self-citationvan Giffen, B., Herhausen, D., & Fahse, T. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.01.076 · Cited by 230 · see citing works
Algorithmic psychometrics and the scalable subject
Stark, L. (2018) · Social Studies of Science · doi:10.1177/0306312718772094 · Cited by 168 · see citing works
Toward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153 · see citing works
“The work of data science has a normative dimension, but there is little agreement about which norms or values should be prioritized [5].”
BackgroundEthical Dimensions of Visualization Research
Correll, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300418 · Cited by 150 · see citing works
“Rather, the process of observing the world and quantifying it is a political act, and deserves ethical consideration [6].”
BackgroundAI Ethics, Ethics Washing, and the Need to Politicize Data Ethics
van Maanen, G. (2022) · Digital Society · doi:10.1007/s44206-022-00013-3 · Cited by 97 · see citing works
The Human in Emotion Recognition on Social Media: Attitudes, Outcomes, Risks
Andalibi, N., & Buss, J. (2020) · International Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376680 · Cited by 125 · see citing works
“Studies and social media corporate experiments interacting with people’s private information (e.g. the “emotion contagion” study [72]) have received public and scholarly attention, and sometimes backlash (e.g., [12,38,55,64,85,86,97,113,130,131]).”
BackgroundHow Different Groups Prioritize Ethical Values for Responsible AI
Jakesch, M., Buçinca, Z., Amershi, S., & Olteanu, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533097 · Cited by 116 · see citing works
“In addition to assessments of values themselves, we asked participants about their preferences in cases of conflicting values [4].”
BackgroundStark, L., & Crawford, K. (2019) · Surveillance & Society · doi:10.24908/ss.v17i3/4.10821 · Cited by 102 · see citing works
Brown, N., Xie, B., Sarder, E., Fiesler, C. L., & Wiese, E. S. (2023) · ACM Transactions on Computing Education · doi:10.1145/3634685 · Cited by 116 · see citing works
“By synthesizing the various approaches, we can contribute to the ongoing dialogue and guide future developments for computing ethics education.”
BackgroundAdvancing algorithmic bias management capabilities in AI-driven marketing analytics research
Akter, S., Sultana, S., Mariani, M. M., Wamba, S. F., Spanaki, K., & Dwivedi, Y. K. (2023) · Industrial Marketing Management · doi:10.1016/j.indmarman.2023.08.013 · Cited by 116 · see citing works
Construction of a quality model for machine learning systems
Siebert, J., Jöckel, L., Heidrich, J., Trendowicz, A., Nakamichi, K., Ohashi, K., Namba, I., Yamamoto, R., & Aoyama, M. (2021) · Software Quality Journal · doi:10.1007/s11219-021-09557-y · Cited by 84 · see citing works
Towards Guidelines for Assessing Qualities of Machine Learning Systems
Siebert, J., Jöckel, L., Heidrich, J., Nakamichi, K., Ohashi, K., Namba, I., Yamamoto, R., & Aoyama, M. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-58793-2_2 · Cited by 56 · see citing works
“ation tasks, the cost of a false-positive may not be the same as the cost of a falsenegative. Some metrics might not be compatible with one another, as is the case, for example, for fairness measures [20, 21]. Fig. 1. The availability of ground truth data (labels) has a direct impact on the analysis or training method…”
Managing Bias in Machine Learning Projects
Fahse, T., Huber, V., & van Giffen, B. (2021) · Lecture notes in information systems and organisation · doi:10.1007/978-3-030-86797-3_7 · Cited by 54 · see citing works
Exploring the ethical implications of business analytics with a business ethics canvas
Vidgen, R. T., Hindle, G. A., & Randolph, I. (2019) · European Journal of Operational Research · doi:10.1016/j.ejor.2019.04.036 · Cited by 51 · see citing works
“Data scientists are involved in making trade-offs, judgments (e.g., is this error rate acceptable), and some will look explicitly at issues of fairness, e.g., by testing for gender bias (Barocas and Boyd, 2017).”
BackgroundInfluentialReconfiguring Diversity and Inclusion for AI Ethics
Chi, N., Lurie, E., & Mulligan, D. K. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462622 · Cited by 51 · see citing works
“particular kinds of allocative, representational, and exclusionary harms [9].”
BackgroundChallenges in Using Big Data to Develop Decision Support Systems for Social Work in Germany
Schneider, D., & Seelmeyer, U. (2019) · Journal of Technology in Human Services · doi:10.1080/15228835.2019.1614513 · Cited by 44 · see citing works
“Previous studies have criticized the failure to consider ethical and social implications when developing technologies (Ackerman, 2000; Barocas & Boyd, 2017).”
BackgroundGood systems, bad data?: Interpretations of AI hype and failures
Slota, S. C., Fleischmann, K. R., Greenberg, S. R., Verma, N., Cummings, B., Li, L., & Shenefiel, C. (2020) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.275 · Cited by 44 · see citing works
“…change — driven by technology but bearing substantive social effects and reflecting a changing conception of the possible — issues of governance and ethics of AI are becoming substantially more important, but not necessarily better articulated or understood (Barocas & boyd, 2017; Kitchin, 2014).”
BackgroundLoi, D., Lodato, T. J., Wolf, C. T., Arar, R., & Blomberg, J. (2018) · Participatory Design Conference · doi:10.1145/3210604.3210614 · Cited by 28 · see citing works
Demographics and discussion influence views on algorithmic fairnessPreprint
Pierson, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.09124 · Cited by 26 · see citing works
“A recent review of the ethical considerations in data science argues, with respect to algorithmic fairness, that “Favoring certain fairness properties over others could just as well have reflected a difference in values” [1].”
BackgroundDon’t Look at the Data! How Differential Privacy Reconfigures the Practices of Data Science
Sarathy, J., Song, S., Haque, A., Schlatter, T., & Vadhan, S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580791 · Cited by 24 · see citing works
“Barocas and boyd emphasize the tensions between ethicists who critique the ‘ethically uninformed’ practices of data science, and data scientists who feel that they already embed nuanced ethical considerations into every step of their analyses [6].”
BackgroundLocating the work of artificial intelligence ethics
Slota, S. C., Fleischmann, K. R., Greenberg, S. R., Verma, N., Cummings, B., Li, L., & Shenefiel, C. (2022) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24638 · Cited by 25 · see citing works
“Ethical deliberation is already embedded in technology design (Barocas & Boyd, 2017), but it is in the deployment, legislation, and implementation of AI that its impacts are realized.”
BackgroundAnalítica del aprendizaje y educación basada en datos: Un campo en expansión
Figaredo, D. D., Reich, J., & Ruipérez‐Valiente, J. A. (2020) · RIED Revista Iberoamericana de Educación a Distancia · doi:10.5944/ried.23.2.27105 · Cited by 23 · see citing works
“Y así, de nuevo surge la necesidad de apoyarse en enfoques analíticos integrales, considerando los diferentes planos que forman parte de un proceso educativo (Barocas y Boyd, 2017).”
BackgroundThe importance of consulting children and young people about data literacy
Robertson, J., & Tisdall, E. K. M. (2020) · Journal of Media Literacy Education · doi:10.23860/jmle-2020-12-3-6 · Cited by 22 · see citing works
“…should be embedded throughout the design, Robertson & Tisdall ǀ Journal of Media Literacy Education, 12(3), 58-74, 2020 67 implementation and testing: “it requires making a series of small decisions that are often fraught, forcing reflection at each step” (Barocas & Boyd, 2017, p. 23).”
BackgroundSame Data, Diverging Perspectives: The Power of Visualizations to Elicit Competing Interpretations
Xiong, C., van Weelden, L., Waytz, A. G., & Franconeri, S. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3388515 · Cited by 21 · see citing works
“The story the visualization designer chooses to tell can profoundly impact on people’s decisions and actions [6,8,33].”
BackgroundResponsible AI in Education: Understanding Teachers’ Priorities and Contextual Challenges
Yin, Y., Karumbaiah, S., & Acquaye, S. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732176 · Cited by 21 · see citing works
“Importantly, these values frequently come into tension with one another [55], highlighting the need for greater teacher involvement in ethical decision-making around AI [8, 13, 18, 36, 40, 53].”
Data Science Ethos Lifecycle: Interplay of Ethical Thinking and Data Science Practice
Boenig-Liptsin, M., Tanweer, A., & Edmundson, A. (2022) · Journal of Statistics and Data Science Education · doi:10.1080/26939169.2022.2089411 · Cited by 19 · see citing works
Data hazards in synthetic biology
Zelenka, N., Cara, N. H. D., Sharma, K., Sarvaharman, S., Ghataora, J. S., Parmeggiani, F., Nivala, J., Abdallah, Z. S., Marucci, L., & Gorochowski, T. E. (2024) · Synthetic Biology · doi:10.1093/synbio/ysae010 · Cited by 16 · see citing works
Leuthe, D., Meyer-Hollatz, T., Plank, T., & Senkmüller, A. (2024) · Information Systems Frontiers · doi:10.1007/s10796-024-10526-6 · Cited by 15 · see citing works
“Organizations must keep the ML development closely aligned to ethics and must reflect on critical voices through conversations (Barocas & Boyd, 2017).”
Digitalisierung und Soziale Arbeit. Transformationen und Herausforderungen
Wunder, M., & Klinkhardt, V. J. (2021) · Verlag Julius Klinkhardt eBooks · doi:10.35468/5911 · Cited by 14 · see citing works
Risk mitigation in algorithmic accountability: The role of machine learning copies
Unceta, I., Nin, J., & Pujol, O. (2020) · PLoS ONE · doi:10.1371/journal.pone.0241286 · Cited by 13 · see citing works
Modeling Ethics: Approaches to Data Creep in Higher Education
Whitman, M. (2021) · Science and Engineering Ethics · doi:10.1007/s11948-021-00346-1 · Cited by 14 · see citing works
“In their work with data scientists, Barocas and boyd (2017) complicate critiques of data scientists by observing that ethical practice is not always apparent as explicit ethical discussion but instead is evident in smaller decisions that practitioners make.”
BackgroundAlonso, J., Toja-Alamancos, J., & Bugarín-Diz, A. (2020) · IEEE International Conference on Fuzzy Systems · doi:10.1109/fuzz48607.2020.9177770 · Cited by 14 · see citing works
“Accordingly, there is an increasing attention to AI Fairness, Accountability, Transparency and Ethics (FATE in short) [4]– [7].”
BackgroundAlzahrani, N., Meccawy, M., Samra, H. E., & El-Sabagh, H. A. (2025) · Electronics · doi:10.3390/electronics14153018 · Cited by 13 · see citing works
Hod, S., Chagal-Feferkorn, K., Elkin-Koren, N., et al. (2022) · Communications of the ACM · doi:10.1145/3506575 · Cited by 11 · see citing works
Beyond Ethics: Considerations for Centering Equity-Minded Data Science
Alexander, N., Eaton, C. D., Shrout, A. H., Tsinnajinnie, B., & Tsosie, K. S. (2022) · Journal of Humanistic Mathematics · doi:10.5642/jhummath.ocys6929 · Cited by 8 · see citing works
Environmental Adaptation and Differential Replication in Machine Learning
Unceta, I., Nin, J., & Pujol, O. (2020) · Entropy · doi:10.3390/e22101122 · Cited by 9 · see citing works
Iceberg Sensemaking: A Process Model for Critical Data Analysis
Berret, C., & Munzner, T. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3486613 · Cited by 10 · see citing works
“Critical studies of data, algorithms, and visualization have emerged as a vital area of academic research to create accountability for the social impact of data science when it harbors harmful, unexamined premises [5,9,10,20,21,25,26,28].”
Dangers of Bias in Data-Intensive Information Systems
Park, B., Rao, D. L., & Gudivada, V. N. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-4851-2_28 · Cited by 9 · see citing works
Ten Guidelines for Intelligent Systems Futures
Loi, D. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-02686-8_59 · Cited by 8 · see citing works
AL-Shboul, M. A. (2024) · Journal of Information Communication and Ethics in Society · doi:10.1108/jices-05-2023-0063 · Cited by 6 · see citing works
Schneider, D., Maier, A., Cimiano, P., et al. (2022) · Sozialer Fortschritt · doi:10.1007/978-3-658-49118-5_11 · Cited by 6 · see citing works
Schneider, D. (2021) · Verlag Julius Klinkhardt eBooks · doi:10.35468/5909-09 · Cited by 6 · see citing works
Quantitative intersectional data (QUINTA): a #metoo case study
Boyd, A. E. (2021) · Digital Commons-DePaul (DePaul University) · Cited by 6 · see citing works
Lund, B. D., & Wang, T. (2022) · Journal of Data and Information Science · doi:10.2478/jdis-2022-0018 · Cited by 5 · see citing works
“Mittelstadt and Floridi (2015) and Barocas and Boyd (2017) survey current ethical beliefs and practices of data scientists and identify gaps that may exacerbate concerns with the next generation of machine learning technologies.”
Reconfiguring Diversity and Inclusion for AI EthicsPreprint
Chi, N., Lurie, E., & Mulligan, D. K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.02407 · Cited by 5 · see citing works
“particular kinds of allocative, representational, and exclusionary harms [9].”
BackgroundKatirai, A., & Nagato, Y. (2024) · AI and Ethics · doi:10.1007/s43681-024-00477-8 · Cited by 5 · see citing works
Ethical requirements in job advertisements: A deep learning approach
Liu, R., Li, M., Sin, S., et al. (2023) · European Management Review · doi:10.1111/emre.12569 · Cited by 4 · see citing works
Learning Analytics and Higher Music Education: Perspectives and Challenges
de Reizábal, M. L., & Benito, M. (2022) · ARTSEDUCA Revista electrónica de educación en las Artes · doi:10.6035/artseduca.6831 · Cited by 4 · see citing works
Datenwissenschaften und Gesellschaft
Saner, P. (2022) · transcript Verlag eBooks · doi:10.1515/9783839462591 · Cited by 4 · see citing works
Ethics and AI StartupsPreprint
Bessen, J., Impink, S. M., Reichensperger, L., & Seamans, R. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3895939 · Cited by 5 · see citing works
“…stemming from the development and use of AI, such as unrepresentative training data, biased programmers, and an overemphasis on prediction accuracy, which may negatively impact decisionmaking or harm a demographic subgroup (Friedman & Nissenbaum 1996, Barocas & boyd 2017, Cowgill & Tucker 2019).”
User-driven prioritization of ethical principles for artificial intelligence systems
Fernholz, Y., Ermakova, T., Fabian, B., & Buxmann, P. (2024) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2024.100055 · Cited by 5 · see citing works
Precision public health, the key for future outbreak management: A scoping review
Rajendran, E. G., Hairi, F. M., Supramaniam, R. K., & Mohd, T. A. M. T. (2024) · Digital Health · doi:10.1177/20552076241256877 · Cited by 5 · see citing works
“79,80 Similarly, navigating regulatory and policy uncertainties involves advocating for adaptive regulations that keep pace with innovation while safeguarding individual rights.”
BackgroundDatenwissenschaften und Gesellschaft
Saner, P. (2022) · Digitale Soziologie · doi:10.14361/9783839462591 · Cited by 4 · see citing works
Data Ethics Club: Creating a collaborative space to discuss data ethics
Cara, N. H. D., Zelenka, N., Day, H., Bennet, E. D., Hanschke, V. A., Maggio, V., Michalec, O., Radclyffe, C. J. T., Shkunov, R., Tonkin, E. L., et al. (2022) · Patterns · doi:10.1016/j.patter.2022.100537 · Cited by 3 · see citing works
“Some data scientists do this already, 6 though there is a spectrum of ethical considerations that may go from nothing at all, to treating bias as a technical issue, to fully engaging with data ethics as a professional practice.”
BackgroundSonar, A., & Weber, K. (2022) · Franz Steiner Verlag eBooks · doi:10.25162/9783515129770 · Cited by 3 · see citing works
Cao, L., & Zhu, C. (2021) · Scientific Reports · doi:10.1038/s41598-021-03443-0 · Cited by 3 · see citing works
“and deep learners are sensitive to data characteristics; (2) it is highly challenging to learn general and robust general-purpose representations on low-quality and ultrahigh-dimensional whole-of-enterprise data; and (3) it is significant to develop universal representation learning to automatically fit specific data…”
BackgroundMethodInfluentialA theoretical model of AI bias mitigation: incentives, regulation and equilibrium
Bera, S., & Bera, I. (2025) · Journal of Information Communication and Ethics in Society · doi:10.1108/jices-06-2025-0138 · Cited by 2 · see citing works
Conflict in Community-Based Design: A Case Study of a Relationship Breakdown
Gandu, A., & Gautam, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757570 · Cited by 2 · see citing works
“In this way, technology can amplify existing inequities rather than correct them, particularly when practices are taken for granted as neutral or beneficial [6, 78, 79].”
Behavior Matters: An Alternative Perspective on Promoting Responsible Data Science
Zhang, D., Patil, A., Shoda, Y., Battle, L., & Wall, E. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710932 · Cited by 2 · see citing works
“While data science can advance important societal goals, such as fighting climate change and species extinction, it can also cause considerable societal harm [6].”
Good classification matters: conceptual engineering in data science
Köhler, S. (2025) · Synthese · doi:10.1007/s11229-024-04850-2 · Cited by 3 · see citing works
“This paper contributes to the growing and vibrant literature that does so (see, e.g., Barocas & Boyd, 2017; Leonelli, 2016b, 2020 ; Symons & Alvarado, 2022 ; Fazelpour & Danks, 2021 ; John - son, 2021 ; Barocas & Nissenbaum, 2014 ; Saltz & Dewar, 2019; Alvarado, 2023; Suresh & Guttag, 2021; Zook et…”
The Need for an Enhanced Process Methodology for Ethical Data Science Projects
Lahiri, S., & Saltz, J. (2023) · IEEE International Symposium on Ethics in Engineering, Science, and Technology (ETHICS) · doi:10.1109/ethics57328.2023.10155007 · Cited by 2 · see citing works
“Given that conflicting values with stakeholders [5] and a misalignment with team culture [13] are two major factors that impede the implementation of fairness and interpretability in ML models, it is necessary that accountability and consensus on defining fairness are taken up as preliminary steps…”
BackgroundInfluentialReflexive Practices in Software Engineering
Boyd, A. E. (2023) · doi:10.1109/fairware59297.2023.00006 · Cited by 4 · see citing works
“Barocas and Boyd [57] cautioned that ethics is not enough, especially when researchers do not acknowledge the power they yield in making decisions in the data-centric process.”
BackgroundL’émergence du mouvement IA responsable dans les organisations : structuration et enjeux
Jeannin, H. (2021) · Communication & management · doi:10.3917/comma.172.0105 · Cited by 2 · see citing works
Differential Replication for Credit Scoring in Regulated Environments
Unceta, I., Nin, J., & Pujol, O. (2021) · Entropy · doi:10.3390/e23040407 · Cited by 2 · see citing works
Ethics washing: een introductie
van Maanen, G. (2020) · Algemeen Nederlands Tijdschrift voor Wijsbegeerte · doi:10.5117/antw2020.4.020.vanm · Cited by 1 · see citing works
Natural Language Explanation Model for Decision Trees
Silva, J., Palma, H. H., Núñez, W. A. N., et al. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1432/1/012074 · Cited by 2 · see citing works
RetractedData Hazards: An open-source vocabulary of ethical hazards for data-intensive projects
Zelenka, N., Cara, N. H. D., Bennet, E. D., Clatworthy, P. L., Day, H., Garcia, I. K., García, S. R., Hanschke, V. A., Kuwertz, E. S., & Garcia, I. K. (2025) · Journal of Responsible Technology · doi:10.1016/j.jrt.2025.100110 · Cited by 6 · see citing works
Data Science: A Natural EcosystemPreprint
Porcu, E., Moukari, R. E., Najman, L., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5346934 · Cited by 1 · see citing works
Data Quality, Data Diversity and Data Provenance: An Ethical Perspective
Ramalli, E. (2024) · SpringerBriefs in applied sciences and technology · doi:10.1007/978-3-031-52962-7_4 · Cited by 2 · see citing works
Pricing and data science: The tale of two accidentally parallel transitions
Wallusch, J. (2023) · Economics and Business Review/The Poznań University of Economics Review · doi:10.18559/ebr.2023.2.739 · Cited by 1 · see citing works
“Some managers who co-exist with data scientists often presume ignorance and corrupt intentions (Barocas & Boyd, 2017).”
BackgroundTowards a Systematic Review of Data Science Programs
Nellore, N., & Zimmer, M. T. (2022) · doi:10.1145/3545947.3576356 · Cited by 1 · see citing works
“Given the increasing focus on the social and ethical implications of data[1, 3], we also focus on whether Data Science programs include such issues in their coursework and learning outcomes.”
BackgroundDifferential Replication in Machine LearningPreprint
Unceta, I., Nin, J., & Pujol, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.07981 · Cited by 2 · see citing works
“This notion of adaptability has been present in the literature since the early times of machine learning, as practitioners have had to devise ways in which to adapt theoretical proposals to their everyday life scenarios [3, 4, 5].”
BackgroundMannov, A., Andersen, A. O., & Bruun, M. H. (2020) · STS Encounters · doi:10.7146/stse.v11i1.135274 · Cited by 3 · see citing works
Azan, W., & Abdesselam, R. (2026) · Management & Prospective · doi:10.3917/g12000.421.0097
Vilkins, S. (2024) · Communication Research and Practice · doi:10.1080/22041451.2024.2351778
“…approach was also motivated by a recent call from the field of critical data science about foregrounding the expertise of professionals in critique: Barocas and Boyd (2017) comment that academics have often flattened the subjects they study, declaring professionals to be unaware of the power they…”
Ulbricht, L., & Katzenbach, C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4850176
Machine Knowledge: Information Studies and Artificial Intelligence in Dialogue
Bullard, J. A., & Abdul-Mageed, M. (2018) · Proceedings of the Annual Conference of CAIS / Actes du congrès annuel de l ACSI · doi:10.29173/cais971
- Authors
- Barocas, S., & Boyd, D.
- Affiliations
- Cornell University (US)
- Microsoft (United States) (US)
- Venue
- Communications of the ACM
- Published
- 2017-10-24
- Volume
- 60
- Issue
- 11
- Pages
- 23-25
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2766542544
- Semantic Scholar
- d770228fb12ae0ee93fd79bcd5c69c6191451f08
- Cited by
- 86
- References
- 9
- Citations by year
- 2026: 4 · 2025: 8 · 2024: 12 · 2023: 7 · 2022: 17 · 2021: 12 · 2020: 15 · 2019: 6 · 2018: 4 · 2017: 1
- FWCI
- 16.30
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Communications of the ACM (publishedVersion)
- Funders
- National Science Foundation (IIS-1413864)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Social Sciences › Decision Sciences › Information Systems and Management › Ethics in Business and Education
- Keywords
- data science practice, data scientists, professional ethics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
