CoChat
Sign Up
Cited by 38Open Source Software InnovationsSoftware Engineering Techniques and PracticesSoftware Engineering Research

On the Relationship Between the Developer’s Perceptible Race and Ethnicity and the Evaluation of Contributions in OSS

Open access
Nadri, Reza, Rodríguez-Pérez, Gema, Nagappan, Meiyappan·2022·IEEE Transactions on Software Engineering
DOI 10.1109/tse.2021.3073773
Open this paper in CoChat — free accountOpen PDF

Abstract

Context:Open Source Software (OSS) projects are typically the result of collective efforts performed by developers with different backgrounds. Although the quality of developers’ contributions should be the only factor influencing the evaluation of the contributions to OSS projects, recent studies have shown that diversity issues are correlated with the acceptance or rejection of developers’ contributions.Objective:This paper assists this emerging state-of-the-art body on diversity research with the first empirical study that analyzes how developers’ perceptible race and ethnicity relates to the evaluation of the contributions in OSS. We also want to create awareness of the racial and ethnic diversity in OSS projects.Methodology:We performed a large-scale quantitative study of OSS projects in GitHub. We extracted the developers’ perceptible race and ethnicity from their names in GitHub using the Name-Prism tool and applied regression modeling of contributions (i.e, pull requests) data from GHTorrent and GitHub.Results:We observed that (1) among the developers whose perceptible race and ethnicity was captured by the tool, only 16.56 percent were perceptible as Non-White developers; (2) contributions from perceptible White developers have about 6–10 percent higher odds of being accepted when compared to contributions from perceptible Non-White developers; and (3) submitters with perceptible non-white races and ethnicities are more likely to get their pull requests accepted when the integrator is estimated to be from their same race and ethnicity rather than when the integrator is estimated to be White.Conclusion:Our initial analysis shows a low number of Non-White developers participating in OSS. Furthermore, the results from our regression analysis lead us to believe that there may exist differences between the evaluation of the contributions from different perceptible races and ethnicities. Thus, our findings reinforce the need for further studies on racial and ethnic diversity in software engineering to foster healthier OSS communities.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tse.2021.3073773

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Both open source and open collaboration in general are surprisingly lacking in diversity, with poor representation of non-male and non-white participants.

    “Yet both open source-and open collaboration in general-is surprisingly lacking in diversity, with poor representation of non-male and non-white participants [33, 73, 95, 109].”

    From Paradoxes of Openness: Trans Experiences in Open Source Software (2024) · cites this paper as [73]

    The Abstract reports that only 16.56% of GitHub developers whose perceptible race or ethnicity could be identified were perceptible as non-White, and Section 2.2 describes a gender imbalance in OSS. These passages support the claim about OSS, but the paper does not establish poor representation across open collaboration in general. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Black and Latinx contributors have especially low involvement numbers in OSS.

    “Black and Latinx contributors have especially low involvement numbers in OSS [73].”

    From Paradoxes of Openness: Trans Experiences in Open Source Software (2024) · cites this paper as [73]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 40

  1. D. R. Avery, P. F. McKay, S. Tonidandel, S. D. Volpone, and M. A. Morris, "Is there method to the madness? examining how racioethnic matching influences retail store productivity," Person- nel Psychology, vol. 65, no. 1, pp. 167-199, 2012.
  2. A. D. Galinsky, A. R. Todd, A. C. Homan, K. W. Phillips, E. P. Apfelbaum, S. J. Sasaki, J. A. Richeson, J. B. Olayon, and W. W. Maddux, "Maximizing the gains and minimizing the pains of di- versity: A policy perspective," Perspectives on Psychological Science, vol. 10, no. 6, pp. 742-748, 2015.
  3. A. Reynolds and D. Lewis, "Teams solve problems faster when they're more cognitively diverse," Harvard Business Review, vol. 23, p. 2019, 2017.
  4. C. P. Earley and E. Mosakowski, "Creating hybrid team cultures: An empirical test of transnational team functioning," Academy of Management journal, vol. 43, no. 1, pp. 26-49, 2000.
  5. B. Vasilescu, D. Posnett, B. Ray, M. G. van den Brand, A. Sere- brenik, P. Devanbu, and V. Filkov, "Gender and tenure diversity in github teams," in Proceedings of the 33rd annual ACM conference on human factors in computing systems. New York, NY: ACM, 2015, pp. 3789-3798.
  6. V. Pieterse, D. G. Kourie, and I. P. Sonnekus, "Software engineer- ing team diversity and performance," in Proceedings of the 2006 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries. Grahamstown, South Africa: South African Institute for Computer Scientists and Information Technologists, 2006, pp. 180-186.
  7. G. Catolino, F. Palomba, D. A. Tamburri, A. Serebrenik, and F. Ferrucci, "Gender diversity and women in software teams: How do they affect community smells?" in Proceedings of the 41st Inter- national Conference on Software Engineering: Software Engineering in Society. New York, NY: IEEE, 2019, pp. 11-20.
  8. W. Scacchi, "Free/open source software development: Recent re- search results and methods," Advances in Computers, vol. 69, pp. 243-295, 2007.
  9. J. Tsay, L. Dabbish, and J. Herbsleb, "Influence of social and tech- nical factors for evaluating contribution in github," in Proceedings of the 36th international conference on Software engineering. New York, NY: ACM, 2014, pp. 356-366.
  10. R. N. Iyer, S. A. Yun, M. Nagappan, and J. Hoey, "Effects of personality traits on pull request acceptance," IEEE Transactions on Software Engineering, 2019.
  11. A. Rastogi, N. Nagappan, G. Gousios, and A. van der Hoek, "Relationship between geographical location and evaluation of developer contributions in github," in Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineer- ing and Measurement. New York, NY: ACM, 2018, p. 22.
  12. B. Vasilescu, V. Filkov, and A. Serebrenik, "Perceptions of diversity on git hub: A user survey," in 2015 IEEE/ACM 8th International Workshop on Cooperative and Human Aspects of Software Engineering. IEEE, 2015, pp. 50-56.
  13. R. T. Fielding, "Shared leadership in the apache project," Commu- nications of the ACM, vol. 42, no. 4, pp. 42-43, 1999.
  14. D. C. Schmidt and A. Porter, "Leveraging open-source commu- nities to improve the quality & performance of open-source soft- ware," in Proceedings of the 1st Workshop on Open Source Software Engineering, vol. 1. Citeseer, 2001.
  15. D. E. Byrne, The attraction paradigm. Cambridge, MA: Academic Pr, 1971, vol. 11.
  16. H. Tajfel, "Social psychology of intergroup relations," Annual review of psychology, vol. 33, no. 1, pp. 1-39, 1982.
  17. J. Ye, S. Han, Y. Hu, B. Coskun, M. Liu, H. Qin, and S. Skiena, "Nationality classification using name embeddings," in Proceed- ings of the 2017 ACM on Conference on Information and Knowledge Management. New York, NY: ACM, 2017, pp. 1897-1906.
  18. S. Cornell and D. Hartmann, Ethnicity and race: Making identities in a changing world. Sage Publications, 2006.
  19. T. Duster, "Debating reality and relevance," Science, vol. 324, no. 5931, p. 1144, 2009.
  20. L. Cosmides, J. Tooby, and R. Kurzban, "Perceptions of race," Trends in cognitive sciences, vol. 7, no. 4, pp. 173-179, 2003.
  21. M. Bertrand and S. Mullainathan, "Are emily and greg more employable than lakisha and jamal? a field experiment on labor market discrimination," American economic review, vol. 94, no. 4, pp. 991-1013, 2004.
  22. J. S. B. Evans, "In two minds: dual-process accounts of reasoning," Trends in cognitive sciences, vol. 7, no. 10, pp. 454-459, 2003.
  23. F. Calefato, F. Lanubile, and B. Vasilescu, "A large-scale, in-depth analysis of developers' personalities in the apache ecosystem," Information and Software Technology, vol. 114, pp. 1-20, 2019.
  24. G. Gousios and A. Zaidman, "A dataset for pull-based develop- ment research," in Proceedings of the 11th Working Conference on Mining Software Repositories. New York, NY: ACM, 2014, pp. 368- 371.
  25. G. Gousios, "The ghtorent dataset and tool suite," in 2013 10th Working Conference on Mining Software Repositories (MSR). New York, NY: IEEE, 2013, pp. 233-236.
  26. N. Munaiah, S. Kroh, C. Cabrey, and M. Nagappan, "Curating github for engineered software projects," Empirical Software Engi- neering, vol. 22, no. 6, pp. 3219-3253, 2017.
  27. J. R. Finkel, T. Grenager, and C. Manning, "Incorporating non- local information into information extraction systems by gibbs sampling," in Proceedings of the 43rd annual meeting on association for computational linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2005, pp. 363-370.
  28. G. King and L. Zeng, "Logistic regression in rare events data," Political Analysis, vol. 9, pp. 137-163, Spring 2001.
  29. H. S. Qiu, A. Nolte, A. Brown, A. Serebrenik, and B. Vasilescu, "Going farther together: The impact of social capital on sustained participation in open source," in 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 2019, pp. 688-699.
  30. C. Casalnuovo, B. Vasilescu, P. Devanbu, and V. Filkov, "Developer onboarding in github: the role of prior social links and language experience," in Proceedings of the 2015 10th joint meeting on founda- tions of software engineering, 2015, pp. 817-828.
  31. A. Cuevas, M. Febrero, and R. Fraiman, "An anova test for functional data," Computational statistics & data analysis, vol. 47, no. 1, pp. 111-122, 2004.
  32. H. Akaike, "A new look at the statistical model identification," IEEE Transactions on Automatic Control, vol. 19, no. 6, pp. 716-723, 1974.
  33. G. Schwarz et al., "Estimating the dimension of a model," The annals of statistics, vol. 6, no. 2, pp. 461-464, 1978.
  34. D. Pontille and D. Torny, "The blind shall see! the question of anonymity in journal peer review." The Question of Anonymity in Journal Peer Review, 2014.
  35. J. S. Ross, C. P. Gross, M. M. Desai, Y. Hong, A. O. Grant, S. R. Daniels, V. C. Hachinski, R. J. Gibbons, T. J. Gardner, and H. M. Krumholz, "Effect of blinded peer review on abstract acceptance," Jama, vol. 295, no. 14, pp. 1675-1680, 2006.
  36. H.-D. Daniel, Guardians of science: Fairness and reliability of peer review. Wiley-VCH, 1993.
  37. E. Ernst and T. Kienbacher, "Chauvinism," Nature, vol. 352, no. 6336, pp. 560-560, 1991.
  38. S. Zhou, B. Vasilescu, and C. Kästner, "What the fork: a study of inefficient and efficient forking practices in social coding," in Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2019, pp. 350-361.
  39. G. Gousios, M. Pinzger, and A. v. Deursen, "An exploratory study of the pull-based software development model," in Proceedings of the 36th International Conference on Software Engineering, 2014, pp. 345-355.
  40. B. K. AlShebli, T. Rahwan, and W. L. Woon, "The preeminence of ethnic diversity in scientific collaboration," Nature communications, vol. 9, no. 1, pp. 1-10, 2018.

Cited by

Sorted by citations

Authors
Nadri, R., Rodríguez-Pérez, G., & Nagappan, M.
Affiliations
University of Waterloo (CA)
ORCID
Venue
IEEE Transactions on Software Engineering
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2022-08-01
Volume
48
Issue
8
Pages
2955-2968
ISSN
  • 0098-5589 (ISSN-L)
  • 1939-3520
  • 2326-3881
DOI
10.1109/tse.2021.3073773
OpenAlex
W3153182107
Cited by
38
References
2
Citations by year
2026: 4 · 2025: 10 · 2024: 14 · 2023: 8 · 2022: 2
FWCI
4.74
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Software Engineering (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • Natural Sciences and Engineering Research Council of Canada
  • Social Sciences and Humanities Research Council of Canada
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
Keywords
open source software, racial and ethnic diversity, pull request acceptance, contribution evaluation, GitHub, open source communities

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review