The Types, Roles, and Practices of Documentation in Data Analytics Open Source Software Libraries: A Collaborative Ethnography of Documentation Work
Open accessPublished version of a preprint
Abstract
Computational research and data analytics increasingly relies on complex ecosystems of open source software (OSS) “libraries” – curated collections of reusable code that programmers import to perform a specific task. Software documentation for these libraries is crucial in helping programmers/analysts know what libraries are available and how to use them. Yet documentation for open source software libraries is widely considered low-quality. This article is a collaboration between CSCW researchers and contributors to data analytics OSS libraries, based on ethnographic fieldwork and qualitative interviews. We examine several issues around the formats, practices, and challenges around documentation in these largely volunteer-based projects. There are many different kinds and formats of documentation that exist around such libraries, which play a variety of educational, promotional, and organizational roles. The work behind documentation is similarly multifaceted, including writing, reviewing, maintaining, and organizing documentation. Different aspects of documentation work require contributors to have different sets of skills and overcome various social and technical barriers. Finally, most of our interviewees do not report high levels of intrinsic enjoyment for doing documentation work (compared to writing code). Their motivation is affected by personal and project-specific factors, such as the perceived level of credit for doing documentation work versus more ‘technical’ tasks like adding new features or fixing bugs. In studying documentation work for data analytics OSS libraries, we gain a new window into the changing practices of data-intensive research, as well as help practitioners better understand how to support this often invisible and infrastructural work in their projects.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A playground or online editor is a common documentation component for external DSL-style libraries [14] (e.g. VEGA/VEGA-LITE, PENROSE, IDYLL, NOTA, etc.).
“A playground or online editor is a common documentation component for external DSL-style libraries [14] (e.g.VEGA/VEGA-LITE, PENROSE, IDYLL, NOTA, etc.).”
From Considering Visualization Example Galleries (2024) · cites this paper as [14]
Section 2.1.1 describes code examples, galleries, and API documentation that development environments can render interactively. It does not address whether playgrounds or online editors are common documentation components for external DSL-style libraries. Searches for those terms found no passage establishing the claim. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449 · see citing works
Lave, J., & Wenger, E. (1991) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815355 · Cited by 33,219 · see citing works
Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions
Ryan, R. M., & Deci, E. L. (2000) · Contemporary Educational Psychology · doi:10.1006/ceps.1999.1020 · Cited by 18,666 · cited 2× · see citing works
“tionwork.Ourintervieweesallbelievedthatdocumentationwas importantandvaluablefortheirprojects,buttherewasarangeofattitudestoward doing documentation work. In line with previous theoretical literature (Ryan and Deci 2000), we found it more useful to put interviewees’ expressed motivations on a spectrum between fully int…”
BackgroundLakhani, K. R., & Wolf, R. G. (2003) · doi:10.2139/ssrn.443040 · Cited by 1,531 · see citing works
Suchman, L. (1995) · Communications of the ACM · doi:10.1145/223248.223263 · Cited by 991 · see citing works
Best Practices for Scientific Computing
Wilson, G., Aruliah, D. A., Brown, C. T., Chue Hong, N. P., Davis, M., Guy, R. T., Haddock, S. H. D., Huff, K. D., Mitchell, I. M., Plumbley, M. D., et al. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001745 · Cited by 723 · see citing works
Storey, M.-A., Zagalsky, A., Filho, F. F., Singer, L., & German, D. M. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2584053 · Cited by 221 · see citing works
Zagalsky, A., Teshima, C. G., German, D. M., Storey, M.-A., & Poo-Caamaño, G. (2016) · Proceedings of the 13th International Conference on Mining Software Repositories · doi:10.1145/2901739.2901772 · Cited by 29 · see citing works
Flagged first, then by citations
Research works printed but not linked: 48
- Balestra, Martina; Coye Cheshire; Ofer Arazy; and Oded Nov (2017). Investigating the Motivational Paths of Peer Production Newcomers. In: CHI 2017. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, 6-11 May 2017. New York, NY: ACM Press, pp. 6381-6385.
- Bangerth, Wolfgang; and Timo Heister (2013). What Makes Computational Open Source Soft- ware Libraries Successful? Computational Science & Discovery, vol. 6, no. 1, pp. 1-18. https://doi.org/10.1088/1749-4699/6/1/015010. · DOI
- Berg, Marc; and Geoffrey Bowker (1997). The Multiple Bodies of the Medical Record. Sociological Quarterly, vol. 38, no. 3, pp. 513-537.
- Bowker, Geoffrey C; and Susan Leigh Star (1999). Sorting Things Out: Classification and Its Consequences. Cambridge, MA: The MIT Press.
- Buckland, Michael K (1997). What is a "Aocument"? Journal of the American Society for Information Science, vol. 48, no. 9, pp. 804-809.
- Cohn, Marisa Leavitt; Susan Elliott Sim; and Charlotte P. Lee (2009). What Counts as Software Process? Negotiating the Boundary of Software Work Through Artifacts and Conversation. Computer Supported Cooperative Work (CSCW), vol. 18, no. 5, pp. 401-443.
- Crouch, Stephen; Neil Chue Hong; Simon Hettrick; Mike Jackson; Aleksandra Pawlik; Shoaib Sufi; Les Carr; David De Roure; Carole Goble; and Mark Parsons (2013). The Software Sustain- ability Institute: Changing Research Software Attitudes and Practices. Computing in Science & Engineering, vol. 15, no. 6, pp. 74-80. https://doi.org/10.1109/MCSE.2013.133. · DOI
- Crowston, Kevin (1997). A Coordination Theory Approach to Organizational Process Design. Organization Science, vol. 8, no. 2, pp. 157-175. https://doi.org/10.1287/orsc.8.2.157. · DOI
- Crowston, Kevin; Hala Annabi; and James Howison (2003). Defining Open Source Software Project Success. In: ICIS 2003. Proceedings of the Twenty-Fourth International Conference on Informa- tion Systems, Seattle, Washington, 15-17 December 2003. pp. 1-14. https://surface.syr.edu/cgi/ viewcontent.cgi?article=1080&context=istpub.
- Dabbish, Laura; Colleen Stuart; Jason Tsay; and Jim Herbsleb (2012). Social Coding in GitHub: Transparency and Collaboration in an Open Software Repository. In: CSCW '12. Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, Seattle, WA, 11-15 February 2012. New York, NY: ACM Press, pp. 1277-1286.
- Darville, Richard (1995). Literacy, Experience, Power. In: M. Campbell and A. Manicom (eds.): Knowledge, Experience and Ruling Relations: Studies in the Social Organization of Knowledge. Toronto: University of Toronto Press, pp. 249-261.
- de Souza, Cleidson R. B.; Fernando Figueira Filho; Müller Miranda; Renato Pina Ferreira; Christoph Treude; and Leif Singer (2016). The Social Side of Software Platform Ecosystems. In: CHI '16. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, 7-12 May 2016. New York, NY: ACM Press, pp. 3204-3214.
- Garfinkel, Harold (1967). Studies in Ethnomethodology. Cambridge, UK: Polity Press.
- Geiger, R. Stuart (2017). Beyond Opening up the Black Box: Investigating the Role of Algorithmic Systems in Wikipedian Organizational Culture. Big Data & Society, vol. 4, no. 2, pp. 1-14. https://doi.org/10.1177/2053951717730735. · DOI
- Geiger, R. Stuart; and David Ribes (2011). Trace Ethnography: Following Coordination through Documentary Practices. In: HICSS 2011. Hawaii International Conference on System Sciences, Lihue, HI, 4-7 January 2011. IEEE, pp. 1-10. http://stuartgeiger.com/papers/trace-ethnography- hicss.pdf.
- Gousios, Georgios; Margaret-Anne Storey; and Alberto Bacchelli (2016). Work Practices and Challenges in Pull-based Development. In: ICSE '16. Proceedings of the 38th International Con- ference on Software Engineering, Austin, Texas, 14-22 May 2016. New York, NY: ACM Press, pp. 285-296.
- Halfaker, Aaron; R. Stuart Geiger; Jonathan T Morgan; and John Riedl (2013). The Rise and Decline of an Open Collaboration System: How Wikipedia's Reaction to Popularity Is Causing Its Decline. American Behavioral Scientist, vol. 57, no. 5, pp. 664-688. http://stuartgeiger.com/ papers/abs-rise-and-decline-wikipedia.pdf.
- Hinds, Pamela; and Diane Bailey (2003). Out of Sight, out of Sync: Understanding Conflict in Distributed Teams. Organization Science, vol. 14, no. 6, pp. 615-632.
- Hovde, Marjorie Rush (2000). Tactics for Building Images of Audience in Organizational Con- texts: An Ethnographic Study of Technical Communicators. Journal of Business and Technical Communication, vol. 14, no. 4, pp. 395-444.
- Kelty, Christopher M (2008). Two Bits: The Cultural Significance of Free Software. Durham, NC: Duke University Press.
- Kitzes, J.; D. Turek; and F. Deniz (eds.) (2018). The Practice of Reproducible Research: Case Studies and Lessons from the Data-Intensive Sciences. Oakland, CA: University of California Press. http://practicereproducibleresearch.org.
- Langtangen, Hans Petter; Timothy J Barth; and Michael Griebel (2006). Python Scripting for Computational Science. Berlin: Springer.
- Lassiter, Luke (2005). The Chicago Guide to Collaborative Ethnography. Chicago: University of Chicago Press.
- Latour, Bruno; and Steve Woolgar (1979). Laboratory Life: The Social Construction of Scientific Facts. Beverly Hills, CA: Sage Publications.
- Lee, Charlotte P.; Paul Dourish; and Gloria Mark (2006). The Human Infrastructure of Cyberin- frastructure. In: CSCW 2006. Proceedings of the 2006 ACM conference on Computer Supported Cooperative Work, Banff, Alberta, Canada, 4-8 November 2006. New York, NY: ACM Press, pp. 483-492.
- Luff, P.; Jon; Hindmarsh; and Christian Heath (eds.) (2000). Workplace Studies: Recovering Work Practice and Informing System Design. Cambridge, UK: Cambridge University Press.
- Menking, Amanda; and Ingrid Erickson (2015). The Heart Work of Wikipedia: Gendered, Emo- tional Labor in the World's Largest Online Encyclopedia. In: CHI '15. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Republic of Korea, 18-23 April 2015. New York, NY: ACM Press, pp. 207-210.
- Muchnik, Lev; Sen Pei; Lucas C Parra; Saulo DS Reis; José S Andrade Jr; Shlomo Havlin; and Hernán A Makse (2013). Origins of Power-Law Degree Distribution in the Heterogene- ity of Human Activity in Social Networks. Scientific Reports, vol. 3, no. 1783, pp. 1-8. https://doi.org/10.1038/srep01783. · DOI
- Osterlund, Carsten; and Paul Carlile (2005). Relations in Practice: Sorting Through Practice Theo- ries on Knowledge Sharing in Complex Organizations. The Information Society, vol. 21, no. 2, pp. 91-107.
- Parkinson, C. N. (1957). Parkinson's Law and Other Studies in Administration. Cambridge, MA: Houghton Milton.
- Parnin, Chris; Christoph Treude; and Margaret-Anne Storey (2013). Blogging Developer Knowl- edge: Motivations, Challenges, and Future Directions. In: ICPC 2013. Proceedings of the 21st International Conference on Program Comprehension, San Francisco, CA, 21-23 May 2013. Piscataway, NJ: IEEE, pp. 211-214. http://ieeexplore.ieee.org/document/6613850.
- Poche, Elizabeth; Nishant Jha; Grant Williams; Jazmine Staten; Miles Vesper; and Anas Mahmoud (2017). Analyzing User Comments on YouTube Coding Tutorial Videos. In: ICPC '17. Proceed- ings of the 2017 IEEE/ACM 25th International Conference on Program Comprehension, Buenos Aires, Argentina, 22-23 May 2017. Piscataway, NJ: IEEE, pp. 196-206. http://ieeexplore.ieee. org/document/7961517.
- Ribes, David; and Thomas A. Finholt (2007). Tensions Across the Scales: Planning Infrastructure for the Long-Term. In: GROUP 2007. Proceedings of the 2007 international ACM conference on Supporting group work, Sanibel Island, FL, 4-7 November 2007. New York, NY: ACM Press, p. 229.
- Sandve, Geir Kjetil; Anton Nekrutenko; James Taylor; and Eivind Hovig (2013). Ten Simple Rules for Reproducible Computational Research. PLOS Computational Biology, vol. 9, no. 10, pp. 1-4.
- Sellen, Abigail J.; and Richard H. R. Harper (2003). The Myth of the Paperless Office. Cambridge, MA: The MIT Press.
- Sharma, Pankajeshwara; Bastin Tony Roy Savarimuthu; Nigel Stanger; Sherlock A. Licorish; and Austen Rainer (2017). Investigating Developers' Email Discussions During Decision-making in Python Language Evolution. In: EASE'17. Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering, Karlskrona, Sweden, 15-16 June 2017. New York, NY: ACM Press, pp. 286-291. https://doi.org/10.1145/3084226.3084271. · DOI
- Singer, Leif; Fernando Figueira Filho; and Margaret-Anne Storey (2014). Software Engineering at the Speed of Light: How Developers Stay Current Using Twitter. In: ICSE 2014. Proceedings of the 36th International Conference on Software Engineering, Hyderabad, India, 31 May -7 June 2014. New York, NY: ACM Press, pp. 211-221.
- Star, Susan Leigh (1999). The Ethnography of Infrastructure. American Behavioral Scientist, vol. 43, no. 3, pp. 377-391.
- Star, Susan Leigh; and Anselm Strauss (1999). Layers of Silence, Arenas of Voice: The Ecology of Visible and Invisible Work. Computer Supported Cooperative Work (CSCW), vol. 8, no. 1, pp. 9-30.
- Steinmacher, Igor; Marco Aurelio; Graciotto Silva; Marco Aurelio Gerosa; and David F. Red- miles (2015). A Systematic Literature Review on the Barriers Faced by Newcomers to Open Source Software Projects. Information and Software Technology, vol. 59, pp. 67-85. https:// www.sciencedirect.com/science/article/pii/S0950584914002390.
- Storey, Margaret-Ae; Jody Ryall; R Ian Bull; Del Myers; and Janice Singer (2008). TODO or to Bug: Exploring How Task Annotations Play a Role in the Work Practices of Software Develop- ers. In: ICSE 2008. Proceedings of the 30th international conference on Software engineering, Leipzig, Germany, 10-18 May 2008. pp. 251-260. http://ieeexplore.ieee.org/document/4814136.
- Strauss, Anselm (1988). The Articulation of Project Work: An Organizational Process. The Sociological Quarterly, vol. 29, no. 2, pp. 163-178.
- Tkacz, Nathaniel (2014). Wikipedia and the Politics of Openness. Chicago: University of Chicago Press.
- Trace, Ciaran B (2011). Documenting Work and Working Documents: Perspectives from Work- place Studies, CSCW, and Genre Studies. In: HICSS 2011. 2011 44th Hawaii International Conference on System Sciences, Kauai, Hawaii, 4-7 January 2011. Piscataway, NJ: IEEE, pp. 1-10.
- Van der Meij, Hans (1995). Principles and Heuristics for Designing Minimalist Instruction. Technical communication, vol. 42, no. 2, pp. 243-261.
- Vaughan, Diane (1997). The Challenger Launch Decision: Risky Technology, Culture, and Deviance at NASA. Chicago, IL: University of Chicago Press.
- Weiss, Edmond H. (1985). How to Write a Usable User Manual. Philadelphia: ISI Press.
- Zagalsky, Alexey; Carlos Gómez Teshima; Daniel M. German; Margaret-Anne Storey; and Germán Poo-Caamaño (2016). How the R Community Creates and Curates Knowledge. In: MSR '16. Proceedings of the 13th International Workshop on Mining Software Repositories, Austin, Texas, 14-22 May 2016. New York, NY: ACM Press, pp. 441-451.
Cited by
Sorted by citations
Wang, A. Y., Wang, D., Drozdal, J., Liu, X., Park, S., Oney, S., & Brooks, C. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451617 · Cited by 31 · see citing works
Muszynski, M., Lugtigheid, S., Castor, F., & Brinkkemper, S. (2022) · 2022 IEEE 19th International Conference on Software Architecture (ICSA) · doi:10.1109/icsa53651.2022.00013 · Cited by 10 · see citing works
Considering Visualization Example Galleries
Yang, J., McNutt, A. M., & Battle, L. (2024) · 2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00043 · Cited by 5 · see citing works
“Galleries are but one part of a constellation of different types and forms of documentation written by software creators and consumed by end users [15].”
Background
- Authors
- Geiger, R. S., Varoquaux, N., Mazel-Cabasse, C., & Holdgraf, C.
- Affiliations
- University of California, Berkeley (US)
- Corresponding author
- Geiger, R. Stuart
- Venue
- Computer Supported Cooperative Work (CSCW)
- Published
- 2018-05-29
- Volume
- 27
- Issue
- 3-6
- Pages
- 767-802
- ISSN
- 0925-9724 (ISSN-L)
- 1573-7551
- OpenAlex
- W4289219086
- Cited by
- 32
- References
- 8
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 6 · 2023: 3 · 2022: 3 · 2021: 8 · 2020: 6 · 2019: 1 · 2018: 1
- FWCI
- 5.05
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Computer Supported Cooperative Work (CSCW) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion, other-oa)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Funders
- Gordon and Betty Moore Foundation (GBMF3834)
- Alfred P. Sloan Foundation (2013-10-27)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- software documentation, open-source library, data analysis software, volunteer contributors, documentation practices, documentation quality, contributor motivation, data-intensive research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review