The Labor of Maintaining and Scaling Free and Open-Source Software Projects
Open accessThe DOI matches its Crossref record; 146 of the 146 references it cites were found in a registry or an index; 5 of the 39 sources its claims cite were checked for support.
Abstract
Free and/or open-source software (or F/OSS) projects now play a major and dominant role in society, constituting critical digital infrastructure relied upon by companies, academics, non-profits, activists, and more. As F/OSS has become larger and more established, we investigate the labor of maintaining and sustaining those projects at various scales. We report findings from an interview-based study with contributors and maintainers working in a wide range of F/OSS projects. Maintainers of F/OSS projects do not just maintain software code in a more traditional software engineering understanding of the term: fixing bugs, patching security vulnerabilities, and updating dependencies. F/OSS maintainers also perform complex and often-invisible interpersonal and organizational work to keep their projects operating as active communities of users and contributors. We particularly focus on how this labor of maintaining and sustaining changes as projects and their software grow and scale across many dimensions. In understanding F/OSS to be as much about maintaining a communal project as it is maintaining software code, we discuss broadly applicable considerations for peer production communities and other socio-technical systems more broadly.
Overview
CoChat Summary
TL;DR
This qualitative interview study examines how the labor of maintaining free and/or open-source software (F/OSS) projects changes as projects scale across users, contributors, code, ecosystems, and institutions. The authors introduce “scalar labor” and “scalar debt” to describe the often-invisible organizational, emotional, and financial work required to sustain projects that become critical infrastructure.
Key findings
- F/OSS maintainers do not only fix bugs, patch security issues, and update dependencies; they also perform user support, code review, governance, fundraising, event organizing, evangelizing, and ecosystem coordination, with these tasks transforming rather than merely increasing as projects scale.
- Based on 37 semi-structured interviews with F/OSS maintainers in 2019–2020 (median interview length 55 minutes), the authors find that larger-scale projects face overwhelming user support, formalized governance, dedicated testing roles, and complex funding/finance labor, while smaller projects often rely on a solo or lead maintainer.
- The paper introduces “scalar labor” for work that builds a project’s capacity to meet growing needs, and “scalar debt” for deferred scaling work that later requires costly payback, especially after “catastrophic success.”
- The authors report that some maintainers become hypervisible “microcelebrities,” contrary to accounts of maintenance as invisible work, while still performing invisible labor such as grants, conflict resolution, and managing harassment; they also cite CI usage rising from 40% of the 34,000 most popular GitHub projects to 70% of the 500 most popular, and Rust running over 126,000 tests per pull request across 20 configurations.
Methods
- Qualitative study based primarily on semi-structured interviews with 37 current or former F/OSS maintainers in 2019–2020; interviews lasted a median of 55 minutes and covered personal history, work practices, governance, funding, burnout, careers, platforms, and work/life balance.
- Recruitment used personal networks, F/OSS conferences and events, a Twitter call, cold emailing, snowball sampling, and trace-ethnography-like identification of core contributors via GitHub timelines, commits, and release notes; sampling sought diversity across geography, national origin, age, employment, sector, and gender.
- Analysis was grounded in symbolic interactionism and used a grounded theory approach with iterative coding in Taguette; the authors also used member-checking by sharing transcripts and findings, and a post-interview survey completed by 85% of interviewees.
Limitations
- Stated: findings are limited by the number of interviews and recruitment methods; the study mostly covers projects that became relied upon as infrastructure and began as volunteer efforts, so it does not speak to the overwhelming majority of single-person F/OSS projects or entirely corporate-developed F/OSS projects.
- Stated: a more traditional longitudinal study would capture scale issues with more depth, and interview-based memories may be less accurate; the authors also note their own participant experience in F/OSS and funding from non-profit foundations that directly fund F/OSS projects may affect responses and analytical distance.
- Evident from the text: the sample is demographically skewed—81% men/male, 19% women/female, 0% non-binary or other, and 72% white/Caucasian—which limits generalizability, though the authors explicitly discuss the structural gender gap in F/OSS as a recruitment challenge.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 12 Not valid · 27 Not checked · 171 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Several interviewees noted that the way users interact with contributors and maintainers was crucial, with a few kind words or less-demanding phrasing going a long way for maintainers, which [132] also finds.”
§ The Labor of Maintaining and Scaling Free and Open-Source Software Projects > 4.2 Maintaining Users: User support
What the sources say
- Supportedcites [132] 132. Kangning Wei, Kevin Crowston, U. Yeliz Eseryel, and Robert Heckman. 2017. Roles and politeness behavior in community-based free/libre open source software development. Information & Management 54, 5 (2017), 573-582. https://doi.org/10.1016/j.im.2016.11.006 source
The Discussion (p. §5) states that peripheral members — the users who report bugs, make suggestions and ask questions of the core developers who write the code — routinely use positive politeness strategies such as phatics, complimenting and appreciation ('thank you. You are great.', 'I appreciate your time…') to show respect and acknowledgement for the core developers' work, and §2.2/§4 show they also rely on negative-politeness, non-imposing devices (hedges/subjunctives, indirect inquiries — the most frequent indicator at 58.9%), i.e. less-demanding phrasing. §5.2 Practical implications frames exactly this interaction as important: peripheral members 'play an important role in relationship building during everyday communication' and developers/project leaders 'should be aware of the nature of FTAs and consider how these are perceived by and the kind of impact they have on others.' The only divergence is methodological — our copy is a content analysis of two community-based FLOSS projects' messages rather than interviews — but the citing sentence attributes to [132] only the convergent finding, which the paper states. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In particular, we find that the ostensibly-technical work of software engineering takes on more organizational, communicative, and even competitive aspects at larger scales.”
§ The Labor of Maintaining and Scaling Free and Open-Source Software Projects > 1 INTRODUCTION
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As the work of investigating, triaging, and resolving issues intensified, maintainers reported feelings of demotivation, exhaustion, and burnout.”
§ The Labor of Maintaining and Scaling Free and Open-Source Software Projects > 4.2 Maintaining Users: User support
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Yet our interviewees who maintained large, well-known projects with many users identified user support as an overwhelming and never-ending chore, particularly for projects that use GitHub-style collaboration platforms.”
§ The Labor of Maintaining and Scaling Free and Open-Source Software Projects > 4.2 Maintaining Users: User support
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, this can be complicated in the global landscape of F/OSS, with several interviewees discussing cross-cultural or language barriers.”
§ The Labor of Maintaining and Scaling Free and Open-Source Software Projects > 4.2 Maintaining Users: User support
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
94 more claims
Create a free account to see all 214 claims and to verify the 8 references still open.
- Probably not a substantial claim. 77
- It may state two separate claims as one. 58
- The quote only partly states this claim. 1
Citations
Cites
Symbolic Interactionism: Perspective and Method.
Kanter, R. M., & Blumer, H. (1971) · American Sociological Review · doi:10.2307/2094060 · Cited by 9,967
Ethnography in/of the World System: The Emergence of Multi-Sited Ethnography
Marcus, G. E. (1995) · Annual Review of Anthropology · doi:10.1146/annurev.an.24.100195.000523 · Cited by 8,885 · cited 2×
“Our findings advance work that interprets scale as a multidimentional quality beyond the number of users/participants [81], and methodologies that use participants’ multiple understandings of scale as an analytic resource [86, 104].”
BackgroundThe Managed Heart: Commercialization of Human Feeling
Tropman, J. E., & Hochschild, A. R. (1984) · Journal of Policy Analysis and Management · doi:10.2307/3324333 · Cited by 8,746
The Matthew Effect in Science: The reward and communication systems of science are considered.
Merton, R. K. (1968) · Science · doi:10.1126/science.159.3810.56 · Cited by 6,720
Sorting Things Out: Classification and Its Consequences
Bowker, G. C., & Star, S. L. (1999) · doi:10.7551/mitpress/6352.001.0001 · Cited by 6,188
(1968) · PubMed · Cited by 4,803
The Ethnography of Infrastructure
STAR, S. L. (1999) · American Behavioral Scientist · doi:10.1177/00027649921955326 · Cited by 4,486
The Wealth of Networks: How Social Production Transforms Markets and Freedom
Benkler, Y. (2017) · Yale University Press eBooks · doi:10.12987/9780300127232 · Cited by 3,680
Wikinomics: how mass collaboration changes everything
(2007) · Choice Reviews Online · doi:10.5860/choice.44-6933 · Cited by 3,367
Raymond, E. (1999) · Knowledge, Technology & Policy · doi:10.1007/s12130-999-1026-0 · Cited by 2,915
Research works printed but not linked: 8
- Fernando Domínguez Rubio. 2020. Ecologies of the Modern Imagination at the Art Museum. University of Chicago Press, Chicago.
- Nadia Eghbal. 2016. Roads and bridges: The unseen labor behind our digital infrastructure. Ford Foundation.
- Nadia Eghbal. 2020. Working in Public: The Making and Maintenance of Open Source Software. Stripe Press.
- Nathan Ensmenger. 2008. Fixing things that can never be broken: Software maintenance as heterogeneous engineering. In Proceedings of the SHOT Conference.
- Dorothy Howard and R. Stuart Geiger. 2019. Ethnography, Genealogy, and Political Economy in the Post-Market Era of Free & Open-Source Software. In Proceedings of CSCW '19 Extended Abstracts.
- 10.5555/1481673 · doi:10.5555/1481673
- Christopher M Kelty. 2013. There is no free software. The Journal of Peer Production (2013). Issue 3. http://peerproduction.net/issues/issue-3-free-software-epistemics/debate/there-is-no-free-software/
- Kazys Varnelis. 2008. Invisible City: Telecommunication. Actar Barcelona, New York.
Cited by
Guo, P. (2021) · The 34th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3472749.3474819 · Cited by 42
Li, H., Vincent, N., Chancellor, S., & Hecht, B. J. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594070 · Cited by 49
Ten simple rules for funding scientific open source software
Strasser, C. A., Hertweck, K. L., Greenberg, J., Taraborelli, D., & Vu, E. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010627 · Cited by 32
A Systematic Literature Review of Infrastructure Studies in SIGCHI
Lyu, Y., Cai, J., & Carroll, J. M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757545 · Cited by 31
Jamieson, J., Yamashita, N., & Foong, E. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623340 · Cited by 26
Arancio, J. (2023) · Environmental Science & Policy · doi:10.1016/j.envsci.2023.103576 · Cited by 16
"Nip it in the Bud": Moderation Strategies in Open Source Software Projects and the Role of Bots
Hsieh, J., Kim, J., Dabbish, L., & Zhu, H. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610092 · Cited by 26
Value Tensions in OpenStreetMap: Openness, Membership, and Policy in Online Communities
Chauhan, A., Sarkar, D., Agrawaal, T. S., & Soden, R. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686919 · Cited by 15
Jamieson, J., Foong, E., & Yamashita, N. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555550 · Cited by 13
Chen, H., & Cai, W. (2023) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2023.3247464 · Cited by 9
Jahn, L., Engelbutzeder, P., Randall, D. W., Bollmann, Y., Ntouros, V., Michel, L. K., & Wulf, V. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642541 · Cited by 13
From Concept to Community: Unpacking the Work of Designing Educational and Activist Toolkits
Wilner, T., Adavi, K. A. K., Mandava, S., Bhimdiwala, A., Frluckaj, H., Turns, J., & Arif, A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642681 · Cited by 11
Plantin, J., & Thomer, A. (2023) · New Media & Society · doi:10.1177/14614448231176758 · Cited by 16
Cheng, K., Zhou, S., & Olechowski, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687008 · Cited by 14
Ethical Tensions, Norms, and Directions in the Extraction of Online Volunteer Work
Li, H., Ajmani, L., Zhou, M., Vincent, N., Hwang, S., Piccardi, T., Narayan, S., Daniel, S. L., & Veselovsky, V. (2022) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3500868.3560923 · Cited by 8
Do We Run How We Say We Run? Formalization and Practice of Governance in OSS Communities
Chakraborti, M., Atkisson, C., Stănciulescu, Ş., Filkov, V., & Frey, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641980 · Cited by 7
*READ**THIS*!! Spam as a threat for open science
Cohoon, J. (2024) · New Media & Society · doi:10.1177/14614448241248655 · Cited by 9
Galaxy single-cell & spatial omics community update: Navigating new frontiers in 2025
Loach, M., Nilchi, A. N., Chiang, D., Howells, M., Heyl, F., Rasche, H., Jakiela, J., Tekman, M., Gamal, M., Moreno, P. A., et al. (2025) · Cell Genomics · doi:10.1016/j.xgen.2025.101005 · Cited by 7
Paxton, A., Varoquaux, N., Holdgraf, C., & Geiger, R. S. (2022) · Cognitive Science · doi:10.1111/cogs.13134 · Cited by 6
Self-citationBig data, sound science, lasting impact: A framework for passive acoustic monitoring
Wall, C. C., McKenna, M. F., Hatch, L. T., Parijs, S. M. V., Bochenek, R., Dugan, P. J., Parcerisas, C., Ryan, J., Anderson, C. D., Becker, K. M., et al. (2025) · Big Earth Data · doi:10.1080/20964471.2025.2583505 · Cited by 7
Software Maintenance as Materialization of Common Knowledge
Ojala, M., & Cohn, M. L. (2024) · Engaging Science Technology and Society · doi:10.17351/ests2023.1325 · Cited by 8
Adopting Third-party Bots for Managing Online Communities
Hwang, S., Kiene, C., Ong, S., & Shaw, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653707 · Cited by 13
Fang, H., Herbsleb, J. D., & Vasilescu, B. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3608142 · Cited by 11
Gangi, P. M. D., Teigland, R., & Yetis, Z. (2022) · Information Technology and People · doi:10.1108/itp-10-2021-0751 · Cited by 6
Khadpe, P., Xu, O., Kaufman, G. F., & Kulkarni, C. E. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710997 · Cited by 5
Hellman, J., Epstein, I., Cheng, J., & Guo, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757443 · Cited by 6
Ecosystem-wide influences on pull request decisions: insights from NPM
Meijer, W., Riveni, M., & Rastogi, A. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10706-1 · Cited by 7
Invisible Labor in Open Source Software Ecosystems
Meluso, J., Casari, A., McLaughlin, K., & Trujillo, M. Z. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757417 · Cited by 15
Tales of Transitions: Seeking Scientific Software Sustainability
Cohoon, J., Du, C., & Howison, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3701208 · Cited by 4
Open Source Software Developers' Views on Public and Private Funding: A Case Study on scikit-learn
Osborne, C. (2024) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3678884.3681844 · Cited by 12
Generative AI and the Nature of WorkPreprint
Hoffmann, M., Boysel, S., Nagle, F., Peng, S., & Xu, K. S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5007084 · Cited by 61
The Emerging Artifacts of Centralized Open-Code
Choksi, M. Z., Mandel, I., Widder, D. G., & Shvartzshnaider, Y. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659019 · Cited by 5
It Is Giving Major Satisfaction: Why Fairness Matters for Software Practitioners
Sesari, E., Sarro, F., & Rastogi, A. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3757060 · Cited by 17
Labor, Capital, and Machine: Toward a Labor Process Theory for HCI
Qin, Y., & Cheon, E. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791147 · Cited by 2
Discovering Ideologies of the Open Source Software Movement
Yue, Y., Wang, Y., & Redmiles, D. (2025) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier66352.2025.00010 · Cited by 6
Exploring the Effectiveness of Open‐Source Donation Platform: An Empirical Study on Opencollective
Zhang, S., Tang, E., Gao, X., Zhang, Z., Shan, Y., Zhang, H., He, Z., Zhao, J., & Li, X. (2025) · Journal of Software Evolution and Process · doi:10.1002/smr.70033 · Cited by 4
Opening Up: Interdisciplinary Guidance for Managing Open EcosystemsPreprint
Meluso, J., Chambers, C. R., Littauer, R., Llamas, N. A., Lingo, E. L., Mhangami, M., Pitt, B., Splitter, V., & Wang, H. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4821969
Opening Software Research Data 5Ws+1H
Terzi, A., & Bibi, S. (2024) · Software · doi:10.3390/software3040021
Plegat, M., & Lewkowicz, M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711003 · Cited by 1
Underproduction analysis of open source software
Champion, K., & Hill, B. M. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10703-4 · Cited by 2
Investigating the Impact of GitHub Discussions on Maintainers’ Workload and Community Dynamics
Maciel, A., Wessel, M. S., Wiese, I., & Steinmacher, I. (2025) · Simpósio Brasileiro de Engenharia de Software (SBES) · doi:10.5753/sbes.2025.9630
Reflection on Code Contributor Demographics and Collaboration Patterns in the Rust Community
Dandamudi, R., Adaji, I., & Rodríguez-Pérez, G. (2025) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3756681.3756979
Nezhad, R. H., Guo, J., & Cheng, J. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3817037
WikiCoach: Scaffolding Learning for Novices in Online Apprenticeship Communities
Zhou, M., Zhao, M., & Terveen, L. (2026) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3774935.3806185 · Cited by 1
Beyond Code: Empirical Insights into How Team Dynamics Influence OSS Project Selection
Nirmani, S., Khalajzadeh, H., Shahin, M., & Liu, X. (2026) · CHASE@ICSE · doi:10.1145/3794860.3794901
What Makes a GitHub Project Popular? A Study of Documentation in Seven Software Ecosystems
Mursetyani, D., & Nugroho, Y. S. (2026) · International Conference on Smart Computing, IoT and Machine Learning (SIML) · doi:10.1109/siml69834.2026.11621825
A Socio-Technical Analysis of Bug-Introducing Commits in the Apache Ecosystem
Nurrofifah, H. F., Nugroho, Y. S., & Rufaida, A. S. R. (2026) · International Conference on Smart Computing, IoT and Machine Learning (SIML) · doi:10.1109/siml69834.2026.11621096
Open practices across science, software, education and other domains
Meluso, J., Chambers, C. R., Littauer, R., Llamas, N. A., Lingo, E. L., Mhangami, M., Pitt, B., Splitter, V., & Wang, H. (2026) · Nature Human Behaviour · doi:10.1038/s41562-026-02526-0
Martin, A. J., Kailash, T., & Lin, C. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3817017
Metadata
- Authors
- Geiger, R. S., Howard, D., & Irani, L.
- Affiliations
- University of California San Diego (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2021-04-13
- Volume
- 5
- Issue
- CSCW1
- Pages
- 1-28
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3137149983
- Cited by
- 49
- References
- 132
- Citations by year
- 2026: 8 · 2025: 15 · 2024: 15 · 2023: 5 · 2022: 5 · 2021: 1
- FWCI
- 5.50
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- Funders
- Alfred P. Sloan Foundation (G-2018-11354)
- Gordon and Betty Moore Foundation (GBMF3834)
- Ford Foundation (G-2018-11354)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Social Sciences › Communication › Wikis in Education and Collaboration
- Keywords
- free and open source software, software maintainers, project scaling, peer production, developer communities, interpersonal work, organizational work, socio-technical systems, digital infrastructure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.