No Single Metric Captures Productivity
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
The urge to measure the productivity of developers is not new. Since it is often the case at organizations that more code needs to be written, many attempts have been made to measure productivity based on lines of code (LOC). For example, in early 1982, the engineering management of developers working on software for the Apple Lisa computer decided to start tracking LOC added by each developer. One week, the main user interface designer, Bill Atkinson, optimized Quickdraw’s region calculation machinery and contributed -2000 LOC. The management stopped asking for his LOC. Although measuring engineer productivity by LOC is clearly fraught, anecdotes like this abound on the internet. Organizations have continued to search for better easy ways to measure developer productivity. We argue that there is no metric that adequately captures the full space of developer productivity, and that attempting to find one is counter-productive. Instead, we encourage the design of a set of metrics tailored for answering a specific goal.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Regardless of multiple approaches, no single metric fully captures productivity in software development (Jaspan & Sadowski, 2019).”
From Developer Productivity With and Without GitHub Copilot: A Longitudinal Mixed-Methods Case Study (2026) · cites this paper as Jaspan & Sadowski, 2019
The cited source says what the claim attributes to it.
In the citing paper
“One key challenge, as noted by Jaspan, Sadowski (2019), is that no single metric can fully capture developer productivity, which depends on many interrelated technical and non-technical factors.”
From Developer Productivity With and Without GitHub Copilot: A Longitudinal Mixed-Methods Case Study (2026) · cites this paper as Jaspan, Sadowski (2019)
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The evaluation of context-aware API completion for Visual Studio IntelliCode considered Recall@5-the proportion of completions for which the correct method call was in the top five suggestions.This metric fell from 90% in offline evaluation to 70% when used online.21”
From Measuring GitHub Copilot's Impact on Productivity (2024) · cites this paper as [21]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Goal Question Metric Approach
Basili, V. R., Caldiera, G., & Rombach, H. D. (1994) · Cited by 1,651
Shin, Y., Meneely, A., Williams, L., & Osborne, J. A. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.81 · Cited by 607
Learning a Metric for Code Readability
Buse, R. P. L., & Weimer, W. R. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.70 · Cited by 396
Summarizing and measuring development activity
Treude, C., Figueira Filho, F., & Kulesza, U. (2015) · Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2786805.2786827 · Cited by 42
Cited by
Pandemic programming: How COVID-19 affects software developers and how their organizations can help
Ralph, P., Baltes, S., Adisaputri, G., Torkar, R., Kovalenko, V., Kalinowski, M., Novielli, N., Yoo, S., Devroey, X., Tan, X., et al. (2020) · Empirical Software Engineering · doi:10.1007/s10664-020-09875-y · Cited by 269
"How Was Your Weekend?" Software Development Teams Working From Home During COVID-19
Miller, C., Rodeghero, P., Storey, M.-A., Ford, D., & Zimmermann, T. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00064 · Cited by 126
Measuring GitHub Copilot's Impact on Productivity
Ziegler, A., Kalliamvakou, E., Li, X. A., Rice, A., Rifkin, D., Simister, S., Sittampalam, G., & Aftandilian, E. (2024) · Communications of the ACM · doi:10.1145/3633453 · Cited by 124
Does working from home work? A natural experiment from lockdowns
Shen, L. (2022) · European Economic Review · doi:10.1016/j.euroecorev.2022.104323 · Cited by 37
Uddin, G., Alam, O., & Serebrenik, A. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10156-z · Cited by 21
Metadata
- Authors
- Jaspan, C., & Sadowski, C.
- Affiliations
- Google (United States) (US)
- Venue
- Rethinking Productivity in Software Engineering
- Publisher
- Apress
- Published
- 2019
- Pages
- 13-20
- OpenAlex
- W2944429713
- Cited by
- 20
- References
- 4
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 4 · 2023: 5 · 2022: 3 · 2021: 1 · 2020: 3
- FWCI
- 5.26
- Type
- book chapter
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- Locations
- Apress eBooks (publishedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Keywords
- developer productivity, productivity indicators, lines of code, software metrics, software engineering management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: No Single Metric Captures Productivity. https://app.cochat.ai/papers/10.1007/978-1-4842-4221-6_2
- The paper
- Jaspan, C., & Sadowski, C. (2019). No Single Metric Captures Productivity. Rethinking Productivity in Software Engineering. https://doi.org/10.1007/978-1-4842-4221-6_2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.