A Comparative Case Study on the Impact of Test-Driven Development on Program Design and Test Coverage
Open accessAbstract
Test-driven development (TDD) is a programming technique in which the tests are written prior to the source code. It is proposed that TDD is one of the most fundamental practices enabling the development of software in an agile and iterative manner. Both the literature and practice suggest that TDD practice yields several benefits. Essentially, it is claimed that TDD leads to an improved software design, which has a dramatic impact on the maintainability and further development of the system. The impact of TDD on program design has seldom come under the researchers' focus. This paper reports the results from a comparative case study of three software development projects where the effect of TDD on program design was measured using object-oriented metrics. The results show that the effect of TDD on program design was not as evident as expected, but the test coverage was significantly superior to iterative test-last development.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
TDD is compatible with Agile, Scrum, XP, and Lean.
“TDD is compatible with various software development paradigms, including Agile, Scrum, XP, and Lean [5].”
From Test Driven Development and Its Impact on Program Design and Software Quality: A Systematic Literature Review (2023) · cites this paper as [5]
The Introduction links TDD to Extreme Programming (XP), and §3.1 describes a project using TDD within an agile development method. These passages support XP and an agile method, but our copy does not establish compatibility with Scrum or Lean (searched: “Agile Scrum XP Lean” and “Scrum Lean compatible software development paradigms”). (LLM, conf=0.88)
Experimental — accuracy not yet measuredTDD is a critical component in ensuring that software is developed and delivered to meet customer requirements and maintain high-quality standards in Agile, Scrum, XP, and Lean paradigms.
“TDD is a critical component in ensuring that software is developed and delivered to meet customer requirements and maintain high-quality standards in these paradigms [5].”
From Test Driven Development and Its Impact on Program Design and Software Quality: A Systematic Literature Review (2023) · cites this paper as [5]
Section 2.5 (Summary; page unavailable) says TDD may significantly improve software quality, and Section 3.2.5 reports high test coverage in a TDD project whose product was delivered to a customer as planned. The paper does not establish that TDD is critical to ensuring customer requirements or quality standards are met across Agile, Scrum, XP, and Lean; its quality finding is more qualified than the citing sentence. (LLM, conf=0.86)
Experimental — accuracy not yet measuredTDD's effectiveness depends on factors like team size, project complexity, and organizational context, requiring careful planning.
“TDD's effectiveness depends on factors like team size, project complexity, and organizational context, requiring careful planning [17, 22].”
From Test Driven Development and Its Impact on Program Design and Software Quality: A Systematic Literature Review (2023) · cites this paper as [22]
Section 2.3 says TDD may not suit all application domains, and Section 3.3 identifies product size and single-team, single-location development as limits on generalizing the study. Those passages support a qualified point about context, but they do not establish that effectiveness depends on team size, project complexity, or organizational context, or prescribe careful planning. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A metrics suite for object oriented design
Chidamber, S. R., & Kemerer, C. F. (1994) · IEEE Transactions on Software Engineering · doi:10.1109/32.295895 · Cited by 5,698 · see citing works
A validation of object-oriented design metrics as quality indicators
Basili, V. R., Briand, L., & Melo, W. L. (1996) · IEEE Transactions on Software Engineering · doi:10.1109/32.544352 · Cited by 1,730 · see citing works
Test Driven Development: By Example
Beck (2002) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 1,278 · see citing works
Iterative and incremental developments. a brief history
Larman, C., & Basili, V. R. (2003) · Computer · doi:10.1109/mc.2003.1204375 · Cited by 1,164 · see citing works
Object-oriented metrics: measures of complexity
Henderson‐Sellers, B. (1995) · Prentice-Hall, Inc eBooks · Cited by 753 · see citing works
Höst, M., Regnell, B., & Wohlin, C. (2000) · Empirical Software Engineering · doi:10.1023/a:1026586415054 · Cited by 640 · see citing works
On the effectiveness of the test-first approach to programming
Erdogmus, H., Morisio, M., & Torchiano, M. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.37 · Cited by 315 · see citing works
Balancing Agility and Discipline: A Guide for the Perplexed
Smith, P. G. (2005) · Journal of Product Innovation Management · doi:10.1111/j.0737-6782.2005.116_3.x · Cited by 282 · see citing works
Test Driven Development: A Practical Guide
Astels, D. (2003) · CERN Document Server (European Organization for Nuclear Research) · Cited by 221 · see citing works
A structured experiment of test-driven development
George, B., & Williams, L. A. (2003) · Information and Software Technology · doi:10.1016/j.infsof.2003.09.011 · Cited by 202 · see citing works
Test-driven development as a defect-reduction practice
Williams, L. A., Maximilien, E. M., & Vouk, M. A. (2005) · doi:10.1109/issre.2003.1251029 · Cited by 183 · see citing works
Extreme Programming Refactored: The Case Against XP
Stephens, M., & Rosenberg, D. (2003) · Apress eBooks · doi:10.1007/978-1-4302-0810-5 · Cited by 170 · see citing works
Experiment about test-first programming
Müller, M. M., & Hagner, O. (2002) · IEE Proceedings - Software · doi:10.1049/ip-sen:20020540 · Cited by 164 · see citing works
Assessing test-driven development at IBM
Maximilien, E. M., & Williams, L. A. (2003) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1109/icse.2003.1201238 · Cited by 138 · see citing works
Beck, K. (2001) · IEEE Software · doi:10.1109/52.951502 · Cited by 87 · see citing works
Evaluating advantages of test driven development
Canfora, G., Cimitile, A., García, F. O., Piattini, M. G., & Visaggio, C. A. (2006) · International Symposium on Empirical Software Engineering · doi:10.1145/1159733.1159788 · Cited by 83 · see citing works
On the Influence of Test-Driven Development on Software Design
Janzen, D. S., & Saiedian, H. (2006) · Proceedings · doi:10.1109/cseet.2006.25 · Cited by 81 · see citing works
Towards empirical evaluation of test-driven development in a university environment
Pančur, M., Ciglarič, M., Trampus, M., & Vidmar, T. (2004) · IEEE EUROCON - International Conference on Computer as a Tool (EUROCON) · doi:10.1109/eurcon.2003.1248153 · Cited by 73 · see citing works
Implications of test-driven development
Kaufmann, R., & Janzen, D. S. (2003) · doi:10.1145/949344.949421 · Cited by 64 · see citing works
Empirical Evaluation of Agile Software Development: The Controlled Case Study Approach
Salo, O., & Abrahamsson, P. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24659-6_29 · Cited by 47 · see citing works
Self-citationA prototype empirical evaluation of test driven development
Geras, A., Smith, M. R., & Miller, J. (2004) · doi:10.1109/metric.2004.1357925 · Cited by 36 · see citing works
Test Driven Development and Software Process Improvement in China
Lui, K. M., & Chan, K. C. C. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24853-8_27 · Cited by 34 · see citing works
Introducing Test Automation and Test-Driven Development: An Experience Report
Damm, L., Lundberg, L., & Olsson, D. (2005) · Electronic Notes in Theoretical Computer Science · doi:10.1016/j.entcs.2004.02.090 · Cited by 34 · see citing works
The Effect of Test-Driven Development on Program Code
Müller, M. (2006) · Lecture notes in computer science · doi:10.1007/11774129_10 · Cited by 11 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- Beck, K., Extreme Programming Explained, Second Edition :Embrace Change, Addison-Wesley, Boston, MA, USA, 2004.
- K. Beck, "Aim, fire", IEEE Software 18(5), IEEE Computer Soc., Los Alamitos, CA, USA, 2001, pp. 87-89.
- Abrahamsson, P., A. Hanhineva and J. Jäälinoja, "Improving Business Agility Through Technical Solutions: A Case Study on Test-Driven Development in Mobile Software Development", in Business Agility and Information Technology Diffusion, R. Baskerville, L. Mathiassen, J. Pries-Heje and J. DeGross, Springer, New York, USA, 2004.
- The definite version was published in: Siniaalto, M., & Abrahamsson, P. (2007). A comparative case study on the impact of test-driven development on program design and test coverage. In Empirical Software Engineering and Measurement, 2007. ESEM 2007. First International Symposium on (pp. 275-284). IEEE.
- T. Ihme and P. Abrahamsson, "Agile Architecting: The Use of Architectural Patterns in Mobile Java Applications", International Journal of Agile Manufacturing 8(2), International Society of Agile Manufacturing, Vaajakoski, Finland, 2005, pp. 97-112.
Cited by
Sorted by citations
Zaidman, A., Rompaey, B. V., Arie Van Deursen, A., & Demeyer, S. (2010) · Empirical Software Engineering · doi:10.1007/s10664-010-9143-7 · Cited by 172 · see citing works
The Effects of Test-Driven Development on External Quality and Productivity: A Meta-Analysis
Rafique, Y., & Mišić, V. B. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.28 · Cited by 121 · see citing works
Factors Limiting Industrial Adoption of Test Driven Development: A Systematic Review
Čaušević, A., Sundmark, D., & Punnekkat, S. (2011) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2011.19 · Cited by 87 · see citing works
A Preliminary Roadmap for Empirical Research on Agile Software Development
Dingsøyr, T., Dybå, T., & Abrahamsson, P. (2008) · Agile Conference · doi:10.1109/agile.2008.50 · Cited by 74 · see citing works
Self-citationImpact of test-driven development on productivity, code and tests: A controlled experiment
Pančur, M., & Ciglarič, M. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2011.02.002 · Cited by 51 · see citing works
On the Effectiveness of Unit Test Automation at Microsoft
Williams, L. A., Kudrjavets, G., & Nagappan, N. (2009) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2009.32 · Cited by 45 · see citing works
Test-Driven Development - Still a Promising Approach?
Kollanus, S. (2010) · International Conference on the Quality of Information and Communications Technology (QUATIC) · doi:10.1109/quatic.2010.73 · Cited by 35 · see citing works
Siniaalto, M., & Abrahamsson, P. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85279-7_12 · Cited by 33 · see citing works
Self-citationAgile Processes in Software Engineering and Extreme Programming
Porto, P. X. (. 2., Garbajosa, J., & Wang, X. (2018) · Lecture notes in business information processing · doi:10.1007/978-3-319-91602-6 · Cited by 23 · see citing works
Obtaining better software product by using test first programming technique
Agarwal, N., & Deep, P. (2014) · International Conference on Cloud Computing, Data Science & Engineering (Confluence) · doi:10.1109/confluence.2014.6949233 · Cited by 20 · see citing works
Teaching TDD, the Coding Dojo Style
da Luz, R. B., Neto, A. G. S. S., & Noronha, R. V. (2013) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2013.114 · Cited by 18 · see citing works
Rug: Turbo Llm for Rust Unit Test Generation
Cheng, X. C., Sang, F., Zhai, Y., Zhang, X., & Kim, T. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00097 · Cited by 17 · see citing works
Wang, Y., & Wagner, S. (2018) · Lecture notes in business information processing · doi:10.1007/978-3-319-91602-6_3 · Cited by 17 · see citing works
Erdogmus, H., Melnik, G., & Jeffries, R. (2010) · CRC Press eBooks · doi:10.1081/e-ese-120044180 · Cited by 15 · see citing works
Critical Issues on Test-Driven Development
Kollanus, S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21843-9_25 · Cited by 13 · see citing works
Overview of the Test Driven Development Research Projects and Experiments
Bulajić, A., Sambasivam, S., & Stojić, R. (2012) · Informing Science and IT Education Conference · doi:10.28945/1647 · Cited by 7 · see citing works
Test-Driven Development Benefits Beyond Design Quality: Flow State and Developer Experience
Calais, P., & Franzini, L. (2023) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier58687.2023.00025 · Cited by 6 · see citing works
A Test-Driven Approach to Improving Student Contributions to Open-Source Projects
Hu, Z., Song, Y., & Gehringer, E. F. (2019) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie43999.2019.9028521 · Cited by 6 · see citing works
Towards automatic scenario generation from coverage information
Diepenbeck, M., Soeken, M., Grosse, D. J., & Drechsler, R. (2013) · International Workshop on Automation of Software Test (AST) · doi:10.1109/iwast.2013.6595796 · Cited by 4 · see citing works
Agha, D., Sohail, R., Meghji, A. F., Qaboolio, R., & Bhatti, S. (2023) · VAWKUM Transactions on Computer Sciences · doi:10.21015/vtcs.v11i1.1494 · Cited by 3 · see citing works
Empirical Study of Test Driven Development with Scrum
Aggarwal, V., & Singhal, A. (2019) · Communications in computer and information science · doi:10.1007/978-981-13-9942-8_2 · Cited by 3 · see citing works
Torchiano, M., & Sillitti, A. (2009) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2009.5090058 · Cited by 2 · see citing works
Digitally Mediated Tools for Facilitating High-Quality Team-Based Programming Projects
Hu, Z., & Gehringer, E. F. (2021) · Educational communications and technology: issues and innovations · doi:10.1007/978-3-030-77614-5_7
Multi-Stage Generation of Rust Unit Tests with LLMs
Chen, Y., Zhang, J., Hu, X., & Xia, X. (2025) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec66846.2025.00049
- Authors
- Siniaalto, M., & Abrahamsson, P.
- Affiliations
- VTT Technical Research Centre of Finland (FI)
- ORCID
- Abrahamsson, Pekka
- Venue
- First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007)
- Publisher
- IEEE
- Published
- 2007-09
- Pages
- 275-284
- ISSN
- 1938-6451 (ISSN-L)
- OpenAlex
- W3105854783
- Cited by
- 26
- References
- 24
- Citations by year
- 2025: 2 · 2023: 2 · 2021: 3 · 2019: 2 · 2018: 2 · 2014: 1 · 2013: 3 · 2012: 1
- FWCI
- 1.70
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- test-driven development, program design, test coverage, object-oriented metrics, software maintainability, agile software development, iterative development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
