Performance Outcomes of Test-Driven Development: An Experimental Investigation
Open accessAbstract
Despite the growing popularity of test-driven development (TDD), there is no empirical confirmation of the benefits that this contemporary practice confers on its users. Prior research findings on its efficacy have largely been inconclusive. We conducted a laboratory experiment to assess the impact of TDD on software quality and task satisfaction. Additionally, we investigated the productivity aspect of TDD as compared to the traditional test-last method of software development. Results indicate that software quality and task satisfaction are significantly improved when TDD is used. Despite the additional requirements of testing, TDD is not more resource intensive than the test-last method. We also examined TDD’s impact on learning post hoc and discuss the implications of our findings and directions for future research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The studies emphasize positive effects on developers, increased productivity, and improved software quality.
“The studies emphasize positive effects on developers, increased productivity, and improved software quality [25].”
From Test Driven Development and Its Impact on Program Design and Software Quality: A Systematic Literature Review (2023) · cites this paper as [25]
The Conclusion and §9 report higher software quality and greater developer task satisfaction with TDD. But §7.1 says TDD produced higher quality without a loss of productivity, not increased productivity; §2.2 also describes prior productivity findings as mixed. (LLM, conf=0.90)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Herne, H., Cooley, W. W., & Lohnes, P. R. (1973) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344428 · Cited by 35,801 · see citing works
The "what" and "why" of goal pursuits: Human needs and the self-determination of behavior
Deci, E. L., & Ryan, R. M. (2000) · Psychological Inquiry · doi:10.1207/s15327965pli1104_01 · Cited by 34,210 · see citing works
Intrinsic Motivation and Self-Determination in Human Behavior
Deci, E. L., & Ryan, R. M. (1985) · doi:10.1007/978-1-4899-2271-7 · Cited by 26,772 · see citing works
The Reflective Practitioner: How Professionals Think in Action
Checkoway, B. N., & Schön, D. A. (1985) · Journal of Policy Analysis and Management · doi:10.2307/3324262 · Cited by 21,311 · see citing works
The reflective practitioner: How professionals think in action
Stewart, C. J. (1984) · Patient Education and Counseling · doi:10.1016/0738-3991(84)90022-3 · Cited by 19,751 · see citing works
Intrinsic Motivation and Self-Determination in Human Behavior
Miller, K. A. J., Deci, E. L., & Ryan, R. M. (1988) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2070638 · Cited by 15,361 · see citing works
Qu, X. (2007) · Technometrics · doi:10.1198/tech.2007.s455 · Cited by 14,758 · see citing works
Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1998) · DigitalCommons - Kennesaw State University (Kennesaw State University) · Cited by 14,054 · see citing works
Organizational Learning: A Theory of Action Perspective
Argyris, C., & Schön, D. A. (1997) · Reis · doi:10.2307/40183951 · Cited by 11,113 · cited 1× · see citing works
“Recognition and correction of errors based on immediate feedback provide an opportunity for learning and reflection (Argyris & Schon, 1978).”
BackgroundApplied Linear Statistical Models
Ziegel, E. R., Neter, J., Wasserman, W., & Kutner, M. (1992) · Technometrics · doi:10.2307/1269588 · Cited by 9,906 · see citing works
Nunnally, J. C. 1.-1., & Bernstein, I. H. (1994) · Cited by 9,365 · see citing works
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771 · see citing works
Software engineering: A practitioner's approach
(1983) · Advances in Engineering Software (1978) · doi:10.1016/0141-1195(83)90118-3 · Cited by 8,227 · see citing works
Stockton, R. A., & Kerlinger, F. N. (1974) · American Educational Research Journal · doi:10.2307/1162206 · Cited by 7,356 · see citing works
Extreme Programming Explained: Embrace Change
Beck, K. L., & Andres, C. (2000) · Cited by 5,855 · see citing works
Applied Linear Statistical Models.
Lorenz, F. O., Neter, J., Wasserman, W., & Kutner, M. (1986) · Journal of the American Statistical Association · doi:10.2307/2289108 · Cited by 5,566 · see citing works
Manifesto for agile software development
Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., & Thomas, D. A. (2001) · Cited by 4,284 · see citing works
The Reflective Practitioner: How Professionals Think in Action
Schön, D. A., & DeSanctis, V. (1986) · The Journal of Continuing Higher Education · doi:10.1080/07377366.1986.10401080 · Cited by 3,234 · see citing works
Foundations of Behavioral Research
Buttram, J. B. (1967) · Journal of Music Therapy · doi:10.1093/jmt/4.4.144-a · Cited by 2,977 · see citing works
Accelerating Adaptive Processes: Product Innovation in the Global Computer Industry
Eisenhardt, K. M., & Tabrizi, B. N. (1995) · Administrative Science Quarterly · doi:10.2307/2393701 · Cited by 2,426 · cited 1× · see citing works
“The applicability of this finding to other domains such as product innovation is supported by a similar observation by Eisenhardt and Tabrizi (1995) that “iterations and testing would rapidly build understanding and create multiple options” (p. 104); they showed that an experiential strategy using…”
MethodTest-Driven Development by Example
Beck, K. L. (2002) · Cited by 1,457 · see citing works
Test Driven Development: By Example
Beck (2002) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 1,278 · see citing works
Iterative and incremental developments. a brief history
Larman, C., & Basili, V. R. (2003) · Computer · doi:10.1109/mc.2003.1204375 · Cited by 1,164 · see citing works
Agile software development, the people factor
Cockburn, A., & Highsmith, J. A. (2001) · Computer · doi:10.1109/2.963450 · Cited by 1,105 · see citing works
Challenges of migrating to agile methodologies
Nerur, S., Mahapatra, R., & Mangalaraj, G. (2005) · Communications of the ACM · doi:10.1145/1060710.1060712 · Cited by 1,018 · see citing works
Self-citationWork Motivation and Satisfaction: Light at the End of the Tunnel
Locke, E. A., & Latham, G. P. (1990) · Psychological Science · doi:10.1111/j.1467-9280.1990.tb00207.x · Cited by 923 · see citing works
Management and the Learning Process
Kolb, D. A. (1976) · California Management Review · doi:10.2307/41164649 · Cited by 770 · see citing works
Generalizing From Laboratory to Field Settings.
Saxton, S. L., & Locke, E. A. (1987) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2069857 · Cited by 753 · see citing works
Mayer, R. E. (1989) · Review of Educational Research · doi:10.3102/00346543059001043 · Cited by 687 · see citing works
Ilardi, B. C., Leone, D., Kasser, T., & Ryan, R. M. (1993) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1993.tb01066.x · Cited by 653 · see citing works
Yi, M. Y., & Davis, F. D. (2003) · Information systems research · doi:10.1287/isre.14.2.146.16016 · Cited by 642 · see citing works
Principles of software engineering management
(1990) · Computer Audit Update · doi:10.1016/s0960-2593(05)80027-8 · Cited by 570 · see citing works
Morris, M. G., & Venkatesh, V. (2010) · MIS Quarterly · doi:10.2307/20721418 · Cited by 437 · see citing works
The Importance of Learning Style in End-User Training1
Bostrom, R. P., Olfman, L., & Sein, M. K. (1990) · MIS Quarterly · doi:10.2307/249313 · Cited by 426 · see citing works
Software Engineering : a practitioner's approach, 4th ed
Pressman, R. S. (1997) · Cited by 318 · see citing works
On the effectiveness of the test-first approach to programming
Erdogmus, H., Morisio, M., & Torchiano, M. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.37 · Cited by 315 · see citing works
Using Self-Regulatory Learning to Enhance E-Learning-Based Information Technology Training
Santhanam, R., Sasidharan, S., & Webster, J. (2008) · Information systems research · doi:10.1287/isre.1070.0141 · Cited by 244 · see citing works
Disciplined Agile Delivery: A Practitioner's Guide to Agile Software Delivery in the Enterprise
Ambler, S. W., & Lines, M. (2012) · Cited by 228 · see citing works
Using software testing to move students from trial-and-error to reflection-in-action
Edwards, S. H. (2004) · ACM SIGCSE Bulletin · doi:10.1145/1028174.971312 · Cited by 213 · see citing works
Understanding Conceptual Schemas: Exploring the Role of Application and IS Domain Knowledge
Khatri, V., Vessey, I., Ramesh, V., Clay, P. F., & Park, S. (2006) · Information Systems Research · doi:10.1287/isre.1060.0081 · Cited by 203 · see citing works
A structured experiment of test-driven development
George, B., & Williams, L. A. (2003) · Information and Software Technology · doi:10.1016/j.infsof.2003.09.011 · Cited by 202 · see citing works
Job Satisfaction in Agile Development Teams: Agile Development as Work Redesign
Tripp, J. F., Rienemschneider, C., & Thatcher, J. B. (2016) · Journal of the Association for Information Systems · doi:10.17705/1jais.00426 · Cited by 199 · see citing works
An experiment to assess the cost-benefits of code inspections in large scale software development
Porter, A. A., Siy, H. P., Toman, C. A., & Votta, L. G. (1997) · IEEE Transactions on Software Engineering · doi:10.1109/32.601071 · Cited by 165 · see citing works
Experiment about test-first programming
Müller, M. M., & Hagner, O. (2002) · IEE Proceedings - Software · doi:10.1049/ip-sen:20020540 · Cited by 164 · see citing works
Burton‐Jones, A., & Meso, P. N. (2006) · Information Systems Research · doi:10.1287/isre.1050.0079 · Cited by 160 · see citing works
Evaluating the efficacy of test-driven development
Bhat, T., & Nagappan, N. (2006) · doi:10.1145/1159733.1159787 · Cited by 142 · see citing works
The Effects of Test-Driven Development on External Quality and Productivity: A Meta-Analysis
Rafique, Y., & Mišić, V. B. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.28 · Cited by 121 · see citing works
Are two heads better than one for software development? the productivity paradox of pair programming
Balijepally, V., Mahapatra, R., Nerur, S., & Price, K. H. (2009) · Journal of the Association for Information Systems · doi:10.5555/2017410.2017418 · Cited by 118 · see citing works
Self-citationBalijepally, V., Mahapatra, R., Nerur, S., & Price (2009) · MIS Quarterly · doi:10.2307/20650280 · Cited by 118 · see citing works
Self-citationA Hidden Markov Model of Developer Learning Dynamics in Open Source Software Projects
Singh, P. V., Tan, Y., & Youn, N. (2010) · Information Systems Research · doi:10.1287/isre.1100.0308 · Cited by 117 · see citing works
Stressor‐burnout relationship in software development teams
Sonnentag, S., Brodbeck, F. C., Heinbokel, T., & Stolte, W. (1994) · Journal of Occupational and Organizational Psychology · doi:10.1111/j.2044-8325.1994.tb00571.x · Cited by 111 · see citing works
Exhaustion from Information System Career Experience: Implications for Turn-Away Intention1
Armstrong, D. J., Brooks, N. G., & Riemenschneider, C. K. (2015) · MIS Quarterly · doi:10.25300/misq/2015/39.3.10 · Cited by 99 · see citing works
Comparative Analysis of Job Satisfaction in Agile and Non-agile Software Development Teams
Melnik, G., & Maurer, F. (2006) · Lecture notes in computer science · doi:10.1007/11774129_4 · Cited by 98 · see citing works
Fucci, D., Erdogmus, H., Turhan, B., Oivo, M., & Juristo, N. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2616877 · Cited by 90 · see citing works
Purao, S., Storey, V. C., & Han, T. (2003) · Information Systems Research · doi:10.1287/isre.14.3.269.16559 · Cited by 82 · see citing works
On the Influence of Test-Driven Development on Software Design
Janzen, D. S., & Saiedian, H. (2006) · Proceedings · doi:10.1109/cseet.2006.25 · Cited by 81 · see citing works
Madeyski, L. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.08.007 · Cited by 74 · see citing works
Driving Software Quality: How Test-Driven Development Impacts Software Quality
Crispin, L. (2006) · IEEE Software · doi:10.1109/ms.2006.157 · Cited by 70 · see citing works
Impact of test-driven development on productivity, code and tests: A controlled experiment
Pančur, M., & Ciglarič, M. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2011.02.002 · Cited by 51 · see citing works
Lean-Agile Software Development: Achieving Enterprise Agility
Shalloway, A., Beaver, G., & Trott, J. R. (2009) · CERN Document Server (European Organization for Nuclear Research) · Cited by 50 · see citing works
Performance and strain levels of it workers engaged in rapidly changing environments
Chilton, M. A., Hardgrave, B. C., & Armstrong, D. J. (2010) · ACM SIGMIS Database the DATABASE for Advances in Information Systems · doi:10.1145/1719051.1719053 · Cited by 44 · see citing works
Madeyski, L. (2005) · Cited by 42 · see citing works
Extreme Programming and Agile Processes in Software Engineering
Abrahamsson, P., Marchesi, M., & Succi, G. (2006) · Lecture notes in computer science · doi:10.1007/11774129 · Cited by 40 · see citing works
Lean – Agile Software Development: Achieving Enterprise Agility
(2010) · Kybernetes · doi:10.1108/03684921011043314 · Cited by 38 · see citing works
Comparing the Defect Reduction Benefits of Code Inspection and Test-Driven Development
Wilkerson, J. W., Nunamaker, J. F., & Mercer, R. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.46 · Cited by 38 · see citing works
Effective Software Testing: 50 Ways to Improve Your Software Testing
Dustin, E. (2002) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 26 · see citing works
Avgar, A. C., Tambe, P. B., & Hitt, L. M. (2018) · MIS Quarterly · doi:10.25300/misq/2018/13668 · Cited by 18 · see citing works
Effects of Animations in Learning—A Cognitive Fit Perspective1
Li, P., Santhanam, R., & Carswell, C. M. (2009) · Decision Sciences Journal of Innovative Education · doi:10.1111/j.1540-4609.2009.00227.x · Cited by 16 · see citing works
Enterprise Integration: An Experiential Learning Model
Cameron, B. H. (2014) · Springer eBooks · doi:10.1007/978-3-319-03656-4_13 · Cited by 9 · see citing works
Does test-driven development improve class design? A qualitative study on developers’ perceptions
Aniche, M. F., & Gerosa, M. A. (2015) · Journal of the Brazilian Computer Society · doi:10.1186/s13173-015-0034-z · Cited by 7 · see citing works
Enterprise Integration: An Experiential Learning Model.
Cameron, B. H., & Purao, S. (2010) · Information Systems Education Journal · Cited by 4 · see citing works
Flagged first, then by citations
2 of the 73 references couldn’t be matched to a known paper.
Research works printed but not linked: 8
- Buchan, J., Li, L., & MacDonell, S. G. (2011). Causal factors, benefits and challenges of test-driven development: Practitioners perceptions. Proceedings of the 18th Asia-Pacific Software Engineering Conference.
- Canfora, A. (2006). Evaluating advantages of TDD: A controlled experiment with professionals, Proceedings of the International Symposium on Empirical Software Engineering.
- Lui, K. M., & Chan, C. C. (2004). Test driven development and software process improvement in China: Extreme programming and agile processes in software engineering (pp. 219-222).
- Mangalaraj, G., Nerur, S., Mahapatra, R. K., & Price, K. H. (2014). Distributed cognition in software development: an experimental investigation of the role of design patterns and collaboration. MIS Quarterly, 38(1), 249-274.
- Moore, J. E. (2000). One road to exhaustion: An examination of work exhaustion in technology professionals. MIS Quarterly, 24(1), 141-168.
- Tripp, J. F., & Riemenschneider, C. K. (2014). Towards an understanding of job satisfaction on agile teams: Agile development as work redesign. Proceedings of the 47th Hawaii International Conference on System Sciences.
- Wastell, D. (1999). Learning dysfunctions in information systems development: Overcoming the social defenses with traditional objects. MIS Quarterly, 23(4), 581-600.
- Westlund, S. G., & Hannon, J. C. (2008). Retaining talent: Assessing job satisfaction facets most significantly related to software developer turnover intentions. Journal of Information Technology Management, 19(4), 1-15.
Cited by
Sorted by citations
Spreadsheet quality assurance: a literature review
Poon, P., Lau, M. F., Yu, Y. T., & Tang, S.-F. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-023-2384-6 · Cited by 11 · see citing works
Automation of Test Skeletons Within Test-Driven Development Projects
Ölmez, M. M., & Gehringer, E. F. (2024) · IEEE International Conference on Software Engineering Education and Training (CSEE&T) · doi:10.1109/cseet62301.2024.10663016 · Cited by 4 · see citing works
A Conceptual Learning Framework of Cybersecurity Education for Military and Law Enforcement
Nag, A. K., Bhadauria, V. S., Gibson, C. A., Neupane, R. C., & Creider, D. (2022) · International Journal of Smart Education and Urban Society · doi:10.4018/ijseus.309953 · Cited by 3 · see citing works
Self-citationAgha, D., Sohail, R., Meghji, A. F., Qaboolio, R., & Bhatti, S. (2023) · VAWKUM Transactions on Computer Sciences · doi:10.21015/vtcs.v11i1.1494 · Cited by 3 · see citing works
Building AI Talent in Organizations – An Experiential Learning Approach
Samuel, J., Nerur, S., Mahapatra, R., & White, B. (2025) · Journal of Information Systems Education · doi:10.62273/nrqw1204 · Cited by 2 · see citing works
Self-citationFrom Letters to LINE to AI: Reframing Organizational Communication Norms in the Digital Era
Lee, G. Y. C., & Luarn, P. (2026) · International Journal of Business Communication · doi:10.1177/23294884261415593 · Cited by 1 · see citing works
A Two-stage Method of Synchronization Prediction Framework in TDD
Hsieh, C.-H., & Wang, Z. (2021) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-021-06207-0
Pfeiffer, R.-H., & Aaen, J. (2024) · International journal of information systems and project management · doi:10.12821/ijispm120102
THE RELATIONSHIP OF HOSPITAL WEBSITE INFORMATION QUALITY AND PATIENT SATISFACTION IN EAST JAVA
Theresia, C., & Syahrir, I. (2025) · Global Journal of Health Administration · doi:10.30651/gjha.v1i1.26864 · Cited by 3 · see citing works
- Authors
- Texas A&M University Texarkana, U., Bhadauria, V., Mahapatra, R., University of Texas Arlington, U., Nerur, S., & University of Texas Arlington, U.
- Affiliations
- Texas A&M University – Texarkana (US)
- The University of Texas at Arlington (US)
- Venue
- Journal of the Association for Information Systems
- Publisher
- Association for Information Systems
- Published
- 2020-07-01
- Volume
- 21
- Pages
- 1045-1071
- ISSN
- 1536-9323 (ISSN-L)
- OpenAlex
- W3046467066
- Cited by
- 9
- References
- 73
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 3 · 2023: 1 · 2022: 1 · 2021: 1
- FWCI
- 0.71
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the Association for Information Systems (publishedVersion)
- Journal of the Association for Information Systems (submittedVersion)
- Funders
- University of Texas at Arlington
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- test-driven development, software quality, task satisfaction, developer productivity, software development practices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
