CoChat

Evaluating and Improving ChatGPT for Unit Test Generation

Open access
Yuan, Zhiqiang, Liu, Mingwei, Ding, Shiji, Wang, Kaixin, Chen, Yixuan, Peng, Xin, Lou, Yiling · 2024 · Proceedings of the ACM on Software Engineering
Cited by 163

The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.

Abstract

Unit testing plays an essential role in detecting bugs in functionally-discrete program units ( e.g. , methods). Manually writing high-quality unit tests is time-consuming and laborious. Although the traditional techniques are able to generate tests with reasonable coverage, they are shown to exhibit low readability and still cannot be directly adopted by developers in practice. Recent work has shown the large potential of large language models (LLMs) in unit test generation. By being pre-trained on a massive developer-written code corpus, the models are capable of generating more human-like and meaningful test code. In this work, we perform the first empirical study to evaluate the capability of ChatGPT ( i.e ., one of the most representative LLMs with outstanding performance in code generation and comprehension) in unit test generation. In particular, we conduct both a quantitative analysis and a user study to systematically investigate the quality of its generated tests in terms of correctness, sufficiency, readability, and usability. We find that the tests generated by ChatGPT still suffer from correctness issues, including diverse compilation errors and execution failures (mostly caused by incorrect assertions); but the passing tests generated by ChatGPT almost resemble manually-written tests by achieving comparable coverage, readability, and even sometimes developers’ preference. Our findings indicate that generating unit tests with ChatGPT could be very promising if the correctness of its generated tests could be further improved. Inspired by our findings above, we further propose ChatTester , a novel ChatGPT-based unit test generation approach, which leverages ChatGPT itself to improve the quality of its generated tests. Chat Tester incorporates an initial test generator and an iterative test refiner. Our evaluation demonstrates the effectiveness of ChatTester by generating 34.3 % more compilable tests and 18.7 % more tests with correct assertions than the default ChatGPT. In addition to ChatGPT, we further investigate the generalization capabilities of ChatTester by applying it to two recent open-source LLMs ( i.e. , CodeLlama-Instruct and CodeFuse) and our results show that ChatTester can also improve the quality of tests generated by these LLMs.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3660783

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 10 references

Research works printed but not linked: 64

  1. Mohammad Moein Almasi Hadi Hemmati Gordon Fraser Andrea Arcuri and Janis Benefelds. 2017. An Industrial Evaluation of Unit Test Generation: Finding Real Faults in a Financial Application. In 39th IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice Track ICSE-SEIP 2017 Buenos Aires Argentina May 20-28 2017. IEEE Computer Society 263–272. · DOI · Paper page
  2. Yuntao Bai Andy Jones Kamal Ndousse Amanda Askell Anna Chen Nova DasSarma Dawn Drain Stanislav Fort Deep Ganguli Tom Henighan et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 (2022). · arXiv
  3. Arianna Blasi Alessandra Gorla Michael D. Ernst and Mauro Pezzè. 2022. Call Me Maybe: Using NLP to Automatically Generate Unit Test Cases Respecting Temporal Constraints. In 37th IEEE/ACM International Conference on Automated Software Engineering ASE 2022 Rochester MI USA October 10-14 2022. ACM 19:1-19:11. · DOI
  4. Arianna Blasi, Alessandra Gorla, Michael D. Ernst, and Mauro Pezzè. 2022. Call Me Maybe: Using NLP to Automatically Generate Unit Test Cases Respecting Temporal Constraints. In 37th IEEE/ACM International Conference on Automated Software Engineering, ASE 2022, Rochester, MI, USA, October 10-14, 2022. ACM, 19:1–19:11.
  5. Eason Chen, Ray Huang, Han-Shin Chen, Yuen-Hsien Tseng, and Liang-Yi Li. 2023. GPTutor: A ChatGPT-Powered Programming Tool for Code Explanation (Communications in Computer and Information Science, Vol. 1831). Springer, 321–327.
  6. Hugh A Chipman Edward I George and Robert E McCulloch. 2010. BART: Bayesian additive regression trees. (2010). · DOI · Paper page
  7. Christoph Csallner Nikolai Tillmann and Yannis Smaragdakis. 2008. DySy: dynamic symbolic execution for invariant inference. In 30th International Conference on Software Engineering (ICSE 2008) Leipzig Germany May 10-18 2008. ACM 281–290. · DOI · Paper page
  8. Andrew M. Dai and Quoc V. Le. 2015. Semi-supervised Sequence Learning. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada. 3079–3087.
  9. Ermira Daka, José Campos, Gordon Fraser, Jonathan Dorn, and Westley Weimer. 2015. Modeling readability to improve unit tests. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, Bergamo, Italy, August 30 - September 4, 2015. ACM, 107–118. · DOI
  10. Ermira Daka and Gordon Fraser. 2014. A Survey on Unit Testing Practices and Problems. In 25th IEEE International Symposium on Software Reliability Engineering, ISSRE 2014, Naples, Italy, November 3-6, 2014. IEEE Computer Society, 201–211.
  11. 10.1109/TSE.2022.3227418 · DOI
  12. Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. 2023. Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2023, Seattle, WA, USA, fuly 17-21, 2023. ACM, 423–435.
  13. Yinlin Deng, Chunqiu Steven Xia, Chenyuan Yang, Shizhuo Dylan Zhang, Shujing Yang, and Lingming Zhang. 2024. Large Language Models are Edge-Case Generators: Crafting Unusual Programs for Fuzzing Deep Learning Libraries. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, ICSE 2024, Lisbon, Portugal, April 14-20, 2024. ACM, 70:1–70:13.
  14. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers). Association for Computational Linguistics, 4171–4186.
  15. Elizabeth Dinella Gabriel Ryan Todd Mytkowicz and Shuvendu K. Lahiri. 2022. TOGA: A Neural Method for Test Oracle Generation. In 44th IEEE/ACM 44th International Conference on Software Engineering ICSE 2022 Pittsburgh PA USA May 25-27 2022. ACM 2130–2141. · DOI
  16. Xueying Du Mingwei Liu Juntao Li Hanlin Wang Xin Peng and Yiling Lou. 2023. Resolving Crash Bugs via Large Language Models: An Empirical Study. CoRR abs/2312.10448 (2023). arXiv:2312.10448 · arXiv
  17. Xueying Du Mingwei Liu Kaixin Wang Hanlin Wang Junwei Liu Yixuan Chen Jiayi Feng Chaofeng Sha Xin Peng and Yiling Lou. 2023. ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation. CoRR abs/2308.01861 (2023). arXiv:2308.01861 · arXiv
  18. Michael D. Ernst, Jeff H. Perkins, Philip J. Guo, Stephen McCamant, Carlos Pacheco, Matthew S. Tschantz, and Chen Xiao. 2007. The Daikon system for dynamic detection of likely invariants. Sci. Comput. Program. 69, 1-3 (2007), 35–45.
  19. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020 (Findings of ACL, Vol. EMNLP 2020). Association for Computational Linguistics, 1536–1547.
  20. Mohammad Fraiwan and Natheer Khasawneh. 2023. A Review of ChatGPT Applications in Education, Marketing, Software Engineering, and Healthcare: Benefits, Drawbacks, and Research Directions. CoRR abs/2305.00237 (2023). arXiv:2305.00237 · arXiv
  21. Gordon Fraser and Andrea Arcuri. 2011. EvoSuite: automatic test suite generation for object-oriented software. In SIGSOFT/FSE’11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC’11: 13th European Software Engineering Conference (ESEC-13), Szeged, Hungary, September 5-9, 2011. ACM, 416–419.
  22. Shuzheng Gao Xin-Cheng Wen Cuiyun Gao Wenxuan Wang Hongyu Zhang and Michael R. Lyu. 2023. What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs?. In 38th IEEE ACM International Conference on Automated Software Engineering ASE 2023 Luxembourg September 11-15 2023. IEEE 761–773. · DOI · Paper page
  23. Shuzheng Gao Hongyu Zhang Cuiyun Gao and Chaozheng Wang. 2023. Keeping Pace with Ever-Increasing Data: Towards Continual Learning of Code Intelligence Models. CoRR abs/2302.03482 (2023). arXiv:2302.03482 · arXiv
  24. Giovanni Grano, Fabio Palomba, Dario Di Nucci, Andrea De Lucia, and Harald C. Gall. 2019. Scented since the beginning: On the diffuseness of test smells in automatically generated test code. J. Syst. Softw. 156 (2019), 312–327.
  25. Giovanni Grano, Simone Scalabrino, Harald C. Gall, and Rocco Oliveto. 2018. An empirical investigation on the readability of manual and generated test cases. In Proceedings of the 26th Conference on Program Comprehension, ICPC 2018, Gothenburg, Sweden, May 27-28, 2018. ACM, 348–351.
  26. Mark Harman and Phil McMinn. 2010. A Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search. IEEE Trans. Software Eng. 36, 2 (2010), 226–247.
  27. Claus Klammer and Albin Kern. 2015. Writing unit tests: It’s now or never!. In Eighth IEEE International Conference on Software Testing Verification and Validation ICST 2015 Workshops Graz Austria April 13-17 2015. IEEE Computer Society 1–4. · doi:10.1109/icstw.2015.7107469
  28. Elson Kurian, Daniela Briola, Pietro Braione, and Giovanni Denaro. 2023. Automatically generating test cases for safety-critical software via symbolic execution. J. Syst. Softw. 199 (2023), 111629. · DOI
  29. Caroline Lemieux Jeevana Priya Inala Shuvendu K. Lahiri and Siddhartha Sen. 2023. CodaMosa: Escaping Coverage Plateaus in Test Generation with Pre-trained Large Language Models. In 45th IEEE/ACM International Conference on Software Engineering ICSE 2023 Melbourne Australia May 14-20 2023. IEEE 919–931. · DOI · Paper page
  30. Bo Li Gexiang Fang Yang Yang Quansen Wang Wei Ye Wen Zhao and Shikun Zhang. 2023. Evaluating ChatGPT’s Information Extraction Capabilities: An Assessment of Performance Explainability Calibration and Faithfulness. CoRR abs/2304.11633 (2023). arXiv:2304.11633 · arXiv
  31. Tsz On Li, Wenxi Zong, Yibo Wang, Haoye Tian, Ying Wang, Shing-Chi Cheung, and Jeff Kramer. 2023. Nuances are the Key: Unlocking ChatGPT to Find Failure-Inducing Tests with Differential Prompting. 38th IEEE ACM International Conference on Automated Software Engineering (ASE 2023), 11-15 September 2023, Kirchberg, Luxembourg (2023).
  32. Stephan Lukasczyk and Gordon Fraser. 2022. Pynguin: Automated Unit Test Generation for Python. In 44th IEEE/ACM International Conference on Software Engineering: Companion Proceedings ICSE Companion 2022 Pittsburgh PA USA May 22-24 2022. ACM/IEEE 168–172. · DOI
  33. Stephan Lukasczyk Florian Kroiß and Gordon Fraser. 2023. An empirical study of automated unit test generation for Python. Empir. Softw. Eng. 28 2 (2023) 36. · DOI
  34. Antonio Mastropaolo Simone Scalabrino Nathan Cooper David Nader-Palacio Denys Poshyvanyk Rocco Oliveto and Gabriele Bavota. 2021. Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks. In 43rd IEEE/ACM International Conference on Software Engineering ICSE 2021 Madrid Spain 22-30 May 2021. IEEE 336–347. · DOI · Paper page
  35. Webb Miller and David L. Spooner. 1976. Automatic Generation of Floating-Point Test Data. IEEE Trans. Software Eng. 2, 3 (1976), 223–226. · DOI · Paper page
  36. Noor Nashid Mifta Sintaha and Ali Mesbah. 2023. Retrieval-Based Prompt Selection for Code-Related Few-Shot Learning. In 45th IEEE/ACM International Conference on Software Engineering ICSE 2023 Melbourne Australia May 14-20 2023. IEEE 2450–2462. · DOI · Paper page
  37. Pengyu Nie Rahul Banerjee Junyi Jessy Li Raymond J. Mooney and Milos Gligoric. 2023. Learning Deep Semantics for Test Completion. In 45th IEEE/ACM International Conference on Software Engineering ICSE 2023 Melbourne Australia May 14-20 2023. IEEE 2111–2123. · DOI
  38. Erik Nijkamp Bo Pang Hiroaki Hayashi Lifu Tu Huan Wang Yingbo Zhou Silvio Savarese and Caiming Xiong. 2023. CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. In The Eleventh International Conference on Learning Representations ICLR 2023 Kigali Rwanda May 1-5 2023. OpenReview.net.
  39. Michael Olan . 2003. Unit testing: test early, test often. Journal of Computing Sciences in Colleges 19, 2 (2003), 319–328.
  40. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, and etc. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
  41. Carlos Pacheco Shuvendu K. Lahiri Michael D. Ernst and Thomas Ball. 2007. Feedback-Directed Random Test Generation. In 29th International Conference on Software Engineering (ICSE 2007) Minneapolis MN USA May 20-26 2007. IEEE Computer Society 75–84. · DOI · Paper page
  42. Fabio Palomba, Dario Di Nucci, Annibale Panichella, Rocco Oliveto, and Andrea De Lucia. 2016. On the diffusion of test smells in automatically generated test code: an empirical study. In Proceedings of the 9th International Workshop on Search-Based Software Testing, SBST@ICSE 2016, Austin, Texas, USA, May 14-22, 2016. ACM, 5–14.
  43. Fabio Palomba, Annibale Panichella, Andy Zaidman, Rocco Oliveto, and Andrea De Lucia. 2016. Automatic test case generation: what if test code quality matters?. In Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, Saarbrücken, Germany, July 18-20, 2016. ACM, 130–141.
  44. Yihao Qin Shangwen Wang Yiling Lou Jinhao Dong Kaixin Wang Xiaoling Li and Xiaoguang Mao. 2024. AgentFL: Scaling LLM-based Fault Localization to Project-Level Context. CoRR abs/2403.16362 (2024). arXiv:2403.16362 · arXiv
  45. Prajit Ramachandran, Peter J. Liu, and Quoc V. Le. 2017. Unsupervised Pretraining for Sequence to Sequence Learning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017. Association for Computational Linguistics, 383–391.
  46. Xiaoxue Ren Xinyuan Ye Dehai Zhao Zhenchang Xing and Xiaohu Yang. 2023. From Misuse to Mastery: Enhancing Code Generation with Knowledge-Driven AI Chaining. In 38th IEEE/ACM International Conference on Automated Software Engineering ASE 2023 Luxembourg September 11-15 2023. IEEE 976–987. · DOI
  47. Per Runeson . 2006. A Survey of Unit Testing Practices. IEEE Softw. 23 4 (2006) 22-29. · DOI · Paper page
  48. Simone Scalabrino Giovanni Grano Dario Di Nucci Michele Guerra Andrea De Lucia Harald C. Gall and Rocco Oliveto. 2018. OCELOT: a search-based test-data generation tool for C. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering ASE 2018 Montpellier France September 3-7 2018. ACM 868–871. · DOI
  49. Ravindra Singh and Naurang Singh Mangat. 2013. Elements of survey sampling. Vol. 15. Springer Science & Business Media.
  50. Dominik Sobania Martin Briesch Carol Hanna and Justyna Petke. 2023. An Analysis of the Automatic Bug Fixing Performance of ChatGPT. CoRR abs/2301.08653 (2023). arXiv:2301.08653 · DOI · Paper page
  51. Michele Tufano Dawn Drain Alexey Svyatkovskiy Shao Kun Deng and Neel Sundaresan. 2020. Unit test case generation with transformers and focal context. arXiv preprint arXiv:2009.05617 (2020). · arXiv
  52. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. 5998–6008.
  53. Dennis Wackerly, William Mendenhall, and Richard L Scheaffer. 2014. Mathematical statistics with applications. Cengage Learning.
  54. Chong Wang Jianan Liu Xin Peng Yang Liu and Yiling Lou. 2023. Boosting Static Resource Leak Detection via LLM-based Resource-Oriented Intention Inference. CoRR abs/2311.04448 (2023). arXiv:2311.04448 · arXiv
  55. Junjie Wang Yuchao Huang Chunyang Chen Zhe Liu Song Wang and Qing Wang. 2023. Software Testing with Large Language Model: Survey Landscape and Vision. CoRR abs/2307.07221 (2023). arXiv:2307.07221 · arXiv
  56. Yue Wang, Weishi Wang, Shafiq R. Joty, and Steven C. H. Hoi. 2021. CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021. Association for Computational Linguistics, 8696–8708.
  57. Cody Watson Michele Tufano Kevin Moran Gabriele Bavota and Denys Poshyvanyk. 2020. On learning meaningful assert statements for unit test cases. In ICSE’20: 42nd International Conference on Software Engineering Seoul South Korea 27 June - 19 July 2020. ACM 1398–1409. · DOI
  58. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
  59. Chunqiu Steven Xia Matteo Paltenghi Jia Le Tian Michael Pradel and Lingming Zhang. 2023. Universal Fuzzing via Large Language Models. CoRR abs/2308.04748 (2023). arXiv:2308.04748 · arXiv
  60. Chunqiu Steven Xia and Lingming Zhang. 2023. Keep the Conversation Going: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT. CoRR abs/2304.00385 (2023). arXiv:2304.00385 · arXiv
  61. Xusheng Xiao Sihan Li Tao Xie and Nikolai Tillmann. 2013. Characteristic studies of loop problems for structural test generation via symbolic execution. In 2013 28th IEEE/ACM International Conference on Automated Software Engineering ASE 2013 Silicon Valley CA USA November 11-15 2013. IEEE 246–256. · DOI
  62. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. 5754–5764.
  63. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10-16, 2023.
  64. Hong Zhu Patrick A. V. Hall and John H. R. May. 1997. Software Unit Test Coverage and Adequacy. ACM Comput. Surv. 29 4 (1997) 366–427. · DOI · Paper page

Cited by

See all 163

Metadata

Authors
Yuan, Z., Liu, M., Ding, S., Wang, K., Chen, Y., Peng, X., & Lou, Y.
Venue
Proceedings of the ACM on Software Engineering
Published
2024-07-12
Volume
1
Issue
FSE
Pages
1703-1726
ISSN
2994-970X (ISSN-L)
OpenAlex
W4400582690
Cited by
163
References
10
FWCI
171.22
Type
journal article
Language
en
Open access
hybrid

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Evaluating and Improving ChatGPT for Unit Test Generation. https://app.cochat.ai/papers/10.1145/3660783
The paper
Yuan, Z., Liu, M., Ding, S., Wang, K., Chen, Y., Peng, X., & Lou, Y. (2024). Evaluating and Improving ChatGPT for Unit Test Generation. Proceedings of the ACM on Software Engineering. https://doi.org/10.1145/3660783

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.