Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 247

The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.

Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks

Open access
Mastropaolo, Antonio, Scalabrino, Simone, Cooper, Nathan, Nader Palacio, David, Poshyvanyk, Denys, Oliveto, Rocco, Bavota, Gabriele · 2021 · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

Abstract

Deep learning (DL) techniques are gaining more and more attention in the software engineering community. They have been used to support several code-related tasks, such as automatic bug fixing and code comments generation. Recent studies in the Natural Language Processing (NLP) field have shown that the Text-To-Text Transfer Transformer (T5) architecture can achieve state-of-the-art performance for a variety of NLP tasks. The basic idea behind T5 is to first pre-train a model on a large and generic dataset using a self-supervised task (e.g., filling masked words in sentences). Once the model is pre-trained, it is fine-tuned on smaller and specialized datasets, each one related to a specific task (e.g., language translation, sentence classification). In this paper, we empirically investigate how the T5 model performs when pre-trained and fine-tuned to support code-related tasks. We pre-train a T5 model on a dataset composed of natural language English text and source code. Then, we fine-tune such a model by reusing datasets used in four previous works that used DL techniques to: (i) fix bugs, (ii) inject code mutants, (iii) generate assert statements, and (iv) generate code comments. We compared the performance of this single model with the results reported in the four original papers proposing DL-based solutions for those four tasks. We show that our T5 model, exploiting additional data for the self-supervised pre-training phase, can achieve performance improvements over the four baselines.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/icse43902.2021.00041

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 13 references

Research works printed but not linked: 57

  1. 10.1145/2786805.2803196 · DOI
  2. 10.1145/3106237.3106280 · DOI
  3. 10.1109/ASE.2013.6693078 · DOI
  4. 10.1109/ICSE.2013.6606624 · DOI
  5. 10.1145/1882291.1882315 · DOI · Paper page
  6. 10.1109/ICPC.2009.5090047 · DOI
  7. 10.1145/2635868.2635898 · DOI · Paper page
  8. 10.1145/2591062.2591114 · DOI
  9. 10.1145/3360585 · DOI · Paper page
  10. 10.1109/SANER.2019.8668043 · DOI · Paper page
  11. 10.1016/j.jss.2013.10.019 · doi:10.1016/j.jss.2013.10.019
  12. graves, 2012, Sequence transduction with recurrent neural networks, CoRR, abs 1211 3711
  13. boulanger-lewandowski, 2013, Audio chord recognition with recurrent neural networks, ISMIR, 335
  14. 10.1109/ICSE.2012.6227211 · DOI · Paper page
  15. 10.1109/TSE.2011.104 · DOI · Paper page
  16. 10.1145/2813885.2737988 · DOI
  17. 0, Google colaboraty
  18. lin, 2004, Rouge: A package for automatic evaluation of summaries, Text Summarization Branches Out, 74
  19. caswell, 2020, Recent advances in google translate
  20. 10.1109/TSE.2019.2940179 · DOI · Paper page
  21. kalchbrenner, 2013, Recurrent continuous translation models, Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 1700
  22. brody, 2020, Neural edit completion, arXiv preprint arXiv 2005 13209
  23. sutskever, 2014, Sequence to sequence learning with neural networks, CoRR, abs 1409 3215
  24. 0, Replication Package
  25. 10.18653/v1/P16-1195 · DOI · Paper page
  26. 10.18653/v1/D18-2012 · DOI · Paper page
  27. srivastava, 2014, Dropout: a simple way to prevent neural networks from overfitting, The Journal of Machine Learning Research, 15, 1929
  28. ba, 2016, Layer normalization, arXiv preprint arXiv 1607 06450
  29. 10.18653/v1/D16-1053 · DOI · Paper page
  30. vaswani, 2017, Attention is all you need, Advances in neural information processing systems, 5998
  31. allamanis, 2016, A convolutional attention network for extreme summarization of source code, CoRR, abs 1602 3001
  32. 10.1145/3238147.3238190 · DOI · Paper page
  33. 10.1145/3379597.3387449 · DOI
  34. 10.1145/2025113.2025179 · DOI · Paper page
  35. 10.1109/ASE.2015.36 · DOI · Paper page
  36. 10.1145/3106237.3106289 · DOI
  37. 10.1145/3196321.3196330 · DOI
  38. 10.1109/ICSME.2019.00046 · DOI
  39. 10.1145/3377811.3380429 · DOI
  40. karampatsis, 2019, Maybe deep neural networks are the best choice for modeling source code, CoRR, abs 1903 5734
  41. alon, 2019, Structural language models of code, ArXiv
  42. hata, 2018, Learning to generate corrective patches using neural machine translation, CoRR, abs 1812 7170
  43. 10.1145/3338906.3340455 · DOI
  44. 10.1145/2591062.2591072 · DOI
  45. tufano, 2019, On learning via neural machine translation, Proceedings of the 41st International Conference on Software Engineering ICSE 2019, 2019, 25
  46. 10.1109/ICSE.2019.00087 · DOI · Paper page
  47. 10.1145/2491411.2494584 · DOI
  48. 10.1145/3196321.3196334 · DOI · Paper page
  49. 10.1109/ASE.2017.8115626 · DOI · Paper page
  50. 10.1109/ICSE.2017.13 · DOI
  51. 10.1109/ICPC.2013.6613830 · DOI · Paper page
  52. 10.1109/TSE.2015.2465386 · DOI · Paper page
  53. 10.1145/1985793.1985808 · DOI · Paper page
  54. 0, Utilizing Fast Testing to Transform Java Development into an Agile Quick Release Low Risk Process
  55. 10.1145/1297846.1297902 · DOI · Paper page
  56. 10.1109/ICPC.2011.28 · DOI
  57. 10.1109/WCRE.2010.13 · DOI · Paper page

Cited by

See all 247

Metadata

Authors
Mastropaolo, A., Scalabrino, S., Cooper, N., Nader Palacio, D., Poshyvanyk, D., Oliveto, R., & Bavota, G.
Venue
2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)
Publisher
IEEE
Published
2021-05
Pages
336-347
OpenAlex
W3127853158
Cited by
247
References
13
FWCI
38.49
Type
conference paper
Language
en
Open access
green
Funders
European Research Council

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks. https://app.cochat.ai/papers/10.1109/icse43902.2021.00041
The paper
Mastropaolo, A., Scalabrino, S., Cooper, N., Nader Palacio, D., Poshyvanyk, D., Oliveto, R., & Bavota, G. (2021). Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks. 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). https://doi.org/10.1109/icse43902.2021.00041

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.