Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 558

The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.

Deep learning code fragments for code clone detection

White, Martin, Tufano, Michele, Vendome, Christopher, Poshyvanyk, Denys·2016·Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering
DOI 10.1145/2970276.2970326
Open this paper in CoChat — free account

Abstract

Code clone detection is an important problem for software maintenance and evolution. Many approaches consider either structure or identifiers, but none of the existing detection techniques model both sources of information. These techniques also depend on generic, handcrafted features to represent code fragments. We introduce learning-based detection techniques where everything for representing terms and fragments in source code is mined from the repository. Our code analysis supports a framework, which relies on deep learning, for automatically linking patterns mined at the lexical level with patterns mined at the syntactic level. We evaluated our novel learning-based approach for code clone detection with respect to feasibility from the point of view of software maintainers. We sampled and manually evaluated 398 file- and 480 method-level pairs across eight real-world Java systems; 93% of the file- and method-level samples were evaluated to be true positives. Among the true positives, we found pairs mapping to all four clone types. We compared our approach to a traditional structure-oriented technique and found that our learning-based approach detected clones that were either undetected or suboptimally reported by the prominent tool Deckard. Our results affirm that our learning-based approach is suitable for clone detection and a tenable technique for researchers.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/2970276.2970326

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 8 references

Research works printed but not linked: 80

  1. 10.5555/1018434.1021568 · doi:10.5555/1018434.1021568
  2. 10.1145/1882291.1882315 · DOI
  3. 10.1109/TSE.2006.28 · DOI · Paper page
  4. 10.1007/s10664-008-9076-6 · DOI · Paper page
  5. 10.1007/s10664-011-9195-3 · DOI · Paper page
  6. 10.1145/1081706.1081737 · DOI · Paper page
  7. C. Roy and J. Cordy . A survey on software clone detection research. Technical report , Queen’s University , 2007 . C. Roy and J. Cordy. A survey on software clone detection research. Technical report, Queen’s University, 2007.
  8. N. Davey , P. Barson , S. Field , R. Frank , and D. Tansley . The development of a software clone detector. IJAST, 1(3/4) , 1995 . N. Davey, P. Barson, S. Field, R. Frank, and D. Tansley. The development of a software clone detector. IJAST, 1(3/4), 1995.
  9. A. Walenstein and A. Lakhotia . The software similarity problem in malware analysis . Dagstuhl Seminar Proceedings , 2007 . A. Walenstein and A. Lakhotia. The software similarity problem in malware analysis. Dagstuhl Seminar Proceedings, 2007.
  10. 10.1109/SCAM.2010.19 · DOI
  11. 10.1145/1287624.1287634 · DOI
  12. Lucia D. Lo L. Jiang and A. Budi. Active refinement of clone anomaly reports. ICSE’12. Lucia D. Lo L. Jiang and A. Budi. Active refinement of clone anomaly reports. ICSE’12.
  13. 10.1145/2351676.2351701 · DOI
  14. 10.1145/2420950.2421004 · DOI
  15. 10.1145/2610384.2610394 · DOI
  16. 10.1145/2635868.2635926 · DOI
  17. 10.1016/j.scico.2009.02.007 · DOI · Paper page
  18. 10.1109/WPC.2005.14 · DOI
  19. 10.1109/ICPC.2006.51 · DOI
  20. Y. Bengio , A. Courville , and P. Vincent . Unsupervised feature learning and deep learning: A review and new perspectives. CoRR, abs/1206.5538 , 2012 . Y. Bengio, A. Courville, and P. Vincent. Unsupervised feature learning and deep learning: A review and new perspectives. CoRR, abs/1206.5538, 2012. · arXiv
  21. B. Baker . A program for identifying duplicated code . In Computer Science and Statistics , 1992 . B. Baker. A program for identifying duplicated code. In Computer Science and Statistics, 1992.
  22. 10.1006/jcss.1996.0003 · DOI · Paper page
  23. 10.1109/TSE.2002.1019480 · DOI · Paper page
  24. 10.1002/spe.4380210706 · DOI · Paper page
  25. 10.1109/WCRE.2006.18 · DOI · Paper page
  26. 10.1109/ICSE.2007.30 · DOI · Paper page
  27. A. Gionis P. Indyk and R. Motwani. Similarity search in high dimensions via hashing. VLDB’99. A. Gionis P. Indyk and R. Motwani. Similarity search in high dimensions via hashing. VLDB’99.
  28. 10.1145/1368088.1368132 · DOI · Paper page
  29. 10.1145/2568225.2568286 · DOI · Paper page
  30. 10.1145/24039.24041 · DOI · Paper page
  31. 10.1109/MCI.2010.938364 · DOI · Paper page
  32. 10.1007/978-3-642-00593-0_31 · DOI
  33. 10.1109/ICSE.2009.5070528 · doi:10.1109/icse.2009.5070528
  34. 10.1109/TSE.2011.90 · DOI
  35. 10.1145/1882291.1882317 · doi:10.1145/1882291.1882317
  36. 10.1145/1985793.1985835 · DOI
  37. F. Hermans B. Sedee M. Pinzger and A. van Deursen. Data clone detection and visualization in spreadsheets. ICSE’13. F. Hermans B. Sedee M. Pinzger and A. van Deursen. Data clone detection and visualization in spreadsheets. ICSE’13.
  38. 10.1145/1572272.1572283 · DOI
  39. D. Jurafsky and J. Martin . Speech and Language Processing. 2 ed ., 2009 . D. Jurafsky and J. Martin. Speech and Language Processing. 2 ed., 2009.
  40. A. Hindle E. Barr Z. Su M. Gabel and P. Devanbu. On the naturalness of software. ICSE’12. A. Hindle E. Barr Z. Su M. Gabel and P. Devanbu. On the naturalness of software. ICSE’12.
  41. C. Franks Z. Tu P. Devanbu and V. Hellendoorn. Cacheca: A cache language model based code suggestion tool. ICSE’15. C. Franks Z. Tu P. Devanbu and V. Hellendoorn. Cacheca: A cache language model based code suggestion tool. ICSE’15.
  42. 10.1109/ICST.2013.11 · DOI
  43. D. Movshovitz-Attias and W. Cohen. Natural language models for predicting programming comments. ACL’13. D. Movshovitz-Attias and W. Cohen. Natural language models for predicting programming comments. ACL’13.
  44. 10.1145/2597073.2597102 · DOI
  45. 10.1145/2568225.2568242 · DOI
  46. 10.1145/2635868.2635883 · DOI · Paper page
  47. 10.1145/2491411.2494584 · DOI
  48. 10.1145/2591062.2591072 · DOI
  49. 10.1145/2642937.2643010 · DOI
  50. 10.1145/2594291.2594321 · DOI · Paper page
  51. B. Ray , V. Hellendoorn , Z. Tu , C. Nguyen , S. Godhane , A. Bacchelli , and P. Devanbu . On the “naturalness ” of buggy code. CoRR , abs/1506.01159, 2015 . B. Ray, V. Hellendoorn, Z. Tu, C. Nguyen, S. Godhane, A. Bacchelli, and P. Devanbu. On the “naturalness” of buggy code. CoRR, abs/1506.01159, 2015. · arXiv
  52. 10.1145/2786805.2786849 · DOI · Paper page
  53. R. Rosenfeld . Two decades of statistical language modeling: Where do we go from here? 88(8) , 2000 . R. Rosenfeld. Two decades of statistical language modeling: Where do we go from here? 88(8), 2000.
  54. A. Mnih and Y. Teh. A fast and simple algorithm for training neural probabilistic language models. ICML’12. A. Mnih and Y. Teh. A fast and simple algorithm for training neural probabilistic language models. ICML’12.
  55. Y. Bengio , R. Ducharme , P. Vincent , and C. Janvin . A neural probabilistic language model. JMLR, 3 , 2003 . Y. Bengio, R. Ducharme, P. Vincent, and C. Janvin. A neural probabilistic language model. JMLR, 3, 2003.
  56. F. Morin and Y. Bengio. Hierarchical probabilistic neural network language model. AISTATS’05. F. Morin and Y. Bengio. Hierarchical probabilistic neural network language model. AISTATS’05.
  57. 10.3115/1220575.1220601 · DOI
  58. 10.1561/2200000006 · DOI · Paper page
  59. T. Mikolov S. Kombrink L. Burget J. Cernock´ y and S. Khudanpur. Extensions of recurrent neural network language model. ICASSP’11. T. Mikolov S. Kombrink L. Burget J. Cernock´ y and S. Khudanpur. Extensions of recurrent neural network language model. ICASSP’11.
  60. R. Socher J. Pennington E. Huang A. Ng and C. Manning. Semi-supervised recursive autoencoders for predicting sentiment distributions. EMNLP’11. R. Socher J. Pennington E. Huang A. Ng and C. Manning. Semi-supervised recursive autoencoders for predicting sentiment distributions. EMNLP’11.
  61. R. Socher A. Perelygin J. Wu J. Chuang C. Manning A. Ng and C. Potts. Recursive deep models for semantic compositionality over a sentiment treebank. EMNLP’13. R. Socher A. Perelygin J. Wu J. Chuang C. Manning A. Ng and C. Potts. Recursive deep models for semantic compositionality over a sentiment treebank. EMNLP’13.
  62. J. Goodman . Classes for fast maximum entropy training. CoRR , cs.CL/0108006 , 2001 . J. Goodman. Classes for fast maximum entropy training. CoRR, cs.CL/0108006, 2001. · arXiv
  63. 10.1145/1273496.1273577 · DOI · Paper page
  64. Y. Shi , W. Zhang , J. Liu , and M. Johnson . Rnn language model with word clustering and class-based output layer. EURASIP , 1 , 2013 . Y. Shi, W. Zhang, J. Liu, and M. Johnson. Rnn language model with word clustering and class-based output layer. EURASIP, 1, 2013.
  65. I. Sutskever J. Martens and G. Hinton. Generating text with recurrent neural networks. ICML’11. I. Sutskever J. Martens and G. Hinton. Generating text with recurrent neural networks. ICML’11.
  66. I. Sutskever J. Martens G. Dahl and G. Hinton. On the importance of initialization and momentum in deep learning. ICML’13. I. Sutskever J. Martens G. Dahl and G. Hinton. On the importance of initialization and momentum in deep learning. ICML’13.
  67. M. Hermans and B. Schrauwen. Training and analysing deep recurrent neural networks. NIPS’13. M. Hermans and B. Schrauwen. Training and analysing deep recurrent neural networks. NIPS’13.
  68. R. Pascanu , C. Gulcehre , K. Cho , and Y. Bengio . How to construct deep recurrent neural networks. CoRR, abs/1312.6026 , 2013 . R. Pascanu, C. Gulcehre, K. Cho, and Y. Bengio. How to construct deep recurrent neural networks. CoRR, abs/1312.6026, 2013. · arXiv
  69. P. Werbos . Backpropagation through time: what it does and how to do it. 78(10) , 1990 . P. Werbos. Backpropagation through time: what it does and how to do it. 78(10), 1990.
  70. A. Aho , M. Lam , R. Sethi , and J. Ullman . Compilers: Principles, Techniques, and Tools. 2 ed ., 2006 . A. Aho, M. Lam, R. Sethi, and J. Ullman. Compilers: Principles, Techniques, and Tools. 2 ed., 2006.
  71. T. Cormen , C. Leiserson , R. Rivest , and C. Stein . Introduction to Algorithms. 3 ed ., 2009 . T. Cormen, C. Leiserson, R. Rivest, and C. Stein. Introduction to Algorithms. 3 ed., 2009.
  72. S. Deerwester S. Dumais G. Furnas T. Landauer and R. Harshman. Indexing by latent semantic analysis. JASIS 41(6). S. Deerwester S. Dumais G. Furnas T. Landauer and R. Harshman. Indexing by latent semantic analysis. JASIS 41(6). · DOI
  73. 10.5555/330775 · doi:10.5555/330775
  74. R. Koschke . Survey of research on software clones . Dagstuhl Seminar Proceedings , 2007 . R. Koschke. Survey of research on software clones. Dagstuhl Seminar Proceedings, 2007.
  75. 10.1145/1985404.1985416 · doi:10.1145/1985404.1985416
  76. 10.1145/2491411.2491420 · DOI · Paper page
  77. 10.1109/ICSME.2014.77 · DOI · Paper page
  78. 10.1109/ICSM.2011.6080795 · DOI
  79. 10.1016/j.ijar.2008.11.006 · DOI · Paper page
  80. 10.1007/978-0-387-84858-7 · DOI · Paper page

Cited by

Metadata

Authors
White, M., Tufano, M., Vendome, C., & Poshyvanyk, D.
Venue
Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering
Publisher
ACM
Published
2016-08-25
Pages
87-98
DOI
10.1145/2970276.2970326
OpenAlex
W2511803001
Cited by
558
References
8
FWCI
42.74
Type
conference paper
Language
en
Open access
closed
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.