CoChat
Sign Up
Cited by 962

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

code2vec: learning distributed representations of code

Alon, Uri, Zilberstein, Meital, Levy, Omer, Yahav, Eran·2019·Proceedings of the ACM on Programming Languages
DOI 10.1145/3290353
Open this paper in CoChat — free account

Abstract

We present a neural model for representing snippets of code as continuous distributed vectors (``code embeddings''). The main idea is to represent a code snippet as a single fixed-length code vector, which can be used to predict semantic properties of the snippet. To this end, code is first decomposed to a collection of paths in its abstract syntax tree. Then, the network learns the atomic representation of each path while simultaneously learning how to aggregate a set of them. We demonstrate the effectiveness of our approach by using it to predict a method's name from the vector representation of its body. We evaluate our approach by training a model on a dataset of 12M methods. We show that code vectors trained on this dataset can predict method names from files that were unobserved during training. Furthermore, we show that our model learns useful method name vectors that capture semantic similarities, combinations, and analogies. A comparison of our approach to previous techniques over the same dataset shows an improvement of more than 75%, making it the first to successfully predict method names based on a large, cross-project corpus. Our trained model, visualizations and vector similarities are available as an interactive online demo at http://code2vec.org. The code, data and trained models are available at https://github.com/tech-srl/code2vec.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3290353

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 67

  1. 10.1145/2635868.2635883 · DOI
  2. 10.1145/2786805.2786849 · DOI
  3. Allamanis Miltiadis, 2017, A Survey of Machine Learning for Big Code and Naturalness. arXiv preprint arXiv:1709.06182 · arXiv
  4. Miltiadis Allamanis Marc Brockschmidt and Mahmoud Khademi. 2018. Learning to Represent Programs with Graphs. In ICLR . Miltiadis Allamanis Marc Brockschmidt and Mahmoud Khademi. 2018. Learning to Represent Programs with Graphs. In ICLR .
  5. Allamanis Miltiadis, 2016, Proceedings of the 33nd International Conference on Machine Learning, ICML 2016
  6. 10.5555/2487085.2487127 · DOI
  7. 10.1145/2635868.2635901 · DOI
  8. Allamanis Miltiadis, 2015, Proceedings of the 32nd International Conference on International Conference on Machine Learning -, 37
  9. 10.1145/3192366.3192412 · DOI · Paper page
  10. Amodio Matthew, 2017, Reps
  11. Thierry, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics . 177–184
  12. Ba Jimmy, 2014, Multiple object recognition with visual attention. arXiv preprint arXiv:1412.7755 · arXiv
  13. Bahdanau Dzmitry, 2014, Neural Machine Translation by Jointly Learning to Align and Translate. CoRR abs/1409.0473 · arXiv
  14. 10.1109/ICASSP.2016.7472618 · DOI
  15. Bengio Yoshua, 2003, A Neural Probabilistic Language Model. J. Mach. Learn. Res. 3 (March
  16. Bielik Pavol, 2016, Proceedings of the 33nd International Conference on Machine Learning, ICML 2016
  17. Callison-Burch Chris, 2006, 11th Conference of the European Chapter of the Association for Computational Linguistics.
  18. Jan K Chorowski Dzmitry Bahdanau Dmitriy Serdyuk Kyunghyun Cho and Yoshua Bengio. 2015. Attention-based models for speech recognition. In Advances in Neural Information Processing Systems. 577–585. Jan K Chorowski Dzmitry Bahdanau Dmitriy Serdyuk Kyunghyun Cho and Yoshua Bengio. 2015. Attention-based models for speech recognition. In Advances in Neural Information Processing Systems. 577–585.
  19. 10.1145/1390156.1390177 · DOI · Paper page
  20. 10.1145/2908080.2908126 · DOI
  21. 10.1145/3062341.3062387 · DOI
  22. 10.1145/2594291.2594343 · DOI
  23. 10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9 · DOI
  24. 10.1145/3236024.3236059 · DOI
  25. 10.3115/v1/P15-1030 · DOI
  26. Firth J.R., 1930, A Synopsis of Linguistic Theory
  27. Fowler Martin, 1999, Refactoring: Improving the Design of Existing Code
  28. Glorot Xavier, 2010, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics . 249–256
  29. Glorot Xavier, 2011, Proceedings of the 28th International Conference on Machine Learning (ICML-11)
  30. 10.1145/2814270.2814295 · DOI
  31. Hermann Karl Moritz, 2015, Proceedings of the 28th International Conference on Neural Information Processing Systems -, 1
  32. 10.5555/2337223.2337322 · DOI
  33. 10.1007/978-3-642-03013-0_14 · DOI
  34. 10.18653/v1/P16-1195 · DOI
  35. 10.1145/2837614.2837674 · DOI
  36. 10.1145/3173162.3173202 · DOI
  37. Kingma Diederik, 2014, Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 · arXiv
  38. Le Quoc, 2014, Proceedings of the 31st International Conference on Machine Learning (ICML-14) , Tony Jebara and Eric P. Xing (Eds.). JMLR Workshop and Conference Proceedings, 1188–1196
  39. 10.3115/v1/W14-1618 · DOI
  40. Levy Omer, 2014, Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014
  41. 10.1145/3133908 · DOI · Paper page
  42. Lu Yanxin, 2017, Data-Driven Program Completion. CoRR abs/1705.09042 · arXiv
  43. Chris, Proceedings of the 31st International Conference on International Conference on Machine Learning -, 32
  44. Mikolov Tomas, 2013, Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS’13)
  45. 10.1145/2384616.2384689 · DOI
  46. Mnih Volodymyr, 2014, Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS’14)
  47. Murali Vijayaraghavan, 2017, Bayesian Sketch Learning for Program Synthesis. CoRR abs/1703.05698 · arXiv
  48. 10.1145/2491411.2491458 · DOI
  49. Pennington Jeffrey, 2014, Manning
  50. 10.1145/2983990.2984041 · DOI
  51. 10.1145/2837614.2837671 · DOI
  52. 10.1145/2676726.2677009 · DOI
  53. 10.1145/2594291.2594321 · DOI · Paper page
  54. 10.1023/A:1010091220143 · DOI
  55. 10.1007/978-1-4757-6594-6_14 · DOI
  56. 10.1145/361219.361220 · DOI · Paper page
  57. Seo Minjoon, 2016, Bidirectional attention flow for machine comprehension. arXiv preprint arXiv:1611.01603 · arXiv
  58. Socher Richard, Proceedings of the 26th International Conference on Machine Learning (ICML).
  59. 10.5555/2627435.2670313 · DOI
  60. Tsoumakas Grigorios, 2006, Multi-label classification: An overview, International Journal of Data Warehousing and Mining, 3
  61. Turian Joseph, 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL ’10)
  62. 10.1162/coli.2006.32.3.379 · DOI
  63. Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 6000–6010. Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 6000–6010.
  64. 10.1561/2500000028 · DOI
  65. 10.5555/2820518.2820559 · DOI
  66. Xu Kelvin, 2015, International Conference on Machine Learning . 2048–2057
  67. 10.1145/2986012.2986013 · DOI

Cited by

Sorted by citations

Metadata

Authors
Alon, U., Zilberstein, M., Levy, O., & Yahav, E.
Affiliations
  • Technion, Israel
  • Facebook AI Research, USA
Venue
Proceedings of the ACM on Programming Languages
Publisher
Association for Computing Machinery (ACM)
Published
2019-01-02
Volume
3
Issue
POPL
Pages
1-29
ISSN
2475-1421
DOI
10.1145/3290353
Cited by
962
References
5
Language
en
Licence
creativecommons.org/licenses/by/4.0/ (vor)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review