The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
A Novel Neural Source Code Representation Based on Abstract Syntax Tree
Abstract
Exploiting machine learning techniques for analyzing programs has attracted much attention. One key problem is how to represent code fragments well for follow-up analysis. Traditional information retrieval based methods often treat programs as natural language texts, which could miss important semantic information of source code. Recently, state-of-the-art studies demonstrate that abstract syntax tree (AST) based neural models can better represent source code. However, the sizes of ASTs are usually large and the existing models are prone to the long-term dependency problem. In this paper, we propose a novel AST-based Neural Network (ASTNN) for source code representation. Unlike existing models that work on entire ASTs, ASTNN splits each large AST into a sequence of small statement trees, and encodes the statement trees to vectors by capturing the lexical and syntactical knowledge of statements. Based on the sequence of statement vectors, a bidirectional RNN model is used to leverage the naturalness of statements and finally produce the vector representation of a code fragment. We have applied our neural network based source code representation method to two common program comprehension tasks: source code classification and code clone detection. Experimental results on the two tasks indicate that our model is superior to state-of-the-art approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · cited 1×
“We use Recurrent Neural Network (RNN) [31] to encode statements and the sequential dependency between the statements into a vector.”
MethodImproved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
Tai, K. S., Socher, R., & Manning, C. D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1150 · Cited by 3,081
The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions
Hochreiter, S. (1998) · International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488598000094 · Cited by 2,939
The program dependence graph and its use in optimization
Ferrante, J., Ottenstein, K. J., & Warren, J. D. (1987) · ACM Transactions on Programming Languages and Systems · doi:10.1145/24039.24041 · Cited by 2,755
CCFinder: a multilinguistic token-based code clone detection system for large scale source code
Kamiya, T., Kusumoto, S., & Inoue, K. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1019480 · Cited by 1,711
Document Modeling with Gated Recurrent Neural Network for Sentiment Classification
Tang, D., Qin, B., & Liu, T. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1167 · Cited by 1,519
Clone detection using abstract syntax trees
Baxter, I. D., Yahin, A., Moura, L., Sant'Anna, M., & Bier, L. (2002) · Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272) · doi:10.1109/icsm.1998.738528 · Cited by 1,277
Asymmetric Transitivity Preserving Graph Embedding
Ou, M., Cui, P., Pei, J., Zhang, Z., & Zhu, W. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939751 · Cited by 1,248
(2011) · Cited by 1,192
DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones
Jiang, L., Misherghi, G., Su, Z., & Glondu, S. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.30 · Cited by 1,039
Automatically finding patches using genetic programming
Weimer, W., Nguyen, T., Le Goues, C., & Forrest, S. (2009) · 2009 IEEE 31st International Conference on Software Engineering · doi:10.1109/icse.2009.5070536 · Cited by 761
A Survey on Software Clone Detection Research
Roy, C. K., & Cordy, J. R. (2007) · Cited by 723
Automatically learning semantic features for defect prediction
Wang, S., Liu, T., & Tan, L. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884804 · Cited by 689
Convolutional Neural Networks over Tree Structures for Programming Language Processing
Mou, L., Li, G., Zhang, L., Wang, T., & Jin, Z. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10139 · Cited by 675
Evaluating defect prediction approaches: a benchmark and an extensive comparison
D’Ambros, M., Lanza, M., & Robbes, R. (2012) · Empirical Software Engineering · doi:10.1007/s10664-011-9173-9 · Cited by 626
An Empirical Comparison of Model Validation Techniques for Defect Prediction Models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2584050 · Cited by 574
Deep learning code fragments for code clone detection
White, M., Tufano, M., Vendome, C., & Poshyvanyk, D. (2016) · Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/2970276.2970326 · Cited by 558
Code completion with statistical language models
Raychev, V., Vechev, M., & Yahav, E. (2014) · Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2594291.2594321 · Cited by 555
Fine-grained and accurate source code differencing
Falleri, J.-R., Morandat, F., Blanc, X., Martinez, M., & Monperrus, M. (2014) · Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering · doi:10.1145/2642937.2642982 · Cited by 549
Gu, X., Zhang, H., & Kim, S. (2018) · Proceedings of the 40th International Conference on Software Engineering · doi:10.1145/3180155.3180167 · Cited by 547
Self-citationOn the naturalness of software
Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Dévanbu, P. (2012) · doi:10.5555/2337223.2337322 · Cited by 541
Zhou, J., Zhang, H., & Lo, D. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227210 · Cited by 507
Self-citationGu, X., Zhang, H., Zhang, D., & Kim, S. (2016) · Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2950290.2950334 · Cited by 485
Self-citationSourcererCC: scaling code clone detection to big-code
Sajnani, H., Saini, V., Svajlenko, J., Roy, C. K., & Lopes, C. V. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884877 · Cited by 447
On the naturalness of software
Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Devanbu, P. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227135 · Cited by 425
Long Short-Term Memory Over Recursive Structures
Zhu, X., Sobihani, P., & Guo, H. (2015) · NPARC · Cited by 386
Zhou, J. H., Zhang, H., & Lo, D. (2012) · doi:10.5555/2337223.2337226 · Cited by 372
Self-citationSuggesting accurate method and class names
Allamanis, M., Barr, E. T., Bird, C., & Sutton, C. (2015) · Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2786805.2786849 · Cited by 369
DeepBugs: a learning approach to name-based bug detection
Pradel, M., & Sen, K. (2018) · Proceedings of the ACM on Programming Languages · doi:10.1145/3276517 · Cited by 342
Towards a Big Data Curated Benchmark of Inter-project Code Clones
Svajlenko, J., Islam, J. F., Keivanloo, I., Roy, C. K., & Mia, M. M. (2014) · 2014 IEEE International Conference on Software Maintenance and Evolution · doi:10.1109/icsme.2014.77 · Cited by 339
Supporting program comprehension with source code summarization
Haiduc, S., Aponte, J., & Marcus, A. (2010) · Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2 · doi:10.1145/1810295.1810335 · Cited by 291
Studying the language and structure in non-programmers' solutions to programming problems
PANE, J. F., RATANAMAHATANA, C. “., & MYERS, B. A. (2001) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.2000.0410 · Cited by 289
Zaremba, W., & Sutskever, I. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.4615 · Cited by 283
SourcererCC: Scaling Code Clone Detection to Big Code
Sajnani, H., Saini, V., Svajlenko, J., Roy, C. K., & Lopes, C. V. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.06448 · Cited by 280
Wei, H., & Li, M. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/423 · Cited by 280
DeepSim: deep learning code functional similarity
Zhao, G., & Huang, J. (2018) · Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3236024.3236068 · Cited by 261
Automatically generating commit messages from diffs using neural machine translation
Jiang, S., Armaly, A., & McMillan, C. (2017) · 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115626 · Cited by 255
On the "naturalness" of buggy code
Ray, B., Hellendoorn, V., Godhane, S., Tu, Z., Bacchelli, A., & Devanbu, P. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884848 · Cited by 252
The Java Language Specification, Java SE 8 Edition
Gosling, J., Joy, B., Steele, G. L., Bracha, G., & Buckley, A. (2013) · Cited by 232
Semantically Enhanced Software Traceability Using Deep Learning Techniques
Guo, J., Cheng, J., & Cleland-Huang, J. (2017) · 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2017.9 · Cited by 232
Panichella, A., Dit, B., Oliveto, R., Di Penta, M., Poshynanyk, D., & De Lucia, A. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606598 · Cited by 216
Supporting Program Comprehension Using Semantic and Structural Information
Maletic, J. I., & Marcus, A. (2001) · Cited by 212
A precise inter-procedural data flow algorithm
Myers, E. M. (1981) · Proceedings of the 8th ACM SIGPLAN-SIGACT symposium on Principles of programming languages - POPL '81 · doi:10.1145/567532.567556 · Cited by 211
Panichella, A., Dit, B., Oliveto, R., Penta, M. D., Poshynanyk, D., & Lucia, A. D. (2013) · Archivio Istituzionale della Ricerca - Università degli Studi del Molise · Cited by 206
Code completion with statistical language models
Raychev, V., Vechev, M., & Yahav, E. (2014) · ACM SIGPLAN Notices · doi:10.1145/2666356.2594321 · Cited by 189
A framework for source code search using program patterns
Paul, S., & Prakash, A. (1994) · IEEE Transactions on Software Engineering · doi:10.1109/32.295894 · Cited by 180
Combining Deep Learning with Information Retrieval to Localize Buggy Files for Bug Reports (N)
Lam, A. N., Nguyen, A. T., Nguyen, H. A., & Nguyen, T. N. (2015) · 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.73 · Cited by 179
Deep learning similarities from different representations of source code
Tufano, M., Watson, C., Bavota, G., Di Penta, M., White, M., & Poshyvanyk, D. (2018) · Proceedings of the 15th International Conference on Mining Software Repositories · doi:10.1145/3196398.3196431 · Cited by 158
The program dependence graph and its use in optimization
Ferrante, J., Ottenstein, K. J., & Warren, J. D. (1984) · Lecture notes in computer science · doi:10.1007/3-540-12925-1_33 · Cited by 147
Xu, B., Ye, D., Xing, Z., Xia, X., Chen, G.-B., & Li, S. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970357 · Cited by 146
MUDABlue: An automatic categorization system for Open Source repositories
Kawaguchi, S., Garg, P. K., Matsushita, M., & Inoue, K. (2006) · Journal of Systems and Software · doi:10.1016/j.jss.2005.06.044 · Cited by 118
On using machine learning to automatically classify software applications into domain categories
Linares‐Vásquez, M., McMillan, C., Poshyvanyk, D., & Grechanik, M. (2012) · Empirical Software Engineering · doi:10.1007/s10664-012-9230-z · Cited by 95
Le, P., & Zuidema, W. H. (2016) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/w16-1610 · Cited by 95
Using IR methods for labeling source code artifacts: Is it worthwhile?
Lucia, A. D., Penta, M. D., Oliveto, R., Panichella, A., & Panichella, S. (2012) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2012.6240488 · Cited by 94
Huo, X., & Li, M. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/265 · Cited by 88
Modeling class cohesion as mixtures of latent topics
Liu, Y., Poshyvanyk, D., Ferenć, R., Gyimóthy, T., & Chrisochoides, N. P. (2009) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2009.5306318 · Cited by 81
Understanding semi-structured merge conflict characteristics in open-source Java projects
Accioly, P., Borba, P., & Cavalcanti, G. (2018) · Empirical Software Engineering · doi:10.1007/s10664-017-9586-1 · Cited by 48
Cho, S., Chi, Z., Siu, W.-C., & Tsoi, A. C. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.813831 · Cited by 41
An empirical assessment of Bellon's clone benchmark
Charpentier, A., Falleri, J., Lo, D., & Réveillère, L. (2015) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/2745802.2745821 · Cited by 24
Cited by
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029
Self-citationUnified Pre-training for Program Understanding and Generation
Ahmad, W., Chakraborty, S., Ray, B., & Chang, K.-W. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.211 · Cited by 642
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536
GraphCodeBERT: Pre-training Code Representations with Data Flow
Guo, D., Ren, S., Lu, S., Feng, Z., Tang, D., Liu, S., Zhou, L., Duan, N., Svyatkovskiy, A., Fu, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.08366 · Cited by 480
DeepWukong: Statically Detecting Software Vulnerabilities Using Deep Graph Neural Network
Cheng, X., Wang, H., Hua, J., Xu, G., & Sui, Y. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3436877 · Cited by 335
Metadata
- Authors
- Zhang, J., Wang, X., Zhang, H., Sun, H., Wang, K., & Liu, X.
- Venue
- 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)
- Publisher
- IEEE
- Published
- 2019-05
- Pages
- 783-794
- OpenAlex
- W2955426500
- Cited by
- 655
- References
- 67
- FWCI
- 52.42
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.