The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
JEMMA: An extensible Java dataset for ML4Code applications
Abstract
Machine Learning for Source Code () is an active research field in which extensive experimentation is needed to discover how to best use source code’s richly structured information. With this in mind, we introduce : An Extensible Java Dataset for Applications, which is a large-scale, diverse, and high-quality dataset targeted at . Our goal with is to lower the barrier to entry in by providing the building blocks to experiment with source code models and tasks. comes with a considerable amount of pre-processed information such as metadata, representations (e.g., code tokens, ASTs, graphs), and several properties (e.g., metrics, static analysis results) for 50,000 Java projects from the dataset, with over 1.2 million classes and over 8 million methods. is also extensible allowing users to add new properties and representations to the dataset, and evaluate tasks on them. Thus, becomes a workbench that researchers can use to experiment with novel representations and tasks operating on source code. To demonstrate the utility of the dataset, we also report results from two empirical studies on our data, ultimately showing that significant work lies ahead in the design of context-aware source code models that can reason over a broader network of source code entities in a software project—the very task that is designed to help with.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.139 · Cited by 2,824
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · arXiv · doi:10.48550/arxiv.2109.00859 · Cited by 1,495
Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., Roman, S., et al. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1425 · Cited by 1,034
code2vec: Learning distributed representations of code
Alon, U., Zilberstein, M., Levy, O., & Yahav, E. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3290353 · Cited by 962
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
McCoy, R. T., Pavlick, E., & Linzen, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1334 · Cited by 944
Annotation Artifacts in Natural Language Inference Data
Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., & Smith, N. A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2017 · Cited by 907
Summarizing Source Code using a Neural Attention Model
Iyer, S., Konstas, I., Cheung, A., & Zettlemoyer, L. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1195 · Cited by 701
Recognition in Terra Incognita
Beery, S., Van Horn, G., & Perona, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_28 · Cited by 679
Research works printed but not linked: 26
- Allamanis M, Brockschmidt M, Khademi M (2017) Learning to represent programs with graphs. arXiv:171100740
- Allamanis M, Barr E T, Devanbu P, Sutton C (2018) A survey of machine learning for big code and naturalness. ACM Computing Surveys (CSUR) 51(4):1–37 · DOI · Paper page
- Alon U, Yahav E (2020) On the bottleneck of graph neural networks and its practical implications. arXiv:200605205
- Alon U, Brody S, Levy O, Yahav E (2018) code2seq: Generating sequences from structured representations of code. arXiv:180801400
- Barone AVM, Sennrich R (2017) A parallel corpus of Python functions and documentation strings for automated code documentation and code generation. arXiv:170702275
- Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, et al. (2020) Language models are few-shot learners. arXiv:200514165
- Chen M, Tworek J, Jun H, Yuan Q, Pinto HPdO, Kaplan J, Edwards H, Burda Y, Joseph N, Brockman G, et al. (2021) Evaluating large language models trained on code. arXiv:210703374
- Guo D, Ren S, Lu S, Feng Z, Tang D, Liu S, Zhou L, Duan N, Svyatkovskiy A, Fu S, et al. (2020) Graphcodebert: Pre-training code representations with data flow. arXiv:200908366
- Hellendoorn V J, Sutton C, Singh R, Maniatis P, Bieber D (2019b) Global relational models of source code. In: International conference on learning representations
- Husain H, Wu HH, Gazit T, Allamanis M, Brockschmidt M (2019) Codesearchnet challenge: Evaluating the state of semantic code search. arXiv:190909436
- Kanade A, Maniatis P, Balakrishnan G, Shi K (2020) Learning and evaluating contextual embedding of source code. In: International Conference on Machine Learning, PMLR, pp 5110–5121
- Kitaev N, Kaiser Ł, Levskaya A (2020) Reformer: The efficient transformer. arXiv:200104451
- Lin XV, Wang C, Zettlemoyer L, Ernst MD (2018) Nl2bash: A corpus and semantic parser for natural language interface to the linux operating system. arXiv:180208979
- Lu S, Guo D, Ren S, Huang J, Svyatkovskiy A, Blanco A, Clement C, Drain D, Jiang D, Tang D, et al. (2021) Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv:210204664
- Nijkamp E, Pang B, Hayashi H, Tu L, Wang H, Zhou Y, Savarese S, Xiong C (2022) A conversational paradigm for program synthesis. arXiv:220313474
- Puri R, Kung DS, Janssen G, Zhang W, Domeniconi G, Zolotov V, Dolby J, Chen J, Choudhury M, Decker L, et al. (2021) Project codenet: A large-scale ai for code dataset for learning a diversity of coding tasks. arXiv:210512655
- Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I, et al. (2019) Language models are unsupervised multitask learners. OpenAI blog 1(8):9
- Tay Y, Dehghani M, Abnar S, Shen Y, Bahri D, Pham P, Rao J, Yang L, Ruder S, Metzler D (2020a) Long range arena: A benchmark for efficient transformers. arXiv:201104006
- Tay Y, Dehghani M, Bahri D, Metzler D (2020b) Efficient transformers: A survey. arXiv:200906732
- Utture A, Kalhauge CG, Liu S, Palsberg J (2020) Njr-1 dataset. https://doi.org/10:5281/zenodo4839913
- Vasic M, Kanade A, Maniatis P, Bieber D, Singh R (2019) Neural program repair by jointly learning to localize and repair. arXiv:190401720
- Wang K, Christodorescu M (2019) Coset: A benchmark for evaluating neural program embeddings. arXiv:190511445
- Wei J, Goyal M, Durrett G, Dillig I (2020) Lambdanet: Probabilistic type inference using graph neural networks. arXiv:200502161
- Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, et al. (2019b) Huggingface’s transformers: State-of-the-art natural language processing. arXiv:191003771 · DOI · Paper page
- Zavershynskyi M, Skidanov A, Polosukhin I (2018) Naps: Natural program synthesis dataset. arXiv:180703168
- Zhou Y, Liu S, Siow J, Du X, Liu Y (2019) Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. arXiv:190903496
Metadata
- Authors
- Karmakar, A., Allamanis, M., & Robbes, R.
- Venue
- Empirical Software Engineering
- Published
- 2023-03
- Volume
- 28
- Issue
- 2
- ISSN
- 1382-3256
- 1573-7616
- Cited by
- 4
- References
- 68
- Language
- en
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor, 9 d)
- Funders
- Libera Università di Bolzano
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: JEMMA: An extensible Java dataset for ML4Code applications. https://app.cochat.ai/papers/10.1007/s10664-022-10275-7
- The paper
- Karmakar, A., Allamanis, M., & Robbes, R. (2023). JEMMA: An extensible Java dataset for ML4Code applications. Empirical Software Engineering. https://doi.org/10.1007/s10664-022-10275-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
