Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 4

The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.

JEMMA: An extensible Java dataset for ML4Code applications

Karmakar, Anjan, Allamanis, Miltiadis, Robbes, Romain·2023·Empirical Software Engineering
DOI 10.1007/s10664-022-10275-7
Open this paper in CoChat — free account

Abstract

Machine Learning for Source Code () is an active research field in which extensive experimentation is needed to discover how to best use source code’s richly structured information. With this in mind, we introduce : An Extensible Java Dataset for Applications, which is a large-scale, diverse, and high-quality dataset targeted at . Our goal with is to lower the barrier to entry in by providing the building blocks to experiment with source code models and tasks. comes with a considerable amount of pre-processed information such as metadata, representations (e.g., code tokens, ASTs, graphs), and several properties (e.g., metrics, static analysis results) for 50,000 Java projects from the dataset, with over 1.2 million classes and over 8 million methods. is also extensible allowing users to add new properties and representations to the dataset, and evaluate tasks on them. Thus, becomes a workbench that researchers can use to experiment with novel representations and tasks operating on source code. To demonstrate the utility of the dataset, we also report results from two empirical studies on our data, ultimately showing that significant work lies ahead in the design of context-aware source code models that can reason over a broader network of source code entities in a software project—the very task that is designed to help with.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10664-022-10275-7

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 26

  1. Allamanis M, Brockschmidt M, Khademi M (2017) Learning to represent programs with graphs. arXiv:171100740
  2. Allamanis M, Barr E T, Devanbu P, Sutton C (2018) A survey of machine learning for big code and naturalness. ACM Computing Surveys (CSUR) 51(4):1–37 · DOI · Paper page
  3. Alon U, Yahav E (2020) On the bottleneck of graph neural networks and its practical implications. arXiv:200605205
  4. Alon U, Brody S, Levy O, Yahav E (2018) code2seq: Generating sequences from structured representations of code. arXiv:180801400
  5. Barone AVM, Sennrich R (2017) A parallel corpus of Python functions and documentation strings for automated code documentation and code generation. arXiv:170702275
  6. Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, et al. (2020) Language models are few-shot learners. arXiv:200514165
  7. Chen M, Tworek J, Jun H, Yuan Q, Pinto HPdO, Kaplan J, Edwards H, Burda Y, Joseph N, Brockman G, et al. (2021) Evaluating large language models trained on code. arXiv:210703374
  8. Guo D, Ren S, Lu S, Feng Z, Tang D, Liu S, Zhou L, Duan N, Svyatkovskiy A, Fu S, et al. (2020) Graphcodebert: Pre-training code representations with data flow. arXiv:200908366
  9. Hellendoorn V J, Sutton C, Singh R, Maniatis P, Bieber D (2019b) Global relational models of source code. In: International conference on learning representations
  10. Husain H, Wu HH, Gazit T, Allamanis M, Brockschmidt M (2019) Codesearchnet challenge: Evaluating the state of semantic code search. arXiv:190909436
  11. Kanade A, Maniatis P, Balakrishnan G, Shi K (2020) Learning and evaluating contextual embedding of source code. In: International Conference on Machine Learning, PMLR, pp 5110–5121
  12. Kitaev N, Kaiser Ł, Levskaya A (2020) Reformer: The efficient transformer. arXiv:200104451
  13. Lin XV, Wang C, Zettlemoyer L, Ernst MD (2018) Nl2bash: A corpus and semantic parser for natural language interface to the linux operating system. arXiv:180208979
  14. Lu S, Guo D, Ren S, Huang J, Svyatkovskiy A, Blanco A, Clement C, Drain D, Jiang D, Tang D, et al. (2021) Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv:210204664
  15. Nijkamp E, Pang B, Hayashi H, Tu L, Wang H, Zhou Y, Savarese S, Xiong C (2022) A conversational paradigm for program synthesis. arXiv:220313474
  16. Puri R, Kung DS, Janssen G, Zhang W, Domeniconi G, Zolotov V, Dolby J, Chen J, Choudhury M, Decker L, et al. (2021) Project codenet: A large-scale ai for code dataset for learning a diversity of coding tasks. arXiv:210512655
  17. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I, et al. (2019) Language models are unsupervised multitask learners. OpenAI blog 1(8):9
  18. Tay Y, Dehghani M, Abnar S, Shen Y, Bahri D, Pham P, Rao J, Yang L, Ruder S, Metzler D (2020a) Long range arena: A benchmark for efficient transformers. arXiv:201104006
  19. Tay Y, Dehghani M, Bahri D, Metzler D (2020b) Efficient transformers: A survey. arXiv:200906732
  20. Utture A, Kalhauge CG, Liu S, Palsberg J (2020) Njr-1 dataset. https://doi.org/10:5281/zenodo4839913
  21. Vasic M, Kanade A, Maniatis P, Bieber D, Singh R (2019) Neural program repair by jointly learning to localize and repair. arXiv:190401720
  22. Wang K, Christodorescu M (2019) Coset: A benchmark for evaluating neural program embeddings. arXiv:190511445
  23. Wei J, Goyal M, Durrett G, Dillig I (2020) Lambdanet: Probabilistic type inference using graph neural networks. arXiv:200502161
  24. Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, et al. (2019b) Huggingface’s transformers: State-of-the-art natural language processing. arXiv:191003771 · DOI · Paper page
  25. Zavershynskyi M, Skidanov A, Polosukhin I (2018) Naps: Natural program synthesis dataset. arXiv:180703168
  26. Zhou Y, Liu S, Siow J, Du X, Liu Y (2019) Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. arXiv:190903496

Metadata

Authors
Karmakar, A., Allamanis, M., & Robbes, R.
Venue
Empirical Software Engineering
Publisher
Springer Science and Business Media LLC
Published
2023-03
Volume
28
Issue
2
ISSN
  • 1382-3256
  • 1573-7616
DOI
10.1007/s10664-022-10275-7
Cited by
4
References
68
Language
en
Licence
Funders
Libera Università di Bolzano

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.