Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)
Open accessThe DOI matches its Crossref record; 79 of the 80 references it cites were found in a registry or an index; 33 of the 57 sources its claims cite were checked for support.
Abstract
Large Language Models (LLM) are a new class of computation engines, "programmed" via prompt engineering. Researchers are still learning how to best "program" these LLMs to help developers. We start with the intuition that developers tend to consciously and unconsciously collect semantics facts, from the code, while working. Mostly these are shallow, simple facts arising from a quick read. For a function, such facts might include parameter and local variable names, return expressions, simple pre- and post-conditions, and basic control and data flow, etc.
Overview
CoChat Summary
## TL;DR
The paper proposes ASAP (Automatic Semantic Augmentation of Prompts), which adds automatically extracted semantic facts from code into few-shot prompts for LLM-based code summarization and code completion. On CodeSearchNet/CodeXGLUE-style evaluations, the augmented prompts generally improved performance, including statistically significant BLEU gains across six languages and improvements in zero-shot line completion.
## Key findings
- On code summarization with `code-davinci-002`, ASAP improved BLEU-4 over BM25 few-shot prompting in all six languages: Java 22.87→25.41 (+11.11%), Python 21.78→24.26 (+11.39%), Ruby 17.21→19.62 (+14.00%), JavaScript 23.27→25.36 (+8.98%), Go 22.67→24.51 (+8.12%), and PHP 28.15→32.73 (+16.27%); overall 22.66→25.32 (+11.74%), with p < 0.01 in each case.
- In same-project summarization, ASAP outperformed vanilla BM25 few-shot learning in 14 of 16 BLEU comparisons (87.5%).
- For line completion on 9,292 Java and 6,550 Python samples, ASAP improved Exact Match from 17.97 to 19.01 overall (+5.79%, p < 0.01) and Edit Similarity from 53.01 to 55.72 overall (+5.11%, p < 0.01).
- ASAP also improved other metrics overall: BLEU-DC and ROUGE-L showed significant gains, while METEOR improved on average but was not significant for 4 of 6 languages.
## Methods
- The paper evaluates few-shot prompting for code summarization on the CodeSearchNet dataset, using 1,000 test samples per language across six languages: Java, Python, Ruby, JavaScript, Go, and PHP; for same-project experiments, it used project-specific subsets and sorted them by creation date to avoid leakage.
- The core method, ASAP, augments each retrieved exemplar and the target code with automatically derived semantic facts: repository name/path, tagged identifiers, and a data flow graph; exemplars are retrieved with BM25 from the training set.
- Main analysis used OpenAI models (`code-davinci-002` primarily, plus `text-davinci-003` and `gpt-3.5-turbo`), with 3-shot prompts, temperature 0, and statistical testing via one-sided pairwise Wilcoxon signed-rank tests (with B-H correction where reported); line completion was evaluated with Exact Match and Edit Similarity.
## Limitations
- The paper states it did not conduct a user study, so the metric gains may not translate directly into improved developer performance.
- It could not directly check whether the model had seen test data during training because the training dataset is inaccessible; this is a limitation stated in the paper.
- Evident from the text: the main experiments were constrained by OpenAI API rate limits and cost, so many analyses used only 1,000 samples per language, and some deeper analyses were restricted mainly to Java and Python.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
17 Valid · 8 Not valid · 11 Not checked · 81 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“In most cases, we see improvements of over 2.0 BLEU, the required threshold for human perception noted by Roy et al. [57].”
§ Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization) > 4.2 A𝑆𝐴𝑃 Prompt Enhancement
What the sources say
- Disputedcites [57] 57. Devjeet Roy, Sarah Fakhoury, and Venera Arnaoudova. 2021. Reassessing auto- matic evaluation metrics for code summarization tasks. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1105-1116. sourceExperimental — accuracy not yet measured
In “Pairwise Model Comparisons” (page not provided), Roy et al. say human assessors perceived differences even when metric scores differed by less than 2 points. They describe 2 points as a boundary below which automatic metrics are unreliable, not a required BLEU threshold for human perception. The paper does not assess ASAP’s improvements. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“De-duplication is key: Code duplication in machine learning models can deceptively inflate performance metrics a lot, when compared to de-duplicated datasets [7,46,59].”
§ Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization) > 3.1 Dataset
What the sources say
- Supportedcites [7] 7. Miltiadis Allamanis. 2019. The adverse effects of code duplication in machine learning models of code. In Proceedings of the 2019 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software. 143-153. source
The Introduction says duplication can cause evaluations to overestimate the performance users actually observe, with observed performance up to 50% worse than reported results. Section 5 adds that end-user performance is often significantly worse than evaluation metrics suggest, supporting the claim that inflation can be substantial. (LLM, conf=0.93)
- Partly supportedcites [46] 46. Cristina V Lopes, Petr Maj, Pedro Martins, Vaibhav Saini, Di Yang, Jakub Zitny, Hitesh Sajnani, and Jan Vitek. 2017. DéjàVu: a map of code duplicates on GitHub. Proceedings of the ACM on Programming Languages 1, OOPSLA (2017), 1-28. sourceExperimental — accuracy not yet measured
Our copy of DéjàVu supports only the general, software-engineering-level point: the Conclusions (Sec. 7) state that 'Code duplication can severely skew the conclusions of those studies' and that the 'assumption of diversity of projects in those datasets may be compromised', and the Introduction (Sec. 1) warns that reuse and file copying 'at scale, may bias results of research'; Sec. 2 adds that duplicate projects in a dataset create a 'lack of diversity ... [that] might lead to incorrect observations' (crediting Nagappan et al. 2013). None of these passages concern machine learning models, nor any comparison of performance metrics on duplicated versus de-duplicated datasets — the paper measures file- and project-level clone rates in a 4.5M-project GitHub corpus (Java, C++, Python, JavaScript), not model accuracy. The ML-specific inflation claim attributed here is therefore not supported by this source; only the background point that duplication distorts dataset validity is. (LLM, conf=0.72)
- Partly supportedcites [59] 59. Ensheng Shi, Yanlin Wang, Lun Du, Junjie Chen, Shi Han, Hongyu Zhang, Dong- mei Zhang, and Hongbin Sun. 2023. On the evaluation of neural code summariza- tion. In Proceedings of the 44th International Conference on Software Engineering. 1597-1608. sourceExperimental — accuracy not yet measured
Our copy supports the direction and significance of the claim: the Abstract (p. 1) and CCS Concepts (p. 2) list duplication ratios among dataset characteristics that 'have a significant impact on model evaluation', and RQ3 (p. 5) states that TL-CodeSum contains 20% exact code duplication while Funcom and CodeSearchNet were deduplicated, motivating experiments at different duplication ratios, with Section 4.1 (p. 5) reporting that most models score lower on the deduplicated TLC Dedup variant. What the source does not say is the claim's strong framing: it hedges that 'ignoring code duplication may result in model performance and generalization ability not being comprehensively evaluated' rather than asserting metrics are 'deceptively inflated a lot', and the explicit overestimation-by-duplication conclusion is credited to another work in Related Work (p. 10) — Allamanis et al. — not reported as this paper's own finding. The magnitude ('a lot') is therefore not established by our copy for this source. (LLM, conf=0.72)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“the 2-BLEU threshold noted by Roy et al. [57], below which BLEU results are unreliable predictors of human preference.”
§ Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization) > 1 INTRODUCTION
What the sources say
- Supportedcites [57] 57. Devjeet Roy, Sarah Fakhoury, and Venera Arnaoudova. 2021. Reassessing auto- matic evaluation metrics for code summarization tasks. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1105-1116. source
In Pairwise Model Comparisons, the paper says that when two code-summarization approaches differ by no more than 2 metric points, automatic metrics cannot reliably distinguish their quality in agreement with human ratings. This includes BLEU. The threshold concerns the difference between BLEU scores, not an absolute BLEU score below 2. (LLM, conf=0.90)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Roy et al. [57] provide arguments, grounded on human subject study that for code summarization (our central task), that a gain of 2.0 or more BLEU is more likely to correspond with human perception of improvement.”
§ Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization) > 3.5 Metrics
What the sources say
- Partly supportedcites [57] 57. Devjeet Roy, Sarah Fakhoury, and Venera Arnaoudova. 2021. Reassessing auto- matic evaluation metrics for code summarization tasks. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1105-1116. sourceExperimental — accuracy not yet measured
The Pairwise Model Comparisons section reports that metric differences of 2 points or less do not consistently align with human judgments, supporting the claim’s general point. But it does not establish 2.0 BLEU as a threshold for perceived improvement: the threshold varies by metric, and BLEU’s disagreement with human assessment is described as erratic even at larger differences. The Implications section recommends METEOR and chrF over BLEU as indicators of human judgment. (LLM, conf=0.88)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“For the tests that we used (Wilcoxon Signed-rank test), we found that the needed sample size was always below the sample size we used for our primary studies, viz., 1000.”
§ Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization) > Sample Size Analysis:
What the sources say
- Could not verifycites [19] 19. Franz Faul, Edgar Erdfelder, Axel Buchner, and Albert-Georg Lang. 2009. Sta- tistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior research methods 41, 4 (2009), 1149-1160. sourceExperimental — accuracy not yet measured
The paper’s sample-size examples concern correlation and regression analyses (Effect size measure; 5.1 Logistic Regression; page numbers unavailable). Searches for Wilcoxon signed-rank tests and the cited studies’ sample size of 1,000 found no passage addressing that comparison. (LLM, conf=0.77)
- Could not verifycites [20] 20. Franz Faul, Edgar Erdfelder, Albert-Georg Lang, and Axel Buchner. 2007. G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods 39, 2 (2007), 175-191. sourceExperimental — accuracy not yet measured
The A Priori Power Analyses section (page unavailable) explains how required sample size is calculated, but it does not report the citing authors’ calculations. Searches for Wilcoxon signed-rank tests and a sample size of 1,000 found no passage establishing that the required size was always below 1,000. (LLM, conf=0.77)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
36 more claims
Create a free account to see all 117 claims and to verify the 9 references still open.
- It may state two separate claims as one. 50
- Probably not a substantial claim. 45
- The quote only partly states this claim. 4
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“ASAP [171] incorporates exemplar tuples, consisting of input code, function definitions, analysis results, and corresponding comments, into prompts to yield better results.”
From Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024) · cites this paper as [171]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971
Statistical power analyses using G∗Power 3.1: tests for correlation and regression analyses
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009) · Behavior Research Methods · doi:10.3758/brm.41.4.1149 · Cited by 37,615
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
The probabilistic relevance framework: BM25 and beyond
Robertson, S., & Zaragoza, H. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000019 · Cited by 3,374 · cited 1×
“BM25 [55] is a frequency-based retrieval method which improves upon TF-IDF [54].”
Method
Research works printed but not linked: 3
- 43. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. 74-81.
- 53. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
- 54. Juan Ramos et al. 2003. Using tf-idf to determine word relevance in document queries. In Proceedings of the first instructional conference on machine learning, Vol. 242. Citeseer, 29-48.
Cited by
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029
Retrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2024) · arXiv · doi:10.48550/arxiv.2402.19473 · Cited by 240
“ASAP [203] add exemplar tuples to the prompt for better results.”
Towards an understanding of large language models in software engineering tasks
Zheng, Z., Ning, K., Zhong, Q., Chen, J., Chen, W., Guo, L., Wang, W., & Wang, Y. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10602-0 · Cited by 116
Generative AI for self-adaptive systems: State of the art and research roadmap
Li, J., Zhang, M., Li, N., Weyns, D., Jin, Z., & Tei, K. (2024) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3686803 · Cited by 77
Prompt Engineering in Clinical Practice: Tutorial for Clinicians
Liu, J., Liu, F., Wang, C., & Liu, S. (2025) · Journal of Medical Internet Research · doi:10.2196/72644 · Cited by 76
Source Code Summarization in the Era of Large Language Models
Sun, W., Miao, Y., Li, Y., Zhang, H., Fang, C., Liu, Y., Deng, G., Liu, Y., & Chen, Z. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00034 · Cited by 213
Shi, J., Yang, Z., & Lo, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708525 · Cited by 43
Revisiting Sentiment Analysis for Software Engineering in the Era of Large Language Models
Zhang, T., Irsan, I. C., Thung, F., & Lo, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3697009 · Cited by 81
“This demonstrates the considerable advantages of fine-tuning sLLMs in terms of all the macro-F1, micro-F1, and AUC scores.”
An Empirical Study on Challenges for LLM Application Developers
Chen, X., Gao, C., Chen, C., Zhang, G., & Liu, Y. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715007 · Cited by 110
Generative AI for Pull Request Descriptions: Adoption, Impact, and Developer Interventions
Xiao, T., Hata, H., Treude, C., & Matsumoto, K. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3643773 · Cited by 29
Agents in software engineering: survey, landscape, and vision
Wang, Y., Zhong, W., Huang, Y., Shi, E., Yang, M., Chen, J., Li, H., Ma, Y., Wang, Q., & Zheng, Z. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00544-2 · Cited by 132
Do Advanced Language Models Eliminate the Need for Prompt Engineering in Software Engineering?
Wang, G., Sun, Z., Ye, S., Gong, Z., Chen, Y., Zhao, Y., Liang, Q., & Hao, D. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3771933 · Cited by 26
AI-Driven Refactoring: A Pipeline for Identifying and Correcting Data Clumps in Git Repositories
Baumgartner, N., Iyenghar, P., Schoemaker, T., & Pulvermüller, E. (2024) · Electronics · doi:10.3390/electronics13091644 · Cited by 27
“[29,30].”
LiSSA: Toward Generic Traceability Link Recovery Through Retrieval-Augmented Generation
Fuchß, D., Hey, T., Keim, J., Liu, H., Ewald, N., Thirolf, T., & Koziolek, A. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00186 · Cited by 52
A review of automatic source code summarization
Zhang, X., Hou, X., Qiao, X., & Song, W. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10553-6 · Cited by 34
An LLM-Based Agent-Oriented Approach for Automated Code Design Issue Localization
Batole, F., O'Brien, D. P., Nguyen, T. N., Dyer, R., & Rajan, H. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00100 · Cited by 34
“The effectiveness of LLMs heavily relies on the quality and relevance of the prompts [23, 24].”
FT2Ra: A Fine-Tuning-Inspired Approach to Retrieval-Augmented Code Completion
Guo, Q., Li, X., Xie, X., Liu, S., Tang, Z. J., Feng, R., Wang, J., Ge, J., & Bu, L. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3652130 · Cited by 28
“Our source code and experimental data are available at [3].”
Hu, S., & Wang, X. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-5803-6_24 · Cited by 34
On the Effectiveness of Large Language Models in Statement-level Code Summarization
Zhu, J., Miao, Y., Xu, T., Zhu, J., & Sun, X. (2024) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs62785.2024.00030 · Cited by 20
Can Large Language Models Serve as Evaluators for Code Summarization?
Wu, Y., Wan, Y., Chu, Z., Zhao, W., Liu, Y., Zhang, H., Shi, X., Jin, H., & Yu, P. S. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3595283 · Cited by 42
“More recently, the emergence of LLMs has garnered significant interest, prompting numerous studies to investigate its potential for code summarization [30], [59], [60], [61], [62], [63].”
LLM-as-a-Judge for Software Engineering: Literature Review, Vision, and the Road Ahead
He, J., Shi, J., Zhuo, T. Y., Treude, C., Sun, J., Du, X., Xing, Z., & Lo, D. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3797276 · Cited by 12
“These models have demonstrated exceptional performance on longstanding challenges, from code generation [52, 131] and code summarization [2, 83] to program translation [63, 114] and repair [36, 38].”
Calibration of Large Language Models on Code Summarization
Virk, Y., Dévanbu, P., & Ahmed, T. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729400 · Cited by 59
“It has infilling capabilities, support for extensive input contexts, and zero-shot instruction-following abilities.”
InfluentialSelf-citationBeyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models
Wang, Y., Jiang, T., Liu, M., Chen, J., Mao, M., Liu, X., Ma, Y., & Zheng, Z. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715749 · Cited by 57
Gao, C., Chen, X., & Zhang, G. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107803 · Cited by 10
“These LLMs have been recently used for improving software engineering tasks (such as code summarization [23, 26, 58], bug fixing [59, 60], and program synthesis [61, 62]).”
Can Developers Prompt? A Controlled Experiment for Code Documentation Generation
Kruse, H.-A., Puhlfürß, T., & Maalej, W. (2024) · 2024 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme58944.2024.00058 · Cited by 8
“Ahmed et al. demonstrated its effectiveness in code summarization [18], [48].”
Enhancing robustness of AI offensive code generators via data augmentation
Improta, C., Liguori, P., Natella, R., Čukić, B., & Cotroneo, D. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10569-y · Cited by 15
“■ CodeT5+ [68] is a new family of Transformer models pre-trained with a diverse set of pretraining tasks including causal language modeling, contrastive learning, and text-code matching to learn rich representations from both unimodal code data and bimodal code-text data.”
Prompt Alchemy: Automatic Prompt Refinement for Enhancing Code Generation
Ye, S., Sun, Z., Wang, G., Guo, L., Liang, Q., Li, Z., & Liu, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3589634 · Cited by 28
“Moreover, human-designed prompts may not align consistently with the precise requirements of LLMs, further compromising performance [15].”
Dockerfile Flakiness: Characterization and Repair
Shabani, T., Nashid, N., Alian, P., & Mesbah, A. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00238 · Cited by 17
“While randomly selected examples can enhance performance, recent studies [34], [35] indicate that choosing examples based on their similarity to the input context can lead to even more significant improvements.”
Challenges and Paths Towards AI for Software EngineeringPreprint
Gu, A., Jain, N., Li, W., Shetty, M., Shao, Y., Li, Z., Yang, D., Ellis, K., Sen, K., & Solar-Lezama, A. (2025) · Qeios · doi:10.32388/vv1661 · Cited by 50
Enhanced Prompting Framework for Code Summarization with Large Language Models
Fang, M., Yuan, X., Li, Y., Li, H., Fang, C., & Du, J. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728949 · Cited by 14
Qiao, Y., Shihab, M. I. H., & Hundhausen, C. D. (2026) · ACM Transactions on Computing Education · doi:10.1145/3785366 · Cited by 31
“Therefore, a key implication is for instructors to shift assignments from product-based (e.g., writing this code) to process-based (e.g., critiquing this GenAI-generated code, or using GenAI to explain this code and find the bug, as seen in [19]).”
InfluentialNjuguna, G. N., & Qingfei, M. (2025) · Technology in Society · doi:10.1016/j.techsoc.2025.103080 · Cited by 7
LLM4SZZ: Enhancing SZZ Algorithm with Context-Enhanced Assessment on Large Language Models
Tang, L., Liu, J., Liu, Z., Yang, X., & Bao, L. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728885 · Cited by 24
Generative AI-Augmented Thematic Analysis
Jayawardene, V., & Ewing, M. T. (2025) · International Journal of Market Research · doi:10.1177/14707853251405043 · Cited by 9
“(1) Human experiences and contextual understanding Few-shot prompting (above) incorporating contextual information within prompts (Ahmed et al., 2024), can enhance gen AI ’ s grasp of complex human experiences.”
Exploring the impact of code review factors on the code review comment generation
Lu, J., Li, Z., Shen, C., Li, Y., & Zuo, C. (2024) · Automated Software Engineering · doi:10.1007/s10515-024-00469-2 · Cited by 8
“Recognizing that automatic metrics alone may not fully capture the effectiveness of comment generation models, we extended our evaluation by incorporating human judgment (Liu et al. 2024; Su and McMillan 2024; Khan and Uddin 2022; Ahmed et al. 2024).”
Large Language Models for Explainable Medical Text Summarization: A Systematic Literature Review
Ayalew, A. M., Hasan, M. R., Seppänen, T., & Oussalah, M. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70089 · Cited by 5
Automated Software Architecture Design Recovery from Source Code Using LLMs
Amalfitano, D., Luca, M. D., Santilli, T., Pelliccione, P., & Fasolino, A. R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02138-0_5 · Cited by 18
Cheng, Y., Zhang, T., Shar, L. K., Yang, S., Dong, C., Lo, D., Lv, S., Shi, Z., & Sun, L. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3599581 · Cited by 26
“Recently, large language models (LLMs) have shown remarkable performance in various code-related tasks, including code generation [19], [20] and code summarization [21], [22].”
Chen, A., & Geng, M. (2025) · Symmetry · doi:10.3390/sym17060934 · Cited by 6
Investigating Execution-Aware Language Models for Code Optimization
Menna, F. D., Traini, L., Bavota, G., & Cortellessa, V. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00029 · Cited by 5
“We exploit a Text-To-Text Transfer Transformer (T5) model, namely CodeT5+ [22], which is specifically pre-trained on source code and widely adopted in software engineering literature [32]–[35].”
An Approach to Intelligent Information Extraction and Utilization from Diverse Documents
Joshi, R., Bubna, Y., Sahana, M., & Shruthiba, A. (2024) · International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS) · doi:10.1109/csitss64042.2024.10816908 · Cited by 3
Leveraging Large Language Models to Detect npm Malicious Packages
Zahan, N., Burckhardt, P., Lysenko, M., Aboukhadijeh, F., & Williams, L. (2024) · arXiv · doi:10.48550/arxiv.2403.12196 · Cited by 3
Banitaan, S., Daoud, M., Alquran, H., & Akour, M. A. (2026) · Information · doi:10.3390/info17010073 · Cited by 8
GNPSum: A code summarization enhancement framework based on Graph Node Position
Cheng, H., Xu, L., Huangfu, L., Liu, C. C., Yan, M., & Lei, Y. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107837 · Cited by 3
VerLog: Enhancing Release Note Generation for Android Apps using Large Language Models
Guo, J., Yang, H., & Cai, H. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728961 · Cited by 8
Chen, J., Liu, J., Wu, J., Lin, D., Wu, J., & Zheng, Z. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3735971 · Cited by 6
“For instance, Ahmed et al. [8] use code search to generate static analysis information and employ LLMs for code summarization tasks.”
Leveraging meta-data of code for adapting prompt tuning for code summarization
Jiang, Z., Wang, D., & Rao, D. (2024) · Applied Intelligence · doi:10.1007/s10489-024-06197-0 · Cited by 3
Williams, D., Kechagia, M., Hort, M., Aleti, A., Petke, J., & Sarro, F. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3786582.3786827 · Cited by 11
Large Language Models in Software Documentation and Modeling: A Literature Review and Findings
Radoský, L., & Polášek, I. (2026) · IEEE World Symposium on Applied Machine Intelligence and Informatics (SAMI) · doi:10.1109/sami68106.2026.11420586 · Cited by 3
Programming Language Techniques for Bridging LLM Code Generation Semantic Gaps
Du, Y., Wang, C., & Wang, H. (2025) · ACM SIGPLAN International Workshop on Language Models and Programming Languages (LMPL) · doi:10.1145/3759425.3763383 · Cited by 5
Issue Retrieval and Verification Enhanced Supplementary Code Comment Generation
Zou, Y., Zhao, X., Pan, X., & Xie, B. (2025) · Asia-Pacific Symposium on Internetware · doi:10.1145/3755881.3755897 · Cited by 1
Guardrails for LLMs: A Comprehensive Review and Case Study on Bias Mitigation for Responsible AI
Park, J., & Mohaghegh, M. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-4575-9_12 · Cited by 1
Icing on the Cake: Automatic Code Summarization at Ericsson
Sridhara, G., Roychowdhury, S., Soman, S., Ranjani, H. G., & Britto, R. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00073 · Cited by 4
“Evaluation Metrics Computation: We use the metrics available in ASAP code [9] to evaluate performance of ASAP and our approaches.”
InfluentialInstructive Code Retriever: Learn from Large Language Model's Feedback for Code Intelligence Tasks
Lu, J., Wang, H., Liu, Z., Liang, K., Bao, L., & Yang, X. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3694997 · Cited by 6
FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
Dai, D., Liu, M., Li, A., Cao, J., Wang, Y., Wang, C., Peng, X., & Zheng, Z. (2025) · arXiv · doi:10.48550/arxiv.2504.06939 · Cited by 1
“In contrast, CoderEval contains more complex, repository-level tasks with broader context and domain-specific logic, exposing errors such as incorrect API usage or hallucinated method calls.”
TreeRanker: Fast and Model-Agnostic Ranking System for Code Suggestions in IDEs
Cipollone, D., Bogomolov, E., van Deursen, A., & Izadi, M. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00290 · Cited by 2
“This allows us to construct a fine-grained ranking signal without requiring beam search [10], [11] or prompt augmentation [12], [13].”
Leveraging Code Slicing for LLMs-Based Malicious NPM Package Detection
Nguyen, D.-K., Ho, G., Pham, Q. N., Anh, T. D.-T., Vu, M.-K., Nguyen, T. C., Tran-Truong, P. T., & Vu, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-35576-8_2
Using Mutation-Analysis to Examine an LLM's Ability to Summarize Code
Khatib, L., Pu, M., Vasilescu, B., & Nagappan, M. (2026) · ACM International Conference on AI-Powered Software (AIware) · doi:10.1145/3805760.3814892 · Cited by 2
Compiling Code LLMs into Lightweight Executables
Shi, J., He, J., Yang, Z., Yang, C., Klymenko, M. V., Hoang, T., Xu, X., Xing, Z., & Lo, D. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808196 · Cited by 1
Kumar, J., Tanguturi, M. R., Janapati, L. S., & Chimalakonda, S. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.113010 · Cited by 1
SmarTracker: Leveraging LLMs to Automate User Behavior Tracking in Mobile Apps
Wu, S., Zhou, H., Zhao, K., Yu, L., Bao, S., & Luo, X. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3701153
Luca, M. D., Santilli, T., Amalfitano, D., Fasolino, A. R., & Pelliccione, P. (2026) · IEEE International Conference on Software Architecture (ICSA) · doi:10.1109/icsa66085.2026.00026 · Cited by 3
“LLMs have been used for summarization and automated repair [9], [10], [31]–[33], with prompt-driven interaction enabling flexible, natural-language guidance [34].”
Contextual Learning Strategies for Dialogue Summarization Using Large Language Models
Penyasov, D. E., & Turalchuk, K. (2026) · Pattern Recognition and Image Analysis · doi:10.1134/s1054661826700252
“The selection of demonstration examples is a critical component of few-shot learning, directly affecting the effectiveness of LLM in completing problems [3, 7, 10].”
Abdul, A. M., Rochan, M., & Roy, C. K. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.112952 · Cited by 1
Kumar, R., & Pathak, H. K. (2026) · International Conference on Artificial Intelligence Systems (AIS) · doi:10.1109/ais69919.2026.11621306
“T. Ahmed et al. (2024) method outperforms the state-of-the-art in a number of contexts indicated by earlier research, including three distinct Large Language Models.”
Optimizing Knowledge Utilization for Multi-Intent Comment Generation With Large Language Models
Li, S., Wang, Z., Du, X., Wu, Z., Yu, J., & Chen, J. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3687241
“With the advent of LLMs, several studies enhanced comment generation through prompt augmentation or task-specific guidance [23], [56].”
Code vs Serialized AST Inputs for LLM-Based Code Summarization: An Empirical Study
Dong, S., Zhao, H., & Harvey, P. (2026) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1145/3786181.3788712 · Cited by 1
Tang, Y., Huang, Z., Xie, J., Yu, Y., Xia, B., Shen, E., & Cao, Y. (2026) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1145/3786181.3788706
SCLA: Automated Smart Contract Summarization via LLMs and Control Flow Prompt
Li, X., Mao, Y., Lu, Z., Li, W., & Li, Z. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3786582.3786812 · Cited by 30
“Ahmed et al. (Ahmed et al., 2024) demonstrated that LLMs struggle with implicit semantics, leading to the loss of critical information.”
Fantino, G., Vetrò, A., Torchiano, M., & Cappelluti, F. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3786582.3786826
Guidelines to Prompt Large Language Models for Code Generation: An Empirical Characterization
Midolo, A., Giagnorio, A., Zampetti, F., Tufano, R., Bavota, G., & Penta, M. D. (2026) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3794763.3794819 · Cited by 18
“Toufique et al. [6] enhance summarization by augmenting prompts with shallow semantic features, demonstrating that even lightweight program analysis improves LLM understanding.”
SkelDPO: A Skeleton-Guided Direct Preference Optimization Framework for Efficient Code Generation
Yu, Y., & Lyu, C. (2026) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3794763.3794824 · Cited by 1
SLMFL: Structural and semantic fusion for fault localization based on Small Language Models
Fan, X., Hu, J., Zheng, W., Liu, Y., & Gao, S. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108143 · Cited by 1
Context-Aware Code Summarization Using Multimodal Transformer
Saif, S., Yaseen, M., Khattak, U. F., & Rahman, G. (2026) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.15857 · Cited by 1
“Authors in [15] incorporated semantic information such as variable names and control-flow into prompts, which enhanced performance at the expense of increased computing time.”
A Systematic Literature Review of Parameter-Efficient Fine-Tuning for Large Code Models
Afrin, S., Haque, M. Z., & Mastropaolo, A. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3796522 · Cited by 19
Trace: Securing Smart Contract Repository Against Access Control Vulnerability
Chen, C., Bao, L., Lo, D., Wang, Y., Shan, Z., Chen, T., Yin, G., Yu, J., Zheng, Z., & Chen, J. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3660900 · Cited by 3
“Large language models (LLMs) have shown promise in tasks like code generation [13], summarization [14], and completion [15].”
Delving into History: Retrieve Less but Augment More on Commit Message Generation
Lan, P., Liu, J., Zhong, X., Gui, N., Bao, L., Lo, D., & Liao, Z. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3794857 · Cited by 2
ALIBIS: Assessing and mitigating the risk of sensitive metadata Leakage In moBile Image Sharing
Thanh, L. N. T., Carminati, B., & FERRARI, E. (2026) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2026.102171 · Cited by 1
Open-Source LLMs for Technical Q&A: Lessons from StackExchange
Babar, Z., Khan, N. I., Hassnain, M., & Filkov, V. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-08977-9_41 · Cited by 4
Hybrid Deep Learning-Based Summarization of Social Media Content Using NLP Techniques
Kumar, C. P., Praveen, V. V., Kaveya, M., Arthi, R., & Jegathesh, P. (2026) · Lecture notes in networks and systems · doi:10.1007/978-981-95-2872-1_20
Framework for Implementing LLMS for Software Quality
Ruiz, J. A., Martínez, M. Á. Q., Gomez-Rios, M.-D., & Arias, S. T. C. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-22641-9_20 · Cited by 1
GP and LLMs for Program Synthesis: No Clear Winners
Hernandez, J. G., Saini, A. K., Ketron, G., & Moore, J. H. (2026) · Genetic and evolutionary computation · doi:10.1007/978-981-95-6398-2_9 · Cited by 1
Tang, Y., Yu, Y., Huang, Z., Xia, B., & Cao, Y. (2025) · World Wide Web · doi:10.1007/s11280-025-01395-3 · Cited by 1
“However, most existing studies integrate LLMs only passively within code representation pipelines—for example, using them as auxiliary comment generators, code encoders, or prompt-driven adapters [ 10–12 ].”
LLMUpdater: Automatic comment synchronization via edit model guided LLMs
Yang, H., Xie, Q., Cai, S., Wang, Z., Kuang, L., & Xia, Y. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10752-9
“Ahmed et al. (2024) introduced the Automatic Semantic Prompt Augmentation(ASPA) method, which enhances semantics by acquiring repository name and paths, tagged identifiers, data flow graphs, and use cases.”
Developer Prompts in Practice: An Empirical Study of Bias, Security, and Optimization
Rzig, D. E., Paul, D. J., Pister, K. S., Henkel, J. M., & Hassan, F. (2025) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem64174.2025.00067 · Cited by 2
“Additionally, Ahmed et al. [76] worked on augmenting LLM prompts for code summarization by adding semantic facts of the code to enhance the prompts.”
Pombo, N. G. C. C., & Ouhbi, S. (2025) · International Conference on Computer, Software and Modeling (ICCSM) · doi:10.1109/iccsm66818.2025.00025 · Cited by 1
“Lastly, Ahmed et al. [4] demonstrated the application of LLMs in automatic semantic augmentation for code summarization, illustrating the models’ ability to improve comprehension and learning outcomes in software engineering.”
RAPRSUM: Retrieval-Augmented Code Summarization with Advanced Prototype Refinement
Lin, Y., Wang, X., Zhan, S., Wei, D., & Chen, B. (2025) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194025500317
Code Retrieval with Mixture of Experts Prototype Learning Based on Classification
Ling, F., Huang, G., Wang, J., Yuan, X., Chen, X., Zhang, X. W., Zhang, F., & Pun, C. (2025) · Asia-Pacific Symposium on Internetware · doi:10.1145/3755881.3755893
Billah, M. M., Rahman, M. S., & Roy, B. (2025) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3756681.3756977 · Cited by 2
Large Language Models as Robust Data Generators in Software Analytics: Are We There Yet?
Abdul, A. M., Rochan, M., & Roy, C. K. (2025) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3756681.3756985 · Cited by 7
“Additionally,wegeneratedandvalidated16KCodeSearchNet summariesusingASAP[3]andSIDE[27],establishingbenchmarks forLLM-generatedcodesummaries.”
InfluentialComCat : Expertise-Guided Context Generation to Enhance Code Comprehension
Grandel, S., Andersen, S. T., Huang, Y., & Leach, K. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3742475 · Cited by 3
“Similarly, Ahmed et al. [4] propose Automatic Semantic Augmentation of Prompts, which enhances code summarization by automatically incorporating semantic facts (such as function names and dataflow) into LLM prompts, thereby improving the accuracy and relevance of the generated summaries.”
Why Personalizing Deep Learning-Based Code Completion Tools Matters
Giagnorio, A., Martin-Lopez, A., & Bavota, G. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3725732 · Cited by 3
Prompt Engineering in Clinical Practice: A Comprehensive Guide for Clinicians (Preprint)Preprint
Liu, J., Wang, C., & Liu, S. (2025) · doi:10.2196/preprints.72644
Code Parameter Summarization Based on Transformer and Fusion Strategy
Zhang, F., Fan, J., Li, W., & Khoo, S. (2024) · IET Software · doi:10.1049/sfw2/3706673
Demonstration Attack against In-Context Learning for Code Intelligence
Ge, Y., Sun, W., Lou, Y., Fang, C., Zhang, Y., Li, Y., Zhang, X., Liu, Y., Zhao, Z., & Chen, Z. (2024) · arXiv · doi:10.48550/arxiv.2410.02841
“LLMs trained on large-scale code repositories, such as Star-Chat [42] and CodeLLama [34], have demonstrated exceptional performance in code intelligence tasks like code generation [8, 45], code summarization [1, 10, 38, 39] and code translation [22, 44].”
Method
Metadata
- Authors
- Ahmed, T., Pai, K. S., Devanbu, P., & Barr, E.
- Affiliations
- University of California, Davis (US)
- UC Davis, Davis, California, USA
- University College London (GB)
- Venue
- Proceedings of the IEEE/ACM 46th International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2024-04-12
- Pages
- 1-13
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W4394745253
- Semantic Scholar
- 0ff8c04c8bdbf93b39b49582c9195cf3fc894d03
- Cited by
- 112
- References
- 39
- Citations by year
- 2026: 33 · 2025: 57 · 2024: 22
- FWCI
- 42.65
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- NSF (National Science Foundation) (CCF (SHF-MEDIUM) No. 2107592)
- Intelligence Advanced Research Projects Agency (IARPA) (W911NF20C0038)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Software › Model-Driven Software Engineering Techniques
- Keywords
- semantic augmentation, prompt engineering, large language models, code summarization, program comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.