Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models
Open accessThe DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index; 3 of the 45 sources its claims cite were checked for support.
Abstract
Deep Learning (DL) systems have received exponential growth in popularity and have become ubiquitous in our everyday life. Such systems are built on top of popular DL libraries, e.g., TensorFlow and PyTorch which provide APIs as building blocks for DL systems. Detecting bugs in these DL libraries is critical for almost all downstream DL systems in ensuring effectiveness/safety for end users. Meanwhile, traditional fuzzing techniques can be hardly effective for such a challenging domain since the input DL programs need to satisfy both the input language (e.g., Python) syntax/semantics and the DL API input/shape constraints for tensor computations. To address these limitations, we propose TitanFuzz – the first approach to directly leveraging Large Language Models (LLMs) to generate input programs for fuzzing DL libraries. LLMs are titanic models trained on billions of code snippets and can autoregressively generate human-like code snippets. Our key insight is that modern LLMs can also include numerous code snippets invoking DL library APIs in their training corpora, and thus can implicitly learn both language syntax/semantics and intricate DL API constraints for valid DL program generation. More specifically, we use both generative and infilling LLMs (e.g., Codex/InCoder) to generate and mutate valid/diverse input DL programs for fuzzing. Our experimental results demonstrate that TitanFuzz can achieve 30.38%/50.84% higher code coverage than state-of-the-art fuzzers on TensorFlow/PyTorch. Furthermore, TitanFuzz is able to detect 65 bugs, with 44 already confirmed as previously unknown bugs. This paper demonstrates that modern titanic LLMs can be leveraged to directly perform both generation-based and mutation-based fuzzing studied for decades, while being fully automated, generalizable, and applicable to domains challenging for traditional approaches (such as DL systems). We hope TitanFuzz can stimulate more work in this promising direction of LLMs for fuzzing.
Overview
CoChat Summary
TL;DR
TitanFuzz is presented as the first approach to fuzz deep-learning libraries by directly using large language models to generate and mutate Python programs that call library APIs. It reports higher API and code coverage than prior fuzzers and finds dozens of previously unknown bugs in TensorFlow and PyTorch.
Key findings
- TitanFuzz covered 2215 TensorFlow APIs and 1329 PyTorch APIs, improving API coverage over the best baseline, DeepREL, by 91.11% and 24.09%, respectively.
- It reached 39.97% line coverage on TensorFlow and 20.98% on PyTorch, improving over DeepREL by 30.38% and 50.84%.
- It detected 65 bugs total, with 55 confirmed and 44 confirmed as previously unknown; 21 of the previously unknown bugs were already fixed.
- Ablations showed the full set of mutation operators and the fitness function combining dataflow depth, unique API calls, and repeated-call penalty performed best; Thompson sampling for operator selection outperformed random selection.
Methods
- TitanFuzz uses Codex as a generative LLM with a step-by-step prompt to create seed programs for each target API, then uses InCoder as an infilling LLM within an evolutionary fuzzing loop.
- Mutation operators include argument replacement, keyword insertion, prefix/suffix edits, and method replacement; operator choice is learned with a multi-armed bandit using Thompson sampling, and generated programs are ranked by a fitness function based on dataflow depth, unique API calls, and repeated API calls.
- Evaluation targets PyTorch v1.12 and TensorFlow v2.10, compares against API-level and model-level fuzzers, and uses differential testing across CPU and GPU backends to detect wrong-computation and crash bugs.
Limitations
- The paper states internal validity threats around implementation correctness and randomness: RQ1 was run once due to the large API set, while RQ2 used five runs on 100 sampled APIs; external validity is limited to PyTorch and TensorFlow.
- The paper acknowledges that differential testing can produce false positives when CPU/GPU differences are tolerated or implementation-defined, as shown by a rejected right-shift bug report.
- The text also makes evident that TitanFuzz has higher execution time than prior tools and depends on access to specific LLMs such as Codex and InCoder.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 1 Not valid · 21 Not checked · 88 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Also, our work shows for the first time that step-by-step prompt engineering [60] can substantially help boost fuzzing.”
§ Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models > 2.3 Testing using Deep Learning Models
What the sources say
- Supportedcites [60] matched to “Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm” (2021) source
The citation identifies step-by-step prompt engineering, not the citing authors’ own finding about fuzzing. Section 4.6 (p. 7) discusses prompts that lead GPT-3 through step-by-step procedures, and the Motivation (p. 2) describes prompts that make a model break a problem into components. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Due to the large amounts of available pre-training data, LLMs without any fine-tuning on specialized datasets can already be directly used for very specific downstream tasks.”
- Also stated in: § 2.2 Large Pre-trained Language Models · plain
“This is accomplished using prompt engineering [47], where a natural language description of the task together with a few demonstrations of the task is provided to the LLM first before the actual input.”
§ Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models > 2.2 Large Pre-trained Language Models
What the sources say
- Partly supportedcites [47] matched to “Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing” (2023) sourceExperimental — accuracy not yet measured
Section 6.1 (Training Settings; page unavailable) says a pretrained language model can be applied as-is to prompts specifying a downstream task, without training it for that task. This supports use without fine-tuning via prompting, but the passage does not establish the sentence’s causal explanation that this is due to the large amount of available pre-training data. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 2.2 Large Pre-trained Language Models · plain
None of the sources cited for this claim have been checked yet.
In the paper
“Compared to our coverage-guided baseline which only adds programs with new coverage to the seed bank for later mutation, our fitness function has minimal additional overhead.”
§ Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models > Additional information
What the sources say
- Not checked yetcites [18]
- Not checked yetcites [35]
- Could not verifycites [42] matched to “FairFuzz: a targeted mutation strategy for increasing greybox fuzz testing coverage” (2018) sourceExperimental — accuracy not yet measured
Our copy of FairFuzz never evaluates or claims anything about the overhead of its rare-branch scoring relative to a coverage-guided baseline: the only 'overhead' statements are generic background — fuzz testing's 'low computational overhead' versus dynamic symbolic execution (1 Introduction, p. 1) and AFL collecting 'some low-overhead program coverage information' (p. 1). The implementation/evaluation section (4, p. 5) reports only that FairFuzz adds ~600 lines of C to AFL and evaluates coverage, proportion of inputs hitting the target branch, and crashes — no executions-per-second or runtime-cost comparison against AFL's baseline. Searches on 'overhead', 'fitness function overhead', and 'throughput / executions per second / cost of rare branch counting' returned no passage on this point, so the claim can be neither confirmed nor contradicted. (LLM, conf=0.68)
- Not checked yetcites [50]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Figure 9c shows a bug when a certain output index is left unspecified for the API ParallelDynamicStitch.”
§ Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models > 5.2.1 Seed Generation.
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In practice, this bug is missed since this API is not covered by any of the previous techniques.”
§ Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models > 5.2.1 Seed Generation.
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
47 more claims
Create a free account to see all 112 claims and to verify the 42 references still open.
- It may state two separate claims as one. 51
- The quote only partly states this claim. 4
- Probably not a substantial claim. 15
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In addition to software issue resolution tasks, LIVE-SWE-AGENT can be easily applied to other challenging software engineering tasks like generating tests [16], detecting and patching vulnerabilities [21], and synthesizing production-ready software from scratch [48].”
From Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly? (2025) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models > Additional information
“| TitanFuzz | 6969 | 18245 | 17411 | 17957 | 5173 | 16865 | 84447 | 86536 |”
§ Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models > Additional information
“| D+U-R | 6960 | 18245 | 17411 | 17957 | 5173 | 16865 | 84447 | 86536 |”
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“Due to the recent advances in Deep Learning (DL), researchers have looked into using DL models to facilitate automated test generation or fuzzing of different software systems.”
BackgroundMethodLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 3×
“Learn&Fuzz [23], DeepSmith [15] and DeepFuzz [47] each trains a RNN to generate programs for PDF file parsers, OpenCL and C, respectively.”
MethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 7×
“Codex can be used to perform auto-completion where the input is simply a description of the task (known as a prompt [10, 43]).”
BackgroundMethodInfluentialAdvances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · cited 1×
“These LLMs are pre-trained on billions of available text on the internet and have been widely used in many different Natural Language Processing (NLP) tasks [10, 48, 79].”
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · cited 3×
“in Figure 5) to improve the model’s performance following [2, 47, 53, 73].”
BackgroundMethodInfluentialLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · cited 3×
“in Figure 5) to improve the model’s performance following [2, 47, 53, 73].”
BackgroundMethodInfluentialChain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · cited 1×
“Codex can be used to perform auto-completion where the input is simply a description of the task (known as a prompt [10, 43]).”
Method
Research works printed but not linked: 13
- Andrea Fioraldi , Dominik Maier , Heiko Eiß feldt, and Marc Heuse . 2020 . AFL++: Combining Incremental Steps of Fuzzing Research . In Proceedings of the 14th USENIX Conference on Offensive Technologies (WOOT’20) . USENIX Association, USA. Article 10, 1 pages. Andrea Fioraldi, Dominik Maier, Heiko Eiß feldt, and Marc Heuse. 2020. AFL++: Combining Incremental Steps of Fuzzing Research. In Proceedings of the 14th USENIX Conference on Offensive Technologies (WOOT’20). USENIX Association, USA. Article 10, 1 pages.
- Sepp Hochreiter and Jürgen Schmidhuber . 1997. Long Short-term Memory. Neural computation, 9 ( 1997 ), 12, 1735–80. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-term Memory. Neural computation, 9 (1997), 12, 1735–80.
- Suyoung Lee , HyungSeok Han , Sang Kil Cha , and Sooel Son . 2020 . Montage: A Neural Network Language $Model-Guided$$JavaScript$ Engine Fuzzer . In 29th USENIX Security Symposium (USENIX Security 20) . 2613–2630. Suyoung Lee, HyungSeok Han, Sang Kil Cha, and Sooel Son. 2020. Montage: A Neural Network Language $Model-Guided$$JavaScript$ Engine Fuzzer. In 29th USENIX Security Symposium (USENIX Security 20). 2613–2630.
- Jiawei Liu Jinkun Lin Fabian Ruffy Cheng Tan Jinyang Li Aurojit Panda and Lingming Zhang. 2023. NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers. In ASPLOS. 530–543. Jiawei Liu Jinkun Lin Fabian Ruffy Cheng Tan Jinyang Li Aurojit Panda and Lingming Zhang. 2023. NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers. In ASPLOS. 530–543.
- Xiao Liu , Xiaoting Li , Rupesh Prajapati , and Dinghao Wu . 2019 . Deepfuzz: Automatic generation of syntax valid c programs for fuzz testing . In Proceedings of the AAAI Conference on Artificial Intelligence. 33 , 1044–1051. Xiao Liu, Xiaoting Li, Rupesh Prajapati, and Dinghao Wu. 2019. Deepfuzz: Automatic generation of syntax valid c programs for fuzz testing. In Proceedings of the AAAI Conference on Artificial Intelligence. 33, 1044–1051.
- Zohar Manna and Richard J . Waldinger . 1971 . Toward Automatic Program Synthesis. Commun. ACM , 14, 3 (1971), mar, 151–165. Zohar Manna and Richard J. Waldinger. 1971. Toward Automatic Program Synthesis. Commun. ACM, 14, 3 (1971), mar, 151–165.
- Pengyu Nie , Rahul Banerjee , Junyi Jessy Li , Raymond J. Mooney , and Milos Gligoric . 2023 . Learning Deep Semantics for Test Completion. In 45th International Conference on Software Engineering. Pengyu Nie, Rahul Banerjee, Junyi Jessy Li, Raymond J. Mooney, and Milos Gligoric. 2023. Learning Deep Semantics for Test Completion. In 45th International Conference on Software Engineering.
- Jiwon Park , Dominik Winterer , Chengyu Zhang , and Zhendong Su . 2021 . Generative type-aware mutation for testing SMT solvers . Proceedings of the ACM on Programming Languages, 5, OOPSLA (2021), 1–19. Jiwon Park, Dominik Winterer, Chengyu Zhang, and Zhendong Su. 2021. Generative type-aware mutation for testing SMT solvers. Proceedings of the ACM on Programming Languages, 5, OOPSLA (2021), 1–19.
- Alec Radford , Jeffrey Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . 2019. Language models are unsupervised multitask learners. OpenAI blog, 1, 8 ( 2019 ), 9. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1, 8 (2019), 9.
- Colin Raffel , Noam Shazeer , Adam Roberts , Katherine Lee , Sharan Narang , Michael Matena , Yanqi Zhou , Wei Li , and Peter J . Liu . 2020 . Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. J. Mach. Learn. Res ., jan. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. J. Mach. Learn. Res., jan.
- Daniel J Russo , Benjamin Van Roy , Abbas Kazerouni, Ian Osband, and Zheng Wen. 2018 . A tutorial on thompson sampling. Foundations and Trends® in Machine Learning , 11, 1 (2018), 1–96. Daniel J Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, and Zheng Wen. 2018. A tutorial on thompson sampling. Foundations and Trends® in Machine Learning, 11, 1 (2018), 1–96.
- Jiannan Wang , Thibaud Lutellier , Shangshu Qian , Hung Viet Pham, and Lin Tan . 2022 . EAGLE : Creating Equivalent Graphs to Test Deep Learning Libraries . Jiannan Wang, Thibaud Lutellier, Shangshu Qian, Hung Viet Pham, and Lin Tan. 2022. EAGLE: Creating Equivalent Graphs to Test Deep Learning Libraries.
- Chenyuan Yang , Yinlin Deng , Jiayi Yao , Yuxing Tu , Hanchi Li , and Lingming Zhang . 2023 . Fuzzing Automatic Differentiation in Deep-Learning Libraries. In International Conference on Software Engineering (ICSE). to appear Chenyuan Yang, Yinlin Deng, Jiayi Yao, Yuxing Tu, Hanchi Li, and Lingming Zhang. 2023. Fuzzing Automatic Differentiation in Deep-Learning Libraries. In International Conference on Software Engineering (ICSE). to appear
Cited by
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650
“Deng et al. [104] propose TitanFuzz, which uses LLMs (i.e., Codex) to generate valid input DL programs to test DL libraries.”
MethodSoftware Testing With Large Language Models: Survey, Landscape, and Vision
Wang, J., Huang, Y., Chen, C., Liu, Z., Wang, S., & Wang, Q. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3368208 · Cited by 573
“Another example is for generating unusual programs for fuzzing deep learning libraries, Deng et al. [58] first generate a possible “bug” (bug description) before generating the actual “bug-triggering” code snippet that invokes the target API.”
InfluentialLarge Language Models for Software Engineering: Survey and Open Problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · arXiv · doi:10.48550/arxiv.2310.03533 · Cited by 523
“Deng et al. [104] propose TitanFuzz, which uses LLMs (i.e., Codex) to generate valid input DL programs to test DL libraries.”
MethodAn Empirical Study of the Non-Determinism of ChatGPT in Code Generation
Ouyang, S., Zhang, J. M., Harman, M., & Wang, M. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3697010 · Cited by 258
Metadata
- Authors
- Deng, Y., Xia, C. S., Peng, H., Yang, C., & Zhang, L.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- University of Science and Technology of China (CN)
- Venue
- Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis
- Publisher
- ACM
- Published
- 2023-07-12
- Pages
- 423-435
- OpenAlex
- W4378591002
- Semantic Scholar
- 66476832701361c9f9b2a7eb2354ee8cd9f72e67
- Cited by
- 326
- References
- 39
- Citations by year
- 2026: 93 · 2025: 137 · 2024: 87 · 2023: 9
- FWCI
- 63.11
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (CCF-2131943,CCF-2141474)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- large language models, fuzzing, deep learning libraries, TensorFlow, PyTorch, bug detection, code generation, code mutation, input generation, code coverage
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models. https://app.cochat.ai/papers/10.1145/3597926.3598067
- The paper
- Deng, Y., Xia, C. S., Peng, H., Yang, C., & Zhang, L. (2023). Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models. Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. https://doi.org/10.1145/3597926.3598067
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.