FairFuzz: a targeted mutation strategy for increasing greybox fuzz testing coverage
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
In recent years, fuzz testing has proven itself to be one of the most effective techniques for finding correctness bugs and security vulnerabilities in practice. One particular fuzz testing tool, American Fuzzy Lop (AFL), has become popular thanks to its ease-of-use and bug-finding power. However, AFL remains limited in the bugs it can find since it simply does not cover large regions of code. If it does not cover parts of the code, it will not find bugs there. We propose a two-pronged approach to increase the coverage achieved by AFL. First, the approach automatically identifies branches exercised by few AFL-produced inputs (rare branches), which often guard code that is empirically hard to cover by naively mutating inputs. The second part of the approach is a novel mutation mask creation algorithm, which allows mutations to be biased towards producing inputs hitting a given rare branch. This mask is dynamically computed during fuzz testing and can be adapted to other testing targets. We implement this approach on top of AFL in a tool named FairFuzz. We conduct evaluation on real-world programs against state-of-the-art versions of AFL. We find that on these programs FairFuzz achieves high branch coverage at a faster rate that state-of-the-art versions of AFL. In addition, on programs with nested conditional structure, it achieves sustained increases in branch coverage after 24 hours (average 10.6% increase). In qualitative analysis, we find that FairFuzz has an increased capacity to automatically discover keywords.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Compared to our coverage-guided baseline which only adds programs with new coverage to the seed bank for later mutation, our fitness function has minimal additional overhead.”
From Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models (2023) · cites this paper as [42]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Symbolic execution and program testing
King, J. C. (1976) · Communications of the ACM · doi:10.1145/360248.360252 · Cited by 3,031 · cited 1×
“While many of the individual test inputs it generates may be garbage, due to its low computational overhead, CGF generates test inputs much faster than more sophisticated methods such as symbolic execution [17, 34] and dynamic symbolic execution (a.”
MethodCUTE: a concolic unit testing engine for C
Sen, K., Marinov, D., & Agha, G. (2005) · Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1081706.1081750 · Cited by 1,651 · cited 2×
“concolic testing) techniques [5, 7, 13, 16, 21, 22, 36, 45, 47].”
MethodSelf-citationDART: directed automated random testing
Godefroid, P., Klarlund, N., & Sen, K. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065036 · Cited by 1,570 · cited 3×
“concolic testing) techniques [5, 7, 13, 16, 21, 22, 36, 45, 47].”
BackgroundMethodSelf-citationEvoSuite: automatic test suite generation for object-oriented software
Fraser, G., & Arcuri, A. (2011) · Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering · doi:10.1145/2025113.2025179 · Cited by 1,110 · cited 1×
“Evosuite [19] uses seed inputs and genetic algorithms to achieve high code coverage.”
MethodAutomated software test data generation
Korel, B. (1990) · IEEE Transactions on Software Engineering · doi:10.1109/32.57624 · Cited by 971 · cited 1×
“Search-based software testing (SBST) [19, 20, 22–24, 29, 31, 32, 42] uses optimization techniques such as hill climbing and genetic algorithms to generate inputs that optimize some observable fitness function.”
MethodDriller: Augmenting Fuzzing Through Selective Symbolic Execution
Stephens, N., Grosen, J., Salls, C., Dutcher, A., Wang, R., Corbetta, J., Shoshitaishvili, Y., Kruegel, C., & Vigna, G. (2016) · Proceedings 2016 Network and Distributed System Security Symposium · doi:10.14722/ndss.2016.23368 · Cited by 887
Finding and understanding bugs in C compilers
Yang, X., Chen, Y., Eide, E., & Regehr, J. (2011) · Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1993498.1993532 · Cited by 825 · cited 3×
“In practice, this trade-off has paid off, and fuzz testers have found numerous correctness bugs and security vulnerabilities in widely used software [9, 14, 25, 26, 28, 34, 39, 41, 43].”
BackgroundMethodBöhme, M., Pham, V.-T., Nguyen, M.-D., & Roychoudhury, A. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134020 · Cited by 747 · cited 3×
“The success of one particular fuzz tester, American Fuzzy Lop (or simply AFL) [43], has gained attention both in practice and in the research community [8, 9, 30, 39].”
BackgroundMethodResultInfluentialCoverage-based Greybox Fuzzing as Markov Chain
Böhme, M., Pham, V.-T., & Roychoudhury, A. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978428 · Cited by 649
The Current State and Future of Search Based Software Engineering
Harman, M. (2007) · Future of Software Engineering (FOSE '07) · doi:10.1109/fose.2007.29 · Cited by 631 · cited 1×
“Search-based software testing (SBST) [24, 25, 27–29, 35, 38, 39, 50] uses optimization techniques such as hill climbing and genetic algorithms to generate inputs that optimize some observable fitness function.”
Method
Research works printed but not linked: 18
- 10.5555/1763507.1763523 · doi:10.5555/1763507.1763523
- [7] Marcel Böhme. 2016. AFLFast.new. https://groups.google.com/d/msg/afl-users/ 1PmKJC-EKZ0/lbzRb8AuAAAJ. (2016). Accessed August 23rd, 2017. -
- [11] Cristian Cadar, Daniel Dunbar, and Dawson Engler. 2008. KLEE: Unassisted and Automatic Generation of High-coverage Tests for Complex Systems Programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI'08) . -
- [13] Chris Evans, Matt Moore, and Tavis Ormandy. 2011. Fuzzing at Scale. https:// security.googleblog.com/2011/08/fuzzing-at-scale.html. (2011). Accessed August 24th, 2017. -
- [19] Mark Grechanik, Chen Fu, and Qing Xie. 2012. Automatically finding performance problems with feedback-directed learning software testing. In 2012 34th International Conference on Software Engineering (ICSE) . IEEE, 156-166. - · doi:10.5555/2337223.2337242
- [21] Istvan Haller, Asia Slowinska, Matthias Neugschwandtner, and Herbert Bos. 2013. Dowsing for Overflows: A Guided Fuzzer to Find Buffer Boundary Violations. In Proceedings of the 22Nd USENIX Conference on Security (SEC'13) . -
- [23] Mark Harman and John Clark. 2004. Metrics are fitness functions too. In Software Metrics, 2004. Proceedings. 10th International Symposium on . IEEE, 58-69. - · doi:10.5555/1018439.1021893
- [24] Mark Harman and Bryan F Jones. 2001. Search-based software engineering. Information and software Technology 43, 14 (2001), 833-839. -
- [25] Sam Hocevar. 2007. zzuf. http://caca.zoy.org/wiki/zzuf/. (2007). Accessed August 22nd, 2017. -
- [26] Christian Holler, Kim Herzig, and Andreas Zeller. 2012. Fuzzing with Code Fragments. In Presented as part of the 21st USENIX Security Symposium (USENIX Security 12) . -
- [28] Allen D. Householder and Jonathan M. Foote. 2012. Probability-Based Parameter Selection for Black-Box Fuzz Testing . Technical Report. Carnegie Mellon University Software Engineering Institute. -
- Guodong Li , Indradeep Ghosh , and Sreeranga P . Rajan . 2011 . KLOVER : A Symbolic Execution and Automatic Test Generation Tool for C++ Programs. In CAV. 609–615. Guodong Li, Indradeep Ghosh, and Sreeranga P. Rajan. 2011. KLOVER: A Symbolic Execution and Automatic Test Generation Tool for C++ Programs. In CAV. 609–615.
- 10.5555/1792786.1792798 · doi:10.5555/1792786.1792798
- [43] Michał Zalewski. 2014. American Fuzzy Lop. http://lcamtuf.coredump.cx/afl. (2014). Accessed August 18th, 2017. -
- [44] Michał Zalewski. 2016. FidgetyAFL. https://groups.google.com/d/msg/afl-users/ fOPeb62FZUg/CES5lhznDgAJ. (2016). Accessed August 23rd, 2017. -
- [45] Michał Zalewski. 2016. Unique crashes as a metric. https://groups.google.com/ d/msg/afl-users/fOPeb62FZUg/LYxgPYheDwAJ. (2016). Accessed August 24th, 2017. -
- [46] Michał Zalewski. 2017. American Fuzzy Lop Technical Details. http://lcamtuf. coredump.cx/afl/technical\_details.txt. (2017). Accessed August 18th, 2017. -
- [47] Andreas Zeller and Ralf Hildebrandt. 2002. Simplifying and Isolating FailureInducing Input. IEEE Transactions on Software Engineering 28, 2 (2002), 183-200.
Cited by
Items 201–300 of 477
Deep Learning for Coverage-Guided Fuzzing: How Far are We?
Li, S., Xie, X., Lin, Y. H., Li, Y., Feng, R., Li, X., Ge, W., & Dong, J. S. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3200525 · Cited by 8
“We select 4 traditional fuzzers: AFL [1], Fairfuzz [43], Angora [22] and Vuzzer [30].”
BackgroundMethodInfluentialCombining BMC and Fuzzing Techniques for Finding Software Vulnerabilities in Concurrent Programs
Aljaafari, F., Menezes, R., Manino, E., Shmarov, F., Mustafa, M., & Cordeiro, L. C. (2022) · IEEE Access · doi:10.1109/access.2022.3223359 · Cited by 8
“Algorithm 1 [23], [41] shows the standard workflow of a gray-box fuzzer.”
BackgroundSquill: Testing DBMS with Correctness Feedback and Accurate Instantiation
Wen, S., Jia, P., Yang, P., & Hu, C. (2023) · Applied Sciences · doi:10.3390/app13042519 · Cited by 8
“Some studies [15–17] have shown that information, such as the rareness of branches, the number of memory reads or writes, and the number of branches that seed changed can guide fuzzer to perform better.”
BackgroundFA-Fuzz: A Novel Scheduling Scheme Using Firefly Algorithm for Mutation-Based Fuzzing
Gao, Z., Xiong, H., Dong, W., Chang, R., Yang, R. G., Zhou, Y., & Jiang, L. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3326144 · Cited by 10
“AFLFast [19] and Fairfuzz [20] allocate more energy to the seeds that trigger low-frequency branches and rare branches with the intention of guiding the direction of fuzzing.”
BackgroundResultEnhancing Coverage-Guided Fuzzing via Phantom Program
Wu, M., Chen, K., Luo, Q., Xiang, J., Qi, J., Chen, J., Cui, H., & Zhang, Y. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616294 · Cited by 15
Energy Distribution Matters in Greybox Fuzzing
Situ, L., Wang, L., Li, X., Guan, L., Zhang, W., & Liu, P. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion.2019.00109 · Cited by 8
BackREST: A Model-Based Feedback-Driven Greybox Fuzzer for Web Applications
Gauthier, F., Hassanshahi, B., Selwyn-Smith, B., Mai, T. N., Schlüter, M., & Williams, M. (2021) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.ecoop.2022.29 · Cited by 19
“Nowadays, most greybox fuzzers use coverage to guide their input generation engine towards producing input that will exercise newly covered code, or branches that will likely lead to new code [66, 65, 36, 80, 64, 26, 25].”
Diver: Oracle-Guided SMT Solver Testing with Unrestricted Random Mutations
Kim, J., So, S., & Oh, H. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00187 · Cited by 9
“F UZZ [49] and VUzzer [50] are designed to achieve high code coverage, and TaintScope [51] is designed to improve security vulnerability-finding ability.”
BackgroundPosFuzz: augmenting greybox fuzzing with effective position distribution
Zou, Y., Zou, W., Zhao, J., Zhong, N., Zhang, Y., Shi, J., & Huo, W. (2023) · Cybersecurity · doi:10.1186/s42400-023-00143-2 · Cited by 9
“This line of research (Rawat et al. 2017; Lemieux and Sen 2018; Chen and Chen 2018; Li et al. 2017; Aschermann et al. 2019; You et al. 2019; Lyu et al. 2019; Wang et al. 2017; Rajpal et al. 2017) assigns mutation operators with different weights by mining the semantics of input bytes.”
BackgroundInfluentialDodrio: Parallelizing Taint Analysis Based Fuzzing via Redundancy-Free Scheduling
Liang, J., Wang, M., Zhou, C., Wu, Z., Liu, J., & Jiang, Y. (2024) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3663529.3663844 · Cited by 8
Multiple Targets Directed Greybox Fuzzing: From Reachable to Exploited
Yu, X., Liang, H., & Wang, C. (2024) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner60148.2024.00099 · Cited by 7
“…basic blocks “a” and “d” are TDBBs, so the execution distances for seeds A and B are calculated as B. Dynamic analysis 1) Branch-sensitive mutation: Inspired by FairFuzz [13], SAFuzz uses lightweight data flow analysis to determine the effect of each byte of seed input on solving path constraints.”
BackgroundMethodScalable Dynamic Analysis of Binary Code
Kargén, U. (2019) · Linköping studies in science and technology. Dissertations · doi:10.3384/diss.diva-157626 · Cited by 8
“Lemieux and Sen [74] use a similar method to approximately determine which parts of an input cannot be mutated in order for a particular branch instruction to be exercised.”
MethodCSEFuzz: Fuzz Testing Based on Symbolic Execution
Xie, Z., Cui, Z., Zhang, J., Liu, X., & Zheng, L. (2020) · IEEE Access · doi:10.1109/access.2020.3030798 · Cited by 7
“In 2018, Caroline Lemieux and Koushik Sen implemented FairFuzz [23] based on AFL.”
MethodMaxAFL: Maximizing Code Coverage with a Gradient-Based Optimization Technique
Kim, Y., & Yoon, J. (2020) · Electronics · doi:10.3390/electronics10010011 · Cited by 7
“We compared MaxAFL with the original AFL [2], AFLFast [3] and FairFuzz [4] on six Linux binaries.”
MethodInfluentialSaieva, A., & Kaiser, G. E. (2022) · Journal of Systems and Software · doi:10.1016/j.jss.2022.111381 · Cited by 7
Yan, Q., Cao, H., Lu, S., & Huang, M. (2023) · International Fuzzing Workshop (FUZZING) · doi:10.1145/3605157.3605847 · Cited by 7
“Various improvement methods have been proposed to enhance the effectiveness of fuzzing, including obtaining information from historical executions [6, 14, 24], using program analysis techniques [9, 18, 21], combining machine learning techniques [10, 19], and learning from human input [20].”
MethodDiPri : Distance-Based Seed Prioritization for Greybox Fuzzing
Qian, R., Zhang, Q., Fang, C., Yang, D., Li, S., Li, B., & Chen, Z. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3654440 · Cited by 12
“Among the 24 evaluated targets, 8 of which are from real-world projects [39, 79], 8 are from the coverage-based benchmark FuzzBench [14, 49], and the remaining 8 are from the bug-based benchmark Magma [28, 30].”
InfluentialDetection of Intrusions and Malware, and Vulnerability Assessment
Nikolić, I., Mantu, R., Shen, S., & Saxena, P. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-80825-9 · Cited by 7
OTA: An Operation-oriented Time Allocation Strategy for Greybox Fuzzing
Li, X., Sun, L., Jiang, R., Qu, H., & Yan, Z. (2021) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner50967.2021.00019 · Cited by 7
“FairFuzz [20] avoids mutating crucial regions that satisfy the complex conditions of rare branches.”
BackgroundInfluentialEvolutionary computation and machine learning in cryptology
Picek, S., & Jakobović, D. (2021) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3449726.3461420 · Cited by 8
BigMap: Future-proofing Fuzzers with Efficient Large Maps
Ahmed, A., Hiser, J. D., Nguyen‐Tuong, A., Davidson, J. W., & Kevin Skadron, Y. (2021) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn48987.2021.00062 · Cited by 6
A Survey of Human-machine Collaboration in Fuzzing
Yan, Q., Huang, M., & Cao, H. (2022) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc55868.2022.00058 · Cited by 6
Uncovering Bugs in Code Coverage Profilers via Control Flow Constraint Solving
Wang, Y., Zhang, P., Sun, M., Lu, Z., Yang, Y., Tang, Y., Qian, J., Li, Z., & Zhou, Y. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3321381 · Cited by 6
“In recent decades, many studies have used code coverage to guide ef fi cient and effective testing (e.g., fuzzing testing [2], [3], [4], [5], compiler testing [6], [7], [8], mutant testing [9], [10], regression testing [11], [12], and test case generation [13], [14], [15]) and debugging (e.g., fault…”
BackgroundSyntax-Aware Mutation for Testing the Solidity Compiler
Mitropoulos, C., Sotiropoulos, T., Ioannidis, S., & Mitropoulos, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-51479-1_17 · Cited by 6
“AFLGo [12] and power schedules able to direct the fuzzing process locations of a programs ( directed fuzzing), based on [13] and fair-fuzz [32], include a scheduling algorithm rarely-exercised branches to achieve higher coverage. fuzzol prioritization algorithm that is able to identify mutations…”
BackgroundFuzzInMem: Fuzzing Programs via In-memory Structures
Liu, X., You, W., Ye, Y., Zhang, Z., Huang, J., & Zhang, X. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639172 · Cited by 6
sqlFuzz: Directed Fuzzing for SQL Injection Vulnerability
Yuan, Y., Lu, Y., Zhu, K.-L., Huang, H., Chen, Y., & Zhang, Y. (2024) · Electronics · doi:10.3390/electronics13152946 · Cited by 6
State Machine Mutation-based Testing Framework for Wireless Communication Protocols
Rashid, S. M. M., Wu, T.-H., Tu, K., Ishtiaq, A. A., Tanvir, R. H., Dong, Y., Chowdhury, O., & Hussain, S. R. (2024) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3658644.3690312 · Cited by 6
SHFuzz: A hybrid fuzzing method assisted by static analysis for binary programs
Wang, W., Tian, D., Ma, R., Wei, H., Ying, Q., Jia, X., & Zuo, L. (2021) · China Communications · doi:10.23919/jcc.2021.08.001 · Cited by 6
Boosting Grey-box Fuzzing for Connected Autonomous Vehicle Systems
Moukahal, L. J., Zulkernine, M., & Soukup, M. (2021) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c55045.2021.00080 · Cited by 9
AMSFuzz: An adaptive mutation schedule for fuzzing
Zhao, X., Qu, H., Xu, J., Li, S., & Wang, G. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118162 · Cited by 11
Evolutionary computation and machine learning in security
Picek, S., & Jakobović, D. (2022) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3520304.3534087 · Cited by 6
SLOPT: Bandit Optimization Framework for Mutation-Based Fuzzing
Koike, Y., Katsura, H., Yakura, H., & Kurogome, Y. (2022) · Annual Computer Security Applications Conference · doi:10.1145/3564625.3564659 · Cited by 6
“puts will trigger rare execution paths [37].”
Evaluating seed selection for fuzzing JavaScript engines
Wen, M., Wang, Y., Xia, Y., & Jin, H. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10340-9 · Cited by 14
“Specifically, out of the nine fuzzers as shown in Table 1, four (i.e., CodeAlChemist, Montage, COMFORT and SoFi) utilize Test262 Official Ecmascript Conformance Test Suite (1997) as the seeds; four (i.”
MethodInfluentialDSFuzz: Detecting Deep State Bugs with Dependent State Exploration
Liu, Y., & Meng, W. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3616594 · Cited by 6
“…cases, existing approaches either fail to distinguish these different data chunks [12, 38, 42], or need to mutate all the possible value combinations of these data chunks [8, 17, 19, 26–28 Third, the dependencies need to be satisfied by mutating different critical input bytes in the correct order.”
BackgroundLAFuzz: Neural Network for Efficient Fuzzing
Wang, X., Hu, C., Ma, R., Li, B., & Wang, X. (2020) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai50040.2020.00098 · Cited by 6
“A significant number of approaches to improve the performance of AFL has been proposed over time, such as AFLFast [10], FairFuzz [6], Vuzzer [11], and QSYM [12].”
BackgroundMethodMooFuzz: Many-Objective Optimization Seed Schedule for Fuzzer
Zhao, X., Qu, H., Lv, W., Li, S., & Xu, J. (2021) · Mathematics · doi:10.3390/math9030205 · Cited by 6
“FairFuzz [13] identifies rare branches in the program and adjusts mutation strategies to increase coverage.”
BackgroundMethodInfluentialDynamic Verification of Approximate Computing Circuits using Coverage-based Grey-box Fuzzing
Yoshisue, K., Masuda, Y., & Ishihara, T. (2021) · IEEE International On-Line Testing Symposium (IOLTS) · doi:10.1109/iolts52814.2021.9486690 · Cited by 8
Compositional Fuzzing Aided by Targeted Symbolic ExecutionPreprint
Ognawala, S., Kilger, F., & Pretschner, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.02981 · Cited by 6
A survey of coverage-guided greybox fuzzing with deep neural models
Qiu, J., Jiang, Y., Miao, Y., Luo, W., Pan, L., & Zheng, X. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107797 · Cited by 8
Go-Sanitizer: Bug-Oriented Assertion Generation for Golang
Wang, C., Sun, H., Xu, Y., Jiang, Y., Zhang, H., & Gu, M. (2019) · IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) · doi:10.1109/issrew.2019.00039 · Cited by 8
“zip performance, researchers have proposed several optimizations based on symbolic execution and taint analysis [9]–[11].”
Mutation-Based Integration Testing of Knowledge Graph Applications
John, T., Johnsen, E. B., & Kamburjan, E. (2024) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre62328.2024.00052 · Cited by 6
“Lemieux and Sen use a branch mask to control where an input byte sequence may be mutated to increase branch coverage of the test suite [23].”
BackgroundS nowplow : Effective Kernel Fuzzing with a Learned White-box Test Mutator
Gong, S., Rui, W., Altınbüken, D., Fonseca, P., & Maniatis, P. (2025) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3676641.3716019 · Cited by 5
“To automatically generate new kernel tests, modern fuzzers rely heavily on mutation techniques [8, 34, 46] that create new tests by progressively applying minor changes—such as new system call invocations and new argument values—to a base test program.”
InsFuzz: Fuzzing Binaries With Location Sensitivity
Zhang, H., Zhou, A., Jia, P., Liu, L., Ma, J., & Liu, L. (2019) · IEEE Access · doi:10.1109/access.2019.2894178 · Cited by 5
“Some other light-weight fuzzers, such as AFLFast [38], Steelix [31] and FairFuzz [28] are proposed to improve the code coverage 22434 2169-3536 2019 IEEE.”
BackgroundInfluentialImproving Fuzzing through Controlled Compilation
Simon, L., & Verma, A. (2020) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp48549.2020.00011 · Cited by 5
“Note that AFL’s default (deterministic) mode is usually less effective than the non-deterministic mode ( -d option) before the 24 -hour mark [23], so it is important to run AFL for at least 24 hours.”
BackgroundGuide Me to Exploit: Assisted ROP Exploit Generation for ActionScript Virtual Machine
Yılmaz, F., Sridhar, M., & Choi, W. (2020) · Annual Computer Security Applications Conference · doi:10.1145/3427228.3427568 · Cited by 5
“libFuzzer [97], AFL [96] and its extensions [9, 12, 13, 27, 55, 56, 74, 75, 92] constitute the most widely-used implementations of CGF.”
MethodVulnerability-oriented directed fuzzing for binary programs
Yu, L., Lu, Y., Shen, Y., Li, Y., & Pan, Z. (2022) · Scientific Reports · doi:10.1038/s41598-022-07355-5 · Cited by 6
Better Pay Attention Whilst Fuzzing
Zhu, S., Wang, J., Sun, J., Yang, J., Lin, X., Wang, T., Zhang, L., & Cheng, P. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3338129 · Cited by 8
“Fairfuzz [20] locates and selects the seeds which trigger low-probability edges.”
BackgroundNo-Fuzz: Efficient Anti-fuzzing Techniques
Zhou, Z., Wang, C., & Zhao, Q. (2023) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-25538-0_38 · Cited by 5
“Besides, researchers also modify the fuzzing mechanisms of some classic works [36,3], to meticulously reallocate the fuzzing resources on some specific tasks (e.g., directly fuzzing a particular code area [7,8,10,23]).”
BackgroundTokenAuditor: Detecting Manipulation Risk in Token Smart Contract by Fuzzing
Cao, M., Zhang, Y., Feng, Z., Hu, J., & Zhu, Y. (2022) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs57517.2022.00071 · Cited by 5
“It has been inherited and developed by CollAFL [32], FairFuzz [33], and other works.”
BackgroundMethodExploring Effective Fuzzing Strategies to Analyze Communication Protocols
Chen, Y., Mei, Y., Lan, T., & Venkataramani, G. (2023) · Digital Threats Research and Practice · doi:10.1145/3526088 · Cited by 5
“Coverage guided fuzzing: Plenty of works focus on smarter testcase mutation/selection or search heuristics, to help the fuzzer generate inputs that explore more/rare/buggy execution paths [15, 22, 26, 35, 38, 39, 43, 48].”
BackgroundMethodGraphuzz: Data-driven Seed Scheduling for Coverage-guided Greybox Fuzzing
Xu, H., Chen, L., Gan, S., Zhang, C., Li, Z. J., Ji, J., Chen, B., & Hu, F. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3664603 · Cited by 5
“…using a Markov chain model; CollAFL [13] evaluates seeds using adjacent uncovered edges; and VUZZER [31] evaluates seeds using path depth; FairFuzz [25] evaluates seeds based on rare edges; EcoFuzz [45] evaluates seeds based on new paths; Entropic [4] uses information entropy of the trace of…”
Song, Y., Chen, W., Sun, X., Qin, Z., Dai, H., Wang, K., Lv, B., & Chen, J. (2025) · Journal of Network and Systems Management · doi:10.1007/s10922-025-09991-4 · Cited by 5
“● AFL [23]: AFL is a widely used fuzzy testing tool that uses genetic algorithms to generate new test cases.”
Evolutionary computation and machine learning in cryptology
Picek, S., & Jakobović, D. (2020) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3377929.3389886 · Cited by 6
GTFuzz: Guard Token Directed Grey-Box Fuzzing
Li, R., Liang, H., Liu, L., Ma, X., Qu, R., Yan, J., & Zhang, J. (2020) · IEEE Pacific Rim International Symposium on Dependable Computing (PRDC) · doi:10.1109/prdc50213.2020.00027 · Cited by 5
“Fairfuzz [16] automatically identifies branches exercised by few AFL-produced inputs (rare branches) and dynamically calculates the mutation masks during fuzzing to help the generated inputs likely cover these branches.”
BackgroundMethodInfluentialA Two-Phase Approach for Conditional Floating-Point Verification
Lohar, D., Jeangoudoux, C., Sobel, J., Darulová, E., & Christakis, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72013-1_3 · Cited by 5
“Apart from AFLGo [12], there is a wide range of targeted greybox fuzzers, such as those targeting specified program locations [18], rare branches [54], unexplored branches [55,73], or potential vulnerabilities [39,45,22,56].”
BackgroundRefined Grey-Box Fuzzing with Sivo
Nikolić, I., Mantu, R., Shen, S., & Saxena, P. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-80825-9_6 · Cited by 5
Bennet: Randomized Specification Testing for Heap-Manipulating Programs
Aamer, Z. K., & Pierce, B. C. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3764115 · Cited by 5
“CGF is an active area with continuous advances from incorporating smarter mutation strategies [10, 48, 51, 73], improving seed prioritization [13, 14, 37, 76], and using better feedback [8, 87].”
Bare-Metal Firmware Fuzzing: A Survey of Techniques and Approaches
Asmita, A., Tsang, R., Ghimire, S., Salehi, S., & Homayoun, H. (2025) · IEEE Access · doi:10.1109/access.2025.3575691 · Cited by 10
“They provide an in-depth discussion of various fuzzing techniques and existing fuzzers, including MOPT-AFL [62], Enfuzz [63], AFLGo [37], VUzzer [34], Angora [64], QSYM [65], FairFuzz [66], Hawkeye [67], FuzzGen [68], AFLNet [69], and others.”
One Mutation Fits All: Exploring Universal Library Fuzzing Based on Exogenous Mutation
Dong, R., Tong, F., Huang, H., Zhu, X., Xiao, X., Wang, S., Wen, S., & Xiang, Y. (2025) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2025.3588878 · Cited by 4
REFLECTA: Reflection-based Scalable and Semantic Scripting Language Fuzzing
Zhang, C., Lee, G., Liu, Q., & Payer, M. (2025) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3708821.3710818 · Cited by 4
“For efficiency, reflection helps in corpus scheduling [25, 44, 57] by prioritizing programs with rare types or attributes.”
DiPri: Distance-Based Seed Prioritization for Greybox Fuzzing (Registered Report)
Qian, R., Zhang, Q., Fang, C., & Chen, Z. (2023) · International Fuzzing Workshop (FUZZING) · doi:10.1145/3605157.3605172 · Cited by 6
Not All Seeds Are Important: Fuzzing Guided by Untouched Edges
Chen, X., Jia, P., Yang, P., Hu, C., Kuang, H., Ye, G., & Hong, X. (2023) · Applied Sciences · doi:10.3390/app132413172 · Cited by 4
“FairFuzz [13] leverages a targeted mutation strategy to prioritize the exploration of rare branches.”
BackgroundChang, Z., Zhang, H., Jia, Y., Xu, S., Li, T., & Liu, Z. (2024) · Computer Communications · doi:10.1016/j.comcom.2024.04.005 · Cited by 5
Scaling Software Security Analysis to Satellites: Automated Fuzz Testing and Its Unique Challenges
Willbold, J., Schloegel, M., Göhler, F., Scharnowski, T., Bars, N., Wörner, S., Schiller, N., & Holz, T. (2024) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero58975.2024.10521316 · Cited by 4
“The most prominent examples of such fuzzers are AFL [3], its successor AFL++ [4], and its derivatives [5], [6], [7], which all use code coverage as their core feedback metric to determine whether a new code path has been covered in the target.”
BackgroundFuzzing API Error Handling Behaviors using Coverage Guided Fault Injection
Sharma, S., Tanksalkar, S. R., Cherupattamoolayil, S., & Machiry, A. (2024) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3634737.3637650 · Cited by 4
“• FairFuzz [43] ( 𝐹𝐹 ): This is a fuzzing tool that uses novel mutation techniques to generate inputs effective at exploring error handling code.”
Fuzzing JavaScript Environment APIs with Interdependent Function Calls
Hodován, R., Vince, D., & Kiss, Á. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-34968-4_12 · Cited by 4
FairFuzz-TC: a fuzzer targeting rare branches
Lemieux, C., & Sen, K. (2020) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-020-00569-w · Cited by 4
Self-citationDRLFCfuzzer: fuzzing with Deep-Reinforcement-Learning under Format Constraints
Gong, K., Yang, W., Cui, B., & Chen, C. (2022) · International Conference on Electronic Information Engineering and Computer Technology (EIECT) · doi:10.1109/eiect58010.2022.00080 · Cited by 4
DAFuzz: data-aware fuzzing of in-memory data stores
Zeng, Y., Zhu, F., Zhang, S., Yu, Y., Yi, S., Pan, Y., Xie, G., & Wu, T. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1592 · Cited by 4
“…2017 ; Honggfuzz, 2023 ; libFuzzer, 2023 ; Fioraldi et al., 2020 ; Google Security Team, 2018 ) and academia ( Böhme, Pham & Roychoudhury, 2016 ; Lemieux & Sen, 2018 ; Gan et al., 2018 ; Lyu et al., 2019 ; Pham, Böhme & Roychoudhury, 2020 ; Aschermann et al., 2020 ; Yue et al., 2020 ; Wang,…”
Co-dependence Aware Fuzzing for Dataflow-Based Big Data Analytics
Humayun, A., Kim, M., & Gulzar, M. A. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616298 · Cited by 5
“1 × faster, and triggering faults that are 0.9 operators deeper than the ones found by the state of the art commercial fuzzer for JVM.”
BackgroundInfluentialDeepDiffer: Find Deep Learning Compiler Bugs via Priority-guided Differential Fuzzing
Lin, K., Song, X., Zeng, Y., & Guo, S. (2023) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs60937.2023.00066 · Cited by 12
“However, generic binary fuzzers [8], [9], [10] face challenges in generating valid inputs and parsing the components of DL models due to the distinct characteristics between traditional compilers and DL compilers [11].”
BackgroundSyzBridge: Bridging the Gap in Exploitability Assessment of Linux Kernel Bugs in the Linux Ecosystem
Zou, X., Yu, H., Zhang, Z., Pu, J., Chen, W., & Qian, Z. (2024) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2024.24926 · Cited by 4
“This has been attempted by several prior work [56], [50], [33], [35].”
ResultHotCFuzz: Enhancing Vulnerability Detection through Fuzzing and Hotspot Code Coverage Analysis
Du, C., Guo, Y., Feng, Y., & Zheng, S. (2024) · Electronics · doi:10.3390/electronics13101909 · Cited by 4
Software Testing or The Bugs’ Nightmare
Menéndez, H. D. (2021) · doi:10.46723/ojse.1.1.1 · Cited by 4
Sleuth: A Switchable Dual-Mode Fuzzer to Investigate Bug Impacts Following a Single PoC
Wei, H., Chen, L., Zhang, Z., Shi, G., & Meng, D. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680316 · Cited by 6
“Moreover, with the help of static analysis techniques, such as taint analysis [21, 36, 41], symbolic execution [17, 43, 50], etc., the performance of detecting memory corruption bugs using fuzzing has also been improving.”
Critical Variable State-Aware Directed Greybox Fuzzing
Xu, C., Cui, N., Pan, Z., Chen, L., Shi, G., & Meng, D. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00219 · Cited by 3
“AFL [4] is a classic greybox fuzzer with edge coverage guidance, and many fuzzers [6], [15], [32], [33] are implemented based on the AFL framework.”
Model-Guided Fuzzing of Distributed Systems
Gulcan, B., Ozkan, B. K., Majumdar, R., & Nagendra, S. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763060 · Cited by 8
Directed Grammar-Based Test Generation
Kirschner, L., & Soremekun, E. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3627220 · Cited by 3
“FairFuzz [47] is a greybox feedback-driven input generation technique built on top of AFL.”
Karim, I., Cicala, F., Hussain, S. R., Chowdhury, O., & Bertino, E. (2020) · Digital Threats Research and Practice · doi:10.1145/3416125 · Cited by 3
A Comprehensive Review of Learning-based Fuzz Testing Techniques
Cheng, H., Li, D., Zhao, M., Li, H., & Wong, W. E. (2024) · International Symposium on System Security, Safety, and Reliability (ISSSR) · doi:10.1109/isssr61934.2024.00024 · Cited by 6
“Grey-box fuzz testing [22][23][24], merging these approaches, utilizes partial internal structure information and external behavior observations to generate more targeted test cases.”
Generator-based Fuzzing with Input Features
Kraus, R., Nguyen, H. L., & Schneider, M. (2024) · IEEE/ACM International Workshop on Search-Based Software Testing (SBST) · doi:10.1145/3643659.3643925 · Cited by 3
“For example, one aim could be to produce more diverse test inputs that reach rarely covered branches within a fuzzing campaign, as done in past work in binary fuzzing [19].”
InfluentialFunFuzz: Greybox Fuzzing with Function Significance
Qian, R., Zhang, Q., Fang, C., Guo, L., & Chen, Z. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3702974 · Cited by 9
“To steer executions to rarely explored code regions, greybox fuzzing leverages code coverage, such as edge coverage, as feedback to schedule the energy (i.e., time budget) assigned to seeds to control test executions [23, 38].”
InfluentialReinforcement Learning-Based Multi-Phase Seed Scheduling for Network Protocol Fuzzing
Cheng, M., Zhu, K.-L., Chen, Y., Lu, Y., Chen, C., & Yu, J. J. (2024) · Electronics · doi:10.3390/electronics13244962 · Cited by 3
“FairFuzz [33] enhances test coverage by allocating power toward seeds that are likely to reach rare branches, thereby controlling the mutation frequency of these seeds.”
The Havoc Paradox in Generator-Based Fuzzing
Li, A., Huang, M., Vikram, V., Lemieux, C., & Padhye, R. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3742894 · Cited by 4
“Similarly, the standard best practice when running AFL variants is to disable the deterministic mutation stage by default [33, 35].”
Self-citationFeedback-Guided Circuit Structure Mutation for Testing Hardware Model Checkers
Zhang, C., Sun, M., Li, J., Su, T., & Pu, G. (2021) · International Conference on Computer Aided Design · doi:10.1109/iccad51958.2021.9643509 · Cited by 4
“For example, FairFuzz [16] and Steelix [17] improve the performance of AFL on the binary level.”
BackgroundSpotFuzz: Fuzzing Based on Program Hot-Spots
Pang, H., Jian, J., Zhuang, Y., Ye, Y., & Li, Z. (2021) · Electronics · doi:10.3390/electronics10243142 · Cited by 3
FuzzingDriver: the Missing Dictionary to Increase Code Coverage in Fuzzers
Ebrahim, A. A., Hazhirpasand, M., Nierstrasz, O., & Ghafari, M. (2022) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner53432.2022.00042 · Cited by 3
CAMFuzz: Explainable Fuzzing with Local Interpretation
Shi, J. F., Zou, W., Zhang, C., Tan, L., Zou, Y., Peng, Y., & Huo, W. (2022) · Cybersecurity · doi:10.1186/s42400-022-00116-x · Cited by 3
“First, to prioritize code blocks to explore, many studies focus on counting unvisited code blocks, e.g., used in CollAFL (Gan et al. 2018) and FairFuzz (Lemieux and Sen 2018).”
BackgroundMethodMemConFuzz: Memory Consumption Guided Fuzzing with Data Flow Analysis
Du, C., Cui, Z., Guo, Y., Xu, G., & Wang, Z. (2023) · Mathematics · doi:10.3390/math11051222 · Cited by 3
RLTG: Multi-targets directed greybox fuzzing
He, Y., & Zhu, Y. (2023) · PLoS ONE · doi:10.1371/journal.pone.0278138 · Cited by 3
“Since AFLGo is developed on AFL, We also compared RLTG with coverage-guided greybox fuzzer AFL [26] and the state-of-the-art AFL-based coverage-guided greybox fuzzer FairFuzz [27].”
MethodInfluentialFCEVAL: An effective and quantitative platform for evaluating fuzzer combinations fairly and easily
Zhao, X., Yang, C., Jia, Z., Wang, Y., & Ma, J. (2023) · Computers & Security · doi:10.1016/j.cose.2023.103354 · Cited by 3
Practical Anti-Fuzzing Techniques With Performance Optimization
Zhou, Z., & Wang, C. (2023) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2023.3301883 · Cited by 3
BSFuzz: Branch-State Guided Hybrid Fuzzing
Hu, Q., Chen, W., Wang, Z., Lu, S., Nie, Y., Li, X., & Kuang, X. (2023) · Electronics · doi:10.3390/electronics12194033 · Cited by 3
LinFuzz: Program-Sensitive Seed Scheduling Greybox Fuzzing Based on LinUCB Algorithm
Su, Y., Xiong, D., Wan, Y., Shi, C., & Zeng, Q. (2024) · IEEE Access · doi:10.1109/access.2024.3404918 · Cited by 6
“The experiment selected open-source fuzzers AFL [12], AFLFast [7], AFLGo [21], FairFuzz [23] and AFL ++ [24] for comparison.”
BackgroundMethodAutoFuzz: automatic fuzzer-sanitizer scheduling with multi-armed bandit
Gao, Y., Zeng, W., Liu, S., & Zeng, Y. (2025) · Software Quality Journal · doi:10.1007/s11219-025-09707-6 · Cited by 4
“…AFL (Zalewski, 2017), a CGF fuzzer, holds widespread adoption in both academia and industry, influencing the development of fuzzers like AFL++ (Fioraldi et al., 2020), libFuzzer (LLVM, 2023a) and others (Li et al., 2021; Lemieux & Sen, 2018; Lyu et al., 2019; Yue et al., 2020) in the literature.”
Adaptive mutation based on multi-population evolution strategy for greybox fuzzing
Jiao, W., Li, X., Li, Q., Cao, F., Li, X., & Yue, S. (2025) · Information Sciences · doi:10.1016/j.ins.2025.121959 · Cited by 3
Finding Bugs with Specification-Based Testing is Easy!
Chin, J., & Pearce, D. J. (2021) · The Art Science and Engineering of Programming · doi:10.22152/programming-journal.org/2021/5/13 · Cited by 4
“For example, American Fuzzy Lop [103] starts from a set of input seeds which it gradually mutates—an approach widely adopted [63, 95].”
BackgroundFuzzing with Multi-dimensional Control of Mutation Strategy
Xu, H., Cui, B., & Chen, C. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-79728-7_27 · Cited by 3
An improved fuzzing approach based on adaptive random testing
Chen, J., Chen, J., Guo, D., & Towey, D. (2020) · IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) · doi:10.1109/issrew51248.2020.00045 · Cited by 3
“Among them, American fuzzy lop (AFL) has become one of the most advanced fuzzing tools, due to its ease of use and powerful ability to detect vulnerabilities [38].”
BackgroundImproving the Effectiveness of Grey-box Fuzzing By Extracting Program Information
Fu, Y. V., Tong, S., Guo, X., Cheng, L., Zhang, Y., & Feng, D. (2020) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom50675.2020.00066 · Cited by 3
“FairFuzz [7] records how many times each branch is hit, selects those inputs that hit rare branches to do further mutations.”
Background
Metadata
- Authors
- Lemieux, C., & Sen, K.
- Affiliations
- University of California, Berkeley (US)
- Venue
- Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering
- Publisher
- ACM
- Published
- 2018-09-03
- Pages
- 475-485
- OpenAlex
- W2757104921
- Cited by
- 477
- References
- 29
- Citations by year
- 2026: 39 · 2025: 60 · 2024: 69 · 2023: 65 · 2022: 62 · 2021: 68 · 2020: 59 · 2019: 47 · 2018: 8
- FWCI
- 30.83
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/ACM International Conference on Automated Software Engineering (ASE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Funders
- National Science Foundation (CCF-1409872, CCF-1423645)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- targeted mutations, branch coverage, code coverage, software testing, bug detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.