FairFuzz: a targeted mutation strategy for increasing greybox fuzz testing coverage
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
In recent years, fuzz testing has proven itself to be one of the most effective techniques for finding correctness bugs and security vulnerabilities in practice. One particular fuzz testing tool, American Fuzzy Lop (AFL), has become popular thanks to its ease-of-use and bug-finding power. However, AFL remains limited in the bugs it can find since it simply does not cover large regions of code. If it does not cover parts of the code, it will not find bugs there. We propose a two-pronged approach to increase the coverage achieved by AFL. First, the approach automatically identifies branches exercised by few AFL-produced inputs (rare branches), which often guard code that is empirically hard to cover by naively mutating inputs. The second part of the approach is a novel mutation mask creation algorithm, which allows mutations to be biased towards producing inputs hitting a given rare branch. This mask is dynamically computed during fuzz testing and can be adapted to other testing targets. We implement this approach on top of AFL in a tool named FairFuzz. We conduct evaluation on real-world programs against state-of-the-art versions of AFL. We find that on these programs FairFuzz achieves high branch coverage at a faster rate that state-of-the-art versions of AFL. In addition, on programs with nested conditional structure, it achieves sustained increases in branch coverage after 24 hours (average 10.6% increase). In qualitative analysis, we find that FairFuzz has an increased capacity to automatically discover keywords.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Compared to our coverage-guided baseline which only adds programs with new coverage to the seed bank for later mutation, our fitness function has minimal additional overhead.”
From Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models (2023) · cites this paper as [42]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Symbolic execution and program testing
King, J. C. (1976) · Communications of the ACM · doi:10.1145/360248.360252 · Cited by 3,031 · cited 1×
“While many of the individual test inputs it generates may be garbage, due to its low computational overhead, CGF generates test inputs much faster than more sophisticated methods such as symbolic execution [17, 34] and dynamic symbolic execution (a.”
MethodCUTE: a concolic unit testing engine for C
Sen, K., Marinov, D., & Agha, G. (2005) · Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1081706.1081750 · Cited by 1,651 · cited 2×
“concolic testing) techniques [5, 7, 13, 16, 21, 22, 36, 45, 47].”
MethodSelf-citationDART: directed automated random testing
Godefroid, P., Klarlund, N., & Sen, K. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065036 · Cited by 1,570 · cited 3×
“concolic testing) techniques [5, 7, 13, 16, 21, 22, 36, 45, 47].”
BackgroundMethodSelf-citationEvoSuite: automatic test suite generation for object-oriented software
Fraser, G., & Arcuri, A. (2011) · Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering · doi:10.1145/2025113.2025179 · Cited by 1,110 · cited 1×
“Evosuite [19] uses seed inputs and genetic algorithms to achieve high code coverage.”
MethodAutomated software test data generation
Korel, B. (1990) · IEEE Transactions on Software Engineering · doi:10.1109/32.57624 · Cited by 971 · cited 1×
“Search-based software testing (SBST) [19, 20, 22–24, 29, 31, 32, 42] uses optimization techniques such as hill climbing and genetic algorithms to generate inputs that optimize some observable fitness function.”
MethodDriller: Augmenting Fuzzing Through Selective Symbolic Execution
Stephens, N., Grosen, J., Salls, C., Dutcher, A., Wang, R., Corbetta, J., Shoshitaishvili, Y., Kruegel, C., & Vigna, G. (2016) · Proceedings 2016 Network and Distributed System Security Symposium · doi:10.14722/ndss.2016.23368 · Cited by 887
Finding and understanding bugs in C compilers
Yang, X., Chen, Y., Eide, E., & Regehr, J. (2011) · Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1993498.1993532 · Cited by 825 · cited 3×
“In practice, this trade-off has paid off, and fuzz testers have found numerous correctness bugs and security vulnerabilities in widely used software [9, 14, 25, 26, 28, 34, 39, 41, 43].”
BackgroundMethodBöhme, M., Pham, V.-T., Nguyen, M.-D., & Roychoudhury, A. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134020 · Cited by 747 · cited 3×
“The success of one particular fuzz tester, American Fuzzy Lop (or simply AFL) [43], has gained attention both in practice and in the research community [8, 9, 30, 39].”
BackgroundMethodResultInfluentialCoverage-based Greybox Fuzzing as Markov Chain
Böhme, M., Pham, V.-T., & Roychoudhury, A. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978428 · Cited by 649
The Current State and Future of Search Based Software Engineering
Harman, M. (2007) · Future of Software Engineering (FOSE '07) · doi:10.1109/fose.2007.29 · Cited by 631 · cited 1×
“Search-based software testing (SBST) [24, 25, 27–29, 35, 38, 39, 50] uses optimization techniques such as hill climbing and genetic algorithms to generate inputs that optimize some observable fitness function.”
Method
Research works printed but not linked: 18
- 10.5555/1763507.1763523 · doi:10.5555/1763507.1763523
- [7] Marcel Böhme. 2016. AFLFast.new. https://groups.google.com/d/msg/afl-users/ 1PmKJC-EKZ0/lbzRb8AuAAAJ. (2016). Accessed August 23rd, 2017. -
- [11] Cristian Cadar, Daniel Dunbar, and Dawson Engler. 2008. KLEE: Unassisted and Automatic Generation of High-coverage Tests for Complex Systems Programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI'08) . -
- [13] Chris Evans, Matt Moore, and Tavis Ormandy. 2011. Fuzzing at Scale. https:// security.googleblog.com/2011/08/fuzzing-at-scale.html. (2011). Accessed August 24th, 2017. -
- [19] Mark Grechanik, Chen Fu, and Qing Xie. 2012. Automatically finding performance problems with feedback-directed learning software testing. In 2012 34th International Conference on Software Engineering (ICSE) . IEEE, 156-166. - · doi:10.5555/2337223.2337242
- [21] Istvan Haller, Asia Slowinska, Matthias Neugschwandtner, and Herbert Bos. 2013. Dowsing for Overflows: A Guided Fuzzer to Find Buffer Boundary Violations. In Proceedings of the 22Nd USENIX Conference on Security (SEC'13) . -
- [23] Mark Harman and John Clark. 2004. Metrics are fitness functions too. In Software Metrics, 2004. Proceedings. 10th International Symposium on . IEEE, 58-69. - · doi:10.5555/1018439.1021893
- [24] Mark Harman and Bryan F Jones. 2001. Search-based software engineering. Information and software Technology 43, 14 (2001), 833-839. -
- [25] Sam Hocevar. 2007. zzuf. http://caca.zoy.org/wiki/zzuf/. (2007). Accessed August 22nd, 2017. -
- [26] Christian Holler, Kim Herzig, and Andreas Zeller. 2012. Fuzzing with Code Fragments. In Presented as part of the 21st USENIX Security Symposium (USENIX Security 12) . -
- [28] Allen D. Householder and Jonathan M. Foote. 2012. Probability-Based Parameter Selection for Black-Box Fuzz Testing . Technical Report. Carnegie Mellon University Software Engineering Institute. -
- Guodong Li , Indradeep Ghosh , and Sreeranga P . Rajan . 2011 . KLOVER : A Symbolic Execution and Automatic Test Generation Tool for C++ Programs. In CAV. 609–615. Guodong Li, Indradeep Ghosh, and Sreeranga P. Rajan. 2011. KLOVER: A Symbolic Execution and Automatic Test Generation Tool for C++ Programs. In CAV. 609–615.
- 10.5555/1792786.1792798 · doi:10.5555/1792786.1792798
- [43] Michał Zalewski. 2014. American Fuzzy Lop. http://lcamtuf.coredump.cx/afl. (2014). Accessed August 18th, 2017. -
- [44] Michał Zalewski. 2016. FidgetyAFL. https://groups.google.com/d/msg/afl-users/ fOPeb62FZUg/CES5lhznDgAJ. (2016). Accessed August 23rd, 2017. -
- [45] Michał Zalewski. 2016. Unique crashes as a metric. https://groups.google.com/ d/msg/afl-users/fOPeb62FZUg/LYxgPYheDwAJ. (2016). Accessed August 24th, 2017. -
- [46] Michał Zalewski. 2017. American Fuzzy Lop Technical Details. http://lcamtuf. coredump.cx/afl/technical\_details.txt. (2017). Accessed August 18th, 2017. -
- [47] Andreas Zeller and Ralf Hildebrandt. 2002. Simplifying and Isolating FailureInducing Input. IEEE Transactions on Software Engineering 28, 2 (2002), 183-200.
Cited by
Deep Learning Based Vulnerability Detection: Are We There Yet?
Chakraborty, S., Krishna, R., Ding, Y., & Ray, B. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3087402 · Cited by 790
“Traditional vulnerability detection systems fall into three major categories – static analysis [26], [65], dynamic analysis [66], [67], and symbolic analysis [68], [69].”
BackgroundKlees, G., Ruef, A., Cooper, B., Wei, S., & Hicks, M. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243804 · Cited by 633
(2021) · doi:10.1109/tse.2019.2946563 · Cited by 574
“FairFuzz [141] H# ✓ ✓ ✓† H#† ✓ ✓ ✓ go-fuzz [225] H# ✓ ✓ ✓ ✓ ✓ ✓ H# ✓ Hawkeye [56] H# ✓ ✓ ✓ ✓ H# ✓ ✓ honggfuzz [213] H# ✓ ✓ ✓ kAFL [191] H# ✓ ✓ LibFuzzer [9] H# ✓ ✓ ✓ ✓ ✓ ✓ MagicFuzzer [50] H# ✓ ✓ ✓ Nautilus [25] H# ✓ ✓ ✓ H# ✓ ✓ RaceFuzzer [197] H# ✓ ✓ ✓ RedQueen [26] H# ✓ ✓ H# ✓”
BackgroundInfluentialQSYM: a practical concolic execution engine tailored for hybrid fuzzing
Yun, I., Lee, S.-H., Xu, M., Jang, Y., & Kim, T. (2018) · USENIX Security Symposium · Cited by 343
Zhu, X., Wen, S., Camtepe, S., & Xiang, Y. (2022) · ACM Computing Surveys · doi:10.1145/3512345 · Cited by 334
“Most fuzzers select bytes heuristically based on execution information [7, 37, 38, 67, 101, 103, 157, 160, 187] or randomly [22, 23, 118, 203, 206].”
BackgroundDeng, Y., Xia, C. S., Peng, H., Yang, C., & Zhang, L. (2023) · Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3597926.3598067 · Cited by 326
“We compare our default tness function against its variants, as well as random selection (Random) and a simplistic coverage guided [18, 35, 42, 50] (Coverage) baseline in Table 4.”
MethodAFL++ : Combining Incremental Steps of Fuzzing Research.
Fioraldi, A., Maier, D., Eißfeldt, H., & Heuse, M. (2020) · USENIX Security Symposium · Cited by 308
Superion: Grammar-Aware Greybox Fuzzing
Wang, J., Chen, B., Wei, L., & Liu, Y. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00081 · Cited by 260
Hawkeye: Towards a Desired Directed Grey-box Fuzzer
Chen, H., Xue, Y., Li, Y., Chen, B., Xie, X., Wu, X., & Liu, Y. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243849 · Cited by 263
Pham, V.-T., Boehme, M., Santosa, A. E., Căciulescu, A. R., & Roychoudhury, A. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2941681 · Cited by 302
SMARTIAN: Enhancing Smart Contract Fuzzing with Static and Dynamic Data-Flow Analyses
Choi, J., Kim, D., Kim, S., Grieco, G., Groce, A., & Cha, S. K. (2021) · 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678888 · Cited by 192
“Fuzzing has become a de facto standard technique for finding security bugs [14], [34], [40], [45], and there has been significant research effort on adopting fuzzing in the domain of smart contracts, too.”
BackgroundDetecting Vulnerability on IoT Device Firmware: A Survey
Feng, X., Zhu, X., Han, Q.-L., Zhou, W., Wen, S., & Xiang, Y. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.105860 · Cited by 185
Magma: A Ground-Truth Fuzzing Benchmark
Hazimeh, A., Herrera, A., & Payer, M. (2020) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3428334 · Cited by 177
“Finally, popular open-source software (OSS) is often used to evaluate fuzzers [10, 29, 30, 33, 38, 54].”
Padhye, R., Lemieux, C., Sen, K., Papadakis, M., & Traon, Y. L. (2019) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3293882.3330576 · Cited by 223
Self-citationRetroWrite: Statically Instrumenting COTS Binaries for Fuzzing and Sanitization
Dinesh, S., Burow, N., Xu, D.-Y., & Payer, M. (2020) · IEEE Symposium on Security and Privacy · doi:10.1109/sp40000.2020.00009 · Cited by 214
Not All Coverage Measurements Are Equal: Fuzzing by Coverage Accounting for Input Prioritization
Wang, Y., Jia, X., Liu, Y., Zeng, K., Bao, T., Wu, D., & Su, P. (2020) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2020.24422 · Cited by 195
“Previous work design comprehensive mutation strategies [12, 26, 31, 58] and optimal mutation scheduling approaches [36].”
BackgroundMethodProFuzzer: On-the-fly Input Type Probing for Better Zero-Day Vulnerability Discovery
You, W., Wang, X., Ma, S., Huang, J., Zhang, X., Wang, X., & Liang, B. (2019) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2019.00057 · Cited by 131
Typestate-guided fuzzer for discovering use-after-free vulnerabilities
Wang, H., Xie, X., Li, Y., Wen, C., Li, Y., Liu, Y., Qin, S., Chen, H., & Sui, Y. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380386 · Cited by 128
BEACON: Directed Grey-Box Fuzzing with Provable Path Pruning
Huang, H., Guo, Y., Shi, Q., Yao, P., Wu, R., & Zhang, C. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833751 · Cited by 188
“Fairfuzz [59] identifies the input offsets where it is not necessary to change the values, thus, minimizing the input search space improves the efficiency of the mutation.”
BackgroundMemLock: memory usage guided fuzzing
Wen, C., Wang, H., Li, Y., Qin, S., Liu, Y., Xu, Z., Chen, H., Xie, X., Pu, G., & Liu, T. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380396 · Cited by 160
“The experiment results show that MemLock substantially outperforms six state-of-the-art tools (i.e., AFL [84], AFLfast [8], PerfFuzz [37], FairFuzz [38], Angora [12] and QSYM [83]), in discovering the memory consumption vulnerabilities.”
BackgroundMethodBoosting fuzzer efficiency: an information theoretic perspective
Böhme, M., Manès, V. J. M., & Kil, S. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409748 · Cited by 168
“FairFuzz [26] and AFLFast [9] are greybox fuzzers that assign more energy to seeds that exercise low-probability paths or branches to discover more low-probability paths or branches.”
BackgroundSQUIRREL: Testing Database Management Systems with Language Validity and Coverage Feedback
Zhong, R., Chen, Y., Hu, H. P., Zhang, H., Lee, W., & Wu, D. (2020) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3372297.3417260 · Cited by 111
DifuzzRTL: Differential Fuzz Testing to Find CPU Bugs
Hur, J., Song, S., Kwon, D., Baek, E., Kim, J., & Lee, B. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00103 · Cited by 126
Cerebro: context-aware adaptive fuzzing for effective vulnerability detection
Li, Y., Xue, Y., Chen, H., Wu, X., Zhang, C., Xie, X., Wang, H., & Liu, Y. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3338975 · Cited by 125
Fuzzing JavaScript Engines with Aspect-preserving Mutation
Park, S., Xu, W. X., Yun, I., Jang, D., & Kim, T. (2020) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40000.2020.00067 · Cited by 111
“Starting from AFL [49], coverageguided fuzzing became very popular among general-purpose fuzzers [7, 27, 36, 38] and also for JavaScript fuzzing [2, 44].”
BackgroundOn the reliability of coverage-based fuzzer benchmarking
Böhme, M., Szekeres, L., & Metzman, J. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510230 · Cited by 104
“AFL [63] AFL++ [15] AFLSmart [53] AFLFast [4] FairFuzz [38] Eclipser [10] MOPT [40] Honggfuzz [58] LibFuzzer [56] Entropic [3] Figure 3: The fuzzers available in Fuzzbench, that we used in our experiments.”
BackgroundMethodPangolin: Incremental Hybrid Fuzzing with Polyhedral Path Abstraction
Huang, H., Yao, P., Wu, R., Shi, Q., & Zhang, C. (2020) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40000.2020.00063 · Cited by 105
“This is not effective because, even with the help of taint analysis [8], [9], which, in this example, determines only z needs to be mutated, the probability of satisfying the constraint 0 < z < 200 at Line 3 is still very low since z ranges over the entire integer domain.”
BackgroundDifFuzz: Differential Fuzzing for Side-Channel Analysis
Nilizadeh, S., Noller, Y., & Păsăreanu, C. S. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00034 · Cited by 100
LibAFL: A Framework to Build Modular and Reusable Fuzzers
Fioraldi, A., Maier, D., Zhang, D., & Balzarotti, D. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3560602 · Cited by 186
ItyFuzz: Snapshot-Based Fuzzer for Smart Contract
Shou, C., Tan, S., & Sen, K. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3598059 · Cited by 100
Self-citationYue, T., Wang, P., Tang, Y., Wang, E., Yu, B., Lü, K., & Zhou, X. (2020) · USENIX Security Symposium · Cited by 94
ParmeSan : Sanitizer-guided greybox fuzzing
Österlund, S., Razavi, K., Bos, H., & Giuffrida, C. (2020) · VU Research Portal · Cited by 84
DocTer: documentation-guided fuzzing for testing deep learning API functions
Xie, D., Li, Y., Kim, M., Pham, H. V., Tan, L., Zhang, X., & Godfrey, M. W. (2022) · Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3533767.3534220 · Cited by 81
“Fuzzers [3, 5, 9] have been adopted to test non-DL libraries [29, 41].”
MethodReinforcement Learning-based Hierarchical Seed Scheduling for Greybox Fuzzing
Wang, J., Song, C., & Yin, H. (2021) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2021.24486 · Cited by 85
“FairFuzz [27] identifies low-frequency edges and prioritizes mutations satisfying these edges.”
BackgroundDiane: Identifying Fuzzing Triggers in Apps to Generate Under-constrained Inputs for IoT Devices
Redini, N., Continella, A., Das, D., Pasquale, G. D., Spahn, N., Machiry, A., Bianchi, A., Kruegel, C., & Vigna, G. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00066 · Cited by 103
EVMFuzzer: detect EVM vulnerabilities via fuzz testing
Fu, Y., Ren, M., Ma, F., Shi, H., Yang, X., Jiang, Y., Li, H., & Shi, X. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3341175 · Cited by 79
FuzzGuard: Filtering out Unreachable Inputs in Directed Grey-box Fuzzing through Deep Learning.
Zong, P., Lv, T., Wang, D., Deng, Z., Liang, R., & Chen, K. (2020) · USENIX Security Symposium · Cited by 79
Automated conformance testing for JavaScript engines via deep compiler fuzzing
Ye, G., Tang, Z., Tan, S. H., Huang, S., Fang, D., Sun, X., Bian, L., Wang, H., & Wang, Z. (2021) · Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation · doi:10.1145/3453483.3454054 · Cited by 77
“AFL [67] and its subsequent works [9, 23, 34, 41, 51, 64]mutate the seed program to improve the test run coverage.”
BackgroundOne fuzzing strategy to rule them all
Wu, M., Jiang, L. L., Xiang, J., Huang, Y., Cui, H., Zhang, L., & Zhang, Y. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510174 · Cited by 100
“Specifically, we set the overall execution time budget for each fuzzer 24 hours following prior works [6, 7, 19, 20, 33, 34].”
BackgroundMethodInfluentialFuzzFactory: domain-specific fuzzing with waypoints
Padhye, R., Lemieux, C., Sen, K., Simon, L. J., & Vijayakumar, H. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3360600 · Cited by 97
Self-citationWindRanger: a directed greybox fuzzer driven by deviation basic blocks
Du, Z., Li, Y., Liu, Y., & Mao, B. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510197 · Cited by 120
“The results in Table 3 demonstrate that by setting vulnerabilities locations as target sites, DGF s (WindRanger and AFLGo) can achieve better performance on exposing relative crashes than CGF s (AFL and FairFuzz).”
BackgroundMethodInfluentialPATA: Fuzzing with Path Aware Taint Analysis
Liang, J., Wang, M., Zhou, C., Wu, Z., Jiang, Y., Liu, J., Liu, Z., & Sun, J. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833594 · Cited by 75
“Inference-based taint analysis [6, 17, 22, 34] perturbs Yu Jiang and Zhe Liu are the corresponding authors. the input bytes repeatedly and collects the variables’ values during program execution.”
BackgroundFuzzGen: Automatic Fuzzer Generation
Ispoglou, K. K., Austin, D. F., Mohan, V., & Payer, M. (2020) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 71
SoK: Prudent Evaluation Practices for Fuzzing
Schloegel, M., Bars, N., Schiller, N., Bernhard, L., Scharnowski, T., Crump, A., Ebrahim, A. A., Bissantz, N., Muench, M., & Holz, T. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00137 · Cited by 152
“When looking at the fuzzers chosen as baselines for comparison, we find that AFL is compared against by 35% (53) of studies, followed by QSym [176] with 15% (23), AFLFast [15] with 14% (21), Angora [30] with 13% (20), FairFuzz [95] with 8% (12), and AFL++ with 9% (14).”
MethodPAFL: extend fuzzing optimizations of single mode to industrial parallel mode
Liang, J., Jiang, Y., Chen, Y., Wang, M., Zhou, C., & Sun, J. (2018) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3236024.3275525 · Cited by 67
“AFLFast [6] and FairFuzz [14] are two extensions of AFL which collect program running information in step 3 to guide fuzzing to explore more areas.”
BackgroundFuzzing: on the exponential cost of vulnerability discovery
Böhme, M., & Falk, B. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409729 · Cited by 105
“Coverage-guided greybox fuzzers [4, 5, 11, 12, 18, 21, 26] use lightweight instrumentation to collect coverage-information during runtime.”
BackgroundSLF: Fuzzing without Valid Seed Inputs
You, W., Liu, X., Ma, S., Perry, D. M., Zhang, X., & Liang, B. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00080 · Cited by 77
UNIFUZZ: A Holistic and Pragmatic Metrics-Driven Platform for Evaluating FuzzersPreprint
Li, Y., Ji, S., Yuan, C., Liang, S., Lee, W., Chen, Y., Lyu, C., Wu, C., Beyah, R., Cheng, P., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01785 · Cited by 72
“UNIFUZZ to date has incorporated 35 usable fuzzers including AFL [1], AFLFast [2], AFLGo [25], AFLPIN [26], AFLSmart [27], Angora [3], CodeAlchemist [28], Driller [17], Domato [29], Dharma [30], Eclipser [31], FairFuzz [32], Fuzzilli [33], Grammarinator [34], Honggfuzz [4], Jsfuzz [35], jsfunfuzz [36], LearnAFL [37],…”
BackgroundEffective Seed Scheduling for Fuzzing with Graph Centrality Analysis
She, D., Shah, A., & Jana, S. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833761 · Cited by 62
Targeted greybox fuzzing with static lookahead analysis
Wüstholz, V., & Christakis, M. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380388 · Cited by 87
One Engine to Fuzz ’em All: Generic Language Processor Testing with Semantic Validation
Chen, Y., Zhong, R., Hu, H. P., Zhang, H., Yang, Y., Wu, D., & Lee, W. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00071 · Cited by 55
“To fully utilize computation power, researchers try to find a better feedback guidance other than naive code coverage [27, 28, 49, 65].”
MethodOn the unusual effectiveness of type-aware operator mutations for testing SMT solvers
Winterer, D., Zhang, C., & Su, Z. (2020) · Proceedings of the ACM on Programming Languages · doi:10.1145/3428261 · Cited by 85
“Follow-upworks, such as FairFuzz [Lemieux and Sen 2018] and Steelix [Li et al. 2017], improved the performance of AFL on the binary level.”
BackgroundFuzz testing in practice: Obstacles and solutions
Liang, J., Wang, M., Chen, Y., Jiang, Y., & Zhang, R. (2018) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2018.8330260 · Cited by 55
Luo, Z., Zuo, F., Jiang, Y., Gao, J., Jiao, X., & Sun, J. (2019) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3358227 · Cited by 53
Binary-level Directed Fuzzing for Use-After-Free VulnerabilitiesPreprint
Nguyen, M.-D., Bardin, S., Bonichon, R., Groz, R., & Lemerre, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10751 · Cited by 82
“Substantial efforts have been conducted in the last few years to improve over it [26, 40, 46].”
BackgroundPluto: Exposing Vulnerabilities in Inter-Contract Scenarios
Ma, F., Xu, Z., Ren, M., Yin, Z., Chen, Y., Qiao, L., Gu, B., Li, H., Jiang, Y., & Sun, J. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3117966 · Cited by 53
“Apart from the above techniques, fuzzing is also a useful way to detect vulnerabilities in programs[48], [49], [50], [51].”
BackgroundAnkou: guiding grey-box fuzzing towards combinatorial difference
Manès, V. J. M., Kim, S., & Kil, S. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380421 · Cited by 82
Coverage-guided tensor compiler fuzzing with joint IR-pass mutation
Liu, J., Wei, Y., Yang, S., Deng, Y., & Zhang, L. (2022) · Proceedings of the ACM on Programming Languages · doi:10.1145/3527317 · Cited by 51
“Fuzzing [Böhme et al. 2017; Fioraldi et al. 2020; Lemieux and Sen 2018; Serebryany 2016; Zalewski 2018], known as an advanced automatic testing technique, has been widely employed to efficiently detect software bugs in the wild.”
BackgroundMethodRULF: Rust Library Fuzzing via API Dependency Graph Traversal
Jiang, J., Xu, H., & Zhou, Y. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678813 · Cited by 49
“Existing investigations mainly aim to improve the efficiency and effectiveness of such tools, such as AFLFast [28], Driller [29], and Fairfuzz [30].”
MethodCoverage guided, property based testing
Lampropoulos, L., Hicks, M., & Pierce, B. C. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3360607 · Cited by 49
“Fairfuzz [Lemieux and Sen 2018] and Vuzzer [Rawat et al. 2017] use static analysis and instrumentation to prioritize rarely covered parts of the program, and they try to mutate parts of an input that drive a program further down rare paths.”
BackgroundDeepFuzzer: Accelerated Deep Greybox Fuzzing
Liang, J., Jiang, Y., Wang, M., Jiao, X., Chen, Y., Song, H. H., & Choo, K. R. (2019) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2019.2961339 · Cited by 48
“FairFuzz [13], ProFuzzer [30], and MOPT [15] focus on improving the seed mutation.”
BackgroundChen, Y., Mu, D., Xu, J., Sun, Z., Shen, W., Xing, X., Lu, L. J., & Mao, B. (2019) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3321705.3329828 · Cited by 46
“Code exploration efficiency: We calculated the code coverage using a representative quantification — number of edges between basic blocks [23] and summarize the results in Figure 8.”
BackgroundMethodVulnerability-Oriented Fuzz Testing for Connected Autonomous Vehicle Systems
Moukahal, L. J., Zulkernine, M., & Soukup, M. (2021) · IEEE Transactions on Reliability · doi:10.1109/tr.2021.3112538 · Cited by 57
VeriFuzz: Program Aware Fuzzing
Chowdhury, A. B., Medicherla, R. K., & Venkatesh, R. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-17502-3_22 · Cited by 59
Zheng, Y., Li, Y., Zhang, C., Zhu, H., Liu, Y., & Sun, L. (2022) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3533767.3534414 · Cited by 51
Sequence Coverage Directed Greybox Fuzzing
Liang, H., Zhang, Y., Yu, Y., Xie, Z., & Jiang, L. C. (2019) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2019.00044 · Cited by 43
“Greybox fuzzers like AFL [13] can generate several orders of magnitude more inputs during the time that symbolic execution generates a single input, so greybox fuzzers can be used to develop lightweight directed fuzzers (e.g. [14], [15], [12], [16]).”
JITfuzz: Coverage-guided Fuzzing for JVM Just-in-Time Compilers
Wu, M., Lu, M., Cui, H., Chen, J., Zhang, Y., & Zhang, L. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00017 · Cited by 58
“Second, while random/probabilistic mutation becomes a major paradigm adopted by many fuzzers [2, 3, 4, 5], it is nevertheless inefficient for JIT testing since massive resulting mutants cannot conform to JVM specification and executing them easily terminates JIT testing early (e.”
BackgroundCoVeriTest: Cooperative Verifier-Based Testing
Beyer, D., & Jakobs, M.-C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-16722-6_23 · Cited by 42
Industry Practice of Coverage-Guided Enterprise-Level DBMS Fuzzing
Wang, M., Wu, Z., Xu, X., Liang, J., Zhou, C., Zhang, H., & Jiang, Y. (2021) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip52600.2021.00042 · Cited by 42
“While symbolic execution [21], [22] and enhanced mutation [23], [24] improved the quality of generated inputs, coverage-guided fuzzing are still blocked by shallow checks in most cases.”
BackgroundMemFuzz: Using Memory Accesses to Guide Fuzzing
Coppik, N., Schwahn, O., & Suri, N. (2019) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2019.00015 · Cited by 53
Fuzzing JavaScript Interpreters with Coverage-Guided Reinforcement Learning for LLM-Based Mutation
Eom, J., Jeong, S., & Kwon, T. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680389 · Cited by 89
“We adopt edge coverage from the AFL’s coverage bitmap, following FairFuzz [30] and Evaluate-Fuzz-Testing [25] settings.”
Sequence Directed Hybrid Fuzzing
Liang, H., Jiang, L. C., Ai, L., & Wei, J. (2020) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner48275.2020.9054807 · Cited by 37
Evaluating and Improving Hybrid Fuzzing
Jiang, L. L., Yuan, H., Wu, M., Zhang, L., & Zhang, Y. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00045 · Cited by 50
“This metric reflects the effort by the fuzzing strategy to explore the given edge, as in [14], [48].”
BackgroundMethodInfluentialAdaptive Grey-Box Fuzz-Testing with Thompson Sampling
Karamcheti, S., Mann, G., & Rosenberg, D. S. (2018) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3270101.3270108 · Cited by 36
1dVul: Discovering 1-Day Vulnerabilities through Binary Patches
Peng, J., Li, F., Liu, B., Xu, L., Liu, B.-H., Chen, K., & Huo, W. (2019) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2019.00066 · Cited by 37
LearnAFL: Greybox Fuzzing With Knowledge Enhancement
Yue, T., Tang, Y., Yu, B., Wang, P., & Wang, E. (2019) · IEEE Access · doi:10.1109/access.2019.2936235 · Cited by 36
“png, which is the same as that of [15].”
BackgroundMethodInfluentialDARWIN: Survival of the Fittest Fuzzing Mutators
Jauernig, P., Jakobović, D., Picek, S., Stapf, E., & Sadeghi, A. (2023) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2023.23159 · Cited by 43
“We further include jhead 3.06.0.1, bsdtar (from libarchive) 3.6.0, tcpdump 4.99.1, and djpeg 2.1.2, as they are also commonly used [67], [4], [39], [37].”
BackgroundMethodInfluentialA systematic review of fuzzing
Zhao, X., Qu, H., Xu, J., Li, X., Lv, W., & Wang, G. (2023) · Soft Computing · doi:10.1007/s00500-023-09306-2 · Cited by 43
“FairFuzz (Lemieux and Sen 2018) uses new mutation strategies to address the limitation of low coverage of AFL.”
Evolutionary Grammar-Based Fuzzing
Eberlein, M., Noller, Y., Vogel, T., & Grunske, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59762-7_8 · Cited by 35
Zhou, C., Wang, M., Liang, J., Liu, Z., & Jiang, Y. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416572 · Cited by 35
“For the seed mutation stage, FairFuzz [28] mutates input seeds in a restricted way so that they are more likely to still explore the rarest branch, MOPT [36] finds the optimal selection probability distribution of operators with respect to fuzzing effectiveness.”
BackgroundMethodContext-Sensitive and Directional Concurrency Fuzzing for Data-Race Detection
Jiang, Z.-M., Bai, J.-J., Lu, K., & Hu, S. (2022) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.24296 · Cited by 35
“Based on AFL and Syzkaller, many approaches [4], [15], [30], [45], [50], [60], [61], [72], [79], [80], [82] have been proposed to further improve fuzzing effectiveness for code coverage and bug detection.”
BackgroundMethodVulnerability Detection of ICS Protocols via Cross-State Fuzzing
Zuo, F., Luo, Z., Yu, J., Chen, T., Xu, Z., Cui, A., & Jiang, Y. (2022) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2022.3201471 · Cited by 40
“To improve the efficiency of AFL, some researchers modified its seed selection or mutation strategies, such as FairFuzz [26].”
MethodBeDivFuzz: integrating behavioral diversity into generator-based fuzzing
Nguyen, H. L., & Grunske, L. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510182 · Cited by 49
“The main idea of FairFuzz is to learn mutation masks and areas that have a higher chance of hitting these rare branches.”
BackgroundMobFuzz: Adaptive Multi-objective Optimization in Gray-box Fuzzing
Zhang, G., Wang, P., Yue, T., Kong, X., Huang, S., Zhou, X., & Lü, K. (2022) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.24314 · Cited by 33
“Following the experiments in the MAGMA paper, AFL[7], AFLFast[1], AFL++[31] (with lto and cmplog enabled), FairFuzz[17], honggfuzz[32], MOPT [33], and SYMCC [34] are used in our evaluation.”
BackgroundMethodInfluentialMagma: A Ground-Truth Fuzzing Benchmark
Hazimeh, A., Herrera, A., & Payer, M. (2021) · ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS) · doi:10.1145/3410220.3456276 · Cited by 46
Learning Seed-Adaptive Mutation Strategies for Greybox Fuzzing
Lee, M., Cha, S., & Oh, H. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00043 · Cited by 32
“F AIR F UZZ [23], for example, guides the seed mutation strategy to mutate the desired locations of the seed inputs by masking the locations not to mutate.”
BackgroundMethodInfluentialUTopia: Automatic Generation of Fuzz Driver using Unit Tests
Jeong, B., Jang, J., Yi, H., Moon, J., Kim, J., Jeon, I., Kim, T., Shim, W., & Hwang, Y. H. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179394 · Cited by 70
“Therefore, several improvements [8, 11, 13–15, 19–22, 22–24, 26] have been introduced to date on seed scheduling, input mutation, and feedback mechanism.”
BackgroundFormal Methods and Validation Techniques for Ensuring Automotive Systems Security
Krichen, M. (2023) · Information · doi:10.3390/info14120666 · Cited by 53
Putting the semantics into semantic versioning
Lam, P., Dietrich, J. B., & Pearce, D. J. (2020) · ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!) · doi:10.1145/3426428.3426922 · Cited by 49
“These tools provide value by helping to avoid errors caused by violating the rules of binary compatibility [Lindholm et al. 2014, Sect.5.4], which many developers struggle to understand [Dietrich et al. 2016].”
MethodInfluentialSGPFuzzer: A State-Driven Smart Graybox Protocol Fuzzer for Network Protocol Implementations
Yu, Y., Chen, Z., Gan, S., & Wang, X. (2020) · IEEE Access · doi:10.1109/access.2020.3025037 · Cited by 31
EVMFuzz: Differential fuzz testing of Ethereum virtual machine
Fu, Y., Ren, M., Ma, F., Yang, X., Shi, H., Li, S., & Liao, X. (2023) · Journal of Software Evolution and Process · doi:10.1002/smr.2556 · Cited by 28
“Some popular AFL [39] family tools [2–4, 17, 21, 23, 31, 33, 35, 40] apply various strategies to boost fuzzing process, including symbolic execution, schedule algorithm and so on.”
MethodPySE: Automatic Worst-Case Test Generation by Reinforcement Learning
Koo, J., Saumya, C., Kulkarni, M., & Bagchi, S. (2019) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2019.00023 · Cited by 32
“While FairFuzz [20] focuses on code coverage, re-”
BackgroundPathAFL: Path-Coverage Assisted Fuzzing
Yan, S., Wu, C., Li, H., Shao, W., & Jia, C. (2020) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3320269.3384736 · Cited by 29
BigFuzz: efficient fuzz testing for data analytics using framework abstraction
Zhang, Q., Wang, J., Gulzar, M. A., Padhye, R., & Kim, M. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416641 · Cited by 50
“Prior work finds that branch coverage is an effective guidance mechanism for feedback-guided fuzz testing, pushing test generation towards hard-to-reach corners [17, 44, 56].”
BackgroundHDTest: Differential Fuzz Testing of Brain-Inspired Hyperdimensional Computing
Ma, D., Guo, J., Jiang, Y., & Jiao, X. (2021) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586169 · Cited by 29
REINAM: reinforcement learning for input-grammar inference
Wu, Z., Johnson, E., Yang, W. M., Bastani, O., Song, D. X., Peng, J., & Xie, T. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3338958 · Cited by 28
“fuzzing has achieved substantial success with tools such as American Fuzzy Lop (AFL)6 and Fairfuzz [25].”
BackgroundMethodIntelliGen: Automatic Driver Synthesis for Fuzz Testing
Zhang, M., Liu, J., Ma, F., Zhang, H., & Jiang, Y. (2021) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip52600.2021.00041 · Cited by 28
POWER: Program Option-Aware Fuzzer for High Bug Detection Ability
Lee, A., Ariq, I., Kim, Y., & Kim, M. (2022) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst53961.2022.00032 · Cited by 36
“FairFuzz [49] and Vuzzer [50] favor input files which execute rarely executed branches and which execute basic blocks located in deep control-structure, respectively.”
BackgroundThe Use of Likely Invariants as Feedback for Fuzzers
Fioraldi, A., D’Elia, D. C., & Balzarotti, D. (2021) · IRIS Research product catalog (Sapienza University of Rome) · Cited by 81
Path transitions tell more: optimizing fuzzing schedules via runtime program states
Zhang, K., Xiao, X., Zhu, X., Sun, R., Xue, M., & Wen, S. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510063 · Cited by 39
“The basic idea of CGF is to exploremore code coverage so that it may trigger a bug hiding in certain code regions [1, 13, 19, 25, 35, 38, 43].”
Background
Metadata
- Authors
- Lemieux, C., & Sen, K.
- Affiliations
- University of California, Berkeley (US)
- Venue
- Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering
- Publisher
- ACM
- Published
- 2018-09-03
- Pages
- 475-485
- OpenAlex
- W2757104921
- Cited by
- 477
- References
- 29
- Citations by year
- 2026: 39 · 2025: 60 · 2024: 69 · 2023: 65 · 2022: 62 · 2021: 68 · 2020: 59 · 2019: 47 · 2018: 8
- FWCI
- 30.83
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/ACM International Conference on Automated Software Engineering (ASE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Funders
- National Science Foundation (CCF-1409872, CCF-1423645)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- targeted mutations, branch coverage, code coverage, software testing, bug detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.