The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
An empirical study of the reliability of UNIX utilities
Open accessAbstract
The following section describes the tools we built to test the utilities. These tools include the fuzz (random character) generator, ptyjig (to test interactive utilities), and scripts to automate the testing process. Next, we will describe the tests we performed, giving the types of input we presented to the utilities. Results from the tests will follow along with an analysis of the results, including identification and classification of the program bugs that caused the crashes. The final section presents concluding remarks, including suggestions for avoiding the types of problems detected by our study and some commentary on the bugs we found. We include an Appendix with the user manual pages for fuzz and ptyjig.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Letters to the editor: go to statement considered harmful
Dijkstra, E. W. (1968) · Communications of the ACM · doi:10.1145/362929.362947 · Cited by 1,026
Spafford, E. H. (1989) · Communications of the ACM · doi:10.1145/63526.63527 · Cited by 170
Cited by
The oracle problem in software testing: A survey
Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2372785 · Cited by 1,104
GenProg: A Generic Method for Automatic Software Repair
Le Goues, C., Nguyen, T., Forrest, S., & Weimer, W. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.104 · Cited by 1,085
Simplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,079
SOK: (State of) The Art of War: Offensive Techniques in Binary Analysis
Shoshitaishvili, Y., Wang, R., Salls, C., Stephens, N., Polino, M., Dutcher, A., Grosen, J., Feng, S., Hauser, C., Kruegel, C., et al. (2016) · 2016 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2016.17 · Cited by 957
Driller: Augmenting Fuzzing Through Selective Symbolic Execution
Stephens, N., Grosen, J., Salls, C., Dutcher, A., Wang, R., Corbetta, J., Shoshitaishvili, Y., Kruegel, C., & Vigna, G. (2016) · Proceedings 2016 Network and Distributed System Security Symposium · doi:10.14722/ndss.2016.23368 · Cited by 887
Feedback-Directed Random Test Generation
Pacheco, C., Lahiri, S. K., Ernst, M. D., & Ball, T. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.37 · Cited by 863
Finding and understanding bugs in C compilers
Yang, X., Chen, Y., Eide, E., & Regehr, J. (2011) · Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1993498.1993532 · Cited by 825
EXE: automatically generating inputs of death
Cadar, C., Ganesh, V., Pawlowski, P. M., Dill, D. L., & Engler, D. R. (2006) · Proceedings of the 13th ACM conference on Computer and communications security · doi:10.1145/1180405.1180445 · Cited by 788
Automatically finding patches using genetic programming
Weimer, W., Nguyen, T., Le Goues, C., & Forrest, S. (2009) · 2009 IEEE 31st International Conference on Software Engineering · doi:10.1109/icse.2009.5070536 · Cited by 761
Böhme, M., Pham, V.-T., Nguyen, M.-D., & Roychoudhury, A. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134020 · Cited by 747
An empirical study of operating systems errors
Chou, A., Yang, J., Chelf, B., Hallem, S., & Engler, D. (2001) · Proceedings of the eighteenth ACM symposium on Operating systems principles · doi:10.1145/502034.502042 · Cited by 713
Coverage-based Greybox Fuzzing as Markov Chain
Böhme, M., Pham, V.-T., & Roychoudhury, A. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978428 · Cited by 649
Finding and understanding bugs in C compilers
Yang, X., Chen, Y., Eide, E., & Regehr, J. (2011) · ACM SIGPLAN Notices · doi:10.1145/1993316.1993532 · Cited by 586
VUzzer: Application-aware Evolutionary Fuzzing
Rawat, S., Jain, V., Kumar, A., Cojocar, L., Giuffrida, C., & Bos, H. (2017) · Proceedings 2017 Network and Distributed System Security Symposium · doi:10.14722/ndss.2017.23404 · Cited by 584
(2021) · doi:10.1109/tse.2019.2946563 · Cited by 574
S2E: a platform for in-vivo multi-path analysis of software systems
Chipounov, V., Kuznetsov, V., & Candea, G. (2011) · Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systems · doi:10.1145/1950365.1950396 · Cited by 514
Grammar-based whitebox fuzzing
Godefroid, P., Kiezun, A., & Levin, M. Y. (2008) · Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1375581.1375607 · Cited by 462
An empirical analysis of flaky tests
Luo, Q., Hariri, F., Eloussi, L., & Marinov, D. (2014) · Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2635868.2635920 · Cited by 450
An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation
Schäfer, M., Nadi, S., Eghbali, A., & Tip, F. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3334955 · Cited by 431
Efficient detection of all pointer and array access errors
Austin, T. M., Breach, S. E., & Sohi, G. S. (1994) · Proceedings of the ACM SIGPLAN 1994 conference on Programming language design and implementation · doi:10.1145/178243.178446 · Cited by 427
Predicting vulnerable software components
Neuhaus, S., Zimmermann, T., Holler, C., & Zeller, A. (2007) · Proceedings of the 14th ACM conference on Computer and communications security · doi:10.1145/1315245.1315311 · Cited by 416
The Qualitas Corpus: A Curated Collection of Java Code for Empirical Studies
Tempero, E., Anslow, C., Dietrich, J., Han, T., Li, J., Lumpe, M., Melton, H., & Noble, J. (2010) · 2010 Asia Pacific Software Engineering Conference · doi:10.1109/apsec.2010.46 · Cited by 407
TaintScope: A Checksum-Aware Directed Fuzzing Tool for Automatic Software Vulnerability Detection
Wang, T., Wei, T., Gu, G., & Zou, W. (2010) · 2010 IEEE Symposium on Security and Privacy · doi:10.1109/sp.2010.37 · Cited by 380
Taint-based directed whitebox fuzzing
Ganesh, V., Leek, T., & Rinard, M. (2009) · 2009 IEEE 31st International Conference on Software Engineering · doi:10.1109/icse.2009.5070546 · Cited by 348
EXE: Automatically Generating Inputs of Death
Cadar, C., Ganesh, V., Pawlowski, P. M., Dill, D. L., & Engler, D. R. (2008) · ACM Transactions on Information and System Security · doi:10.1145/1455518.1455522 · Cited by 342
Coverage-Based Greybox Fuzzing as Markov Chain
Bohme, M., Pham, V.-T., & Roychoudhury, A. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2785841 · Cited by 340
Zhu, X., Wen, S., Camtepe, S., & Xiang, Y. (2022) · ACM Computing Surveys · doi:10.1145/3512345 · Cited by 334
Skyfire: Data-Driven Seed Generation for Fuzzing
Wang, J., Chen, B., Wei, L., & Liu, Y. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.23 · Cited by 315
Automatic Patch-Based Exploit Generation is Possible: Techniques and Implications
Brumley, D., Poosankam, P., Song, D., & Zheng, J. (2008) · 2008 IEEE Symposium on Security and Privacy (sp 2008) · doi:10.1109/sp.2008.17 · Cited by 281
Christodorescu, M., & Jha, S. (2004) · Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis · doi:10.1145/1007512.1007518 · Cited by 271
Steelix: program-state based binary fuzzing
Li, Y., Chen, B., Chandramohan, M., Lin, S.-W., Liu, Y., & Tiu, A. (2017) · Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering · doi:10.1145/3106237.3106295 · Cited by 269
Hawkeye: Towards a Desired Directed Grey-box Fuzzer
Chen, H., Xue, Y., Li, Y., Chen, B., Xie, X., Wu, X., & Liu, Y. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243849 · Cited by 263
Superion: Grammar-Aware Greybox Fuzzing
Wang, J., Chen, B., Wei, L., & Liu, Y. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00081 · Cited by 260
Execution Generated Test Cases: How to Make Systems Code Crash Itself
Cadar, C., & Engler, D. (2005) · Lecture Notes in Computer Science · doi:10.1007/11537328_2 · Cited by 254
What You Corrupt Is Not What You Crash: Challenges in Fuzzing Embedded Devices
Muench, M., Stijohann, J., Kargl, F., Francillon, A., & Balzarotti, D. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23166 · Cited by 219
Program Logic Based Software Plagiarism Detection
Zhang, F., Wu, D., Liu, P., & Zhu, S. (2014) · 2014 IEEE 25th International Symposium on Software Reliability Engineering · doi:10.1109/issre.2014.18 · Cited by 57
Coverage guided, property based testing
Lampropoulos, L., Hicks, M., & Pierce, B. C. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3360607 · Cited by 49
Ming, J., Zhang, F., Wu, D., Liu, P., & Zhu, S. (2016) · IEEE Transactions on Reliability · doi:10.1109/tr.2016.2570554 · Cited by 33
The Relevance of Classic Fuzz Testing: Have We Solved This One?
Miller, B. P., Zhang, M., & Heymann, E. R. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.3047766 · Cited by 26
Self-citationZero-Trust Principles for Legacy Components: 12 Rules for Legacy Devices: An Antidote to Chaos
Køien, G. M. (2021) · Wireless Personal Communications · doi:10.1007/s11277-021-09055-1 · Cited by 24
Learning Program Behavioral Models from Synthesized Input–Output Pairs
Mammadov, T., Klakow, D., Koller, A., & Zeller, A. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3748720 · Cited by 5
Learning Program Behavioral Models from Synthesized Input-Output Pairs
Mammadov, T., Klakow, D., Koller, A., & Zeller, A. (2024) · arXiv · doi:10.48550/arxiv.2407.08597
Metadata
- Authors
- Miller, B. P., Fredriksen, L., & So, B.
- Venue
- Communications of the ACM
- Published
- 1990-12
- Volume
- 33
- Issue
- 12
- Pages
- 32-44
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2002934700
- Cited by
- 1,281
- References
- 2
- FWCI
- 6.73
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.