The DOI matches its Crossref record; 17 of the 20 references it cites were found in a registry or an index, and 3 are not checked yet.
BugsInPy: a database of existing bugs in Python programs to enable controlled testing and debugging studies
Open accessAbstract
The 2019 edition of Stack Overflow developer survey highlights that, for the first time, Python outperformed Java in terms of popularity. The gap between Python and Java further widened in the 2020 edition of the survey. Unfortunately, despite the rapid increase in Python's popularity, there are not many testing and debugging tools that are designed for Python. This is in stark contrast with the abundance of testing and debugging tools for Java. Thus, there is a need to push research on tools that can help Python developers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We mitigate this by evaluating our approach on Defects4J [13] and BugsInPy [35], two widely used benchmarks containing diverse real-world faults in Java and Python projects.”
From Order Matters! An Empirical Study on Large Language Models' Input Order Bias in Software Fault Localization (2024) · cites this paper as [35]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Matplotlib: a 2D graphics environment
Hunter, J. D. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.55 · Cited by 39,249
Reback, J., McKinney, W., Bossche, J. V. D., Jbrockmendel, Augspurger, T., Cloud, P., Gfyoung, Sinhrks, Klein, A., Tratner, J., et al. (2019) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3509134 · Cited by 4,909
Defects4J: a database of existing faults to enable controlled testing studies for Java programs
Just, R., Jalali, D., & Ernst, M. D. (2014) · Proceedings of the 2014 International Symposium on Software Testing and Analysis · doi:10.1145/2610384.2628055 · Cited by 1,447
Do, H., Elbaum, S. G., & Rothermel, G. (2005) · Empirical Software Engineering · doi:10.1007/s10664-005-3861-2 · Cited by 1,161
Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria
Hutchins, M., Foster, H. D., Goradia, T., & Ostrand, T. J. (1994) · doi:10.5555/257734.257766 · Cited by 783
Are mutants a valid substitute for real faults in software testing?
Just, R., Jalali, D., Inozemtseva, L., Ernst, M. D., Holmes, R., & Fraser, G. (2014) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2635868.2635929 · Cited by 602
Honnibal, M., Montani, I., Honnibal, M., Peters, H., Samsonov, M., Geovedi, J., Regan, J., Orosz, G., Kristiansen, S. L., R., et al. (2018) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1212303 · Cited by 543
Le, X.-B. D., Lo, D., & Goues, C. L. (2016) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2016.76 · Cited by 341
Self-citationPrecise Condition Synthesis for Program Repair
Xiong, Y., Wang, J., Yan, R., Zhang, J., Shi, H., Huang, G., & Zhang, L. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2017.45 · Cited by 308
FLUCCS: using code and change metrics to improve fault localization
Sohn, J., & Yoo, S. (2017) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3092703.3092717 · Cited by 252
Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset
Martínez, M., Durieux, T., Sommerard, R., Xuan, J., & Monperrus, M. (2016) · Empirical Software Engineering · doi:10.1007/s10664-016-9470-4 · Cited by 249
QuixBugs: a multi-lingual program repair benchmark set based on the quixey challenge
Lin, D., Koppel, J., Chen, A., & Solar-Lezama, A. (2017) · Proceedings Companion of the 2017 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity · doi:10.1145/3135932.3135941 · Cited by 242
A learning-to-rank based fault localization approach using likely invariants
Le, T.-D. B., Lo, D., Goues, C. L., & Grunske, L. (2016) · International Symposium on Software Testing and Analysis · doi:10.1145/2931037.2931049 · Cited by 206
Self-citationSaha, R. K., Lyu, Y., Lam, W., Yoshida, H., & Prasad, M. R. (2018) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3196398.3196473 · Cited by 168
How does regression test prioritization perform in real-world software evolution?
Lu, Y., Lou, Y., Cheng, S., Zhang, L., Hao, D., Zhou, Y., & Zhang, L. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884874 · Cited by 110
Xia, X., Bao, L., Lo, D., & Li, S. (2016) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2016.67 · Cited by 104
Self-citationBugsJS: a Benchmark of JavaScript Bugs
Gyimesi, P., Vancsics, B., Stocco, A., Mazinanian, D., Beszédes, Á., Ferenć, R., & Mesbah, A. (2019) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2019.00019 · Cited by 99
BugSwarm: Mining and Continuously Growing a Dataset of Reproducible Failures and Fixes
Tomassi, D. A., Dmeiri, N., Wang, Y., Bhowmick, A., Liu, Y.-C., Dévanbu, P., Vasilescu, B., & Rubio-González, C. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00048 · Cited by 91
GRT: Program-Analysis-Guided Random Testing (T)
Ma, L., Artho, C. V., Zhang, C., Satō, H., Gmeiner, J., & Ramler, R. (2015) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.49 · Cited by 77
Defexts: A Curated Dataset of Reproducible Real-World Bugs for Modern JVM Languages
Benton, S., Ghanbari, A., & Zhang, L. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion.2019.00035 · Cited by 23
Research works printed but not linked: 1
- Durieux Thomas, 1905, Critical Review of BugSwarm for Fault Localization and Program Repair. arXiv preprint arXiv
Cited by
Sorted by citations
CodaMosa: Escaping Coverage Plateaus in Test Generation with Pre-trained Large Language Models
Lemieux, C., Inala, J. P., Lahiri, S. K., & Sen, S. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00085 · Cited by 306
Large Language Models for Test-Free Fault Localization
Yang, A. Z. H., Le Goues, C., Martins, R., & Hellendoorn, V. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3623342 · Cited by 144
A Quantitative and Qualitative Evaluation of LLM-Based Explainable Fault Localization
Kang, S., An, G., & Yoo, S. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3660771 · Cited by 128
Code-Aware Prompting: A Study of Coverage-Guided Test Generation in Regression Setting using LLM
Ryan, G., Jain, S., Shang, M., Wang, S., Ma, X., Ramanathan, M. K., & Ray, B. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3643769 · Cited by 180
An Empirical Study on Fine-Tuning Large Language Models of Code for Automated Program Repair
Huang, K., Meng, X., Zhang, J., Liu, Y., Wang, W., Li, S., & Zhang, Y. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00181 · Cited by 113
Metadata
- Authors
- Widyasari, R., Sim, S. Q., Lok, C., Qi, H., Phan, J., Tay, Q., Tan, C., Wee, F., Tan, J. E., Yieh, Y., Goh, B., Thung, F., Kang, H. J., Hoang, T., Lo, D., & Ouh, E. L.
- Affiliations
- Singapore Management University (SG)
- Venue
- Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2020-11-08
- Pages
- 1556-1560
- OpenAlex
- W3105943882
- Cited by
- 150
- References
- 17
- Citations by year
- 2026: 36 · 2025: 44 · 2024: 29 · 2023: 20 · 2022: 12 · 2021: 9
- FWCI
- 6.80
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Singapore Management University
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- Python, software defects, debugging tools, software testing, programming language popularity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
