Model-based testing for software safety: a systematic mapping study
Open accessThe DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
Abstract
Testing safety-critical systems is crucial since a failure or malfunction may result in death or serious injuries to people, equipment, or environment. An important challenge in testing is the derivation of test cases that can identify the potential faults. Model-based testing adopts models of a system under test and/or its environment to derive test artifacts. This paper aims to provide a systematic mapping study to identify, analyze, and describe the state-of-the-art advances in model-based testing for software safety. The systematic mapping study is conducted as a multi-phase study selection process using the published literature in major software engineering journals and conference proceedings. We reviewed 751 papers and 36 of them have been selected as primary studies to answer our research questions. Based on the analysis of the data extraction process, we discuss the primary trends and approaches and present the identified obstacles. This study shows that model-based testing can provide important benefits for software safety testing. Several solution directions have been identified, but further research is critical for reliable model-based testing approach for safety.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Therefore, we constructed a review protocol before conducting the review, based on the guidelines from Kitchenham and Charters [4], Ali et al. [31], and Gurbuz and Tekinerdogan [5].”
From Automation of systematic literature reviews: A systematic literature review (2021) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Grading quality of evidence and strength of recommendations
(2004) · BMJ · doi:10.1136/bmj.328.7454.1490 · Cited by 8,477 · cited 4×
“In this work, we used the definitions from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) (Atkins et al. 2004) working group which is developed for grading the quality of evidence and strength of recommendations.”
MethodInfluentialExperimentation in Software Engineering
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2012) · doi:10.1007/978-3-642-29044-2 · Cited by 4,460 · cited 2×
“We consider the categorization in Wohlin et al. (2012) to extract research method during the review process.”
BackgroundMethodGuidelines for conducting systematic mapping studies in software engineering: An update
Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2015.03.007 · Cited by 2,669 · cited 2×
“We carry out the SMS to provide an overview of existing research directions regarding the MBT for software safety by following the guidelines and process proposed by (Petersen et al. 2015).”
MethodJensen, K. (1987) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-47919-2_10 · Cited by 1,744
Experimentation in software engineering
(2013) · Choice Reviews Online · doi:10.5860/choice.50-5018 · Cited by 1,517
The guide to the Software Engineering Body of Knowledge
Bourque, P., Dupuis, R., Abran, A., Moore, J. W., & Tripp, L. (1999) · IEEE Software · doi:10.1109/52.805471 · Cited by 1,083
A taxonomy of model‐based testing approaches
Utting, M., Pretschner, A., & Legeard, B. (2012) · Software Testing, Verification and Reliability · doi:10.1002/stvr.456 · Cited by 632
Identifying relevant studies in software engineering
Zhang, H., Babar, M. A., & Tell, P. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.010 · Cited by 557
Introduction to automata theory, languages, and computation, 2nd edition
Hopcroft, J. E., Motwani, R., & Ullman, J. D. (2001) · ACM SIGACT News · doi:10.1145/568438.568455 · Cited by 510
Guide to the Software Engineering Body of Knowledge : 2004 Version
Abran, A., Moore, J. W., Bourque, P., Dupuis, R., & Tripp, L. L. (2005) · Medical Entomology and Zoology · Cited by 375
Cited by
Predictive maintenance using digital twins: A systematic literature review
van Dinter, R., Tekinerdogan, B., & Catal, C. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.107008 · Cited by 540
“While developing the quality assessment criteria, the summary quality checklist for quantitative and qualitative studies has been adopted, as proposed by [7] and [36].”
MethodAutomation of systematic literature reviews: A systematic literature review
van Dinter, R., Tekinerdogan, B., & Catal, C. (2021) · Information and Software Technology · doi:10.1016/j.infsof.2021.106589 · Cited by 432
“The success of SLR in evidence-based medicine triggered various other research areas to adopt similar SLR approaches [5].”
BackgroundMethodInfluentialObstacles and features of Farm Management Information Systems: A systematic literature review
Tummers, J., Kassahun, A., & Tekinerdoğan, B. (2019) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2018.12.044 · Cited by 153
Digital twin for ship life-cycle: A critical systematic review
Mauro, F., & Kana, A. A. (2022) · Ocean Engineering · doi:10.1016/j.oceaneng.2022.113479 · Cited by 158
“To this end, the quality assessment proposed by Kietchenham and Charters (2007) and Gurbuz and Tekinerdogan (2018) is a suitable example, requiring answering the following quality index (QI): QI-1 Are the aims of the study clearly stated?”
MethodWomen’s Participation in Open Source Software: A Survey of the Literature
Trinkenreich, B., Wiese, I., Sarma, A., Gerosa, M., & Steinmacher, I. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3510460 · Cited by 100
“The GRADE approach is widely used by secondary studies in software engineering to evaluate the strength of evidence found in the primary studies [1, 2, 30, 47, 48, 96].”
Method
Metadata
- Authors
- Gurbuz, H. G., & Tekinerdogan, B.
- ORCID
- Gurbuz, Havva Gulay
- Venue
- Software Quality Journal
- Published
- 2017-09-25
- Volume
- 26
- Issue
- 4
- Pages
- 1327-1372
- ISSN
- 0963-9314 (ISSN-L)
- 1573-1367
- OpenAlex
- W2759114566
- Cited by
- 78
- References
- 64
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 8 · 2023: 12 · 2022: 14 · 2021: 13 · 2020: 5 · 2019: 6 · 2018: 4
- FWCI
- 13.60
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- Locations
- Software Quality Journal (publishedVersion, cc-by)
- Socio-Environmental Systems Modeling (acceptedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- Funders
- Wageningen University
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Physical Sciences › Computer Science › Software › Model-Driven Software Engineering Techniques
- Keywords
- model-based testing, software safety, safety-critical systems, test case derivation, fault detection, software testing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.