The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Variability in Software Systems—A Systematic Literature Review
Abstract
Context: Variability (i.e., the ability of software systems or artifacts to be adjusted for different contexts) became a key property of many systems. Objective: We analyze existing research on variability in software systems. We investigate variability handling in major software engineering phases (e.g., requirements engineering, architecting). Method: We performed a systematic literature review. A manual search covered 13 premium software engineering journals and 18 premium conferences, resulting in 15,430 papers searched and 196 papers considered for analysis. To improve reliability and to increase reproducibility, we complemented the manual search with a targeted automated search. Results: Software quality attributes have not received much attention in the context of variability. Variability is studied in all software engineering phases, but testing is underrepresented. Data to motivate the applicability of current approaches are often insufficient; research designs are vaguely described. Conclusions: Based on our findings we propose dimensions of variability in software engineering. This empirically grounded classification provides a step towards a unifying, integrated perspective of variability in software systems, spanning across disparate or loosely coupled research themes in the software engineering community. Finally, we provide recommendations to bridge the gap between research and practice and point to opportunities for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2004) · Cited by 5,198
Software Architecture in Practice
Bass, L., Clements, P., & Kazman, R. (1997) · Cited by 4,995
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · cited 2×
“…and lessons learnt (e.g., Staples and Niazi [6], Biolchini et al. [7], Riaz et al. [8], Zhang and Babar [9], Brereton et al. [10]) as well as on meta-studies of systematic reviews (Kitchenham et al. [11], [12]) when designing and conducting our review and when developing our search strategy.”
Empirical studies of agile software development: A systematic review
Dybå, T., & Dingsøyr, T. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.01.006 · Cited by 2,668 · cited 1×
“[29], we adopted the quality assessment instrument used in [41].”
Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M., & Khalil, M. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2006.07.009 · Cited by 2,394
Software Product Line Engineering
Pohl, K., Böckle, G., & van der Linden, F. (2005) · Springer eBooks · doi:10.1007/3-540-28901-1 · Cited by 2,358
Service-Oriented Computing: State of the Art and Research Challenges
Papazoglou, M. P., Traverso, P., Dustdar, S., & Leymann, F. (2007) · Computer · doi:10.1109/mc.2007.400 · Cited by 1,950
Systematic literature reviews in software engineering – A tertiary study
Kitchenham, B., Pretorius, R., Budgen, D., Pearl Brereton, O., Turner, M., Niazi, M., & Linkman, S. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.03.006 · Cited by 1,230 · cited 1×
“…and lessons learnt (e.g., Staples and Niazi [6], Biolchini et al. [7], Riaz et al. [8], Zhang and Babar [9], Brereton et al. [10]) as well as on meta-studies of systematic reviews (Kitchenham et al. [11], [12]) when designing and conducting our review and when developing our search strategy.”
Automated analysis of feature models 20 years later: A literature review
Benavides, D., Segura, S., & Ruiz-Cortés, A. (2010) · Information Systems · doi:10.1016/j.is.2010.01.001 · Cited by 1,204
Software Engineering for Self-Adaptive Systems: A Research Roadmap
Cheng, B. H. C., de Lemos, R., Giese, H., Inverardi, P., Magee, J., Andersson, J., Becker, B., Bencomo, N., Brun, Y., Cukic, B., et al. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-02161-9_1 · Cited by 1,094
Self-citationA systematic review of software development cost estimation studies
Jørgensen, M., & Shepperd, M. (2007) · Brunel University Research Archive (BURA) (Brunel University London) · doi:10.1109/tse.2007.3 · Cited by 742
A Systematic Review of Software Development Cost Estimation Studies
Jorgensen, M., & Shepperd, M. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256943 · Cited by 729
Systematic Review in Software Engineering
Biolchini, J. C. D. A., & Mian, P. G. (2005) · Cited by 561
On the notion of variability in software product lines
van Gurp, J., Bosch, J., & Svahnberg, M. (2002) · Working IEEE/IFIP Conference on Software Architecture (WICSA) · doi:10.1109/wicsa.2001.948406 · Cited by 516
A taxonomy of variability realization techniques
Svahnberg, M., van Gurp, J., & Bosch, J. (2005) · Software: Practice and Experience · doi:10.1002/spe.652 · Cited by 430
Dynamic Software Product Lines
Hallsteinsen, S. O., Hinchey, M., Park, S., & Schmid, K. (2008) · Computer · doi:10.1109/mc.2008.123 · Cited by 398
Applying Systematic Reviews to Diverse Study Types: An Experience Report
Dyba, T., Dingsoyr, T., & Hanssen, G. K. (2007) · First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007) · doi:10.1109/esem.2007.59 · Cited by 391
A Visual Text Mining approach for Systematic Reviews
Malheiros, V., Hohn, E., Pinho, R., Mendonca, M., & Maldonado, J. C. (2007) · First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007) · doi:10.1109/esem.2007.21 · Cited by 329
Experiences using systematic review guidelines
Staples, M., & Niazi, M. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2006.09.046 · Cited by 313
Software product line testing – A systematic mapping study
Engström, E., & Runeson, P. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.05.011 · Cited by 307
A method for evaluating rigor and industrial relevance of technology evaluations
Ivarsson, M., & Gorschek, T. (2011) · Empirical Software Engineering · doi:10.1007/s10664-010-9146-4 · Cited by 279
A systematic review of evaluation of variability management approaches in software product lines
Chen, L., & Ali Babar, M. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.006 · Cited by 247
Variability management in software product lines: a systematic review
Chen, L., Babar, M. A., & Alí, N. (2009) · Arrow@dit (Dublin Institute of Technology) · doi:10.5555/1753235.1753247 · Cited by 231
A systematic mapping study of software product lines testing
da Mota Silveira Neto, P. A., Carmo Machado, I. D., McGregor, J. D., de Almeida, E. S., & de Lemos Meira, S. R. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.003 · Cited by 218
Requirements engineering for software product lines: A systematic literature review
Alves, V., Niu, N., Alves, C., & Valença, G. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.03.014 · Cited by 193
Modeling Dimensions of Self-Adaptive Software Systems
Andersson, J., de Lemos, R., Malek, S., & Weyns, D. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-02161-9_2 · Cited by 180
Self-citationCharacterizing software architecture changes: A systematic review
Williams, B. J., & Carver, J. C. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.07.002 · Cited by 158
On the success of empirical studies in the international conference on software engineering
Zannier, C., Melnik, G., & Maurer, F. (2006) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1134285.1134333 · Cited by 124
Refining the systematic literature review process—two participant-observer case studies
Kitchenham, B. A., Pearl Brereton, A., Turner, M., Niazi, M., Linkman, S., Pretorius, R., & Budgen, D. (2010) · Empirical Software Engineering · doi:10.1007/s10664-010-9134-8 · Cited by 120
Rabiser, R., Grünbacher, P., & Dhungana, D. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.11.001 · Cited by 109
Variability in quality attributes of service-based software systems: A systematic literature review
Mahdavi-Hezavehi, S., Galster, M., & Avgeriou, P. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2012.08.010 · Cited by 95
Self-citationA systematic review of comparative evidence of aspect-oriented programming
Ali, M. S., Ali Babar, M., Chen, L., & Stol, K.-J. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.05.003 · Cited by 89
On Searching Relevant Studies in Software Engineering
Zhang, H., & Babar, M. A. (2010) · Electronic workshops in computing · doi:10.14236/ewic/ease2010.14 · Cited by 75
Managing Variability in Software Product Lines
Babar, M. A., Chen, L., & Shull, F. J. (2010) · IEEE Software · doi:10.1109/ms.2010.77 · Cited by 66
Kitchenham, B. A., Pearl Brereton, A., Turner, M., Niazi, M., Linkman, S., Pretorius, R., & Budgen, D. (2009) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5314238 · Cited by 64
Experiences Conducting Systematic Reviews from Novices’ Perspective
Riaz, M., Sulayman, M., Salleh, N., & Mendes, E. (2010) · Electronic workshops in computing · doi:10.14236/ewic/ease2010.6 · Cited by 56
Requirements and Tools for Variability Management
Aiello, M., Bulanov, P., & Groefsema, H. (2010) · IEEE Annual Computer Software and Applications Conference Workshops (COMPSACW) · doi:10.1109/compsacw.2010.50 · Cited by 46
Lessons Learnt Undertaking a Large-scale Systematic Literature Review
Turner, M., Kitchenham, B. A., Budgen, D., & Pearl Brereton, A. (2008) · Electronic workshops in computing · doi:10.14236/ewic/ease2008.12 · Cited by 46
A Preliminary Review on the Application of Feature Diagrams in Practice
Hubaux, A., Claßen, A., Mendonça, M., & Heymans, P. (2010) · Repository of the University of Namur · Cited by 45
An Evaluation of Quality Checklist Proposals - A participant-observer case study
Kitchenham, B. A., Brereton, O. P., Budgen, D., & Li, Z. (2009) · Electronic workshops in computing · doi:10.14236/ewic/ease2009.7 · Cited by 30
Computer Software and Applications Conference Workshops (COMPSACW)
Aiello, M., Bulanov, P., & Groefsema, H. (2010) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 29
Evaluation of Quality Attribute Variability in Software Product Families
Etxeberria, L., & Sagardui, G. (2008) · IEEE International Conference and Workshop on the Engineering of Computer-Based Systems (ECBS) · doi:10.1109/ecbs.2008.14 · Cited by 26
Handling Variability in Software Architecture: Problems and Implications
Galster, M., & Avgeriou, P. (2011) · Working IEEE/IFIP Conference on Software Architecture (WICSA) · doi:10.1109/wicsa.2011.30 · Cited by 24
Self-citationA Status Report on the Evaluation of Variability Management Approaches
Chen, L., Babar, M. A., & Cawley, C. (2009) · Electronic workshops in computing · doi:10.14236/ewic/ease2009.14 · Cited by 23
An integrity constraint for database systems containing embedded static neural networks
Millns, I., & Eaglestone, B. (2001) · International Journal of Intelligent Systems · doi:10.1002/1098-111x(200103)16:3<391::aid-int1014>3.0.co;2-2 · Cited by 2
Cited by
A systematic mapping study on technical debt and its management
Li, Z., Avgeriou, P., & Liang, P. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2014.12.027 · Cited by 743
Self-citationAmpatzoglou, A., Bibi, S., Avgeriou, P., Verbeek, M., & Chatzigeorgiou, A. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2018.10.006 · Cited by 305
Self-citationExploring the ambient assisted living domain: a systematic review
Calvaresi, D., Cesarini, D., Sernani, P., Marinoni, M., Dragoni, A. F., & Sturm, A. (2017) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-016-0374-3 · Cited by 268
Content-Based Fake News Detection With Machine and Deep Learning: a Systematic Review
Capuano, N., Fenza, G., Loia, V., & Nota, F. D. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.02.005 · Cited by 183
Metadata
- Authors
- Galster, M., Weyns, D., Tofan, D., Michalik, B., & Avgeriou, P.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2014-03
- Volume
- 40
- Issue
- 3
- Pages
- 282-306
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- OpenAlex
- W2139380319
- Cited by
- 259
- References
- 46
- FWCI
- 9.69
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.