Tools to support systematic reviews in software engineering: a feature analysis
Abstract
Background The labour intensive and error prone nature of the systematic review process has led to the development and use of a range of tools to provide automated support.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many SLR tools have undergone significant development and advancements in the last few years.
“Comparing our results with the previous studies in the literature (Marshall et al. 2014; Van der Mierden et al. 2019; Harrison et al. 2020; Cowie et al. 2022), we observed that many SLR tools have undergone significant development and advancements in the last few years.”
From Artificial intelligence for literature reviews: opportunities and challenges (2024) · cites this paper as Marshall et al. 2014
Cites
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · see citing works
Self-citationBrereton, P., Kitchenham, B. A., Budgen, D., Turner, M., & Khalil, M. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2006.07.009 · Cited by 2,394 · see citing works
Self-citation(2004) · doi:10.5555/998675.999432 · Cited by 656 · see citing works
Self-citationUsing mapping studies as the basis for further research – A participant-observer case study
Kitchenham, B. A., Budgen, D., & Brereton, O. P. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.011 · Cited by 639 · see citing works
Self-citationA Visual Text Mining approach for Systematic Reviews
Malheiros, V., Höhn, E. N., de Pinho, R., Mendonça, M., & Maldonado, J. C. (2007) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2007.21 · Cited by 334 · see citing works
Experiences using systematic review guidelines
Staples, M., & Niazi, M. (2006) · Journal of Systems and Software · doi:10.1016/j.jss.2006.09.046 · Cited by 313 · see citing works
DESMET: a methodology for evaluating software engineering methods and tools
Kitchenham, B. A., Linkman, S., & Law, D. (1997) · Computing & Control Engineering Journal · doi:10.1049/cce:19970304 · Cited by 247 · see citing works
Self-citationDeveloping search strategies for detecting relevant experiments
Dieste, O., Grimán, A., & Juristo, N. (2009) · Empirical Software Engineering · doi:10.1007/s10664-008-9091-7 · Cited by 200 · see citing works
Using GQM and TAM to evaluate StArt – a tool that supports Systematic Review
Hernandes, E. C. M., Zamboni, A., Fabbri, S., & Thommazo, A. D. (2012) · CLEI electronic journal · doi:10.19153/cleiej.15.1.2 · Cited by 146 · see citing works
Babar, M. A., & Zhang, H. (2009) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5314235 · Cited by 97 · see citing works
Identifying Barriers to the Systematic Literature Review Process
Carver, J. C., Hassler, E. E., Hernandes, E. C. M., & Kraft, N. A. (2013) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2013.28 · Cited by 95 · see citing works
Tools to Support Systematic Literature Reviews in Software Engineering: A Mapping Study
Marshall, C., & Brereton, P. (2013) · 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2013.32 · Cited by 87 · see citing works
Self-citationExperiences Conducting Systematic Reviews from Novices’ Perspective
Riaz, M., Sulayman, M., Salleh, N., & Mendes, E. (2010) · Electronic workshops in computing · doi:10.14236/ewic/ease2010.6 · Cited by 55 · see citing works
Search Engine Overlaps : Do they agree or disagree?
Bailey, J., Zhang, C., Budgen, D., Turner, M., & Charters, S. (2007) · doi:10.1109/rebse.2007.4 · Cited by 44 · see citing works
SLR-TOOL - A Tool for Performing Systematic Literature Reviews
Fernández-Sáez, A. M., Bocco, M. G., & Romero, F. P. (2010) · International Conference on Software and Data Technologies · doi:10.5220/0003003601570166 · Cited by 42 · see citing works
Bowes, D., Hall, T., & Beecham, S. (2012) · International Workshop on Evidential Assessment of Software Technologies (EAST) · doi:10.1145/2372233.2372243 · Cited by 39 · see citing works
Evaluating software engineering methods and tools, part 7
Kitchenham, B. A. (1997) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/263244.263251 · Cited by 19 · see citing works
Self-citationA Preliminary Evaluation of Software Inspection Tools,with the DESMET Method
Hedberg, H., & Lappalainen, J. (2006) · Proceedings of the ... International Conference on Quality Software/Proceedings · doi:10.1109/qsic.2005.7 · Cited by 18 · see citing works
Feature analysis for architectural evaluation methods
Grimán, A., Pérez, M. E. D. M., Mendoza, L. E., & Losavio, F. (2006) · Journal of Systems and Software · doi:10.1016/j.jss.2005.12.015 · Cited by 17 · see citing works
Evaluating SW Eng. methods and tools, part 8
Kitchenham, B. A., & Jones, L. R. (1997) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/270849.270851 · Cited by 10 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Harrison, H., Griffin, S. J., Kuhn, I., & Usher-Smith, J. A. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-0897-3 · Cited by 449 · see citing works
Improvements in the StArt tool to better support the systematic review process
Fabbri, S., Silva, C. C., Hernandes, E. C. M., Octaviano, F., Thommazo, A. D., & Belgamo, A. (2016) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/2915970.2916013 · Cited by 240 · see citing works
Artificial intelligence for literature reviews: opportunities and challenges
Bolaños, F., Salatino, A., Osborne, F., & Motta, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10902-3 · Cited by 246 · see citing works
Goal-oriented requirements engineering: an extended systematic mapping study
Horkoff, J., Aydemir, F. B., Cardoso, E. C. S., Li, T., Maté, A., Paja, E., Salnitri, M., Piras, L., Mylopoulos, J., & Giorgini, P. (2017) · Requirements Engineering · doi:10.1007/s00766-017-0280-z · Cited by 212 · see citing works
On the pragmatic design of literature studies in software engineering: an experience-based guideline
Kuhrmann, M., Méndez, D., & Daneva, M. (2017) · Empirical Software Engineering · doi:10.1007/s10664-016-9492-y · Cited by 161 · see citing works
Software Tools for Literature Screening in Systematic Reviews in Biomedical Research
van der Mierden, S. (2019) · ALTEX · doi:10.14573/altex.1902131 · Cited by 99 · see citing works
Cierco Jimenez, R., Lee, T., Rosillo, N., Cordova, R., Cree, I. A., Gonzalez, A., & Indave Ruiz, B. I. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01805-4 · Cited by 85 · see citing works
Finding better active learners for faster literature reviews
Yu, Z., Kraft, N. A., & Menzies, T. (2018) · Empirical Software Engineering · doi:10.1007/s10664-017-9587-0 · Cited by 83 · see citing works
A Machine Learning Approach for Semi-Automated Search and Selection in Literature Studies
Ros, R., Bjarnason, E., & Runeson, P. (2017) · Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/3084226.3084243 · Cited by 80 · see citing works
Text-Mining Techniques and Tools for Systematic Literature Reviews: A Systematic Literature Review
Feng, L., Chiam, Y. K., & Lo, S. K. (2017) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec.2017.10 · Cited by 75 · see citing works
Cowie, K., Rahmatullah, A., Hardy, N., Holub, K., & Kallmes, K. (2022) · JMIR Medical Informatics · doi:10.2196/33219 · Cited by 68 · see citing works
Digitalising the Systematic Literature Review process: the MySLR platform
Ammirato, S., Felicetti, A. M., Rogano, D., Linzalone, R., & Corvello, V. (2022) · Knowledge Management Research & Practice · doi:10.1080/14778238.2022.2041375 · Cited by 64 · see citing works
Marshall, C., Brereton, P., & Kitchenham, B. (2015) · Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2745802.2745827 · Cited by 46 · see citing works
Self-citationIdentification and prioritization of SLR search tool requirements: an SLR and a survey
Al-Zubidy, A., & Carver, J. C. (2018) · Empirical Software Engineering · doi:10.1007/s10664-018-9626-5 · Cited by 49 · see citing works
Olorisade, B. K., de Quincey, E., Brereton, P., & Andras, P. (2016) · Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2915970.2915982 · Cited by 42 · see citing works
Automating Systematic Literature Review
Felizardo, K. R., & Carver, J. C. (2020) · Springer eBooks · doi:10.1007/978-3-030-32489-6_12 · Cited by 41 · see citing works
Identification of SLR tool needs – results of a community workshop
Hassler, E., Carver, J. C., Hale, D., & Al-Zubidy, A. (2016) · Information and Software Technology · doi:10.1016/j.infsof.2015.10.011 · Cited by 35 · see citing works
Reproducibility and replicability of software defect prediction studies
Mahmood, Z., Bowes, D., Hall, T., Lane, P. C. R., & Petrić, J. (2018) · Information and Software Technology · doi:10.1016/j.infsof.2018.02.003 · Cited by 29 · see citing works
CERSE - Catalog for empirical research in software engineering: A Systematic mapping study
Molléri, J. S., Petersen, K., & Mendes, E. (2018) · Information and Software Technology · doi:10.1016/j.infsof.2018.08.008 · Cited by 26 · see citing works
Felizardo, K. R., Lima, M. H. T. D. F., Deizepe, A., Conte, T. U., & Steinmacher, I. (2024) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3674805.3686666 · Cited by 29 · see citing works
Vision for SLR tooling infrastructure: Prioritizing value-added requirements
Al-Zubidy, A., Carver, J. C., Hale, D. P., & Hassler, E. E. (2017) · Information and Software Technology · doi:10.1016/j.infsof.2017.06.007 · Cited by 27 · see citing works
Analysis of the Tools to Support Systematic Literature Review in Software Engineering
Stefanović, D., Havzi, S., Nikolić, D., Dakić, D., & Lolić, T. (2021) · IOP Conference Series Materials Science and Engineering · doi:10.1088/1757-899x/1163/1/012013 · Cited by 27 · see citing works
Saraiva, J. D. A. G., de França, M. S., Soares, S., Castor, F., & de Souza, R. M. C. R. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2015.01.014 · Cited by 24 · see citing works
Analysis of the Tools for Static Code Analysis
Nikolić, D., Stefanović, D., Dakić, D., Sladojević, S., & Ristić, S. (2021) · International Symposium INFOTEH-JAHORINA (INFOTEH) · doi:10.1109/infoteh51037.2021.9400688 · Cited by 19 · see citing works
Exploring Automatic Search in Digital Libraries
Singh, P., & Singh, K. (2017) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3084226.3084275 · Cited by 18 · see citing works
Major, L., Kyriacou, T., & Pearl Brereton, A. (2014) · Computer Science Education · doi:10.1080/08993408.2014.963362 · Cited by 17 · see citing works
An SLR-tool: search process in practice
Hinderks, A., Mayo, F. J. D., Thomaschewski, J., & Escalona, M. J. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377812.3382137 · Cited by 15 · see citing works
Tool Features to Support Systematic Reviews in Software Engineering - A Cross Domain Study
Marshall, C., Kitchenham, B. A., & Pearl Brereton, A. (2018) · Keele Research Repository (Keele University) · doi:10.5277/e-inf180104 · Cited by 14 · see citing works
Self-citationUsing the SCAS strategy to perform the initial selection of studies in systematic reviews
Octaviano, F., Silva, C. C., & Fabbri, S. (2016) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/2915970.2916000 · Cited by 12 · see citing works
Using Q&A Websites as a Method for Assessing Systematic Reviews
Cartaxo, B., Pinto, G., Ribeiro, D., Kamei, F., Santos, R. E. S., da Silva, F. Q. B., & Soares, S. (2017) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2017.5 · Cited by 11 · see citing works
A Design Thinking Techniques Recommendation Tool: An Initial and On-going Proposal
Parizi, R., Moreira, M., Couto, I., Marczak, S. D. S., & Conte, T. U. (2020) · Brazilian Symposium on Software Quality (SBQS) · doi:10.1145/3439961.3439997 · Cited by 11 · see citing works
Gonzalez-Toral, S., Freire, R., Gualán, R., & Saquicela, V. (2019) · Latin American Computer Conference (CLEI) · doi:10.1109/clei47609.2019.235079 · Cited by 11 · see citing works
Supporting systematic literature reviews using deep-learning-based language models
Alchokr, R., Borkar, M., Thotadarya, S., Saake, G., & Leich, T. (2022) · IEEE/ACM International Workshop on Natural Language-Based Software Engineering (NLBSE) · doi:10.1145/3528588.3528658 · Cited by 10 · see citing works
Proposal and Evaluation of a Collaborative IS to Support Systematic Reviews and Mapping Studies
de Campos, T. P., Damasceno, E. F., & Valentim, N. (2022) · Anais do Simpósio Brasileiro de Sistemas de Informação (SBSI) · doi:10.1145/3535511.3535531 · Cited by 10 · see citing works
Prehospital Care Post-Road-Crash: A Systematic Review of the Literature
Cuthbertson, J. L., & Drummond, G. (2025) · Prehospital and Disaster Medicine · doi:10.1017/s1049023x25000202 · Cited by 9 · see citing works
Designing service-based applications in the presence of non-functional properties: A mapping study
Neto, P. A. S., Solar, G. V., da Costa, U. S., & Musicante, M. A. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2015.09.004 · Cited by 9 · see citing works
How to Read Less: Better Machine Assisted Reading Methods for Systematic Literature Reviews.Preprint
Yu, Z., Kraft, N. A., & Menzies, T. (2016) · arXiv (Cornell University) · Cited by 9 · see citing works
A Workflow-Based Methodological Framework for Hybrid Human-AI Enabled Scientometrics
Correia, A., Jameel, S., Schneider, D., Paredes, H., & Fonseca, B. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378096 · Cited by 8 · see citing works
Peftitsis, D., & Mavroudi, A. (2022) · IEEE Access · doi:10.1109/access.2022.3183584 · Cited by 9 · see citing works
Adopting Digital-Oriented Work Practices That Facilitate Work Satisfaction
Jafari, P., & Looy, A. V. (2022) · IEEE Access · doi:10.1109/access.2022.3192853 · Cited by 6 · see citing works
Correia, A., Paredes, H., & Fonseca, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99504-5_5 · Cited by 6 · see citing works
Zhang, L., Hu, C., & Quan, Z. (2024) · North American Actuarial Journal · doi:10.1080/10920277.2024.2416903 · Cited by 5 · see citing works
Testing in Blockchain-based Systems: A Systematic Review
Arsat, N., Bakar, N. S. A. A., & Yahya, N. (2022) · International Conference on Cyber and IT Service Management (CITSM) · doi:10.1109/citsm56380.2022.9935846 · Cited by 5 · see citing works
Mapping the landscape of functionality reviews in HCI: trends, methods, and contributions
Hossain, E., Ahmetoglu, Y., Chen, S., Dowthwaite, A., Lascău, L., Mella, J., Stern, N., & Cox, A. L. (2025) · Interacting with Computers · doi:10.1093/iwc/iwaf027 · Cited by 4 · see citing works
Alves, L. F., Vasconcellos, F. J. S., & Nogueira, B. M. (2022) · Empirical Software Engineering · doi:10.1007/s10664-021-10084-4 · Cited by 4 · see citing works
Tell, P., Cholewa, J. B., Nellemann, P., & Kuhrmann, M. (2016) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/2915970.2916011 · Cited by 4 · see citing works
Managing the State of the Art of Engineering: Learning from Medicine
Silva, É. R., Bartholo, R., & Júnior, D. P. (2018) · Philosophy of engineering and technology · doi:10.1007/978-3-319-91029-1_15 · Cited by 3 · see citing works
Evidence-Based Systematic Literature Reviews in the Cloud
Ruiz‐Rube, I., Person, T., Mota, J. M., Dodero, J. M., & González-Toro, Á. R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-03496-2_12 · Cited by 2 · see citing works
Integrating user experience assessment in Re-CRUD console framework development
Anuar, A. W., Azmi, A., Kama, N., Rusli, H. M., Bakar, N. A. A., & Mohamed, N. (2022) · Wireless Networks · doi:10.1007/s11276-022-03098-3 · Cited by 2 · see citing works
Rainer, A., Williams, A. H., Garousi, V., & Felderer, M. (2020) · IET Software · doi:10.1049/iet-sen.2020.0109 · Cited by 2 · see citing works
A Sustainable Data Extraction Documentation Process for Literature Studies
Kaplan, A., Konersmann, M., Kühn, T., Mirandola, R., & Reussner, R. (2026) · IEEE/ACM International Workshop on Methodological Issues with Empirical Studies in Software Engineering (WSESE) · doi:10.1145/3786149.3788301 · Cited by 1 · see citing works
The Diffusion of News Applying Sentiment Analysis and Impact on Human Behavior Through Social Media
Peñafiel, M., Navarrete, R., Tenemaza, M., Vásquez-Peñafiel, M.-S., Vasquéz, D., & Luján‐Mora, S. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-20454-9_25 · Cited by 1 · see citing works
Automation Tool for the Planning Phase of the Systematic Review
Zumaeta, R., Cuayla, E., & Mauricio, D. (2019) · International Engineering, Sciences and Technology Conference (IESTEC) · doi:10.1109/iestec46403.2019.00-62 · Cited by 1 · see citing works
Teaching Systematic Literature Reviews: Strategies and Best Practices
Genero, M., & Piattini, M. G. (2024) · Springer eBooks · doi:10.1007/978-3-031-71769-7_20
- Authors
- Marshall, C., Brereton, P., & Kitchenham, B.
- Affiliations
- Keele University (GB)
- Keele University, Staffordshire UK
- ORCID
- Kitchenham, Barbara
- Venue
- Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering
- Publisher
- ACM
- Published
- 2014-05-13
- Pages
- 1-10
- OpenAlex
- W2043634247
- Cited by
- 56
- References
- 20
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 4 · 2022: 10 · 2021: 2 · 2020: 6 · 2019: 4 · 2018: 8 · 2017: 7 · 2016: 7 · 2015: 4 · 2014: 1
- FWCI
- 2.51
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Conference on Evaluation and Assessment in Software Engineering (EASE) (publishedVersion)
- Funders
- Keele University
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- systematic reviews, software engineering, review tools, automated support, literature searching, software quality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
