When and what to automate in software testing? A multi-vocal literature review

Garousi, Vahid, Mäntylä, Mika V.·2016·Information and Software Technology
DOI 10.1016/j.infsof.2016.04.015

Cited by (181)

Items 101–181 of 181

  1. An Industrial Case-Study on GUI Testing With RPA
    Mats Holmberg, Felix Dobslaw · 2022 · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)
  2. PLATEM: a method for mobile applications testing
    Euler Horta Marinho, Rodolfo Ferreira Resende · 2017 · IET Software
  3. Estimating Return on Investment for GUI Test Automation Tools
    Felix Dobslaw, ROBERT H. FELDT, David Michaelsson, Haar, Patrick, Francisco Gomes de Oliveira Neto, Richard Torkar · 2019 · arXiv (Cornell University)
  4. Data catalog tools: A systematic multivocal literature review
    Marco Tonnarelli, Indika Kumara, Stefan W. Driessen, Damian A. Tamburri, Willem‐Jan van den Heuvel, Patrick Oor · 2025 · Journal of Systems and Software
  5. What We Know about AIBOMs: Results from a Multivocal Literature Review on Artificial Intelligence Bill of Materials
    Sabato Nocera, M. D. Di Penta, Fatima Ahmed, Simone Romano, Giuseppe Scanniello · 2025 · ACM Transactions on Software Engineering and Methodology
  6. Artificial intelligence for context-aware visual change detection in software test automation
    Milad Moradi, Ke Yan, David Colwell, Rhona Asgari · 2025 · Progress in Artificial Intelligence
  7. Extraction of test cases procedures from textual use cases to reduce test effort
    Erick Barros dos Santos, Lucas Simão da Costa, Bruno Sabóia Aragão, Ismayle de Sousa Santos, Rossana M. C. Andrade · 2019 · Brazilian Symposium on Software Quality (SBQS)
  8. Cracks in the Foundation: Issues with Diversity and the Hiring Process in Computing Fields
    Stephanie Jill Lunn, Monique S. Ross · 2021 · ASEE Annual Conference & Exposition
  9. A Grey Literature Review on Data Stream Processing applications testing
    Alexandre Vianna, Fernando Kamei, Kiev Santos da Gama, Carlos Zimmerle, João Alexandre Neto · 2023 · Journal of Systems and Software
  10. Towards Automating Integration Testing of .NET Applications Using Roslyn
    Mehrdad Saadatmand · 2017 · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C)
  11. Transitioning from Manual to Automated Software Regression Testing: Experience from the Banking Domain
    Abdurrahman Akin, Şerafettin Şentürk, Vahid Garousi · 2018 · Asia-Pacific Software Engineering Conference (APSEC)
  12. Finding high-quality grey literature for use as evidence in software engineering research.
    Ashley H. Williams · 2019 · UC Research Repository (University of Canterbury)
  13. Coping with Verification in Complex Engineered Product Development
    Ola Skreddernes, Rune André Haugen, Cecilia Haskins · 2023 · INCOSE International Symposium
  14. Implementation of the 5IR for efficient service delivery in the public sector, South Africa
    Amos Shibambu · 2025 · Africa’s Public Service Delivery and Performance Review
  15. Test Automation with the Gauge Framework: Experience and Best Practices
    Vahid Garousi, Alper Buğra Keleş, Yunus Balaman, Zeynep Özdemir Güler · 2020 · Lecture notes in computer science
  16. The Impact of NLP on Software Testing
    Sai Deepak Reddy Konreddy · 2021 · Journal of University of Shanghai for Science and Technology
  17. A multivocal literature review on record management potential components in CRUD operation for web application development
    Asyraf Wahi Anuar, Nazri Kama, Azri Azmi, Hazlifah Mohd Rusli · 2022 · Advances in Complex Systems
  18. Classification and Evaluation Framework of Automated testing tools for agile software: Technical Review
    Mogeeb A. A. Mosleh, Nashwan Amin Al-khulaidi, Abdu Gumaei, Ayman Alsabry, Ali A. A. Musleh · 2024 · International Conference on Emerging Smart Technologies and Applications (eSmarTA)
  19. Enhancing integration testing efficiency through AI-driven combined structural and textual class coupling metric
    Iyad Alazzam, Anas Alsobeh, Basil Bani Melhem · 2024 · Online Journal of Communication and Media Technologies
  20. Evaluating a Test Automation Decision Support Tool
    Kesina Baral, Rasika Mohod, Jennifer Flamm, Seth Goldrich, Paul Ammann · 2019 · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)
  21. Experimenting with the Peanut Butter and Jelly Sandwich Challenge to Introduce Algorithmic Thinking and Test Case Writing
    Kiev Santos da Gama, Andrew Diniz da Costa, Hendi Lemos Coelho, Ricardo Almeida Venieris, Carlos J. P. de Lucena · 2019 · Brazilian Symposium on Software Engineering
  22. Overlap between Automated Unit and Acceptance Testing – a Systematic Literature Review
    Jack van Heugten Breurkes, Fabian Gilson, Matthias Galster · 2022 · International Conference on Evaluation and Assessment in Software Engineering (EASE)
  23. Quadruple Factors Interference and Its Effects on Quality of Outsourcing Testing: A Case Study
    2021 · International Journal of Advanced Trends in Computer Science and Engineering
  24. CLG: Automated checklist generation for improved pull request quality
    Shuangwei Bai, Chenkun Meng, Guodong Li, Huaxiao Liu, Lei Liu · 2025 · Expert Systems with Applications
  25. Data Quality Model-based Testing of Information Systems: the Use-case of E-scooters
    Anastasija Nikiforova, Jānis Bičevskis, Zane Bičevska, Ivo Odītis · 2020 · International Conference on Internet of Things: Systems, Management and Security (IOTSMS)
  26. Test Case Auto-Generation For Web Applications: A Model-Based Approach
    Refat Othman, Samer Zein · 2022 · International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT)
  27. Feedback in Virtual Software Development Teams
    Ana Beatriz Cavalcanti Ribeiro, Carina Frota Alves · 2023 · Iberian Conference on Information Systems and Technologies (CISTI)
  28. SoK: Automated Software Testing for TLS Libraries
    Ben Swierzy, Felix Boes, Timo Pohl, Christian Bungartz, Michael Meier · 2024 · International Conference on Availability, Reliability and Security (ARES)
  29. Developing SPIM-TA: a maturity-level framework for systematic process improvement in software testing automation
    Muhammad Faisal Abrar, Yasser Obaid Alharbi, Muhammad Alsaffar, Shah Hussain, Muhammad Saqib, Jawad Khan, Youngmoon Lee · 2025 · Ain Shams Engineering Journal
  30. Critical Influential Factors For Software Testing-as-a-Service Adoption: Preliminary Findings From Systematic Literature Review
    Sikandar Ali, Niamat Ullah · 2019 · International Conference on Electrical, Communication, and Computer Engineering (ICECCE)
  31. Recent Trends in Software Testing - A Case Study with Google Calendar.
    Б. К. Попов, Bojana Koteska, Anastas Mishev · 2019
  32. Maintainability of Automatic Acceptance Tests for Web Applications—A Case Study Comparing Two Approaches to Organizing Code of Test Cases
    Aleksander Sadaj, Mirosław Ochodek, Sylwia Kopczyńska, Jerzy R. Nawrocki · 2020 · Lecture notes in computer science
  33. COMPARISON OF JAVA PROGRAMMING TESTING TOOLS
    Shikha Gautam · 2020 · International Journal of Engineering Technologies and Management Research
  34. Retrieving and mining professional experience of software practice from grey literature: an exploratory review
    Austen Rainer, Ashley H. Williams, Vahid Garousi, Michael Felderer · 2020 · IET Software
  35. Comparative analysis of selected tools for test automation of web applications
    Michał Pojęta, Franciszek Wąsik, Małgorzata Plechawska–Wójcik · 2023 · Journal of Computer Sciences Institute
  36. Global software development agile planning model: challenges and current trends
    Hajar Lamsellak, Mohammed Ghaouth Belkasmi · 2023 · Indonesian Journal of Electrical Engineering and Computer Science
  37. PLATOOL
    Euler Horta Marinho, Eduardo Figueiredo · 2020 · Brazilian Symposium on Software Engineering
  38. Special issue on automation of software testing: improving practical applicability
    Christof J. Budnik, Gordon Fraser, Francesca Lonetti, Hong Zhu · 2018 · Software Quality Journal
  39. Visual GUI testing in practice: An extended industrial case study
    Vahid Garousi, Wasif Afzal, Adem Çağlar, İhsan Berk Işık, Berker Baydan, Seçkin Çaylak, Ahmet Zeki Boyraz, Burak Yolaçan, Kadir Herkiloğlu · 2020 · arXiv (Cornell University)
  40. AUTOMATED TESTING OF WEB PROJECT FUNCTIONALITY WITH USING OF ERROR PROPAGATION ANALYSIS
    Iryna ZASORNOVA, Mykola Fedula, Olga Pavlova, Tetiana Kysil · 2023 · Computer Systems and Information Technologies
  41. Building the Landing Zone for Site Reliability Engineering: A Multivocal Literature Review
    Luiz Alexandre Costa, Awdren Fontão, Eleni Constantinou, Rodrigo Pereira dos Santos, Alexander Serebrenik · 2025 · Simpósio Brasileiro de Engenharia de Software (SBES)
  42. Reliability of Software Correlated Components Failure in Pakistan Industry
    Darakshan Anjum, Tasleem Mustafa · 2017 · International Journal of Information Engineering and Electronic Business
  43. The Use of Grey Literature Review as Evidence for Software Engineering
    Fernando Kamei, Sérgio Soares, Gustavo Pinto · 2019 · Congresso Brasileiro de Software: Teoria e Prática (CBSoft)
  44. Evidence and perceptions on GUI test automation - An Exploratory Study
    Chahna Polepalle, Ravi Shankar Kondoju, Deepika Badampudi · 2022 · Innovations in Software Engineering Conference (ISEC)
  45. Automated Game Testing With Online Search Agent and Model Construction, a Study
    Samira Shirzadehhajimahmood, I. S. W. B. Prasetya, Davide Prandi, Frank P. M. Dignum, Mehdi M. Dastani, Gabriele Keller · 2025 · Software Testing Verification and Reliability
  46. AI-Assisted Test Script Generation for GUI Applications
    Saurabh Kapoor · 2025 · International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT)
  47. Continuous integration and delivery applied to large-scale software-intensive embedded systems
    Torvald Mårtensson · 2019 · University of Groningen research database (University of Groningen / Centre for Information Technology)
  48. What Techniques Can Be Used for GUI Risk-based Testing?
    Behzad Nazarbakhsh, Dietmar Pfahl · 2017 · International Conference on Software Technologies (ICSOFT)
  49. AutoJet: Web Application Automation Tool
    Sheetika Kapoor, Kalpna Sagar · 2018 · Springer eBooks
  50. IntegrationDistiller: Automating Integration Analysis and Testing of Object-Oriented Applications
    Mehrdad Saadatmand · 2019 · IEEE International Conference on Industrial Technology (ICIT)
  51. Towards Automated Test Case Generation Maturity
    Urko Rueda, Fitsum Meshesha Kifetew, Xavier Devroey · 2019 · IEEE/ACM International Workshop on Search-Based Software Testing (SBST)
  52. The Peanut Butter and Jelly Sandwich Challenge as an Approach to Improve Students Abilities in Test Case Writing
    Kiev Santos da Gama, Andrew Diniz, Ricardo Froitzheim Rinelli de Almeida, Hendi Lemos Coelho · 2019 · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE)
  53. Automated Office Software Test Method
    Lin Zhang, Ning Li, Xia Hou, Yingai Tian · 2020 · International Conference on Software and Computer Applications (ICSCA)
  54. Software Testing - An Important Phase of Software Development Life Cycle
    Vijayashree Shetty · 2020 · International Journal for Research in Applied Science and Engineering Technology
  55. Practitioner-generated blog posts as evidence for software engineering research: attitudinal survey and preliminary checklist
    Austen Rainer, Ashley H. Williams · 2021 · arXiv (Cornell University)
  56. Extraction of test cases procedures from textual use cases: is it worth it?
    Erick Barros dos Santos, Lucas Simão da Costa, Thaís Marinho De Amorim, Bruno Sabóia Aragão, Ismayle de Sousa Santos, Danilo Vasconcelos, Rossana M. C. Andrade · 2020 · Journal of Software Engineering Research and Development
  57. Generating Test Cases using Eclipse Environment – A Case Study of Mobile Application
    Rosziati Ibrahim, Nurul Ain, Sapiee Haji Jamel, Jahari Abdul · 2021 · International Journal of Advanced Computer Science and Applications
  58. A Scoping Review on Automated Software Testing with Special Reference to Android Based Mobile Application Testing
    Fathima Naja Musthafa, S M Hasan Mansur, Andika Wibawanto, Owais Qureshi · 2021 · International Journal on Advances in ICT for Emerging Regions (ICTer)
  59. Model-driven software architecture conformance analysis
    Burak Uzun · 2021
  60. RETRACTED ARTICLE: A Novel Framework in Software Engineering for Deep Learning
    Tao Lin, Xue Fu · 2022 · SN Computer Science · Retracted
  61. PopArt: Ranked Testing Efficiency
    Iason Papapanagiotakis-Bousy, Earl T. Barr, David Clark · 2022 · IEEE Transactions on Software Engineering
  62. Uma abordagem de teste de software para aplicações de realidade virtual utilizando testes metamórficos
    Stevão Alves de Andrade · 2022 · Digital Library of Theses and Dissertations (Universidade de São Paulo)
  63. ANALYSIS OF THE ECONOMIC EFFICIENCY OF AUTOMATED TESTING IMPLEMENTATION IN SOFTWARE DEVELOPMENT
    T. R. Masalimov, Sofiya F. Sayfullina · 2023 · Business Strategies
  64. Exploring Coexistence of Software Architecture Development and Agility through a Multivocal Literature Review
    Gülşah Kırtıloğlu, Özden Özcan‐Top · 2023 · International Conference on Control, Decision and Information Technologies (CoDIT)
  65. Technical debt prioritization: methods, techniques, and a large exploratory study
    Diogo de Jesus Pina · 2023
  66. CHALLENGES OF USING QUANTUM COMPUTING TECHNOLOGY: A SOFTWARE DEVELOPERS PERSPECTIVE
    TIAC, Luka Radujević, Vladimir Mandić · 2023 · International Scientific Conference on Industrial Systems (IS)
  67. A Mapping of Recording-based Game Test Automation Tools
    Vinícius Mioto, Fábio dos Santos Petrillo · 2025 · IEEE/ACM International Workshop on Games and Software Engineering (GAS)
  68. Research and Development of Test Automation Maturity Model Building and Assessment Methods for E2E Testing
    Daiju Kato, Ayane Mogi, Hiroshi Ishikawa, Yasufumi Takama · 2025 · Software
  69. Towards automation for rework reduction in software development: An approach
    V. Konashchuk, Rob Bemthuis · 2025 · Procedia Computer Science
  70. A Case Study on CrowdStrike's Cybersecurity Incident
    Ivan Livingstone Zziwa, Anthoinette Ilolo, Astha Sharma, Brian Serwadda, Maurice Eugene Dawson · 2025 · IGI Global eBooks
  71. A Comprehensive Framework for Testing Goal‐Oriented NFPs in Software Product Lines
    Ibtesam Gwasem · 2025 · Journal of Software Evolution and Process
  72. Manoel 2.0: An approach to the execution of manual tests using Device Farm
    Thiago T. Gavanski, Lorena P. de Figueiredo, Jonathan P. Gomes, Rene L. de Almeida, Welton De A. Seabra, Dinara M. Araujo, Andre F. Da Cruz, Erick C. Bezerra · 2025 · International Conference on Algorithms, Computing and Artificial Intelligence (ACAI)
  73. Les pratiques de code : de la documentation à la détection
    Corentin Latappy · 2024 · theses.fr (ABES)
  74. On the emergence of testing strategies: A socio-technical grounded theory
    Mark Swillus, Rashina Hoda, Andy Zaidman · 2026 · Empirical Software Engineering
  75. CONCEPTUAL MODEL FOR THE USE OF DEVOPS AND AI TOOLS IN AGILE PROJECTS
    Damartine Naiane Martins Feitosa, Renato Penha, José Romualdo da Costa Filho, Geovane Soares Galvão, Vlamir Fernandes da Silva · 2026 · SSRN Electronic Journal
  76. DevSecOpsUX: An Integrated Framework for Security, Quality Assurance, and User Experience in Continuous Software Delivery
    Jonathan A. Lopez, Gabriel Villegas, Cristian Barría Huidobro · 2026 · IEEE Access
  77. Coverage-Aware Guidance for Novelty-Driven Exploration in Automated Game Testing Under Sparse-Reward 3-D Environments
    Tae-Hyeon Jang, Hyeon-Uk Lee, Hyunseok Kim · 2026 · IEEE Access
  78. Automating software testing using artificial intelligence and machine learning
    Carlos Hernández Salas, Carmen Carolina Ortega Hernández · 2026 · CognitivaTech Ingeniería de Software Inteligente y Sistemas Adaptativos
  79. Uma Abordagem Unificada para Testes Automatizados em Arquiteturas Web Modernas
    João Victor de Morais, Rafael S. Durelli, Ricardo Terra · 2026 · Simpósio Brasileiro de Testes de Software Sistemático e Automatizado (SAST)
  80. An Empirical Study on the CI/CD Pipeline Evolution
    Luka Četina, Luka Pavlič · 2026 · Journal of Software Evolution and Process
  81. A Design Science Approach to Traceability-Aware Visual Browser Test Automation
    Yusuf Köse, Stiphen Chowdhury, Sin-Chun Ng, Man-Fai Leung · 2026 · International Conference on Information Science and Technology