On using grey literature and google scholar in systematic literature reviews in software engineering
Open accessAbstract
Context: The inclusion of grey literature (GL) is important to remove publication bias while gathering available evidence regarding a certain topic. The number of systematic literature reviews (SLRs) in Software Engineering (SE) is increasing but we do not know about the extent of GL usage in these SLRs. Moreover, Google Scholar is rapidly becoming a search engine of choice for many researchers but the extent to which it can find the primary studies is not known. Objective: This tertiary study is an attempt to i) measure the usage of GL in SLRs in SE. Furthermore this study proposes strategies for categorizing GL and a quality checklist to use for GL in future SLRs; ii) explore if it is feasible to use only Google Scholar for finding scholarly articles for academic research. Method: We have conducted a systematic mapping study to measure the extent of GL usage in SE SLRs as well as to measure the feasibility of finding primary studies using Google Scholar. Results and conclusions: a) Grey Literature: 76.09% SLRs (105 out of 138) in SE have included one or more GL studies as primary studies. Among total primary studies across all SLRs (6307), 582 are classified as GL, making the frequency of GL citing as 9.23%. The intensity of GL use indicate that each SLR contains 5 primary studies on average (total intensity of GL use being 5.54). The ranking of GL tells us that conference papers are the most used form 43.3% followed by technical reports 28.52%. Universities, research institutes, labs and scientific societies together make up 67.7% of GL used, indicating that these are useful sources for searching GL. We additionally propose strategies for categorizing GL and criteria for evaluating GL quality, which can become a basis for more detailed guidelines for including GL in future SLRs. b) Google Scholar Results: The results show that Google Scholar was able to retrieve 96% of primary studies of these SLRs. Most of the primary studies that were not found using Google Scholar were from grey sources.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Results from Google Scholar in combination with grey literature extracted from Google sufficiently extracts necessary sources similar to searches with other databases.
“According to Yasin et al. [55] results from Google Scholar in combination with grey literature extracted from Google sufficiently extracts necessary sources similar to searches with other databases, such as ScienceDirect, IEEE, ACM digital library and Springer Link.”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [55]
We used Google Scholar as the search engine to retrieve an unbiased start data set.
“We used Google Scholar as the search engine to retrieve an unbiased start data set.”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [55]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · see citing works
Empirical studies of agile software development: A systematic review
Dybå, T., & Dingsøyr, T. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.01.006 · Cited by 2,668 · see citing works
Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M., & Khalil, M. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2006.07.009 · Cited by 2,394 · see citing works
Systematic literature reviews in software engineering – A tertiary study
Kitchenham, B., Pretorius, R., Budgen, D., Pearl Brereton, O., Turner, M., Niazi, M., & Linkman, S. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.03.006 · Cited by 1,230 · see citing works
Automated analysis of feature models 20 years later: A literature review
Benavides, D., Segura, S., & Ruiz–Cortés, A. (2010) · Information Systems · doi:10.1016/j.is.2010.01.001 · Cited by 1,200 · see citing works
Arcuri, A., & Briand, L. (2011) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1985793.1985795 · Cited by 989 · see citing works
Does the technology acceptance model predict actual use? A systematic literature review
Turner, M., Kitchenham, B., Brereton, P., Charters, S., & Budgen, D. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2009.11.005 · Cited by 851 · see citing works
Garousi, V., Felderer, M., & Mäntylä, M. V. (2019) · Information and Software Technology · doi:10.1016/j.infsof.2018.09.006 · Cited by 831 · see citing works
A systematic review of software development cost estimation studies
Jørgensen, M., & Shepperd, M. (2007) · Brunel University Research Archive (BURA) (Brunel University London) · doi:10.1109/tse.2007.3 · Cited by 748 · see citing works
A Systematic Review of Software Development Cost Estimation Studies
Jørgensen, M., & Shepperd, M. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256943 · Cited by 736 · see citing works
A review of studies on expert estimation of software development effort
Jørgensen, M. (2003) · Journal of Systems and Software · doi:10.1016/s0164-1212(02)00156-5 · Cited by 618 · see citing works
A systematic review of software fault prediction studies
Catal, C., & Di̇ri̇, B. (2008) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.10.027 · Cited by 533 · see citing works
Systematic literature review of machine learning based software development effort estimation models
Wen, J., Li, S., Lin, Z., Hu, Y., & Huang, C. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2011.09.002 · Cited by 531 · see citing works
Motivation in software engineering: A systematic literature review
Beecham, S., Baddoo, N., Hall, T., Robinson, H., & Sharp, H. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2007.09.004 · Cited by 506 · see citing works
Usability evaluation methods for the web: A systematic mapping study
Fernandez, A., Insfrán, E., & Abrahão, S. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2011.02.007 · Cited by 497 · see citing works
Ali, S., Briand, L., Hemmati, H., & Panesar-Walawege, R. K. (2009) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.52 · Cited by 452 · see citing works
A systematic review of search-based testing for non-functional system properties
Afzal, W., Torkar, R., & FELDT, R. H. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.12.005 · Cited by 447 · see citing works
Cross versus Within-Company Cost Estimation Studies: A Systematic Review
Kitchenham, B. A., Mendes, E., & Travassos, G. H. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.1001 · Cited by 425 · see citing works
A Systematic Survey of Program Comprehension through Dynamic Analysis
Cornelissen, B., Zaidman, A., Arie Van Deursen, A., Moonen, L., & Koschke, R. (2009) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.28 · Cited by 421 · see citing works
Evaluating prediction systems in software project estimation
Shepperd, M., & MacDonell, S. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2011.12.008 · Cited by 408 · see citing works
Ahmed, I., Sutton, A. J., & Riley, R. D. (2012) · BMJ · doi:10.1136/bmj.d7762 · Cited by 404 · see citing works
Scientific research ontology to support systematic review in software engineering
Biolchini, J. C. D. A., Mian, P. G., Natali, A. C. C., Conte, T. U., & Travassos, G. H. (2007) · Advanced Engineering Informatics · doi:10.1016/j.aei.2006.11.006 · Cited by 400 · see citing works
Applying Systematic Reviews to Diverse Study Types: An Experience Report
Dyba, T., Dingsoyr, T., & Hanssen, G. K. (2007) · First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007) · doi:10.1109/esem.2007.59 · Cited by 391 · see citing works
Bjørnson, F. O., & Dingsøyr, T. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.03.006 · Cited by 366 · see citing works
Salleh, N., Mendes, E., & Grundy, J. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.59 · Cited by 361 · see citing works
Davis, A. M., Dieste, Ó., Hickey, A. M., Juristo, N., & Moreno, A. M. (2006) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2006.17 · Cited by 341 · see citing works
Research synthesis in software engineering: A tertiary study
Cruzes, D. S., & Dybå, T. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2011.01.004 · Cited by 339 · see citing works
A Visual Text Mining approach for Systematic Reviews
Malheiros, V., Höhn, E. N., de Pinho, R., Mendonça, M., & Maldonado, J. C. (2007) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2007.21 · Cited by 334 · see citing works
Using Scrum in Global Software Development: A Systematic Literature Review
Hossain, E., Babar, M. A., & Paik, H.-Y. (2009) · IEEE International Conference on Global Software Engineering (ICGSE) · doi:10.1109/icgse.2009.25 · Cited by 331 · see citing works
Experiences using systematic review guidelines
Staples, M., & Niazi, M. (2006) · Journal of Systems and Software · doi:10.1016/j.jss.2006.09.046 · Cited by 313 · see citing works
A systematic review on regression test selection techniques
Engström, E., Runeson, P., & Skoglund, M. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.07.001 · Cited by 312 · see citing works
Software product line testing – A systematic mapping study
Engström, E., & Runeson, P. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.05.011 · Cited by 308 · see citing works
User-Centered Design and Agile Methods: A Systematic Review
da Silva, T. S., Martin, A., Maurer, F., & Silveira, M. S. (2011) · Agile Conference · doi:10.1109/agile.2011.24 · Cited by 300 · see citing works
Evaluation and Measurement of Software Process Improvement—A Systematic Literature Review
Unterkalmsteiner, M., Gorschek, T., Islam, A., Cheng, C. K., Permadi, R. B., & FELDT, R. H. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.26 · Cited by 290 · see citing works
An analysis of research in computing disciplines
Glass, R. L., Ramesh, V., & Vessey, I. (2004) · Communications of the ACM · doi:10.1145/990680.990686 · Cited by 284 · see citing works
The effectiveness of pair programming: A meta-analysis
Hannay, J. E., Dybå, T., Arisholm, E., & Sjøberg, D. I. K. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.02.001 · Cited by 268 · see citing works
Evaluating and selecting software packages: A review
Jadhav, A. S., & Sonar, R. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.003 · Cited by 249 · see citing works
A systematic review of evaluation of variability management approaches in software product lines
Chen, L., & Babar, M. A. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.006 · Cited by 247 · see citing works
Hauge, Ø., Ayala, C. P., & Conradi, R. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.05.008 · Cited by 238 · see citing works
A systematic review of software maintainability prediction and metrics
Riaz, M., Mendes, E., & Tempero, E. D. (2009) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5314233 · Cited by 234 · see citing works
Variability management in software product lines: a systematic review
Chen, L., Babar, M. A., & Alí, N. (2009) · Arrow@dit (Dublin Institute of Technology) · doi:10.5555/1753235.1753247 · Cited by 232 · see citing works
A systematic mapping study of software product lines testing
Neto, P. A. D. M. S., Machado, I., McGregor, J. D., de Almeida, E. S., & Meira, S. R. D. L. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.003 · Cited by 219 · see citing works
Garousi, V., Felderer, M., & Mäntylä, M. V. (2016) · Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2915970.2916008 · Cited by 210 · see citing works
A systematic review of UML model consistency management
Lucas, F. J., Molina, F., & Toval, A. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.04.009 · Cited by 198 · see citing works
A systematic review of security requirements engineering
Mellado, D., Blanco, C., Sanchez, L. E., & Fernández‐Medina, E. (2010) · Computer Standards & Interfaces · doi:10.1016/j.csi.2010.01.006 · Cited by 197 · see citing works
A systematic literature review to identify and classify software requirement errors
Walia, G., & Carver, J. C. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.01.004 · Cited by 194 · see citing works
Requirements engineering for software product lines: A systematic literature review
Alves, V., Niu, N., Alves, C. F., & Valença, G. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.03.014 · Cited by 193 · see citing works
Models of motivation in software engineering
Sharp, H., Baddoo, N., Beecham, S., Hall, T., & Robinson, H. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.05.009 · Cited by 177 · see citing works
A systematic review of transformation approaches between user requirements and analysis models
Yue, T., Briand, L., & Labiche, Y. (2010) · Requirements Engineering · doi:10.1007/s00766-010-0111-y · Cited by 171 · see citing works
Six years of systematic literature reviews in software engineering: An updated tertiary study
da Silva, F. Q. B., Santos, A. L. M., Soares, S., França, A. C. C., Monteiro, C. V. F., & Maciel, F. F. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2011.04.004 · Cited by 167 · see citing works
Khan, S. U., Niazi, M., & Ahmad, R. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2010.12.010 · Cited by 167 · see citing works
Agile Practices in Global Software Engineering - A Systematic Map
Jalali, S., & Wohlin, C. (2010) · IEEE International Conference on Global Software Engineering (ICGSE) · doi:10.1109/icgse.2010.14 · Cited by 164 · see citing works
A systematic review on strategic release planning models
Svahnberg, M., Gorschek, T., FELDT, R. H., Torkar, R., Saleem, S. B., & Shafique, M. U. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.11.006 · Cited by 162 · see citing works
Khan, S. U., Niazi, M., & Ahmad, R. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.08.003 · Cited by 161 · see citing works
Characterizing software architecture changes: A systematic review
Williams, B. J., & Carver, J. C. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.07.002 · Cited by 158 · see citing works
Systematic review and aggregation of empirical studies on elicitation techniques
Dieste, Ó., & Juristo, N. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.33 · Cited by 156 · see citing works
A systematic review of software architecture evolution research
Breivold, H. P., Crnković, I., & Larsson, M. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2011.06.002 · Cited by 155 · see citing works
Heckman, S. S., & Williams, L. A. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.007 · Cited by 151 · see citing works
Systematic review of organizational motivations for adopting CMM-based SPI
Staples, M., & Niazi, M. (2007) · Information and Software Technology · doi:10.1016/j.infsof.2007.07.003 · Cited by 151 · see citing works
A systematic review and an expert survey on capabilities supporting multi product lines
Holl, G., Grünbacher, P., & Rabiser, R. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2012.02.002 · Cited by 148 · see citing works
Mandavilli, A. (2011) · Nature · doi:10.1038/469286a · Cited by 138 · see citing works
Software engineering research for computer games: A systematic review
Ampatzoglou, A., & Stamelos, I. G. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.05.004 · Cited by 133 · see citing works
Mohagheghi, P., Dehlen, V., & Neple, T. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.04.004 · Cited by 133 · see citing works
Nicolás, J., & Toval, A. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.04.001 · Cited by 133 · see citing works
Measuring and predicting software productivity: A systematic map and review
Petersen, K. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.001 · Cited by 128 · see citing works
Tools used in Global Software Engineering: A systematic mapping review
Portillo-Rodríguez, J., Vizcaíno, A., Piattini, M. G., & Beecham, S. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2012.02.006 · Cited by 128 · see citing works
A Systematic Review and Comparison of Security Ontologies
Blanco, C., Lasheras, J., Valencia‐García, R., Fernández‐Medina, E., Toval, A., & Piattini, M. G. (2008) · International Conference on Availability, Reliability and Security (ARES) · doi:10.1109/ares.2008.33 · Cited by 119 · see citing works
IT Service Management Process Improvement based on ISO/IEC 15504: A systematic review
Mesquida, A., Mas, A., Amengual, E., & Calvo-Manzano, J. A. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2011.11.002 · Cited by 118 · see citing works
Empirical evaluations of regression test selection techniques
Engström, E., Skoglund, M., & Runeson, P. (2008) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/1414004.1414011 · Cited by 103 · see citing works
Software effort estimation terminology: The tower of Babel
Grimstad, S., Jørgensen, M., & Molokken-Ostvold, K. (2005) · Information and Software Technology · doi:10.1016/j.infsof.2005.04.004 · Cited by 98 · see citing works
Hossain, E., Babar, M. A., Paik, H.-Y., & Verner, J. M. (2009) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec.2009.56 · Cited by 94 · see citing works
Ivarsson, M., & Gorschek, T. (2009) · Requirements Engineering · doi:10.1007/s00766-009-0080-1 · Cited by 94 · see citing works
Afzal, W., & Torkar, R. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.03.041 · Cited by 93 · see citing works
Motivation in software engineering: a systematic review update
França, A. C. C., Gouveia, T. B., Santos, P. C. F., Santana, C., & da Silva, F. Q. B. (2011) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1049/ic.2011.0019 · Cited by 91 · see citing works
A systematic review of comparative evidence of aspect-oriented programming
Ali, M. S., Ali Babar, M., Chen, L., & Stol, K.-J. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.05.003 · Cited by 89 · see citing works
Factors Limiting Industrial Adoption of Test Driven Development: A Systematic Review
Čaušević, A., Sundmark, D., & Punnekkat, S. (2011) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2011.19 · Cited by 87 · see citing works
A systematic mapping study on the combination of static and dynamic quality assurance techniques
Elberzhager, F., Münch, J., & Nha, V. T. N. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2011.06.003 · Cited by 86 · see citing works
Signs of Agile Trends in Global Software Engineering Research: A Tertiary Study
Hanssen, G. K., Šmite, D., & Moe, N. B. (2011) · IEEE International Conference on Global Software Engineering Workshops (ICGSEW) · doi:10.1109/icgse-w.2011.12 · Cited by 78 · see citing works
A comparative study of challenges in integrating Open Source Software and Inner Source Software
Stol, K.-J., Babar, M. A., Avgeriou, P., & Fitzgerald, B. F. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2011.06.007 · Cited by 75 · see citing works
Using Visual Text Mining to Support the Study Selection Activity in Systematic Literature Reviews
Felizardo, K. R., Salleh, N., Martins, R. M., Mendes, E., MacDonell, S. G., & Maldonado, J. C. (2011) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2011.16 · Cited by 70 · see citing works
Synthesizing evidence in software engineering research
Cruzes, D. S., & Dybå, T. (2010) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/1852786.1852788 · Cited by 70 · see citing works
A Systematic Review of the Use of Requirements Engineering Techniques in Model-Driven Development
Loniewski, G., Insfrán, E., & Abrahão, S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-16129-2_16 · Cited by 70 · see citing works
Steinmacher, I., Chaves, A. P., & Gerosa, M. A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15714-1_15 · Cited by 69 · see citing works
Systematic literature review: teaching novices programming using robots
Major, L., Kyriacou, T., & Brereton, O. P. (2012) · IET Software · doi:10.1049/iet-sen.2011.0125 · Cited by 69 · see citing works
Prikladnicki, R., & Audy, J. L. N. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.03.009 · Cited by 68 · see citing works
A systematic review of domain analysis solutions for product lines
Khurum, M., & Gorschek, T. (2009) · Journal of Systems and Software · doi:10.1016/j.jss.2009.06.048 · Cited by 68 · see citing works
Reconciling software development models: A quasi-systematic review
Magdaleno, A. M., Werner, C. M. L., & de Araújo, R. M. (2011) · Journal of Systems and Software · doi:10.1016/j.jss.2011.08.028 · Cited by 61 · see citing works
Software quality trade-offs: A systematic map
Barney, S., Petersen, K., Svahnberg, M., Aurum, A., & Barney, H. T. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2012.01.008 · Cited by 61 · see citing works
Preparing Students and Engineers for Global Software Development: A Systematic Review
Monasor, M. J., Vizcaíno, A., Piattini, M. G., & Caballero, I. (2010) · IEEE International Conference on Global Software Engineering (ICGSE) · doi:10.1109/icgse.2010.28 · Cited by 60 · see citing works
Kitchenham, B. A., Pearl Brereton, A., Turner, M., Niazi, M., Linkman, S., Pretorius, R., & Budgen, D. (2009) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5314238 · Cited by 60 · see citing works
Basis for an integrated security ontology according to a systematic review of existing proposals
Blanco, C., Lasheras, J., Fernández‐Medina, E., Valencia‐García, R., & Toval, A. (2011) · Computer Standards & Interfaces · doi:10.1016/j.csi.2010.12.002 · Cited by 57 · see citing works
A systematic review of research on open source software in commercial software product development
Höst, M., & Oručević-Alagić, A. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.009 · Cited by 54 · see citing works
A systematic literature review on software measurement programs
Tahir, T., Rasool, G., & Gencel, Ç. (2016) · Information and Software Technology · doi:10.1016/j.infsof.2016.01.014 · Cited by 53 · see citing works
A new research agenda for tool integration
Wicks, M., & Dewar, R. G. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2007.03.089 · Cited by 50 · see citing works
Testing in Service Oriented Architectures with dynamic binding: A mapping study
Palacios, M., García-Fanjul, J., & Tuya, J. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.11.014 · Cited by 49 · see citing works
Ontologies application in organizational learning: A literature review
Valaski, J., Malucelli, A., & Reinehr, S. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2012.01.075 · Cited by 48 · see citing works
What Do We Know about Scientific Software Development's Agile Practices?
Sletholt, M. T., Hannay, J. E., Pfahl, D., & Langtangen, H. P. (2011) · Computing in Science & Engineering · doi:10.1109/mcse.2011.113 · Cited by 47 · see citing works
Service Identification Methods: A Systematic Literature Review
Gu, Q., & Lago, P. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-17694-4_4 · Cited by 47 · see citing works
Towards a Defect Prevention Based Process Improvement Approach
Kalinowski, M., Travassos, G. H., & Card, D. N. (2008) · EUROMICRO Conference · doi:10.1109/seaa.2008.47 · Cited by 46 · see citing works
Bibliometrics and internet: Some observations and expectations
van Raan, A. F. J. (2001) · Scientometrics · doi:10.1023/a:1005690102069 · Cited by 45 · see citing works
A systematic literature review of software quality cost research
Karg, L. M., Grottke, M., & Beckhaus, A. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2010.11.904 · Cited by 44 · see citing works
An overview on test generation from functional requirements
Escalona, M. J., Gutiérrez, J., Mejías, M., Aragón, G., Ramos, I. M., Valderrama, J. T., & Mayo, F. J. D. (2011) · Journal of Systems and Software · doi:10.1016/j.jss.2011.03.051 · Cited by 44 · see citing works
Process models for service-based applications: A systematic literature review
Lane, S., & Richardson, I. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.005 · Cited by 44 · see citing works
Empirical Studies on Quality in Agile Practices: A Systematic Literature Review
Sfetsos, P., & Stamelos, I. G. (2010) · International Conference on the Quality of Information and Communications Technology (QUATIC) · doi:10.1109/quatic.2010.17 · Cited by 44 · see citing works
Managing Quality Requirements: A Systematic Review
Svensson, R. B., Höst, M., & Regnell, B. (2010) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa.2010.55 · Cited by 42 · see citing works
Transforming e-services evaluation data into business analytics using value models
Loukis, E., Παζάλος, Κ., & Salagara, A. (2011) · Electronic Commerce Research and Applications · doi:10.1016/j.elerap.2011.12.004 · Cited by 41 · see citing works
Software source code sizing using fuzzy logic modeling
MacDonell, S. G. (2003) · Information and Software Technology · doi:10.1016/s0950-5849(03)00011-9 · Cited by 40 · see citing works
Software Process Simulation Modeling: Facts, Trends and Directions
Zhang, H., Kitchenham, B. A., & Pfahl, D. (2008) · Proceedings - Asia Pacific Software Engineering Conference/Proceedings · doi:10.1109/apsec.2008.50 · Cited by 40 · see citing works
In search of `architectural knowledge'
de Boer, R. C., & Farenhorst, R. (2008) · doi:10.1145/1370062.1370080 · Cited by 38 · see citing works
Comparing Local and Global Software Effort Estimation Models -- Reflections on a Systematic Review
MacDonell, S. G., & Shepperd, M. (2007) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2007.45 · Cited by 36 · see citing works
A Frame of Reference for SOA Migration
Razavian, M., & Lago, P. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-17694-4_13 · Cited by 35 · see citing works
Test-Driven Development - Still a Promising Approach?
Kollanus, S. (2010) · International Conference on the Quality of Information and Communications Technology (QUATIC) · doi:10.1109/quatic.2010.73 · Cited by 35 · see citing works
The role of software process simulation modeling in software risk management: A systematic review
Liu, D., Wang, Q., & Xiao, J. (2009) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5315982 · Cited by 34 · see citing works
Electronic whiteboards in emergency medicine
Rasmussen, R. O. (2012) · International Health Informatics Symposium · doi:10.1145/2110363.2110418 · Cited by 32 · see citing works
What do we know about software product management? - a systematic mapping study
Maglyas, A., Nikula, U., & Smolander, K. (2011) · International Workshop on Software Product Management (IWSPM) · doi:10.1109/iwspm.2011.6046201 · Cited by 31 · see citing works
Adopting software product lines: a systematic mapping study
Bastos, J. F., Neto, P. A. D. M. S., de Almeida, E. S., & Meira, S. R. D. L. (2011) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1049/ic.2011.0002 · Cited by 30 · see citing works
The use of empirical methods in Open Source Software research: Facts, trends and future directions
Stol, K.-J., Babar, M. A., Russo, B., & Fitzgerald, B. F. (2009) · ICSE Workshop on Emerging Trends in Free/Libre/Open Source Software Research and Development (FLOSS) · doi:10.1109/floss.2009.5071355 · Cited by 30 · see citing works
Mitchell, S. M., & Seaman, C. B. (2009) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2009.5314228 · Cited by 29 · see citing works
Pair programming as a teaching tool: a student review of empirical studies
Pearl Brereton, A., Turner, M., & Kaur, R. (2009) · IEEE International Conference on Software Engineering Education and Training (CSEE&T) · doi:10.1109/cseet.2009.11 · Cited by 29 · see citing works
Applicability of the semiotic inspection method: a systematic literature review
Reis, S., & Prates, R. O. (2011) · Latin American conference on Human-Computer Interaction · doi:10.5555/2254436.2254468 · Cited by 28 · see citing works
Are electronic theses and dissertations (still) grey literature in the digital age? A FAIR debate
Schöpfel, J., & Rasuli, B. (2018) · The Electronic Library · doi:10.1108/el-02-2017-0039 · Cited by 28 · see citing works
The effects of open access on un-published documents: A case study of economics working papers
Frandsen, T. F. (2009) · Journal of Informetrics · doi:10.1016/j.joi.2008.12.002 · Cited by 27 · see citing works
Research in concurrent software testing
Souza, S. R. S., Brito, M. A. S., Silva, R. A., de Souza, P. S. L., & Zaluska, E. (2011) · Parallel and Distributed Systems: Testing, Analysis, and Debugging · doi:10.1145/2002962.2002964 · Cited by 26 · see citing works
A Systematic Review of Studies of Open Source Software Evolution
Breivold, H. P., Chauhan, M. A., & Babar, M. A. (2010) · Proceedings - Asia Pacific Software Engineering Conference/Proceedings · doi:10.1109/apsec.2010.48 · Cited by 25 · see citing works
Influence seekers: The production of grey literature for policy and practice
Lawrence, A. (2017) · Information Services & Use · doi:10.3233/isu-170857 · Cited by 25 · see citing works
A comparative evaluation on the accuracies of software effort estimates from clustered data
Huang, S.-J., Chiu, N., & Liu, Y. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.02.005 · Cited by 24 · see citing works
A systematic literature survey of integration testing in component-based software engineering
Shashank, S. P., Chakka, P., & Damera, V. K. (2010) · International Conference on Computer and Communication Technology · doi:10.1109/iccct.2010.5640467 · Cited by 23 · see citing works
An extended systematic review of software process Improvement in small and medium web companies
Sulayman, M., & Mendes, E. (2011) · doi:10.1049/ic.2011.0017 · Cited by 22 · see citing works
Crafting a Global Teaming Model for Architectural Knowledge
Beecham, S., Noll, J., Richardson, I., & Alí, N. (2010) · IEEE International Conference on Global Software Engineering (ICGSE) · doi:10.1109/icgse.2010.15 · Cited by 22 · see citing works
A systematic literature review of interoperable architecture for e-government portals
Sedek, K. A., Sulaiman, S., & Omar, M. A. (2011) · doi:10.1109/mysec.2011.6140648 · Cited by 21 · see citing works
A Framework for Defect Prediction in Specific Software Project Contexts
Wahyudin, D., Ramler, R., & Biffl, S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-22386-0_20 · Cited by 19 · see citing works
A Systematic Mapping Study on Software Engineering Testbeds
Barreiros, E. F. S., Almeida, A., Saraiva, J. D. A. G., & Soares, S. (2011) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2011.19 · Cited by 18 · see citing works
Grey Literature Acquisition and Management: Challenges in Academic Libraries in Africa
Osayande, O., & Ukpebor, C. O. (2012) · Covenant University Repository (Covenant University) · Cited by 17 · see citing works
A framework for awareness maintenance
Talaei‐Khoei, A., RAY, P. K., Parameshwaran, N., & Lewis, L. M. (2011) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2011.06.011 · Cited by 16 · see citing works
Requirements Engineering in the Development of Multi-Agent Systems: A Systematic Review
Blanes, D., Insfrán, E., & Abrahão, S. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04394-9_62 · Cited by 16 · see citing works
Khan, Y. A., Elish, M. O., & El‐Attar, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-31128-4_19 · Cited by 15 · see citing works
A Systematic Literature Review on Service Description Methods
Teka, A. Y., Condorí-Fernández, N., & Sapkota, B. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-28714-5_22 · Cited by 14 · see citing works
Characterising and Analysing Security Requirements Modelling Initiatives
Kárpáti, P., Sindre, G., & Opdahl, A. L. (2011) · International Conference on Availability, Reliability and Security (ARES) · doi:10.1109/ares.2011.113 · Cited by 14 · see citing works
Testability and software performance
Hassan, M. M., Afzal, W., Lindström, B., Shah, S. M. A., Andler, S. F., & Blom, M. (2016) · ACM Symposium on Applied Computing · doi:10.1145/2851613.2851978 · Cited by 14 · see citing works
Critical Issues on Test-Driven Development
Kollanus, S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21843-9_25 · Cited by 13 · see citing works
Vertical Test Reuse for Embedded Systems: A Systematic Mapping Study
Flemström, D., Sundmark, D., & Afzal, W. (2015) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa.2015.46 · Cited by 12 · see citing works
Quality of grey literature in the open access era: Privilege and responsibility
Castro, P. D., & Salinetti, S. (2004) · Publishing Research Quarterly · doi:10.1007/bf02910856 · Cited by 12 · see citing works
Domain-specific language modelling with UML profiles by decoupling abstract and concrete syntaxes
Pardillo, J., & Cachero, C. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2010.08.019 · Cited by 12 · see citing works
Huang, R. Y., & Symonds, J. (2009) · doi:10.1109/edocw.2009.5332001 · Cited by 12 · see citing works
An Evaluation of Mutation and Data-Flow Testing: A Meta-analysis
Kakarla, S., Momotaz, S., & Namin, A. S. (2011) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw.2011.51 · Cited by 12 · see citing works
USING GREY LITERATURE IN INFORMAL INFORMATION SERVICES IN AFRICA
Sturges, P. (1994) · Journal of Documentation · doi:10.1108/eb026934 · Cited by 11 · see citing works
Shivaram, B. S., & Biradar, B. S. (2018) · The Electronic Library · doi:10.1108/el-05-2018-0100 · Cited by 11 · see citing works
The Use of Grey Literature Review as Evidence for Practitioners
Kamei, F. (2019) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/3356773.3356797 · Cited by 11 · see citing works
Fugitive Literature in the Cross Hairs
Denda, K. (2002) · Collection Management · doi:10.1300/j105v27n02_07 · Cited by 10 · see citing works
Pragmatic assessment of research intensive areas in cloud
Hasteer, N., Bansal, A., & Murthy, B. K. (2013) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/2464526.2464533 · Cited by 10 · see citing works
Procedures and Conditions that Influence on the Efficiency of Some Agile Practices
Landim, H. F., Albuquerque, A. B., & Macedo, T. C. (2010) · International Conference on the Quality of Information and Communications Technology (QUATIC) · doi:10.1109/quatic.2010.70 · Cited by 9 · see citing works
Practical relevance of experiments in comprehensibility of requirements specifications
Condorí-Fernández, N., Daneva, M., Sikkel, K., & Herrmann, A. (2011) · doi:10.1109/empire.2011.6046251 · Cited by 8 · see citing works
Context-Awareness on Software Artifacts in Distributed Software Development: A Systematic Review
Vivian, R. L., Huzita, E. H. M., Leal, G. C. L., & Chaves, A. P. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23801-7_3 · Cited by 7 · see citing works
Monteiro, C. V. F., Arcoverde, D. F., da Silva, F. Q. B., & Ferreira, H. S. (2010) · IEEE International Conference on Management of Innovation and Technology (ICMIT) · doi:10.1109/icmit.2010.5492781 · Cited by 7 · see citing works
Towards the Discovery of Data Quality Attributes for Web Portals
Moraga, C., Moraga, M. Á., Calero, C., & Caro, L. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02818-2_20 · Cited by 6 · see citing works
Hyrkkänen, U., & Nenonen, S. P. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21616-9_8 · Cited by 5 · see citing works
Evaluation of SPL Approaches for WebGIS Development: SIGTel, a Case Study
Moreno-Rivera, J. M., & Navarro, E. (2011) · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.1109/hicss.2011.195 · Cited by 3 · see citing works
Grey Literature Publishing in Public Policy: Production and Management, Costs and Benefits
Lawrence, A. (2017) · IOS Press eBooks · doi:10.3233/978-1-61499-769-6-85 · Cited by 3 · see citing works
Systematic Reviews in Evidence-based Software Technology and Software Engineering
(2004) · Information and Software Technology · doi:10.1016/j.infsof.2004.10.006 · Cited by 3 · see citing works
Systematic literature review of virtual role
Noponen, S., Salonen, J., Sihvonen, H., & Kurki, T. A. (2011) · International Conference for Internet Technology and Secured Transactions · Cited by 2 · see citing works
Gelfand, J. M., & King, J. L. (2002) · doi:10.1109/sedep.1998.730719 · Cited by 2 · see citing works
Czech National Repository of Grey Literature
Pejšová, P. (2010) · Institutional digital repository NTL (National Library of Technology (NTK), Czech Republic) · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- A. Mandavilli, ''Trial by Twitter,'' Nature, vol. 469, no. 1, pp. 286-287, 2011.
- O. Osayande and C. O. Ukpebor, ''Grey literature acquisition and man- agement: Challenges in academic libraries in Africa,'' Grey Literature Acquisition Manage., Challenges Acad. Libraries Africa, 2012, pp. 1-13.
- P. De Castro and S. Salinetti, ''Quality of grey literature in the open access era: Privilege and responsibility,'' Publishing Res. Quart., vol. 20, no. 1, pp. 4-12, Mar. 2004.
- B. Kitchenham, P. Brereton, M. Turner, M. Niazi, S. Linkman, R. Pretorius, and D. Budgen, ''The impact of limited search procedures for systematic literature reviews-A participant-observer case study,'' in Proc. 3rd Int. Symp. Empirical Softw. Eng. Meas. Lake Buena Vista, FL, USA: IEEE Computer Society, Oct. 2009, pp. 336-345, doi: 10.1109/esem.2009.5314238. · DOI
- Y. A. Khan, M. O. Elish, and M. El-Attar, ''A systematic review on the impact of CK metrics on the functional correctness of object-oriented classes,'' in Proc. 12th Int. Conf., Comput. Sci. Appl. (ICCSA) (Lec- ture Notes in Computer Science), vol. 7336.
- D. Wahyudin, R. Ramler, and S. Biffl, ''A framework for defect prediction in specific software project contexts,'' in Proc. Softw. Eng. Techn.-3rd IFIP TC 2 Central East Eur. Conf. CEE-SET, Brno, Czech Republic, vol. 4980, in Lecture Notes in Computer Science, Z.
Cited by
Sorted by citations
“We Need To Talk About ChatGPT”: The Future of AI and Higher Education
Neumann, M., Rauschenberger, M., & Schön, E.-M. (2023) · Hochschule Hannover · doi:10.25968/opus-2467 · Cited by 212 · see citing works
Stige, Å., Zamani, E. D., Mikalef, P., & Zhu, Y. (2023) · Information Technology and People · doi:10.1108/itp-07-2022-0519 · Cited by 203 · see citing works
Steidl, M., Felderer, M., & Ramler, R. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111615 · Cited by 126 · see citing works
Reuse assessment of WEEE: Systematic review of emerging themes and research directions
Anandh, G., Venkatesan, S. P., Goh, M., & Mathiyazhagan, K. (2021) · Journal of Environmental Management · doi:10.1016/j.jenvman.2021.112335 · Cited by 82 · see citing works
Grey Literature in Software Engineering: A critical review
Kamei, F., Wiese, I., Lima, C., Polato, I., Nepomuceno, V., Ferreira, W., Ribeiro, M., Pena, C., Cartaxo, B., Pinto, G., et al. (2021) · Information and Software Technology · doi:10.1016/j.infsof.2021.106609 · Cited by 70 · see citing works
Sentiment analysis tools in software engineering: A systematic mapping study
Obaidi, M., Nagel, L., Specht, A., & Klünder, J. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.107018 · Cited by 53 · see citing works
Wilson, R., & Winnard, Y. (2022) · Journal of Health Organization and Management · doi:10.1108/jhom-11-2021-0425 · Cited by 47 · see citing works
Obaidi, M., & Klünder, J. (2021) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3463274.3463328 · Cited by 44 · see citing works
Dynamic Security Analysis on Android: A Systematic Literature Review
Sutter, T. S., Kehrer, T., Rennhard, M., Tellenbach, B., & Klein, J. (2024) · IEEE Access · doi:10.1109/access.2024.3390612 · Cited by 41 · see citing works
Cardiovascular load assessment in the workplace: A systematic review
Dias, M., Silva, L., Folgado, D., Nunes, M. L., Cepeda, C., Cheetham, M., & Gamboa, H. F. S. (2023) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2023.103476 · Cited by 40 · see citing works
Zhang, X., Zhang, C., Li, X., Du, Z., Mao, B., Li, Y., Zheng, Y., Li, Y., Pan, L., Liu, Y., et al. (2024) · ACM Computing Surveys · doi:10.1145/3696788 · Cited by 39 · see citing works
A survey of AI ethics in business literature: Maps and trends between 2000 and 2021
Daza, M. T., & Ilozumba, U. J. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1042661 · Cited by 35 · see citing works
Hill, J. R., Brown, J. C., Campbell, N. L., & Holden, R. J. (2021) · JMIR Formative Research · doi:10.2196/26181 · Cited by 29 · see citing works
How Should Software Engineering Secondary Studies Include Grey Material?
Kitchenham, B. A., Madeyski, L., & Budgen, D. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3165938 · Cited by 30 · see citing works
Test flakiness’ causes, detection, impact and responses: A multivocal review
Tahir, A., Rasheed, S., Dietrich, J. B., Hashemi, N., & Zhang, L. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111837 · Cited by 24 · see citing works
Sam, S., Lehberger, M., & Sparke, K. (2022) · Journal of Rural Studies · doi:10.1016/j.jrurstud.2022.09.016 · Cited by 22 · see citing works
What Evidence We Would Miss If We Do Not Use Grey Literature?
Kamei, F., Pinto, G., Wiese, I., Ribeiro, M., & Soares, S. (2021) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3475716.3475777 · Cited by 21 · see citing works
How Can We Develop Explainable Systems? Insights from a Literature Review and an Interview Study
Chazette, L., Klünder, J., Balci, M., & Schneider, K. A. (2022) · doi:10.1145/3529320.3529321 · Cited by 21 · see citing works
Which Challenges Do Exist With Agile Culture in Practice?
Kuchel, T., Neumann, M., Diebold, P., & Schön, E.-M. (2023) · ACM Symposium on Applied Computing · doi:10.1145/3555776.3578726 · Cited by 21 · see citing works
Agile culture clash: Unveiling challenges in cultivating an agile mindset in organizations
Neumann, M., Kuchel, T., Diebold, P., & Schön, E.-M. (2024) · Computer Science and Information Systems · doi:10.2298/csis230715029n · Cited by 21 · see citing works
da Silva, J. A. T. (2021) · Scientometrics · doi:10.1007/s11192-020-03831-9 · Cited by 19 · see citing works
Software solutions for newcomers’ onboarding in software projects: A systematic literature review
Santos, Í., Felizardo, K. R., Steinmacher, I., & Gerosa, M. A. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107568 · Cited by 22 · see citing works
bin Ali, N., & Tanveer, B. (2022) · e-Informatica Software Engineering Journal · doi:10.37190/e-inf220106 · Cited by 19 · see citing works
On the Use of Grey Literature: A Survey with the Brazilian Software Engineering Research Community
Kamei, F., Wiese, I., Pinto, G., Ribeiro, M., & Soares, S. (2020) · Proceedings of the XXXIV Brazilian Symposium on Software Engineering · doi:10.1145/3422392.3422442 · Cited by 15 · see citing works
The Impact of Covid 19 on Agile Software Development: A Systematic Literature Review
Neumann, M., & Bogdanov, Y. (2022) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2022.882 · Cited by 15 · see citing works
Improving test automation maturity: A multivocal literature review
Wang, Y., Mäntylä, M., Liu, Z., Markkula, J., & Raulamo-Jurvanen, P. (2022) · Software Testing Verification and Reliability · doi:10.1002/stvr.1804 · Cited by 12 · see citing works
Logging requirement for continuous auditing of responsible machine learning-based applications
Foalem, P. L., Silva, L. D., Khomh, F., Li, H., & Merlo, E. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10656-8 · Cited by 10 · see citing works
Sağlam, R. B., Yuan, H., Heering, M. S., Ashraf, R., & Li, S. (2025) · Information · doi:10.3390/info16070514 · Cited by 10 · see citing works
Toward Sustainable Development: Exploring the Value and Benefits of Digital Twins
Weichbroth, P., Jandy, K., & Zurada, J. M. (2024) · Telecom · doi:10.3390/telecom5030039 · Cited by 11 · see citing works
What drives citations of frontier application publications?
Djokoto, J. G., Agyei-Henaku, K. A. A.-O., Afrane-Arthur, A. A., Badu-Prah, C., Gidiglo, F. K., & Srofenyoh, F. Y. (2020) · Heliyon · doi:10.1016/j.heliyon.2020.e05428 · Cited by 10 · see citing works
On the utilization of non‐quality assessed literature in software engineering research
Yasin, A., Fatima, R., Liu, L., Khan, J. A., Ali, R., & Wang, J. (2022) · Journal of Software Evolution and Process · doi:10.1002/smr.2464 · Cited by 9 · see citing works
Welsch, D., Burk, L., Mötefindt, D., & Neumann, M. (2024) · ACM Symposium on Applied Computing · doi:10.1145/3605098.3635988 · Cited by 9 · see citing works
Bervar, M., Bertoncel, T., & Bach, M. P. (2026) · Systems · doi:10.3390/systems14020138 · Cited by 7 · see citing works
A Systematic Review on the Sufficiency of PubMed and Google Scholar for Biosciences
Teo, Y. H., & Ling, M. H. (2020) · Acta Scientific Medical Sciences · doi:10.31080/asms.2020.04.0786 · Cited by 6 · see citing works
A Systematic Literature Review on a Decade of Industrial TLA+ Practice
Bögli, R., Lerena, L., Tsigkanos, C., & Kehrer, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-76554-4_2 · Cited by 6 · see citing works
Nägele, S., Schenk, N., & Matthes, F. (2023) · IEEE Conference on Business Informatics · doi:10.1109/cbi58679.2023.10187439 · Cited by 6 · see citing works
Psychosocial Distress Screening Among Interprofessional Palliative Care Teams: A Narrative Review
Brown, C. K., & Wallace, C. L. (2024) · Journal of Social Work in End-of-Life & Palliative Care · doi:10.1080/15524256.2024.2343052 · Cited by 6 · see citing works
Grey Literature in Software Engineering: A Critical ReviewPreprint
Kamei, F., Wiese, I., Lima, C., Polato, I., Nepomuceno, V., Ferreira, W., Ribeiro, M., Pena, C., Cartaxo, B., Pinto, G., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.13435 · Cited by 6 · see citing works
A SWOT Analysis of Software Development Life Cycle Security Metrics
Khalid, A., Raza, M., Afsar, P., Khan, R. A., Mohmand, M. I., & Rahman, H. U. (2024) · Journal of Software Evolution and Process · doi:10.1002/smr.2744 · Cited by 5 · see citing works
Szilágyi, S., Körei, A., & Vaičiulytė, I. (2025) · Education Sciences · doi:10.3390/educsci15081030 · Cited by 5 · see citing works
Design and Implementation of Social Safety Nets in Tanzania: A Systematic Review
Msuha, B., Kissoly, L., & Kihaule, A. (2024) · International Journal of Economics and Financial Issues · doi:10.32479/ijefi.15523 · Cited by 5 · see citing works
The Use of Grey Literature and Google Scholar in Software Engineering Systematic Literature Reviews
Fatima, R., Yasin, A., Liu, L., & Wang, J. (2020) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac48688.2020.0-121 · Cited by 5 · see citing works
On the Use of Grey Literature: A Survey with the Brazilian Software Engineering Research CommunityPreprint
Kamei, F., Wiese, I., Pinto, G., Ribeiro, M., & Soares, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05926 · Cited by 5 · see citing works
Eaton, C., Campo, K. X., Longhurst, A., Mesmer, B. L., & Ferreira, S. (2024) · Systems Engineering · doi:10.1002/sys.21799 · Cited by 4 · see citing works
Cultivating Digital Pathways: A Systematic Review of e-Commerce Adoption in Asian Agriculture
Jusoh, A., Abbas, A. F., Yazid, M. H. A., Latip, H. B. A., & Hussin, A. R. C. (2025) · Journal of Agricultural & Food Information · doi:10.1080/10496505.2025.2553282 · Cited by 5 · see citing works
Supporting systematic review workflows in computing: a functional evaluation of electronic databases
Kurdi, G. R. (2025) · Journal of Documentation · doi:10.1108/jd-05-2025-0128 · Cited by 4 · see citing works
Zagulova, D., & Popova, Y. (2021) · Strategic decisions and risk management · doi:10.17747/2618-947x-2021-1-34-49 · Cited by 4 · see citing works
Parrales–Bravo, F., Tolozano-Benites, R., Jadán-Guerrero, J. L., Vasquez-Cevallos, L., & Gómez-Rodríguez, V. G. (2026) · Computers · doi:10.3390/computers15040229 · Cited by 3 · see citing works
Ogunleye, O. S. (2023) · 2023 International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci62032.2023.00278 · Cited by 2 · see citing works
Shukla, M., Johnson, S. D., & Jones, P. (2025) · Intelligent Transportation Infrastructure · doi:10.1093/iti/liaf007 · Cited by 2 · see citing works
Investigating the Competencies and Skills of Computer Professionals
Gonçalves, R. F., Passos, O. M., & de Amorim, R. X. (2024) · Anais do Simpósio Brasileiro de Sistemas de Informação (SBSI) · doi:10.1145/3658271.3658308 · Cited by 2 · see citing works
Exploring the impact of feedback on remote SW development teams
Cavalcanti, A. B., Alves, C. F., & Araújo, J. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107649 · Cited by 2 · see citing works
Análise da cadeia de valor de órgão de governo como apoio organização de biblioteca digital
Shintaku, M. (2023) · AtoZ novas práticas em informação e conhecimento · doi:10.5380/atoz.v12i0.84584 · Cited by 2 · see citing works
Revisitando Processos de Construção de Requisitos: Uma Perspectiva de Transparência
Leite, J. C. S. D. P. (2023) · Workshop em Engenharia de Requisitos (WER) · doi:10.29327/1298356.26-14 · Cited by 2 · see citing works
Query Processing of Geosocial Data in Location-Based Social Networks
D’Ulizia, A., Grifoni, P., & Ferri, F. (2021) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi11010019 · Cited by 2 · see citing works
Common names of the Asiatic ibex superspecies at a turning point in its taxonomy and management
Sarasa, M. (2023) · Animal Biodiversity and Conservation · doi:10.32800/abc.2023.46.0079 · Cited by 1 · see citing works
Yasin, A., Fatima, R., Puspitasari, I., JiangBin, Z., & Li, Z. (2025) · Journal of Software Evolution and Process · doi:10.1002/smr.2753 · Cited by 1 · see citing works
Kurdi, G. R., & Allehyani, B. A. (2025) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.9654 · Cited by 1 · see citing works
Building the Landing Zone for Site Reliability Engineering: A Multivocal Literature Review
Costa, L. A., Fontão, A., Constantinou, E., dos Santos, R. P., & Serebrenik, A. (2025) · Simpósio Brasileiro de Engenharia de Software (SBES) · doi:10.5753/sbes.2025.9772 · Cited by 1 · see citing works
Developing a fraud prevention framework for universities: a systematic literature review
Irianto, G., Abdullah, S. N., Setyawan, R. A., Amirya, M., Wahyuni, S., Purwanto, P., & Hanun, N. R. (2025) · Cogent Business & Management · doi:10.1080/23311975.2025.2593736 · Cited by 1 · see citing works
Sentiment Analysis On Tourist Attraction Reviews In Purwokerto Using Naive Bayes Algorithm
Nugraha, S. S., Fathoni, M. Y., & Afrad, M. (2025) · International Conference of Science and Information Technology in Smart Administration (ICSINTESA) · doi:10.1109/icsintesa68165.2025.11413798 · Cited by 1 · see citing works
Crowd Simulation: A Multi-Dimensional Systematic Mapping Study and Taxonomy
Felemban, E. A., Hammad, M., & Rehman, F. U. (2026) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi15050223 · Cited by 1 · see citing works
A systematic review of geographical information systems for the conservation of historic settlements
Mukhopadhyay, N., Bharule, S. P., & Sen, J. (2026) · Discover Sustainability · doi:10.1007/s43621-026-03664-1 · Cited by 1 · see citing works
Melegati, J., Guerra, E., Wiese, I., & Wang, X. (2022) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3530019.3535308 · Cited by 1 · see citing works
A Systematic Review of Grey Literature on Outreach Activity Management Tools
Fell, L., Dalepiane, I., Fialho, M. L., & Bernardino, M. (2024) · Anais do ... Simpósio Brasileiro de Informática na Educação/Anais do Simpósio Brasileiro de Informática na Educação · doi:10.5753/sbie.2024.244978 · Cited by 1 · see citing works
What Evidence We Would Miss If We Do Not Use Grey Literature?Preprint
Kamei, F., Pinto, G., Wiese, I., Ribeiro, M., & Soares, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.05792
de Lima, L. F., & Peres, L. M. (2021) · doi:10.5753/opensciense.2021.17138
Shanmugapriya, S., & P.Devika, M. (2023) · International Journal of Engineering Technology and Management Sciences · doi:10.46647/ijetms.2023.v07i02.077
Pérez, M. R., & Lee-Maturana, S. (2023) · International Journal of Membrane Science and Technology · doi:10.15379/ijmst.v10i2.2591
Privacy-Decision Assisting Techniques
Kqiku, L. (2024) · doi:10.53846/goediss-10852
Current Design Practices in Applied Augmented Reality Research: A Methodological Review
Teutenberg, L. D., Fitz, L. R. G., & Scheeg, J. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-82633-7_9
Shukla, A., Santosh, B., & Vinod, S. (2026) · F1000Research · doi:10.12688/f1000research.171576.2
Louw, L., & Smuts, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-06164-5_3
Zeybek, H. (2025) · Lecture notes in mobility · doi:10.1007/978-3-032-04774-8_81
Shukla, A., Santosh, B., & Vinod, S. (2025) · F1000Research · doi:10.12688/f1000research.171576.1
The innovation in Indonesian language for foreign speaker teaching: A systematic literature review
Faizin, Isnaini, M., & Khoiriyah (2025) · Multidisciplinary Reviews · doi:10.31893/multirev.2026331
Adapting Globally Distributed Scrum Team Strategies for University Classrooms
Ramos, C., Reynolds, S., Vonderhaar, L., Newcomb, A., Ochoa, O., Towhidnejad, M., & Pembridge, J. J. (2025) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie63693.2025.11328543
Accounting for secure software development: Status, trends and business focus: A scoping study
Abeysekera, I. (2026) · Array · doi:10.1016/j.array.2026.100754
Industrial practices for evaluating quality of open source software: a practitioner survey
Yılmaz, N., Kolukısa, A., & Özkaya, M. (2026) · Software Quality Journal · doi:10.1007/s11219-026-09757-4
Shankar, G., Uddin, M. R., Mukta, S. H., Kumar, P., Islam, S., & Islam, N. (2026) · Data & Policy · doi:10.1017/dap.2026.10072
Non-functional requirements: The issues online forum users are discussing about
Mombrey, C., Kollikowski, L., & Herzwurm, G. (2026) · IWSiB@ICSE · doi:10.1145/3786158.3788559
Kalatehjari, R., Rotimi, F. E., Markose, S., & Moshood, T. D. (2026) · Sustainability · doi:10.3390/su18147151
Mahmoudi, B., Chenail-Larcher, Z., Cherief, H. A., Stiévenart, Q., Moha, N., & Avellaneda, F. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793304
Uncertainty in Process Mining: Survey and Conceptual Framework
Lepsien, A., Aleknonytė‐Resch, M., Weidlich, M., & Koschmider, A. (2026) · ACM Computing Surveys · doi:10.1145/3839360
Javvadi, V. P., Ganapathy, R. S., Sri, M. S., Nekkalapudi, K., Pochincharla, S., & Akula, T. V. (2026) · International Conference on Intelligent and Sustainable AI Systems (ICOSAAS) · doi:10.1109/icosaas68663.2026.11648759
- Authors
- Yasin, A., Fatima, R., Wen, L., Afzal, W., Azhar, M., & Torkar, R.
- Affiliations
- Blekinge Institute of Technology (SE)
- Tsinghua University (CN)
- Mälardalen University (SE)
- Shenzhen University (CN)
- University of Gothenburg (SE)
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 36226-36243
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3004454911
- Cited by
- 85
- References
- 162
- Citations by year
- 2026: 13 · 2025: 17 · 2024: 15 · 2023: 13 · 2022: 11 · 2021: 11 · 2020: 5
- FWCI
- 2.07
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- IEEE Access (publishedVersion, other-oa)
- DiVA (Blekinge Institute of Technology) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- European Union’s Horizon 2020 Research and Innovation Programme (871319)
- Knowledge Foundation (20160139 (TestMine))
- National Basic Research Program of China (2019YFB1704003)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Business, Management and Accounting › Management Information Systems › Optics and Image Analysis
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- grey literature, Google Scholar, systematic reviews, software engineering, publication bias, search engine effectiveness, conference papers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
