The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Natural Language Processing for Requirements Engineering: A Systematic Mapping Study
Open accessAbstract
Natural Language Processing for Requirements Engineering (NLP4RE) is an area of research and development that seeks to apply natural language processing (NLP) techniques, tools, and resources to the requirements engineering (RE) process, to support human analysts to carry out various linguistic analysis tasks on textual requirements documents, such as detecting language issues, identifying key domain concepts, and establishing requirements traceability links. This article reports on a mapping study that surveys the landscape of NLP4RE research to provide a holistic understanding of the field. Following the guidance of systematic review, the mapping study is directed by five research questions, cutting across five aspects of NLP4RE research, concerning the state of the literature, the state of empirical research, the research focus, the state of tool development, and the usage of NLP technologies. Our main results are as follows: (i) we identify a total of 404 primary studies relevant to NLP4RE, which were published over the past 36 years and from 170 different venues; (ii) most of these studies (67.08%) are solution proposals, assessed by a laboratory experiment or an example application, while only a small percentage (7%) are assessed in industrial settings; (iii) a large proportion of the studies (42.70%) focus on the requirements analysis phase, with quality defect detection as their central task and requirements specification as their commonly processed document type; (iv) 130 NLP4RE tools (i.e., RE specific NLP tools) are extracted from these studies, but only 17 of them (13.08%) are available for download; (v) 231 different NLP technologies are also identified, comprising 140 NLP techniques, 66 NLP tools, and 25 NLP resources, but most of them—particularly those novel NLP techniques and specialized tools—are used infrequently; by contrast, commonly used NLP technologies are traditional analysis techniques (e.g., POS tagging and tokenization), general-purpose tools (e.g., Stanford CoreNLP and GATE) and generic language lexicons (WordNet and British National Corpus). The mapping study not only provides a collection of the literature in NLP4RE but also, more importantly, establishes a structure to frame the existing literature through categorization, synthesis and conceptualization of the main theoretical concepts and relationships that encompass both RE and NLP aspects. Our work thus produces a conceptual framework of NLP4RE. The framework is used to identify research gaps and directions, highlight technology transfer needs, and encourage more synergies between the RE community, the NLP one, and the software and systems practitioners. Our results can be used as a starting point to frame future studies according to a well-defined terminology and can be expanded as new technologies and novel solutions emerge.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Case Study Research: Design and Methods
Chaiklin, H. (1991) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199102000-00025 · Cited by 39,535
(1998) · Sage eBooks · Cited by 38,631
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
Experimentation in Software Engineering
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2012) · doi:10.1007/978-3-642-29044-2 · Cited by 4,460
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921
The long tail: why the future of business is selling less of more
(2007) · Choice Reviews Online · doi:10.5860/choice.44-2783 · Cited by 2,869
Guidelines for conducting systematic mapping studies in software engineering: An update
Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2015.03.007 · Cited by 2,669
Brereton, P., Kitchenham, B. A., Budgen, D., Turner, M., & Khalil, M. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2006.07.009 · Cited by 2,394
Natural language processing: an introduction
Nadkarni, P. M., Ohno-Machado, L., & Chapman, W. W. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000464 · Cited by 1,975
Advances in natural language processing
Hirschberg, J., & Manning, C. D. (2015) · Science · doi:10.1126/science.aaa8685 · Cited by 1,712
Experimentation in software engineering
(2013) · Choice Reviews Online · doi:10.5860/choice.50-5018 · Cited by 1,517
Preliminary guidelines for empirical research in software engineering
Kitchenham, B. A., Pfleeger, S. L., Pickard, L. M., Jones, P. W., Hoaglin, D. C., El Emam, K., & Rosenberg, J. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2002.1027796 · Cited by 1,510
Boote, D. N., & Beile, P. (2005) · Educational Researcher · doi:10.3102/0013189x034006003 · Cited by 1,212
Jumping NLP Curves: A Review of Natural Language Processing Research [Review Article]
Cambria, E., & White, B. (2014) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2014.2307227 · Cited by 1,102
Recommended Steps for Thematic Synthesis in Software Engineering
Cruzes, D. S., & Dyba, T. (2011) · 2011 International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2011.36 · Cited by 960
Requirements engineering paper classification and evaluation criteria: a proposal and a discussion
Wieringa, R., Maiden, N., Mead, N., & Rolland, C. (2006) · Requirements Engineering · doi:10.1007/s00766-005-0021-6 · Cited by 940
Evidence-Based Software Engineering and Systematic Reviews
Kitchenham, B. A., Budgen, D., & Brereton, P. (2015) · Chapman & Hall/CRC eBooks · doi:10.1201/b19467 · Cited by 759
Using mapping studies as the basis for further research – A participant-observer case study
Kitchenham, B. A., Budgen, D., & Pearl Brereton, O. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.011 · Cited by 621
ERASER: A Benchmark to Evaluate Rationalized NLP Models
DeYoung, J., Jain, S., Rajani, N. F., Lehman, E., Xiong, C., Socher, R., & Wallace, B. C. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.408 · Cited by 561
Advancing candidate link generation for requirements tracing: the study of methods
Hayes, J. H., Dekhtyar, A., & Sundaram, S. K. (2006) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2006.3 · Cited by 481
A Survey of App Store Analysis for Software Engineering
Martin, W., Sarro, F., Jia, Y., Zhang, Y., & Harman, M. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2630689 · Cited by 459
Cross versus Within-Company Cost Estimation Studies: A Systematic Review
Kitchenham, B. A., Mendes, E., & Travassos, G. H. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.1001 · Cited by 424
The Future of Empirical Methods in Software Engineering Research
Sjoberg, D. I. K., Dyba, T., & Jorgensen, M. (2007) · Future of Software Engineering (FOSE '07) · doi:10.1109/fose.2007.30 · Cited by 389
Analyzing Regulatory Rules for Privacy and Security Requirements
Breaux, T. D., & Anton, A. I. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.70746 · Cited by 363
Naming the pain in requirements engineering: Contemporary problems, causes, and effects in practice
Fernández, D. M., Wagner, S., Kalinowski, M., Felderer, M., Mafra, P., Vetrò, A., Conte, T., Christiansson, M.-T., Greer, D., Lassenius, C., et al. (2017) · Empirical Software Engineering · doi:10.1007/s10664-016-9451-7 · Cited by 300
Automated classification of non-functional requirements
Cleland-Huang, J., Settimi, R., Zou, X., & Solc, P. (2007) · Requirements Engineering · doi:10.1007/s00766-007-0045-1 · Cited by 294
Priest, H., Roberts, P., & Woods, L. (2002) · Nurse Researcher · doi:10.7748/nr2002.10.10.1.30.c5877 · Cited by 278
Empirical software engineering experts on the use of students and professionals in experiments
Falessi, D., Juristo, N., Wohlin, C., Turhan, B., Münch, J., Jedlitschka, A., & Oivo, M. (2018) · Empirical Software Engineering · doi:10.1007/s10664-017-9523-3 · Cited by 257
A systematic review of evaluation of variability management approaches in software product lines
Chen, L., & Ali Babar, M. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.006 · Cited by 247
Goal-oriented requirements engineering: an extended systematic mapping study
Horkoff, J., Aydemir, F. B., Cardoso, E., Li, T., Maté, A., Paja, E., Salnitri, M., Piras, L., Mylopoulos, J., & Giorgini, P. (2019) · Requirements Engineering · doi:10.1007/s00766-017-0280-z · Cited by 211
On the reliability of mapping studies in software engineering
Wohlin, C., Runeson, P., da Mota Silveira Neto, P. A., Engström, E., do Carmo Machado, I., & de Almeida, E. S. (2013) · Journal of Systems and Software · doi:10.1016/j.jss.2013.04.076 · Cited by 206
Encyclopedia of Library and Information Science
(2020) · Lexikon des gesamten Buchwesens Online · doi:10.1163/9789004337862__com_050369 · Cited by 198
Applications of ontologies in requirements engineering: a systematic review of the literature
Dermeval, D., Vilela, J., Bittencourt, I. I., Castro, J., Isotani, S., Brito, P., & Silva, A. (2016) · Requirements Engineering · doi:10.1007/s00766-015-0222-6 · Cited by 198
Binkhonain, M., & Zhao, L. (2019) · Expert Systems with Applications: X · doi:10.1016/j.eswax.2019.100001 · Cited by 194
Self-citationAnt Colony Optimization for Software Project Scheduling and Staffing with an Event-Based Scheduler
Chen, W.-N., & Zhang, J. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.17 · Cited by 180
Casamayor, A., Godoy, D., & Campo, M. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2009.10.010 · Cited by 177
Rapid quality assurance with Requirements Smells
Femmer, H., Méndez Fernández, D., Wagner, S., & Eder, S. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2016.02.047 · Cited by 176
A systematic review of transformation approaches between user requirements and analysis models
Yue, T., Briand, L. C., & Labiche, Y. (2011) · Requirements Engineering · doi:10.1007/s00766-010-0111-y · Cited by 170
A natural language approach for Requirements Engineering
Rolland, C., & Proix, C. (1992) · Notes on Numerical Fluid Mechanics and Multidisciplinary Design · doi:10.1007/bfb0035136 · Cited by 167
Cooper, I. D. (2016) · Journal of the Medical Library Association : JMLA · doi:10.3163/1536-5050.104.1.013 · Cited by 164
On the pragmatic design of literature studies in software engineering: an experience-based guideline
Kuhrmann, M., Fernández, D. M., & Daneva, M. (2017) · Empirical Software Engineering · doi:10.1007/s10664-016-9492-y · Cited by 161
On-demand feature recommendations derived from mining public product descriptions
Dumitru, H., Gibiec, M., Hariri, N., Cleland‐Huang, J., Mobasher, B., Castro-Herrera, C., & Mirakhorli, M. T. (2011) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1985793.1985819 · Cited by 146
Nicolás, J., & Toval, A. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2009.04.001 · Cited by 144
Empirical research methods reported in high-profile LIS journal literature
Hider, P. M., & Pymm, B. (2008) · Library & Information Science Research · doi:10.1016/j.lisr.2007.11.007 · Cited by 129
Proceedings of the 11th International Conference on Computational Semantics
Hunter, J., Asher, N. M., & Lascarides, A. (2015) · The Association for Computational Linguistics · Cited by 119
Falessi, D., Cantone, G., & Canfora, G. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.122 · Cited by 118
Automated Extraction and Clustering of Requirements Glossary Terms
Arora, C., Sabetzadeh, M., Briand, L., & Zimmer, F. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2635134 · Cited by 111
Detecting requirements defects with NLP patterns: an industrial experience in the railway domain
Ferrari, A., Gori, G., Rosadini, B., Trotta, I., Bacherini, S., Fantechi, A., & Gnesi, S. (2018) · Empirical Software Engineering · doi:10.1007/s10664-018-9596-7 · Cited by 111
Self-citationBakar, N. H., Kasirun, Z. M., & Salleh, N. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2015.05.006 · Cited by 101
State of practice in requirements engineering: contemporary data
Kassab, M., Neill, C. J., & Laplante, P. A. (2014) · Innovations in Systems and Software Engineering · doi:10.1007/s11334-014-0232-4 · Cited by 98
NLP-assisted software testing: A systematic mapping of the literature
Garousi, V., Bauer, S., & Felderer, M. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106321 · Cited by 96
Resolving Ambiguities in Natural Language Software Requirements
Shah, U. S., & Jinwala, D. C. (2015) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/2815021.2815032 · Cited by 94
Proceedings of the 28th IEEE International Requirements Engineering Conference, RE’20
Breaux, T. D., Zisman, A., Fricker, S., & Glinz, M. (2020) · Requirements Engineering · Cited by 91
Shallow knowledge as an aid to deep understanding in early phase requirements engineering
Sawyer, P., Rayson, P., & Cosh, K. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.129 · Cited by 88
Detecting terminological ambiguity in user stories: Tool and experimentation
Dalpiaz, F., van der Schalk, I., Brinkkemper, S., Aydemir, F. B., & Lucassen, G. (2018) · Information and Software Technology · doi:10.1016/j.infsof.2018.12.007 · Cited by 87
Summarizing Software Artifacts: A Literature Review
Nazar, N., Hu, Y. Z., & Jiang, H. (2016) · Journal of Computer Science and Technology · doi:10.1007/s11390-016-1671-1 · Cited by 85
REQUIREMENTS TRACEABILITY: A SYSTEMATIC REVIEW AND INDUSTRY CASE STUDY
Torkar, R., Gorschek, T., FELDT, R. H., Svahnberg, M., Raja, U. A., & Kamran, K. (2012) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s021819401250009x · Cited by 80
A systematic literature review of software requirements reuse approaches
Irshad, M., Petersen, K., & Poulding, S. (2017) · Information and Software Technology · doi:10.1016/j.infsof.2017.09.009 · Cited by 76
Natural Language Requirements Processing: A 4D Vision
Ferrari, A., Dell’Orletta⋄, F., Esuli, A., Gervasi, V., & Gnesi, S. (2017) · IEEE Software · doi:10.1109/ms.2017.4121207 · Cited by 65
Self-citationTavakoli, M. A., Zhao, L., Heydari, A., & Nenadić, G. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.05.037 · Cited by 61
Self-citationEmpirical research methodologies and studies in Requirements Engineering: How far did we come?
Daneva, M., Damian, D., Marchetto, A., & Pastor, Ó. (2014) · Journal of Systems and Software · doi:10.1016/j.jss.2014.06.035 · Cited by 54
Perryman, C. (2016) · Journal of the Medical Library Association JMLA · doi:10.3163/1536-5050.104.1.014 · Cited by 42
Empirical comparison of text-based mobile apps similarity measurement techniques
Al-Subaihin, A. A., Sarro, F., Black, S., & Capra, L. (2019) · Empirical Software Engineering · doi:10.1007/s10664-019-09726-5 · Cited by 37
Meidan, A., García-García, J. A., Ramos, I. M., & Escalona, M. J. (2018) · ACM Computing Surveys · doi:10.1145/3186888 · Cited by 33
Mining textual requirements to assist architectural software design: a state of the art review
Casamayor, A., Godoy, D. L., & Campo, M. (2011) · Artificial Intelligence Review · doi:10.1007/s10462-011-9237-7 · Cited by 23
User Studies on End-User Service Composition: A Literature Review and a Design Framework
Zhao, L., Loucopoulos, P., Kavakli, E., & Letsholo, K. J. (2019) · ACM Transactions on the Web · doi:10.1145/3340294 · Cited by 12
Self-citation
Cited by
An Empirical Study on Using Large Language Models to Analyze Software Supply Chain Security Failures
Singla, T., Anandayuvaraj, D., Kalu, K. G., Schorlemmer, T. R., & Davis, J. C. (2023) · Proceedings of the 2023 Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses · doi:10.1145/3605770.3625214 · Cited by 26
Pre-trained model-based NFR classification: Overcoming limited data challenges
Rahman, K., Ghani, A., Alzahrani, A., Tariq, M. U., & Rahman, A. U. (2023) · IEEE Access · doi:10.1109/access.2023.3301725 · Cited by 26
Metadata
- Authors
- Zhao, L., Alhoshan, W., Ferrari, A., Letsholo, K. J., Ajagbe, M. A., Chioasca, E.-V., & Batista-Navarro, R. T.
- Venue
- ACM Computing Surveys
- Published
- 2021-04-17
- Volume
- 54
- Issue
- 3
- Pages
- 1-41
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3153389197
- Cited by
- 322
- References
- 74
- FWCI
- 40.08
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.