The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews
Abstract
App stores allow users to submit feedback for downloaded apps in form of star ratings and text reviews. Recent studies analyzed this feedback and found that it includes information useful for app developers, such as user requirements, ideas for improvements, user sentiments about specific features, and descriptions of experiences with these features. However, for many apps, the amount of reviews is too large to be processed manually and their quality varies largely. The star ratings are given to the whole app and developers do not have a mean to analyze the feedback for the single features. In this paper we propose an automated approach that helps developers filter, aggregate, and analyze user reviews. We use natural language processing techniques to identify fine-grained app features in the reviews. We then extract the user sentiments about the identified features and give them a general score across all reviews. Finally, we use topic modeling techniques to group fine-grained features into more meaningful high-level features. We evaluated our approach with 7 apps from the Apple App Store and Google Play Store and compared its results with a manually, peer-conducted analysis of the reviews. On average, our approach has a precision of 0.59 and a recall of 0.51. The extracted features were coherent and relevant to requirements evolution tasks. Our approach can help app developers to systematically analyze user opinions about single features and filter irrelevant reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 3×
“We group collocations whose pairs of words are synonyms and use Wordnet as a synonym dictionary.”
MethodInfluentialFoundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
The Content Analysis Guidebook
Neuendorf, K. A. (2017) · doi:10.4135/9781071802878 · Cited by 9,286 · cited 2×
“The methodological cornerstone for the creation of the truth set was the use of content analysis techniques as described by Neuendorf [20] and by Maalej and Robillard [16].”
MethodFeature-Oriented Domain Analysis (FODA) Feasibility Study
Kang, K. C., Cohen, S. G., Hess, J. A., Novak, W. E., & Peterson, A. S. (1990) · Defense Technical Information Center (DTIC) · doi:10.21236/ada235785 · Cited by 4,282
Natural language processing with Python
Bird, S., Klein, E., & Loper, E. (2009) · CERN Document Server (European Organization for Nuclear Research) · Cited by 3,955
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/jmlr.2003.3.4-5.993 · Cited by 3,024
Extracting product features and opinions from reviews
Popescu, A.-M., & Etzioni, O. (2005) · Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05 · doi:10.3115/1220575.1220618 · Cited by 1,698
Sentiment strength detection in short informal text
Thelwall, M., Buckley, K., Paltoglou, G., Cai, D., & Kappas, A. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21416 · Cited by 1,449
Mining opinion features in customer reviews
Hu, M., & Liu, B. (2004) · National Conference on Artificial Intelligence · Cited by 1,341
Sentiment strength detection for the social web
Thelwall, M., Buckley, K., & Paltoglou, G. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21662 · Cited by 1,062
Topic sentiment mixture: modeling facets and opinions in weblogs
Mei, Q., Ling, X., Wondra, M., Su, H., & Zhai, C. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242596 · Cited by 801
User feedback in the appstore: An empirical study
Pagano, D., & Maalej, W. (2013) · 2013 21st IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2013.6636712 · Cited by 540
Self-citationRetrieving and analyzing mobile apps feature requests from online reviews
Iacob, C., & Harrison, R. (2013) · 2013 10th Working Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2013.6624001 · Cited by 333
App store mining and analysis: MSR for app stores
Harman, M., Jia, Y., & Zhang, Y. (2012) · 2012 9th IEEE Working Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2012.6224306 · Cited by 254
Beginning Behavioral Research: A Conceptual Primer
Rosnow, R. L., & Rosenthal, R. (1993) · Medical Entomology and Zoology · Cited by 239
App StoreMining and Analysis: MSR for App Stores
Harman, M., Jia, Y., & Zhang, Y. (2012) · Cited by 235
Analysis of user comments: An approach for software requirements evolution
Carreno, L. V. G., & Winbladh, K. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606604 · Cited by 216
Patterns of Knowledge in API Reference Documentation
Maalej, W., & Robillard, M. P. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.12 · Cited by 183
Self-citationA large-scale sentiment analysis for Yahoo! answers
Küçüktunç, O., Cambazoglu, B. B., Weber, I. G., & Ferhatosmanoğlu, H. (2012) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2124295.2124371 · Cited by 183
On-demand feature recommendations derived from mining public product descriptions
Dumitru, H., Gibiec, M., Hariri, N., Cleland‐Huang, J., Mobasher, B., Castro-Herrera, C., & Mirakhorli, M. T. (2011) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1985793.1985819 · Cited by 166
Beyond the stars: exploiting free-text user reviews to improve the accuracy of movie recommendations
Jakob, N., Weber, S. H., Müller, M. C., & Gurevych, I. (2009) · Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion · doi:10.1145/1651461.1651473 · Cited by 158
Supporting Domain Analysis through Mining and Recommending Features from Online Product Listings
Hariri, N., Castro-Herrera, C., Mirakhorli, M. T., Cleland‐Huang, J., & Mobasher, B. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2013.39 · Cited by 102
Using Mobile RE Tools to Give End-Users Their Own Voice
Seyff, N., Gräf, F., & Maiden, N. (2010) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re.2010.15 · Cited by 94
When users become collaborators
Maalej, W., Happel, H.-J., & Rashid, A. (2009) · Companion to the ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA Companion) · doi:10.1145/1639950.1640068 · Cited by 86
Self-citationA user satisfaction analysis approach for software evolution
Li, H., Li, Z., Li, Z., et al. (2010) · IEEE International Conference on Progress in Informatics and Computing · doi:10.1109/pic.2010.5687999 · Cited by 46
Cited by
A survey on sentiment analysis challenges
Hussein, D. M. E.-D. M. (2018) · Journal of King Saud University - Engineering Sciences · doi:10.1016/j.jksues.2016.04.002 · Cited by 659
A Survey of App Store Analysis for Software Engineering
Martin, W., Sarro, F., Jia, Y., Zhang, Y., & Harman, M. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2630689 · Cited by 459
“[36], 2014 g PIR 6,938 Guzman and Maalej [88], 2014 a,g RE 7 Khalid et al.”
BackgroundMethodHow can i improve my app? Classifying user reviews for software maintenance and evolution
Panichella, S., Di Sorbo, A., Guzman, E., Visaggio, C. A., Canfora, G., & Gall, H. C. (2015) · 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsm.2015.7332474 · Cited by 455
Self-citationBug report, feature request, or simply praise? On automatically classifying app reviews
Maalej, W., & Nabil, H. (2015) · 2015 IEEE 23rd International Requirements Engineering Conference (RE) · doi:10.1109/re.2015.7320414 · Cited by 453
Self-citationA survey of the use of crowdsourcing in software engineering
Mao, K., Capra, L., Harman, M., & Jia, Y. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2016.09.015 · Cited by 385
Metadata
- Authors
- Guzman, E., & Maalej, W.
- Venue
- 2014 IEEE 22nd International Requirements Engineering Conference (RE)
- Publisher
- IEEE
- Published
- 2014-08
- Pages
- 153-162
- OpenAlex
- W2074975950
- Cited by
- 590
- References
- 26
- FWCI
- 23.72
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.