The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
User feedback in the appstore: An empirical study
Abstract
Application distribution platforms - or app stores - such as Google Play or Apple AppStore allow users to submit feedback in form of ratings and reviews to downloaded applications. In the last few years, these platforms have become very popular to both application developers and users. However, their real potential for and impact on requirements engineering processes are not yet well understood. This paper reports on an exploratory study, which analyzes over one million reviews from the Apple AppStore. We investigated how and when users provide feedback, inspected the feedback content, and analyzed its impact on the user community. We found that most of the feedback is provided shortly after new releases, with a quickly decreasing frequency over time. Reviews typically contain multiple topics, such as user experience, bug reports, and feature requests. The quality and constructiveness vary widely, from helpful advices and innovative ideas to insulting offenses. Feedback content has an impact on download numbers: positive messages usually lead to better ratings and vice versa. Negative feedback such as shortcomings is typically destructive and misses context details and user experience. We discuss our findings and their impact on software and requirements engineering teams.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mining association rules between sets of items in large databases
Agrawal, R., Imieliński, T., & Swami, A. (1993) · Proceedings of the 1993 ACM SIGMOD international conference on Management of data · doi:10.1145/170035.170072 · Cited by 14,907 · cited 1×
“Itemset mining [1] is a data mining method for discovering relationships between different variables based on their co-occurrence in databases.”
MethodThe Content Analysis Guidebook
Neuendorf, K. A. (2017) · doi:10.4135/9781071802878 · Cited by 9,286 · cited 1×
“To explore the feedback content and find included topics, two researchers independently performed a manual content analysis [11] of a random sample from our data set.”
MethodLead Users: A Source of Novel Product Concepts
von Hippel, E. (1986) · Management Science · doi:10.1287/mnsc.32.7.791 · Cited by 4,650
Hartigan, J. A., & Hartigan, P. M. (1985) · The Annals of Statistics · doi:10.1214/aos/1176346577 · Cited by 1,973
Scalable algorithms for association mining
Zaki, M. J. (2000) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/69.846291 · Cited by 1,751
Selecting Empirical Methods for Software Engineering Research
Easterbrook, S., Singer, J., Storey, M.-A., & Damian, D. (2008) · Guide to Advanced Empirical Software Engineering · doi:10.1007/978-1-84800-044-5_11 · Cited by 1,167
Detecting repackaged smartphone applications in third-party android marketplaces
Zhou, W., Zhou, Y., Jiang, X., & Ning, P. (2012) · Proceedings of the second ACM conference on Data and Application Security and Privacy · doi:10.1145/2133601.2133640 · Cited by 590
Zimmermann, T., Premraj, R., Bettenburg, N., Just, S., Schroter, A., & Weiss, C. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2010.63 · Cited by 337
App store mining and analysis: MSR for app stores
Harman, M., Jia, Y., & Zhang, Y. (2012) · 2012 9th IEEE Working Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2012.6224306 · Cited by 254
App StoreMining and Analysis: MSR for App Stores
Harman, M., Jia, Y., & Zhang, Y. (2012) · Cited by 235
Cited by
How Do Users Like This Feature? A Fine Grained Sentiment Analysis of App Reviews
Guzman, E., & Maalej, W. (2014) · 2014 IEEE 22nd International Requirements Engineering Conference (RE) · doi:10.1109/re.2014.6912257 · Cited by 590
Self-citationAR-miner: mining informative reviews for developers from mobile app marketplace
Chen, N., Lin, J., Hoi, S. C. H., Xiao, X., & Zhang, B. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568263 · Cited by 517
A Survey of App Store Analysis for Software Engineering
Martin, W., Sarro, F., Jia, Y., Zhang, Y., & Harman, M. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2630689 · Cited by 459
“[42], 2013 a,g WWW 5,059 Pagano and Maalej [175], 2013 a RE 1,100 Hoon et al.”
BackgroundHow can i improve my app? Classifying user reviews for software maintenance and evolution
Panichella, S., Di Sorbo, A., Guzman, E., Visaggio, C. A., Canfora, G., & Gall, H. C. (2015) · 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsm.2015.7332474 · Cited by 455
Bug report, feature request, or simply praise? On automatically classifying app reviews
Maalej, W., & Nabil, H. (2015) · 2015 IEEE 23rd International Requirements Engineering Conference (RE) · doi:10.1109/re.2015.7320414 · Cited by 453
Self-citationA survey of the use of crowdsourcing in software engineering
Mao, K., Capra, L., Harman, M., & Jia, Y. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2016.09.015 · Cited by 385
A systematic literature review: Opinion mining studies from mobile app store user reviews
Genc-Nayebi, N., & Abran, A. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2016.11.027 · Cited by 358
What would users change in my app? summarizing app reviews for recommending software changes
Di Sorbo, A., Panichella, S., Alexandru, C. V., Shimagaki, J., Visaggio, C. A., Canfora, G., & Gall, H. C. (2016) · Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering · doi:10.1145/2950290.2950299 · Cited by 294
On the automatic classification of app reviews
Maalej, W., Kurtanović, Z., Nabil, H., & Stanik, C. (2016) · Requirements Engineering · doi:10.1007/s00766-016-0251-9 · Cited by 280
Self-citationRelease planning of mobile apps based on user reviews
Villarroel, L., Bavota, G., Russo, B., Oliveto, R., & Di Penta, M. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884818 · Cited by 261
Dehling, T., Gao, F., Schneider, S., & Sunyaev, A. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3672 · Cited by 243
Nicholas, J., Fogarty, A. S., Boydell, K., & Christensen, H. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7273 · Cited by 158
Ensemble Methods for App Review Classification: An Approach for Software Evolution (N)
Guzman, E., El-Haliby, M., & Bruegge, B. (2015) · 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.88 · Cited by 143
Studying the dialogue between users and developers of free apps in the Google Play Store
Hassan, S., Tantithamthavorn, C., Bezemer, C.-P., & Hassan, A. E. (2018) · Empirical Software Engineering · doi:10.1007/s10664-017-9538-9 · Cited by 100
Malik, T., Ambrose, A. J., & Sinha, C. (2022) · JMIR Human Factors · doi:10.2196/35668 · Cited by 83
Generating product reviews from aspect-based ratings using large language models
Pandey, P., & Singh, J. P. (2025) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2025.104244 · Cited by 19
Du, Y., Choe, S., Vega, J., Liu, Y., & Trujillo, A. (2022) · JMIR Pediatrics and Parenting · doi:10.2196/28661 · Cited by 14
Metadata
- Authors
- Pagano, D., & Maalej, W.
- Venue
- 2013 21st IEEE International Requirements Engineering Conference (RE)
- Publisher
- IEEE
- Published
- 2013-07
- Pages
- 125-134
- OpenAlex
- W2085465189
- Cited by
- 540
- References
- 17
- FWCI
- 25.56
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.