Towards understanding and detecting fake reviews in app stores
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
App stores include an increasing amount of user feedback in form of app ratings and reviews. Research and recently also tool vendors have proposed analytics and data mining solutions to leverage this feedback to developers and analysts, e.g., for supporting release decisions. Research also showed that positive feedback improves apps’ downloads and sales figures and thus their success. As a side effect, a market for fake, incentivized app reviews emerged with yet unclear consequences for developers, app users, and app store operators. This paper studies fake reviews, their providers, characteristics, and how well they can be automatically detected. We conducted disguised questionnaires with 43 fake review providers and studied their review policies to understand their strategies and offers. By comparing 60,000 fake reviews with 62 million reviews from the Apple App Store we found significant differences, e.g., between the corresponding apps, reviewers, rating distribution, and frequency. This inspired the development of a simple classifier to automatically detect fake reviews in app stores. On a labelled and imbalanced dataset including one-tenth of fake reviews, as reported in other domains, our classifier achieved a recall of 91% and an AUC/ROC value of 98%. We discuss our findings and their impact on software engineering, app users, and app store operators.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Effect size estimates: Current use, calculations, and interpretation.
Fritz, C. O., Morris, P. E., & Richler, J. J. (2012) · Journal of Experimental Psychology: General · doi:10.1037/a0024338 · Cited by 5,199 · cited 2×
“For Wilcoxon tests we calculated the r value, dividing the z distribution by the square root of the number of samples (Fritz et al., 2012).”
BackgroundMethodJindal, N., & Liu, B. (2008) · Proceedings of the international conference on Web search and web data mining - WSDM '08 · doi:10.1145/1341531.1341560 · Cited by 1,542
Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud
Luca, M., & Zervas, G. (2016) · Management Science · doi:10.1287/mnsc.2015.2304 · Cited by 1,263 · cited 2×
“With an amount of 10% fake reviews (101), as reported to be typical for other domains (Sussin and Thompson 2018; Luca and Zervas 2016; Mukherjee et al. 2013a), the recall of the RF and DT algorithm are nearly identical (RF: 0.”
BackgroundUncovering social spammers: social honeypots + machine learning
Lee, K., Caverlee, J., & Webb, S. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835522 · Cited by 619 · cited 2×
“Other researchers (Dickerson et al. 2014; Lee et al. 2010; Feng et al. 2012; Park et al. 2016) followed this approach and were also able to improve their classification results.”
BackgroundA Survey of App Store Analysis for Software Engineering
Martin, W., Sarro, F., Jia, Y., Zhang, Y., & Harman, M. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2630689 · Cited by 459
Follow the green: growth and dynamics in twitter follower markets
Stringhini, G., Wang, G., Egele, M., Kruegel, C., Vigna, G., Zheng, H., & Zhao, B. Y. (2013) · Proceedings of the 2013 conference on Internet measurement conference · doi:10.1145/2504730.2504731 · Cited by 169 · cited 1×
“10,000 Twitter followers can be bought for a price of $4 (Stringhini et al., 2013).”
Background
Cited by
Meel, P., & Vishwakarma, D. K. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112986 · Cited by 505
Haque, M. D. R., & Rubya, S. (2023) · JMIR mHealth and uHealth · doi:10.2196/44838 · Cited by 404
“• Length: As shorter reviews might not provide deeper insights in general and are frequently false or promotional in nature [70], the minimum character length was considered 200 for the scope of our study.”
InfluentialFake online reviews: Literature review, synthesis, and directions for future research
Wu, Y., Ngai, E. W. T., Wu, P., & Wu, C. (2020) · Decision Support Systems · doi:10.1016/j.dss.2020.113280 · Cited by 359
Talwar, S., Dhir, A., Singh, D., Virk, G. S., & Salo, J. (2020) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2020.102197 · Cited by 289
SoK: Content Moderation in Social Media, from Guidelines to Enforcement, and Research to Practice
Singhal, M., Ling, C., Paudel, P., Thota, P., Kumarswamy, N., Stringhini, G., & Nilizadeh, S. (2023) · 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp57164.2023.00056 · Cited by 57
“Most social media platforms do not publicly share the details of their tools used to detect misinformation or offensive content, however, in the literature, AI-based models are proposed to detect specific types of misinformation about COVID19 and vaccinations [186], [273], security tools [287], hate speech, and offens…”
Background
Metadata
- Authors
- Martens, D., & Maalej, W.
- Venue
- Empirical Software Engineering
- Published
- 2019-05-10
- Volume
- 24
- Issue
- 6
- Pages
- 3316-3355
- ISSN
- 1382-3256 (ISSN-L)
- 1573-7616
- OpenAlex
- W2941394980
- Cited by
- 137
- References
- 7
- FWCI
- 18.74
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Towards understanding and detecting fake reviews in app stores. https://app.cochat.ai/papers/10.1007/s10664-019-09706-9
- The paper
- Martens, D., & Maalej, W. (2019). Towards understanding and detecting fake reviews in app stores. Empirical Software Engineering. https://doi.org/10.1007/s10664-019-09706-9
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.