AppWatcher: unveiling the underground market of trading mobile app reviews
Abstract
Driven by huge monetary reward, some mobile application (app) developers turn to the underground market to buy positive reviews instead of doing legal advertisements. These promotion reviews are either directly posted in app stores like iTunes and Google Play, or published on some popular websites that have many app users. Until now, a clear understanding of this app promotion underground market is still lacking. In this work, we focus on unveiling this underground market and statistically analyzing the promotion incentives, characteristics of promoted apps and suspicious reviewers. To collect promoted apps, we built an automatic data collection system, AppWatcher, which monitored 52 paid review service providers for four months and crawled all the app metadata from their corresponding app stores. Finally, AppWatcher exposes 645 apps promoted in app stores and 29, 680 apps promoted in some popular websites. The current underground market is then reported from various perspectives (e.g., service price, app volume). We identified some interesting features of both promoted apps and suspicious reviewers, which are significantly different from those of randomly chosen apps. Finally, we built a simple tracer to narrow down the suspect list of promoted apps in the underground market.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Identifying organic ASO activities is an open problem due to the ability of such workers to evade existing detection solutions.
“Identifying organic ASO activities is an open problem due to the ability of such workers to evade existing detection solutions, e.g., that leverage lockstep behaviors [32, 43, 51, 72, 73, 78, 85, 86, 89, 92, 93] or review bursts [27, 28, 35, 39, 40, 42, 44, 45, 50-53, 53, 5759, 65, 66, 84, 87, 88, 92].”
From RacketStore (2021) · cites this paper as [86]
Cites
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172 · see citing works
Jindal, N., & Liu, B. (2008) · Proceedings of the international conference on Web search and web data mining - WSDM '08 · doi:10.1145/1341531.1341560 · Cited by 1,542 · see citing works
Finding Deceptive Opinion Spam by Any Stretch of the Imagination
Ott, M., Choi, Y., Cardie, C., & Hancock, J. T. (2011) · arXiv · doi:10.48550/arxiv.1107.4557 · Cited by 1,008 · see citing works
Who are the crowdworkers?: shifting demographics in mechanical turk
Ross, J., Irani, L., Silberman, M. S., Zaldivar, A., & Tomlinson, B. (2010) · CHI '10 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1753846.1753873 · Cited by 1,004 · see citing works
Analyzing the Amazon Mechanical Turk marketplace
Ipeirotis, P. G. (2010) · XRDS Crossroads The ACM Magazine for Students · doi:10.1145/1869086.1869094 · Cited by 890 · see citing works
Spotting fake reviewer groups in consumer reviews
Mukherjee, A., Liu, B., & Glance, N. (2012) · Proceedings of the 21st international conference on World Wide Web · doi:10.1145/2187836.2187863 · Cited by 773 · see citing works
Demographics of Mechanical Turk
Ipeirotis, P. G. (2010) · Faculty Digital Archive (New York University Florence) · Cited by 494 · see citing works
Learning to identify review spam
Li, F., Huang, M., Yang, Y., & Zhu, X. (2011) · doi:10.5591/978-1-57735-516-8/ijcai11-414 · Cited by 391 · see citing works
Review Graph Based Online Store Review Spammer Detection
Wang, G., Xie, S., Liu, B., & Yu, P. S. (2011) · Proceedings · doi:10.1109/icdm.2011.124 · Cited by 390 · see citing works
Exploiting Burstiness in Reviews for Review Spammer Detection
Fei, G., Mukherjee, A., Liu, B., Hsu, M., Castellanos, M., & Ghosh, R. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14400 · Cited by 352 · see citing works
Finding unusual review patterns using unexpected rules
Jindal, N., Liu, B., & Lim, E. (2010) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1871437.1871669 · Cited by 345 · see citing works
Trafficking fraudulent accounts: the role of the underground market in Twitter spam and abuse
Thomas, K., McCoy, D., Grier, C., Kolcz, A., & Paxson, V. (2013) · Cited by 239 · see citing works
Mukherjee, A., Liu, B., Wang, J., Glance, N., & Jindal, N. (2011) · International World Wide Web Conference (WWW) · doi:10.1145/1963192.1963240 · Cited by 235 · see citing works
Blocking Blog Spam with Language Model Disagreement
Mishne, G., Carmel, D., & Lempel, R. (2005) · UvA-DARE (University of Amsterdam) · Cited by 210 · see citing works
Serf and turf: crowdturfing for fun and profit
Wang, G., Wilson, C., Zhao, X., Zhu, Y., Mohanlal, M., Zheng, H., & Zhao, B. Y. (2012) · Proceedings of the 21st international conference on World Wide Web · doi:10.1145/2187836.2187928 · Cited by 197 · see citing works
Re: CAPTCHAs – Understanding CAPTCHA-Solving Services in an Economic Context
Motoyama, M., Levchenko, K., Kanich, C., McCoy, D., Voelker, G. M., & Savage, S. (2010) · Cited by 188 · see citing works
SVMs for the Blogosphere: Blog Identification and Splog Detection
Kolari, P., Finin, T., & Joshi, A. (2006) · Cited by 154 · see citing works
Identifying spam in the iOS app store
Chandy, R., & Gu, H. (2012) · doi:10.1145/2184305.2184317 · Cited by 64 · see citing works
GroupTie: toward hidden collusion group discovery in app stores
Xie, Z., & Zhu, S. (2014) · Proceedings of the 2014 ACM conference on Security and privacy in wireless & mobile networks · doi:10.1145/2627393.2627409 · Cited by 19 · see citing works
Self-citationNEIGHBORWATCHER: A Content-Agnostic Comment Spam Inference System.
Zhang, J., & Gu, G. (2013) · Cited by 13 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Survey of App Store Analysis for Software Engineering
Martin, W., Sarro, F., Jia, Y., Zhang, Y., & Harman, M. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2630689 · Cited by 459 · see citing works
Sadeghi, A., Bagheri, H., Garcia, J., & Malek, S. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2615307 · Cited by 144 · see citing works
Toward Engineering a Secure Android Ecosystem
Xu, M., Song, C., Yang, J., Shih, M.-W., Lu, K., Zheng, C., Duan, R., Jang, Y., Lee, B., Qian, C., et al. (2016) · ACM Computing Surveys · doi:10.1145/2963145 · Cited by 90 · see citing works
App Store Effects on Software Engineering Practices
Al-Subaihin, A. A., Sarro, F., Black, S., Capra, L., & Harman, M. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2891715 · Cited by 90 · see citing works
Design and testing of a mobile health application rating tool
Levine, D. M., Co, Z., Newmark, L. P., Groisser, A. R., Holmgren, A. J., Haas, J. S., & Bates, D. W. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0268-9 · Cited by 71 · see citing works
Dating with Scambots: Understanding the Ecosystem of Fraudulent Dating Applications
Hu, Y., Wang, H., Zhou, Y., Guo, Y., Li, L., Luo, B., & Xu, F. (2019) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2019.2908939 · Cited by 43 · see citing works
Toward Detecting Collusive Ranking Manipulation Attackers in Mobile App Markets
Chen, H., He, D., Zhu, S., & Yang, J. (2017) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3052973.3053022 · Cited by 33 · see citing works
Self-citationSmartphone psychology: New approaches towards safe and efficacious mobile mental health apps.
Marshall, J. M., Dunstan, D. A., & Bartik, W. (2020) · Professional Psychology: Research and Practice · doi:10.1037/pro0000278 · Cited by 25 · see citing works
The Art and Craft of Fraudulent App Promotion in Google Play
Rahman, M., Hernandez, N., Recabarren, R., Ahmed, S. I., & Carbunar, B. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3345658 · Cited by 24 · see citing works
Marshall, J. M., Dunstan, D. A., & Bartik, W. (2020) · JMIR Research Protocols · doi:10.2196/17159 · Cited by 24 · see citing works
A Multi-modal Neural Embeddings Approach for Detecting Mobile Counterfeit Apps
Rajasegaran, J., Karunanayake, N., Gunathillake, A., Seneviratne, S., & Jourjon, G. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313427 · Cited by 19 · see citing works
Uncovering download fraud activities in mobile app markets
Dou, Y., Li, W., Liu, Z., Dong, Z., Luo, J., & Yu, P. S. (2019) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/3341161.3345306 · Cited by 18 · see citing works
Karunanayake, N., Rajasegaran, J., Gunathillake, A., Seneviratne, S., & Jourjon, G. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3007260 · Cited by 15 · see citing works
Crowdsourced App Review Manipulation
Li, S., Caverlee, J., Niu, W., & Kaghazgaran, P. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080741 · Cited by 14 · see citing works
You can promote, but you can't hide
Xie, Z., Zhu, S., Li, Q., & Wang, W. (2016) · Annual Computer Security Applications Conference · doi:10.1145/2991079.2991099 · Cited by 12 · see citing works
Self-citationTreating Psychological Trauma in the Midst of COVID-19: The Role of Smartphone Apps
Marshall, J. M., Dunstan, D. A., & Bartik, W. J. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00402 · Cited by 12 · see citing works
Understanding the Market-Level and Network-Level Behaviors of the Android Malware Ecosystem
Yang, C., Zhang, J., & Gu, G. (2017) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2017.297 · Cited by 11 · see citing works
Characterizing the dynamics and evolution of incentivized online reviews on Amazon
Jamshidi, S., Rejaie, R., & Li, J. (2019) · Social Network Analysis and Mining · doi:10.1007/s13278-019-0563-0 · Cited by 10 · see citing works
Understanding Promotion-as-a-Service on GitHub
Du, K., Yang, H., Zhang, Y., Duan, H., Wang, H., Hao, S., Li, Z., & Yang, M. (2020) · Annual Computer Security Applications Conference · doi:10.1145/3427228.3427258 · Cited by 9 · see citing works
Search Rank Fraud De-Anonymization in Online Systems
Rahman, M., Hernandez, N., Carbunar, B., & Chau, D. H. (2018) · Proceedings of the 29th on Hypertext and Social Media · doi:10.1145/3209542.3209555 · Cited by 8 · see citing works
A Typology of Socialbots (Abbrev.)
Maus, G. (2017) · ACM Web Science Conference (WebSci) · doi:10.1145/3091478.3098860 · Cited by 8 · see citing works
What You See Isn't Always What You Get
Liu, W., Zhang, Y., Li, Z., & Duan, H. (2016) · doi:10.1145/2994459.2994472 · Cited by 8 · see citing works
Quality Prediction of Wearable Apps in the Google Play Store
Siddiqui, S., Faisal, M. S., Khurram, S., Irshad, A., Baz, M., Hamam, H., Iqbal, N., & Shafiq, M. (2021) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2022.022266 · Cited by 8 · see citing works
Detecting incentivized review groups with co-review graph
Zhang, Y., Hao, S., & Wang, H. (2021) · High-Confidence Computing · doi:10.1016/j.hcc.2021.100006 · Cited by 7 · see citing works
Trojan Horses in Amazon's Castle: Understanding the Incentivized Online Reviews
Jamshidi, S., Rejaie, R., & Li, J. (2018) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2018.8508267 · Cited by 6 · see citing works
A multiview clustering framework for detecting deceptive reviews
Zhang, Y., Wang, H., & Stavrou, A. (2023) · Journal of Computer Security · doi:10.3233/jcs-220001 · Cited by 6 · see citing works
Stratify Mobile App Reviews: E-LDA Model Based on Hot "Entity" Discovery
Liu, Y., Li, Y., Guo, Y., & Zhang, M. (2016) · International Conference on Signal-Image Technology & Internet-Based Systems (SITIS) · doi:10.1109/sitis.2016.97 · Cited by 5 · see citing works
Characterizing Promotional Attacks in Mobile App Store
Sun, B., Luo, X., Akiyama, M., Watanabe, T., & Mori, T. (2017) · Communications in computer and information science · doi:10.1007/978-981-10-5421-1_10 · Cited by 4 · see citing works
Dating with Scambots: Understanding the Ecosystem of Fraudulent Dating ApplicationsPreprint
Hu, Y., Wang, H., Zhou, Y., Guo, Y., Li, L., Luo, B., & Xu, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.04901 · Cited by 4 · see citing works
Hernandez, N., Recabarren, R., Carbunar, B., & Ahmed, S. I. (2021) · Proceedings of the 21st ACM Internet Measurement Conference · doi:10.1145/3487552.3487837 · Cited by 3 · see citing works
Sun, B., Luo, X., Akiyama, M., Watanabe, T., & Mori, T. (2018) · Journal of Information Processing · doi:10.2197/ipsjjip.26.212 · Cited by 3 · see citing works
A Hybrid Health Journey Recommender System Using Electronic Medical Records.
Jamshidi, S., Torkamani, M. A., Mellen, J., Jhaveri, M. A., Pan, P., Chung, J., & Kardes, H. (2018) · Cited by 3 · see citing works
Review Trade: Everything Is Free in Incentivized Review Groups
Zhang, Y., Hao, S., & Wang, H. (2020) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-63086-7_19 · Cited by 1 · see citing works
A Neural Embeddings Approach for Detecting Mobile Counterfeit AppsPreprint
Rajasegaran, J., Seneviratne, S., & Jourjon, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.09882 · Cited by 1 · see citing works
A Case Study for the Enhanced Performance Measurement Model in Mobile Business Environment
Kim, A., Lee, J., & Kim, M. (2016) · International Conference on Platform Technology and Service · doi:10.1109/platcon.2016.7456773
EFEKTIVITAS APLIKASI TUMBASIN.ID PADA MASA PANDEMI COVID-19 DI KOTA SEMARANG
Nafiah, Z., & Hartarini, Y. M. (2022) · EKOMA Jurnal Ekonomi Manajemen Akuntansi · doi:10.56799/ekoma.v2i1.878
Measuring and Understanding Crowdturfing in the App Store
Hu, Q., Zhang, X., Li, F., Tang, Z., & Wang, S. (2023) · Information · doi:10.3390/info14070393
Understanding Mobile App Review Ranking Manipulation for Illicit Online Promotion
Wu, Y., Chi, J., Liao, X., Zhao, Y. J., Hong, Z., & Ji, S. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3698591
- Authors
- Xie, Z., & Zhu, S.
- Affiliations
- Pennsylvania State University (US)
- ORCID
- Zhu, Sencun
- Venue
- Proceedings of the 8th ACM Conference on Security & Privacy in Wireless and Mobile Networks
- Publisher
- ACM
- Published
- 2015-06-22
- Pages
- 1-11
- OpenAlex
- W2019333963
- Cited by
- 38
- References
- 20
- Citations by year
- 2026: 1 · 2023: 2 · 2022: 1 · 2021: 3 · 2020: 6 · 2019: 7 · 2018: 6 · 2017: 5 · 2016: 7
- FWCI
- 5.21
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec) (publishedVersion)
- Funders
- National Science Foundation (CCF- 1320605)
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Information Systems › Cybercrime and Law Enforcement Studies
- Social Sciences › Sociology and Political Science › Digital Marketing and Social Media
- Keywords
- review spam, promotion incentives
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
