CoChat
Sign Up
Cited by 79Privacy, Security, and Data ProtectionMobile Health and mHealth ApplicationsPrivacy-Preserving Technologies in Data

Assessing the Privacy of mHealth Apps for Self-Tracking: Heuristic Evaluation Approach

Open access
Hutton, Luke, Price, Blaine A, Kelly, Ryan, McCormick, Ciaran, Bandara, Arosha K, Hatzakis, Tally, Meadows, Maureen, Nuseibeh, Bashar·2018·JMIR mHealth and uHealth
DOI 10.2196/mhealth.9217PMC PMC6231850

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: The recent proliferation of self-tracking technologies has allowed individuals to generate significant quantities of data about their lifestyle. These data can be used to support health interventions and monitor outcomes. However, these data are often stored and processed by vendors who have commercial motivations, and thus, they may not be treated with the sensitivity with which other medical data are treated. As sensors and apps that enable self-tracking continue to become more sophisticated, the privacy implications become more severe in turn. However, methods for systematically identifying privacy issues in such apps are currently lacking. OBJECTIVE: The objective of our study was to understand how current mass-market apps perform with respect to privacy. We did this by introducing a set of heuristics for evaluating privacy characteristics of self-tracking services. METHODS: Using our heuristics, we conducted an analysis of 64 popular self-tracking services to determine the extent to which the services satisfy various dimensions of privacy. We then used descriptive statistics and statistical models to explore whether any particular categories of an app perform better than others in terms of privacy. RESULTS: We found that the majority of services examined failed to provide users with full access to their own data, did not acquire sufficient consent for the use of the data, or inadequately extended controls over disclosures to third parties. Furthermore, the type of app, in terms of the category of data collected, was not a useful predictor of its privacy. However, we found that apps that collected health-related data (eg, exercise and weight) performed worse for privacy than those designed for other types of self-tracking. CONCLUSIONS: Our study draws attention to the poor performance of current self-tracking technologies in terms of privacy, motivating the need for standards that can ensure that future self-tracking apps are stronger with respect to upholding users' privacy. Our heuristic evaluation method supports the retrospective evaluation of privacy in self-tracking apps and can be used as a prescriptive framework to achieve privacy-by-design in future apps.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/mhealth.9217

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Recent studies report privacy violations being a common occurrence with health and wellbeing (HWB) applications.

    “The current situation is alarming, as recent studies report on privacy violations being a common occurrence with health and wellbeing (HWB) applications [26], [27].”

    From Security and Privacy of mHealth Applications: A Scoping Review (2020) · cites this paper as [27]

    The Abstract (Results) and Discussion (Principal Findings) report widespread privacy shortcomings in self-tracking apps, including insufficient consent and limited control over data disclosures; health-related apps performed worse than other self-tracking apps. The Results describe these findings as suggesting a heightened risk of privacy violations, rather than establishing that actual violations commonly occurred. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 40

  1. [2]Jones SL Kelly R Dealing With Information Overload in Multifaceted Personal Informatics Systems Human–Computer Interaction 2017 05 16 33 1 1 48 10.1080/07370024.2017.1302334 · DOI
  2. [3]Rooksby J Rost M Morrison A Chalmers M Personal tracking as lived informatics 2014 CHI '14 the SIGCHI Conference on Human Factors in Computing Systems April 26 - May 01, 2014 Toronto, Canada ACM Press 1163 1172
  3. [4]Stone AA Shiffman S Schwartz JE Broderick JE Hufford MR Patient compliance with paper and electronic diaries Controlled Clinical Trials 2003 4 24 2 182 199 10.1016/S0197-2456(02)00320-3 12689739 · DOI · Paper page
  4. [5]Price BA Kelly R Mehta V McCormick C Ahmed H Pearce O Feel My Pain: Design and Evaluation of Painpad, a Tangible Device for Supporting Inpatient Self-Logging of Pain 2018 CHI '18 the 2018 CHI Conference on Human Factors in Computing Systems April 21 - 26, 2018 Montreal, Canada New York ACM Press 169
  5. [6]Shklovski I Mainwaring S Skúladóttir H Borgthorsson H Leakiness and creepiness in app space: perceptions of privacy and mobile app use 2014 CHI '14 the SIGCHI Conference on Human Factors in Computing Systems April 26 - May 01, 2014 Toronto, Canada New York ACM Press 2347 2356
  6. [7]Rosenfeld L Torous J Vahia IV Data Security and Privacy in Apps for Dementia: An Analysis of Existing Privacy Policies Am J Geriatr Psychiatry 2017 08 25 8 873 877 10.1016/j.jagp.2017.04.009 28645535 S1064-7481(17)30301-9 28645535 · DOI · Paper page
  7. [8]Dehling T Gao F Schneider S Sunyaev A Exploring the Far Side of Mobile Health: Information Security and Privacy of Mobile Health Apps on iOS and Android JMIR Mhealth Uhealth 2015 01 19 3 1 e8 10.2196/mhealth.3672 25599627 v3i1e8 25599627 PMC4319144 · DOI · Paper page
  8. [9]Paul G Irvine J Privacy Implications of Wearable Health Devices 2014 SIN '14 the 7th International Conference on Security of Information and Networks September 09 - 11, 2014 Glascow, Scotland New York ACM Press 117 10.1145/2659651.2659683 · DOI
  9. [13]Hutton L Henderson T Cerquitelli T Quercia D Pasquale F Beyond the EULA: Improving Consent for Data Mining Transparent Data Mining for Big and Small Data. Studies in Big Data 2017 New York Springer 147 167
  10. [14]Bellotti V Sellen A Design for privacy in ubiquitous computing environments 1993 ECSCW'93 the Third European Conference on Computer-Supported Cooperative Work 13–17 September 1993 Milan, Italy Dordrecht Springer 77 92
  11. [15]Cavoukian A Stoddart J Dix A Nemec I Peep V Shroff M Resolution on Privacy by Design 2010 10 27 32nd International Conference of Data Protection and Privacy Commissioners 27-29 Oct 2010 Jerusalem, Israel
  12. [16]Hoepman JH Cuppens-Boulahia N Cuppens F Jajodia S Abou El Kalam A Sans T Privacy Design Strategies ICT Systems Security and Privacy Protection. SEC 2014. IFIP Advances in Information and Communication Technology 2014 Berlin Springer 446 459
  13. [17]Clarke R Privacy impact assessment: Its origins and development Computer Law & Security Review 2009 1 25 2 123 135 10.1016/j.clsr.2009.02.002 · DOI · Paper page
  14. [18]Oetzel MC Spiekermann S A systematic methodology for privacy impact assessments: a design science approach Eur J Inf Syst 2013 7 23 23 2 126 150 10.1057/ejis.2013.18 · DOI
  15. [19]Wright D Should privacy impact assessments be mandatory? Commun. ACM 2011 08 01 54 8 121 131 10.1145/1978542.1978568 · DOI
  16. [21]Perera C McCormick C Bandara A Price B Nuseibeh B Privacy-by-Design Framework for Assessing Internet of Things Applications and Platforms 2016 6th Int Conf on the Internet of Things 7-9 Nov 2016 Stuttgart, Germany New York ACM Press 83 92 10.1145/2991561.2991566 · DOI
  17. [22]Jensen C Potts C Experimental Evaluation of a Lightweight Method for Augmenting Requirements Analysis 2007 1st ACM Int Workshop Empir Assess Softw Eng Lang Technol 5 Nov 2007 Atlanta, Georgia New York ACM Press 49 54 10.1145/1353673.1353684 · DOI
  18. [23]Jamal A Cole M A Heuristic Evaluation of the Facebook's Advertising Tool Beacon 2009 First Int Conf Inf Sci Eng Internet IEEE 2009 Nanjing, China 1527 1530 10.1109/icise.2009.45 · DOI
  19. [24]Nielsen J Molich R Heuristic evaluation of user interfaces 1990 CHI '90 the SIGCHI Conference on Human Factors in Computing Systems 1-5 April 1990 Seattle, Washington New York ACM Press 10.1145/97243.97281 · DOI · Paper page
  20. [25]Sutcliffe A Gault B Heuristic evaluation of virtual reality applications Interacting with Computers 2004 8 16 4 831 849 10.1016/j.intcom.2004.05.001 · DOI
  21. [26]Furano RF Kushniruk A Barnett J Deriving a Set of Privacy Specific Heuristics for the Assessment of PHRs (Personal Health Records) Stud Health Technol Inform 2017 234 125 130 28186028 28186028
  22. [27]Hutton L Quantified Self Privacy Heuristics v1 2017 10 23 2018-09-19 10.21954/ou.rd.5514082 · DOI
  23. [29]Paul G Irvine J Privacy Implications of Wearable Health Devices ACM Press 2014 SIN '14 the 7th International Conference on Security of Information and Networks 9-11 Sept 2014 Glascow, Scotland New York ACM Press
  24. [30]Tankard C What the GDPR means for businesses Network Security 2016 06 2016 6 5 8 10.1016/s1353-4858(16)30056-3 · DOI
  25. [31]Inostroza R Rusu C Roncagliolo S Rusu V Usability Heuristics for Touchscreen-based Mobile Devices: Update 2013 ChileCHI '13 the 2013 Chilean Conference on Human - Computer Interaction 11-15 Nov 2013 Temuco, Chile New York ACM Press 24 29 10.1145/2535597.2535602 · DOI
  26. [33]Mohamed I Patel D Android vs iOS Security: A Comparative Study 2015 12th International Conference on Information Technology - New Generations 13-15 April 2015 Las Vegas New York IEEE A 10.1109/ITNG.2015.123 · DOI
  27. [34]London Quantified Self Meetup Group 2014 Survey Data Internet 2014 2014 10.21954/ou.rd.5505805 · DOI
  28. [35]Wearables unit shipments worldwide by vendor from 1Q'14 to 2Q'18 (in millions) 2018 2009-09-20 Statista https://www.statista.com/statistics/435933/quarterly-wearables-shipments-worldwide-by-vendor/
  29. [39]Cohen J A Coefficient of Agreement for Nominal Scales Educational and Psychological Measurement 1960 04 01 20 1 37 46 10.1177/001316446002000104 · DOI · Paper page
  30. [40]Agresti A Categorical Data Analysis, 3rd Edition 2013 01 London Wiley
  31. [41]Racherla P Furner C Babb J Conceptualizing the Implications of Mobile App Usage and Stickiness: A Research Agenda SSRN Journal 2012 12 9 10.2139/ssrn.2187056 · DOI
  32. [42]Luger E Moran S Rodden T Consent for all: revealing the hidden complexity of terms and conditions 2013 CHI '13 the SIGCHI Conference on Human Factors in Computing Systems April 27 - May 02, 2013 Paris, France New York ACM Press 2687 2696 10.1145/2470654.2481371 · DOI
  33. [43]Obar JA Oeldorf-Hirsch A The Biggest Lie on the Internet: Ignoring the Privacy Policies and Terms of Service Policies of Social Networking Services 2018 08 18 TPRC 44: The 44th Research Conference on Communication, Information and Internet Policy 30 Sep - 1 Oct 2016 Arlington, VA 1 20 10.2139/ssrn.2757465 · DOI
  34. [45]Ayalon O Toch E Retrospective Privacy: Managing Longitudinal Privacy in Online Social Networks 2013 SOUPS '13 the Ninth Symposium on Usable Privacy and Security 24-26 July 2013 Newcastle, UK New York ACM Press 4 10.1145/2501604.2501608 · DOI
  35. [46]Hutton L Henderson T I didn't sign up for this!: Informed consent in social network research 2015 the 9th International AAAI Conference on Web and Social Media (ICWSM) 26-29 May 2015 Oxford, UK Palo Alto, CA The AAAI Press 178 187
  36. [47]Phua J Jin SV Kim JJ Uses and gratifications of social networking sites for bridging and bonding social capital: A comparison of Facebook, Twitter, Instagram, and Snapchat Computers in Human Behavior 2017 07 72 115 122 10.1016/j.chb.2017.02.041 · DOI · Paper page
  37. [48]Kamal N Fels S Ho K Online Social Networks for Personal Informatics to Promote Positive Health Behavior 2010 WSM '10 the second ACM SIGMM workshop on Social media 25 Oct 2010 Firenze, Italy New York ACM Press 47 52 10.1145/1878151.1878167 · DOI
  38. [50]Martin K Nissenbaum H Measuring privacy: an empirical test using context to expose confounding variables Columbia Science and Technology Law Review 2016 18 176
  39. [51]Nissenbaum H Privacy in context: technology, policy, and the integrity of social life 2010 Stanford, CA Stanford University Press
  40. [52]Slavin R Wang X Hosseini M Hester J Krishnan R Bhatia J Breaux T Niu J Toward a framework for detecting privacy policy violations in android application code 2016 ICSE '16 the 38th International Conference on Software Engineering May 14 - 22, 2016 Austin, TX New York ACM Press 25 36

Cited by

Sorted by citations

Authors
Hutton, L., Price, B. A., Kelly, R., McCormick, C., Bandara, A. K., Hatzakis, T., Meadows, M., & Nuseibeh, B.
Affiliations
  • The Open University (GB)
  • The University of Melbourne (AU)
  • Coventry (United Kingdom) (GB)
Corresponding author
Price, Blaine A
ORCID
Venue
JMIR mHealth and uHealth
Publisher
JMIR Publications Inc.
Published
2018-10-22
Volume
6
Issue
10
Pages
e185
ISSN
2291-5222 (ISSN-L)
DOI
10.2196/mhealth.9217
PMC
PMC6231850
OpenAlex
W2844894817
Cited by
79
References
1
Citations by year
2026: 7 · 2025: 8 · 2024: 8 · 2023: 10 · 2022: 12 · 2021: 12 · 2020: 14 · 2019: 8
FWCI
19.34
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR mhealth and uhealth (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Arrow@dit (Dublin Institute of Technology) (acceptedVersion)
  • University of Limerick Institutional Repository (University of Limerick) (acceptedVersion, cc-by-nc-sa)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
  • Open Research Online (The Open University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by-nc-sa)
  • Pure (Coventry University) (submittedVersion, cc-by)
  • University of Limerick Institutional Repository (University of Limerick) (submittedVersion, cc-by-nc-sa)
  • University of Limerick Institutional Repository (University of Limerick)
Funders
  • Science Foundation Ireland (3/RC/2094)
  • Engineering and Physical Sciences Research Council (EP/P01013X/1, EP/R013144/1, EP/L021285/1)
  • FP7 Ideas: European Research Council (291652)
Topics
  • Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
Keywords
self-tracking, mobile health apps, privacy, heuristic evaluation, privacy by design, user consent, data access control, health data, privacy standards

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review