CoChat
Sign Up
Cited by 659Mobile Health and mHealth ApplicationsHealth, Technology, Consumer BehaviorDigital Mental Health Interventions

Mobile medical and health apps: state of the art, concerns, regulatory control and certification

Open access
Kamel Boulos, Maged N., Brewer, Ann C., Karimkhani, Chante, Buller, David B., Dellavalle, Robert P.·2014·Online Journal of Public Health Informatics
DOI 10.5210/ojphi.v5i3.4814PMC PMC3959919
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper examines the state of the art in mobile clinical and health-related apps. A 2012 estimate puts the number of health-related apps at no fewer than 40,000, as healthcare professionals and consumers continue to express concerns about the quality of many apps, calling for some form of app regulatory control or certification to be put in place. We describe the range of apps on offer as of 2013, and then present a brief survey of evaluation studies of medical and health-related apps that have been conducted to date, covering a range of clinical disciplines and topics. Our survey includes studies that highlighted risks, negative issues and worrying deficiencies in existing apps. We discuss the concept of 'apps as a medical device' and the relevant regulatory controls that apply in USA and Europe, offering examples of apps that have been formally approved using these mechanisms. We describe the online Health Apps Library run by the National Health Service in England and the calls for a vetted medical and health app store. We discuss the ingredients for successful apps beyond the rather narrow definition of 'apps as a medical device'. These ingredients cover app content quality, usability, the need to match apps to consumers' general and health literacy levels, device connectivity standards (for apps that connect to glucometers, blood pressure monitors, etc.), as well as app security and user privacy. 'Happtique Health App Certification Program' (HACP), a voluntary app certification scheme, successfully captures most of these desiderata, but is solely focused on apps targeting the US market. HACP, while very welcome, is in ways reminiscent of the early days of the Web, when many "similar" quality benchmarking tools and codes of conduct for information publishers were proposed to appraise and rate online medical and health information. It is probably impossible to rate and police every app on offer today, much like in those early days of the Web, when people quickly realised the same regarding informational Web pages. The best first line of defence was, is, and will always be to educate consumers regarding the potentially harmful content of (some) apps.

Integrity signals

Registry

Crossref · 10.5210/ojphi.v5i3.4814

Cites

86 of 86 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 138 cited sources checked · 0 supported · 0 flagged

  • Findingnone given

    The majority of evaluation studies of health and healthcare-related apps have been conducted on small patient populations to compare app efficacy versus conventional resources.

    “The majority of these studies have been conducted on small patient populations to compare the efficacy of a smartphone/tablet app versus conventional resources.”
    Quote in the paperNot a substantial claim
  • Premisecitation

    A study found 55 unique mobile apps for HIV and STD education, prevention, testing, and resources.

    “A study of available apps demonstrated 55 unique mobile apps for HIV (human immunodeficiency virus) and STD education, prevention, testing and resources [41].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [41] Muessig KE Pike EC Legrand S Hightow-Weidman LB 2013 Mobile phone applications for the care and prevention of HIV and other sexually transmitted diseases: a review. J Med Internet Res. 15(1), e1 10.2196/jmir.2301 23291245 PMC3636069

      Not checked in the free run.

  • Premisecitation

    A 2012 survey of healthcare workers found that the most popular mobile application functions were drug referencing, clinical decision support, communication, electronic health-record access, and medical education materials.

    “In a study published in 2012, a group of surveyed healthcare workers indicated the most popular categories of mobile applications functions include drug-referencing tools, clinical decision-support tools, communication, electronic health-record system access and medical education materials [7].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Conn J 2012 Most-healthful apps. Mod Healthc. 42(50), 30- 32 23323343

      Not checked in the free run.

  • Premisecitation

    Text message reminders may improve antibiotic compliance.

    “Text message reminders to patients may also improve antibiotic compliance [48] or even prevent recurrent cardiovascular events [49].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [48] Suffoletto B Calabria J Ross A Callaway C Yealy DM 2012 A mobile phone text message program to measure oral antibiotic use and provide feedback on adherence to patients discharged from the emergency department. Acad Emerg Med. 19(8), 949- 58 10.1111/j.1553-2712.2012.01411.x 22849787

      Not checked in the free run.

  • Premisecitation

    The top medical provider apps included Epocrates, Lexicomp, UpToDate, and Medscape.

    “The top apps were drug-reference guides such as ‘Epocrates’ (Epocrates, Inc.) [8] and ‘Lexicomp’ (Wolters Kluwer) [9], as well as clinical decision-support reference tools such as ‘UpToDate’ (Wolters Kluwer) [10] and ‘Medscape’ (WebMD LLC) [11].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [8] Epocrates - http://www.epocrates.com/

      Not checked in the free run.

    • cites [9] Lexicomp - http://www.lexi.com/

      Not checked in the free run.

    • cites [10] UpToDate - http://www.uptodate.com/

      Not checked in the free run.

  • May combine two claims. 27
  • Classified as probably not a substantial claim. 39

  • May combine two claims.

    Premisecitation

    Evaluations of text messaging show promising results for clinical monitoring, counselling, appointment keeping, smoking cessation, weight loss, chronic disease management, and sun protection reminders.

    “Evaluation of text messaging shows promising results for assisting with clinical monitoring and counselling, keeping medical appointments, smoking cessation, weight loss, chronic disease management (e.g., diabetes care monitoring), or reminding people to use sun protection to prevent sunburns in the near term and skin cancer in the long term [43-46].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [43] Fjeldsoe BS Marshall AL Miller YD 2009 Behavior change interventions delivered by mobile telephone short-message service. Am J Prev Med. 36(2), 165- 73.. 10.1016/j.amepre.2008.09.040 19135907

      Not checked in the free run.

    • cites [44] Riley WT Rivera DE Atienza AA Nilsen W Allison SM 2011 Health behavior models in the age of mobile interventions: are our theories up to the task? Transl Behav Med. 1(1), 53- 71 10.1007/s13142-011-0021-7 21796270 PMC3142960

      Not checked in the free run.

    • cites [45] Klasnja P Pratt W 2012 Healthcare in the pocket: mapping the space of mobile-phone health interventions. J Biomed Inform. 45(1), 184- 98.. 10.1016/j.jbi.2011.08.017 21925288 PMC3272165

      Not checked in the free run.

    • cites [46] Armstrong AW Watson AJ Makredes M Frangos JE Kimball AB 2009 Text-message reminders to improve sunscreen use: a randomized, controlled trial using electronic monitoring. Arch Dermatol. 145(11), 1230- 36.. 10.1001/archdermatol.2009.269 19917951

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In the United States, 85% of adults own a cell phone and 53% of adults own a smartphone.

    “85% of US adults own a cell phone; of them, 53% own smartphones.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] Fox S, Duggan M. Mobile Health 2012 Washington, DC, Pew Research Center’s Internet & American Life Project 2012. Available at http://pewinternet.org/~/media//Files/Reports/2012/PIP_MobileHealth2012_FINAL.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    One-fifth of US smartphone owners have health apps on their devices.

    “One-fifth of smartphone owners have health apps on their devices [5].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] Fox S, Duggan M. Mobile Health 2012 Washington, DC, Pew Research Center’s Internet & American Life Project 2012. Available at http://pewinternet.org/~/media//Files/Reports/2012/PIP_MobileHealth2012_FINAL.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Half of US smartphone owners use their devices to obtain health information.

    “Half of smartphone owners use their devices to get health information.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] Fox S, Duggan M. Mobile Health 2012 Washington, DC, Pew Research Center’s Internet & American Life Project 2012. Available at http://pewinternet.org/~/media//Files/Reports/2012/PIP_MobileHealth2012_FINAL.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A 2012 estimate places the number of health-related apps at no fewer than 40,000.

    “A 2012 estimate puts the number of health-related apps at no fewer than 40,000 [6].”
    • Also stated in: § Abstract · hedged
      “A 2012 estimate puts the number of health-related apps at no fewer than 40,000”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] Pelletier SG. Explosive Growth in Health Care Apps Raises Oversight Questions. AAMC Reporter 2012, Oct. Available at https://www.aamc.org/newsroom/reporter/october2012/308516/health-care-apps.html

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Smartphones out-shipped feature phones worldwide for the first time in the first quarter of 2013.

    “Smartphones out-shipped feature phones worldwide for the first time in Q1 2013 [3,4].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [3] More Smartphones Were Shipped in Q1 2013 Than Feature Phones, An Industry First According to IDC (Press Release, 25 Apr 2013) - http://www.idc.com/getdoc.jsp?containerId=prUS24085413

      Not checked in the free run.

    • cites [4] Millar M. Market for feature phone apps is low-tech goldmine (BBC News, 7 May 2013) - http://www.bbc.co.uk/news/business-22390444

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    At least 47 iPhone apps for smoking cessation are available.

    “At least 47 iPhone apps for smoking cessation are available [30].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] Abroms LC Padmanabhan N Thaweethai L Phillips T 2011 iPhone apps for smoking cessation: a content analysis. Am J Prev Med. 40(3), 279- 85 10.1016/j.amepre.2010.10.032 21335258 PMC3395318

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    On the Android platform alone, over 80 diabetes apps offer functions including self-monitoring blood glucose recording, medication or insulin logs, and prandial insulin dose calculators.

    “On the Android platform alone, over 80 diabetes apps offer a variety of functions, including self-monitoring blood glucose recording, medication or insulin logs, and prandial insulin dose calculators [25].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [25] Demidowich AP Lu K Tamler R Bloomgarden Z 2012 An evaluation of diabetes self-management applications for Android smartphones. J Telemed Telecare. 18(4), 235- 38 10.1258/jtt.2012.111002 22604278

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    A 2012 survey of ACGME programmes found that over 85% of respondents used a smartphone and over half used mobile applications daily.

    “A survey published in 2012 of ACGME (US Accreditation Council for Graduate Medical Education) programmes demonstrated over 85% of respondents used a smartphone and over half of respondents used mobile applications on a daily basis, most commonly drug guides, medical calculators, coding/billing apps and pregnancy wheels [18].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18] Franko OI Tirrell TF 2012 Smartphone app use among medical providers in ACGME training programs. J Med Syst. 36(5), 3135- 39 10.1007/s10916-011-9798-7 22052129

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    HACP successfully captures most of the desiderata for successful apps, but is solely focused on apps targeting the US market.

    “‘Happtique Health App Certification Program’ (HACP), a voluntary app certification scheme, successfully captures most of these desiderata, but is solely focused on apps targeting the US market.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The Lifelens app captures high-resolution images of cells in a drop of blood using a Windows Phone with a micro lens and analyses them to detect malaria.

    “Using a Windows Phone with a micro lens mounted over the camera, the ‘Lifelens’ app captures high resolution images of cells in a drop of blood and then analyses them to detect to existence of malaria [70].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [70] Clayton S. Lifelens is testing for Malaria – on a Windows Phone (Next at Microsoft, 2 Aug 2011) - http://blogs.technet.com/b/next/archive/2011/08/02/lifelens-is-testing-for-malaria-on-a-windows-phone.aspx

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In a study of 40 patients, MMS-based teledermatology diagnosis was correct for 78% and provided management recommendations for 98% of patients.

    “The diagnosis based on multimedia message referral was correct for 78% and provided management recommendations for 98% of patients.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [63] Börve A Holst A Gente-Lidholm A Molina-Martinez R Paoli J 2012 Use of the mobile phone multimedia messaging service for teledermatology. J Telemed Telecare. 18(5), 292- 96 10.1258/jtt.2012.120206 22802521

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    An adaptor with ECG electrodes may transmit electrical data to detect abnormal heart rhythms in a non-hospital setting.

    “For example, an adaptor with electrocardiogram (ECG) electrodes may transmit electrical data to detect abnormal heart rhythms in a non-hospital setting [66].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [66] Lau JK Lowres N Neubeck L Brieger DB Sy RW 2013 iPhone ECG application for community screening to detect silent atrial fibrillation: A novel technology to prevent stroke. Int J Cardiol. 165(1), 193- 94.. 10.1016/j.ijcard.2013.01.220 23465249

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The industry requests concerned defining boundaries between FDA-regulated and non-regulated apps, clarifying medical versus wellness apps and diagnosing versus monitoring, setting a risk threshold, defining limits for device accessories, and handling modular apps.

    “The requests deal essentially with the need to crisply define the boundaries between apps that have to be regulated by the FDA and apps that do not have to go through the certification process.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [104] Strickland E. The FDA Takes On Mobile Health Apps--Regulation is coming soon to health apps. Here’s what to expect. IEEE Spectrum 2012, Sep 12: online - http://spectrum.ieee.org/biomedical/devices/the-fda-takes-on-mobile-health-apps

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The US FDA defined a small subset of mobile medical apps that may affect the performance or functionality of regulated medical devices and therefore require FDA oversight.

    “Through guidance first released in draft form in July 2011, the US FDA defined “a small subset of mobile medical apps that may impact on the performance or functionality of currently regulated medical devices and as such, will require FDA oversight” [99,100].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [99] US Food and Drug Administration. (FDA): Mobile Medical Applications (Last Updated: 22 Dec 2011) - http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/ConnectedHealth/MobileMedicalApplications/default.htm

      Not checked in the free run.

    • cites [100] US Food and Drug Administration. (FDA): Mobile Medical Applications: Draft Guidance for Industry and Food and Drug Administration Staff Silver Spring, MD: US Department of Health & Human Services; 2011 (July). Available at http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM263366.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Kharrazi et al. mention 19 apps that allow patients to store personal health records on their mobile devices.

    “Kharrazi et al. mention 19 apps that allow patients to store personal health records on their mobile devices [50].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [50] Kharrazi H Chisholm R VanNasdale D Thompson B 2012 Mobile personal health records: an evaluation of features and functionality. Int J Med Inform. 81(9), 579- 93 10.1016/j.ijmedinf.2012.04.007 22809779

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    A review by Demidowich et al. of 42 Android diabetes self-management apps found a mean composite usability score of 11.3 out of 30, and no apps offered direct data input from glucometers.

    “A review conducted by Demidowich et al. [25] investigated 42 Android apps for diabetes self-management. The mean composite usability score which evaluated six standard features per app was 11.3 out of a possible 30, and no apps offered direct data input from glucometers.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [25] Demidowich AP Lu K Tamler R Bloomgarden Z 2012 An evaluation of diabetes self-management applications for Android smartphones. J Telemed Telecare. 18(4), 235- 38 10.1258/jtt.2012.111002 22604278

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In that study, concordance between the two dermatologists on the MMS arm was 68%, compared to 88% between the face-to-face dermatologists.

    “Interestingly, there was a lower concordance (68%) between the two dermatologists on the MMS arm of the study compared to 88% between the face-to-face dermatologists [63].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [63] Börve A Holst A Gente-Lidholm A Molina-Martinez R Paoli J 2012 Use of the mobile phone multimedia messaging service for teledermatology. J Telemed Telecare. 18(5), 292- 96 10.1258/jtt.2012.120206 22802521

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Existing asthma self-management apps have worrying deficiencies, and none combine reliable information with appropriate supportive tools.

    “Huckvale et al. [96] and McKinstry [97] expressed their concerns about worrying deficiencies in existing asthma self-management apps, concluding that none combine reliable information and appropriate supportive tools, and that some are even unsafe.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [96] Huckvale K Car M Morrison C Car J 2012 Apps for asthma self-management: a systematic assessment of content and tools. BMC Med. 10, 144.. 10.1186/1741-7015-10-144 23171675 PMC3523082

      Not checked in the free run.

    • cites [97] McKinstry B: Currently available smartphone apps for asthma have worrying deficiencies. Evid Based Med 2013, Feb (Published Online First 16 Feb 2013). 10.1136/eb-2012-101194 23416417

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Patients with diabetes may synchronize a glucometer attachment to their mobile device to track blood glucose and share data through an internet connection.

    “Patients with diabetes may synchronize a glucometer attachment to their mobile device to track blood glucose and share the data through an internet connection [67,68].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [67] Kamel Boulos MN. Smartphones in diabetes care and management. CyberTherapy & Rehabilitation (C&R) Magazine 2011, 4(3):20-21. http://www.cybertherapyandrehabilitation.com/past-issues/2011/issue-3-2011/smartphones-in-diabetes-care-and-management/

      Not checked in the free run.

    • cites [68] Cafazzo JA Casselman M Hamming N Katzman DK Palmert MR 2012 Design of an mHealth app for the self-management of adolescent type 1 diabetes: a pilot study. J Med Internet Res. 14(3), e70 10.2196/jmir.2058 22564332 PMC3799540

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    A physical therapy app provided virtual-reality-based balance exercises through a mobile device, and remote physiotherapists could adjust exercise levels.

    “A physical therapy app provided virtual-reality-based balance exercises through a mobile device. Remote physiotherapists with access to the results could adjust the level of exercises accordingly [64].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [64] Krpič A Savanovic A Cikajlo I 2013 Telerehabilitation: remote multimedia-supported assistance and mobile monitoring of balance training outcomes can facilitate the clinical staff's effort. Int J Rehabil Res. 36(2), 162- 71.. 10.1097/MRR.0b013e32835dd63b 23337324

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A 2012 IEEE Spectrum article by Strickland presented top-five requests from industry experts for FDA rules on medical apps.

    “In a must-read IEEE Spectrum article published in September 2012, Strickland presented a list of top-five requests to the FDA that came out of a survey of key industry figures on what they would like to see in FDA rules for medical apps [104].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [104] Strickland E. The FDA Takes On Mobile Health Apps--Regulation is coming soon to health apps. Here’s what to expect. IEEE Spectrum 2012, Sep 12: online - http://spectrum.ieee.org/biomedical/devices/the-fda-takes-on-mobile-health-apps

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The iExaminer system allows clinicians to image the eye and save images for later review or sharing.

    “The combined system and app allows clinicians to image the eye (fundus exams) and save the images for later review or sharing with colleagues [103].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [103] FDA approves Welch Allyn foray into iPhone imaging--iExaminer iPhone PanOptic Ophthalmoscope (24 Jan 2013) - http://www.medgadget.com/2013/01/iexaminer-iphone-adapter-for-welch-allyns-panoptic-ophthalmoscope-cleared-in-u-s.html

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In psoriasis patients, 12 weeks of daily text message reminders and educational tools led to significantly better improvement in disease severity and quality of life, superior adherence, and optimised patient-physician communication compared to a control group with no text message intervention.

    “Following 12 weeks of daily text message reminders/educational tools, patients demonstrated significantly better improvement of disease severity and quality of life, with superior adherence to therapy and optimised patient-physician communication compared to a control group with no text message intervention [47].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [47] Balato N Megna M Di Costanzo L Balato A Ayala F 2013 Educational and motivational support service: a pilot study for mobile-phone-based interventions in patients with psoriasis. Br J Dermatol. 168(1), 201- 05 10.1111/j.1365-2133.2012.11205.x 23240729

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Visitors can review and rate apps in the NHS Health Apps Library, and many other app directory sites and app stores allow users to review and rate apps.

    “Visitors can also review and rate apps in the NHS Health Apps Library. Many other app directory sites (and the app stores run by Amazon, Apple, Google, Microsoft and Research In Motion [RIM/BlackBerry]) allow users, including healthcare professionals, to review and rate apps, e.g., the ‘Medical App Journal’ directory [22].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] Microbiology and Immunology Wiz - http://www.medicalappjournal.com/app_details.php?appsname=Microbiology+and+Immunology+Wiz+2013

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    App advice may not align with users' expectations or life activities.

    “Advice may not align with users’ expectations or life activities.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Frohner and colleagues describe an app, and Microsoft HealthVault apps for Windows Phone and Windows 8/RT already support Continua certified devices.

    “Frohner et al. [131] describe one such an app, and Microsoft ‘HealthVault’ apps for Windows Phone and Windows 8/RT [132] already support Continua certified devices [133].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [131] Frohner M Urbauer P Forjan M Pohn B Gerbovics F 2012 Development of an Android App in Compliance with the Continua Health Alliance Design Guidelines for Medical Device Connectivity in mHealth. Biomed Tech (Berl). 57(Suppl 1.).. 10.1515/bmt-2012-4203 23096286

      Not checked in the free run.

    • cites [132] Microsoft HealthVault. (Windows 8/RT) - http://apps.microsoft.com/windows/en-US/app/healthvault/728f1c88-7e2f-4b40-95c1-74fc09983689

      Not checked in the free run.

    • cites [133] Van Hoof B. Connecting Devices to HealthVault (June 2009) - http://download.microsoft.com/download/0/4/9/0498cecf-d0b1-4a72-b9b7-17eb7d7ada98/Van_Hoof_Connecting_Devices.pdf

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Developers wanting the CE mark must notify the MHRA or a corresponding agency, produce a declaration of conformity with technical documentation, and undertake a controlled test and risk assessment.

    “Developers and publishers wanting to have the ‘CE mark’ on their apps need to notify the MHRA in the UK (or a corresponding agency in other European countries), producing a ‘declaration of conformity’ that includes detailed technical documentation of how their app design conforms to the Medical Device Directive MDD 93/42/EEC.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [109] When is an app classed as a medical device? (Mar 2013) - http://www.geneticdigital.co.uk/2013/03/when-should-an-app-be-classed-as-a-device/

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    HACP assesses operability, privacy, security and content.

    “They assess operability, privacy, security (collectively referred to as the ‘Technical Standards’) and content (‘Content Standards’), and are available at [116,117,135].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [116] Horowitz BT. Happtique Publishes Final Standards for Mobile Health App Certification (4 Mar 2013) - http://www.eweek.com/mobile/happtique-publishes-final-standards-for-mobile-health-app-certification/

      Not checked in the free run.

    • cites [117] Happtique: Happtique Health App Certification Standards (Final, 27 Feb 2013) - http://www.happtique.com/app-certification/ - Direct URL to PDF document (October 2013): http://www.happtique.com/docs/HACP_Certification_Standards.pdf

      Not checked in the free run.

    • cites [135] Happtique: Health App Certification Program Standards & Performance Requirements - A Side-By-Side Comparison Of Published Draft And Final Standards/Performance Requirements (27 Feb 2013) - http://www.happtique.com/wp-content/uploads/Happtique-Side-by-side.pdf

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Health disparities and low health literacy and numeracy may negatively affect app use.

    “Health disparities and low health literacy and numeracy may negatively affect use [128].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [128] Kamel Boulos MN. Social media and mobile health (Chapter 13, under Part B: Taking action to create and strengthen health literacy-friendly settings). In: Ilona Kickbusch, Jürgen Pelikan, Franklin Apfel and Agis Tsouros (Editors). THE SOLID FACTS - Health Literacy Copenhagen, Denmark: World Health Organization Regional Office for Europe, 2013, pp. 63-67. Available at http://www.euro.who.int/__data/assets/pdf_file/0008/190655/e96854.pdf - URL of unabridged chapter version: http://ow.ly/kpn43

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The final HACP Certification Standards and Associated Performance Requirements were released on 27 February 2013.

    “The final HACP Certification Standards and Associated Performance Requirements were released on 27 February 2013.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [116] Horowitz BT. Happtique Publishes Final Standards for Mobile Health App Certification (4 Mar 2013) - http://www.eweek.com/mobile/happtique-publishes-final-standards-for-mobile-health-app-certification/

      Not checked in the free run.

    • cites [117] Happtique: Happtique Health App Certification Standards (Final, 27 Feb 2013) - http://www.happtique.com/app-certification/ - Direct URL to PDF document (October 2013): http://www.happtique.com/docs/HACP_Certification_Standards.pdf

      Not checked in the free run.

    • cites [135] Happtique: Health App Certification Program Standards & Performance Requirements - A Side-By-Side Comparison Of Published Draft And Final Standards/Performance Requirements (27 Feb 2013) - http://www.happtique.com/wp-content/uploads/Happtique-Side-by-side.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    HACP Content Standards require app content to be written and presented appropriately for the intended audience.

    “App Content (C) Standards (C1 to C11) require (among other things) that content of apps “be written and presented in a manner that is appropriate for the intended audience”.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [135] Happtique: Health App Certification Program Standards & Performance Requirements - A Side-By-Side Comparison Of Published Draft And Final Standards/Performance Requirements (27 Feb 2013) - http://www.happtique.com/wp-content/uploads/Happtique-Side-by-side.pdf

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Problems may arise from selective or subjective patient reporting and from patient compliance in maintaining manual logs, leading to incomplete data being submitted to clinicians.

    “Problems may arise in relation to selective and subjective reporting of data by patients and/or patient compliance in maintaining such manual logs, resulting in incomplete data being submitted to clinicians.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    HACP is meant to help healthcare providers and consumers identify medical, health and fitness apps that deliver credible content, contain safeguards for user data, and function as described.

    “HACP is meant to assist healthcare providers and consumers in identifying medical, health and fitness apps that “deliver credible content, contain safeguards for user data, and function as described”.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [116] Horowitz BT. Happtique Publishes Final Standards for Mobile Health App Certification (4 Mar 2013) - http://www.eweek.com/mobile/happtique-publishes-final-standards-for-mobile-health-app-certification/

      Not checked in the free run.

    • cites [117] Happtique: Happtique Health App Certification Standards (Final, 27 Feb 2013) - http://www.happtique.com/app-certification/ - Direct URL to PDF document (October 2013): http://www.happtique.com/docs/HACP_Certification_Standards.pdf

      Not checked in the free run.

    • cites [135] Happtique: Health App Certification Program Standards & Performance Requirements - A Side-By-Side Comparison Of Published Draft And Final Standards/Performance Requirements (27 Feb 2013) - http://www.happtique.com/wp-content/uploads/Happtique-Side-by-side.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Happtique partnered with third parties; Intertek tests the Technical Standards, and AAMC and CGFNS International evaluate the Content Standards with clinical specialists.

    “Happtique has partnered with third party organisations to serve as HACP Partners for the evaluation of apps against the Certification Standards, based upon each organisation’s area of expertise. The testing of the Technical Standards was assigned to Intertek, a multinational inspection, product testing and certification company headquartered in London, United Kingdom [136].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [136] Intertek - http://www.intertek.com/

      Not checked in the free run.

    • cites [137] Association of American Medical Colleges (AAMC) - https://www.aamc.org/

      Not checked in the free run.

    • cites [138] CGFNS International - http://cgfns.org/

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    App development, support, maintenance and regular updating may entail significant costs.

    “App development, support, maintenance and regular updating may entail significant costs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationnone given

    Apps may have deficiencies and limits.

    “Apps may have deficiencies and limits.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    HACP is voluntary and requires submitted apps to have English content, run natively on iOS, Android, Blackberry or Windows, be intended for US sale or use, pertain to healthcare, health monitoring or medical knowledge, and obtain any required prior approvals.

    “HACP remains a voluntary programme. In order to qualify for submission, an app’s content must be written in English, run natively on iOS (Apple), Android, Blackberry or Windows devices, be intended for sale or use within the US (Happtique is currently solely focused on the US market), and pertain to at least one of the following areas: provision of healthcare, health monitoring and management, and”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [117] Happtique: Happtique Health App Certification Standards (Final, 27 Feb 2013) - http://www.happtique.com/app-certification/ - Direct URL to PDF document (October 2013): http://www.happtique.com/docs/HACP_Certification_Standards.pdf

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Presenting correct, unbiased information in a way that is hard for the intended audience to understand makes the information useless and makes misunderstanding likely, which can have serious negative health consequences.

    “Presenting correct, unbiased information but in a way that is hard to understand by the intended audience not only renders this information useless, but also makes misunderstanding a likely possibility, which can have serious negative health consequences [128].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [128] Kamel Boulos MN. Social media and mobile health (Chapter 13, under Part B: Taking action to create and strengthen health literacy-friendly settings). In: Ilona Kickbusch, Jürgen Pelikan, Franklin Apfel and Agis Tsouros (Editors). THE SOLID FACTS - Health Literacy Copenhagen, Denmark: World Health Organization Regional Office for Europe, 2013, pp. 63-67. Available at http://www.euro.who.int/__data/assets/pdf_file/0008/190655/e96854.pdf - URL of unabridged chapter version: http://ow.ly/kpn43

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The NHS in England runs an online Health Apps Library that lists and recommends selected apps and provides links to third-party stores.

    “The National Health Service (NHS) in England runs an online Health Apps Library [111], where it lists and recommends some carefully selected apps such as ‘iBreastCheck’ (Breakthrough Breast Cancer) [112] and the NHS’ own ‘Health Choices’ app [113] (not all of the listed apps can be considered as medical devices).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [111] NHS Health Apps Library - http://apps.nhs.uk/

      Not checked in the free run.

    • cites [112] iBreastCheck (NHS page) - http://apps.nhs.uk/app/ibreastcheck/

      Not checked in the free run.

    • cites [113] NHS Health Choices app - http://apps.nhs.uk/app/health-choices/

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Apps meeting all HACP standards receive the Happtique Certification and Seal, valid for two years and tied to the submitted app version.

    “Apps successfully meeting all of HACP’s Standards and Associated Performance Requirements are granted the Happtique Certification and Seal, valid for a two-year period and specifically associated with the app version that was submitted for evaluation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [117] Happtique: Happtique Health App Certification Standards (Final, 27 Feb 2013) - http://www.happtique.com/app-certification/ - Direct URL to PDF document (October 2013): http://www.happtique.com/docs/HACP_Certification_Standards.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    Interactions with apps may require substantial effort.

    “Interactions may require substantial effort.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationcitation

    Not all app content may benefit all users, and getting users to download and engage with mobile apps is an art.

    “Not all content may benefit all users, and getting users to download and engage with mobile apps is an art [1].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] Heron KE Smyth JM 2010 Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol. 15(1), 1- 39.. 10.1348/135910709X466063 19646331 PMC2800172

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    HACP Operability Standards cover ensuring that the app installs, launches and runs consistently on target devices and operating systems.

    “App Operability (OP) Standards (OP1 to OP9) cover issues such as ensuring the app installs, launches, and runs consistently on target device(s) and operating system(s).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [135] Happtique: Health App Certification Program Standards & Performance Requirements - A Side-By-Side Comparison Of Published Draft And Final Standards/Performance Requirements (27 Feb 2013) - http://www.happtique.com/wp-content/uploads/Happtique-Side-by-side.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    HACP Security Standards deal with verifying that the app and any advertisements are free of malware.

    “App Security (S) Standards (S1 to S7) deal with issues such as verifying that the app, including without limitation, any advertisement displayed or supported through it, is free of malware.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [135] Happtique: Health App Certification Program Standards & Performance Requirements - A Side-By-Side Comparison Of Published Draft And Final Standards/Performance Requirements (27 Feb 2013) - http://www.happtique.com/wp-content/uploads/Happtique-Side-by-side.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    HACP Content Standards require apps with advertisements to clearly identify the advertising and comply with applicable regulatory requirements, especially for clinical or health-related products or services.

    “Content Standards also cover in-app advertisements, demanding that “an app that contains advertisements clearly identifies the advertising and complies with any and all applicable regulatory requirements, particularly advertisements that involve or relate to products or services that are clinical or related to health” [135].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [135] Happtique: Health App Certification Program Standards & Performance Requirements - A Side-By-Side Comparison Of Published Draft And Final Standards/Performance Requirements (27 Feb 2013) - http://www.happtique.com/wp-content/uploads/Happtique-Side-by-side.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Wearable devices offer new form factors and affordances, requiring dedicated apps to address their usability opportunities and limitations.

    “These devices offer new, unique form factors and affordances, requiring dedicated apps to properly address the associated usability opportunities and limitations [143,144].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [143] Google. + posts by MN Kamel Boulos on smart watches - https://plus.google.com/u/0/100165894171992699139/posts/ir1iRBjbm4t

      Not checked in the free run.

    • cites [144] Google. + posts by MN Kamel Boulos on Google Glass - https://plus.google.com/u/0/100165894171992699139/posts/7Y1G13Dphrg

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Voluntary app certification schemes such as HACP are reminiscent of early Web quality benchmarking tools and codes of conduct for online medical and health information.

    “Voluntary app certification schemes such as HACP, while very welcome (as standard setters in the domain), are reminiscent of the early days of the Web, when many “similar” quality benchmarking tools and codes of conduct for information publishers were proposed and developed to appraise and rate online medical and health information [141].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [141] Boulos MN Roudsari AV Gordon C Muir Gray JA 2001 The use of quality benchmarking in assessing web resources for the dermatology virtual branch library of the National electronic Library for Health (NeLH). J Med Internet Res. 3(1), e5.. 10.2196/jmir.3.1.e5 11720947 PMC1761889

      Not checked in the free run.

    • Rests on an assumption: Voluntary certification schemes such as HACP can act as standard setters and improve app quality.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Smart watches are being used as physical activity trackers.

    “Smart watches are being used as physical activity trackers [143], while Google Glass has been experimentally deployed as ‘surgical assistant’ and for medical education purposes [144,145].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [143] Google. + posts by MN Kamel Boulos on smart watches - https://plus.google.com/u/0/100165894171992699139/posts/ir1iRBjbm4t

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Google Glass has been experimentally deployed as a surgical assistant and for medical education.

    “Smart watches are being used as physical activity trackers [143], while Google Glass has been experimentally deployed as ‘surgical assistant’ and for medical education purposes [144,145].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [144] Google. + posts by MN Kamel Boulos on Google Glass - https://plus.google.com/u/0/100165894171992699139/posts/7Y1G13Dphrg

      Not checked in the free run.

    • cites [145] Vallurupalli S Paydak H Agarwal SK Agrawal M Assad-Kottner C 2013 Wearable technology to improve education and patient outcomes in a cardiology fellowship program - a feasibility study. Health Technol. 3(4), 267- 70.. 10.1007/s12553-013-0065-4

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A new class of wearable smartphone-connected devices is emerging, including smart watches and optical head-mounted displays such as Google Glass.

    “A new class of wearable smartphone-connected devices is emerging, namely smart watches (worn like a standard watch around the wrist) and optical head-mounted displays such as Google Glass (worn like eyeglasses).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [143] Google. + posts by MN Kamel Boulos on smart watches - https://plus.google.com/u/0/100165894171992699139/posts/ir1iRBjbm4t

      Not checked in the free run.

    • cites [144] Google. + posts by MN Kamel Boulos on Google Glass - https://plus.google.com/u/0/100165894171992699139/posts/7Y1G13Dphrg

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationnone given

    Many apps on offer today are simply custom informational websites displayed in dedicated app screens.

    “Indeed, many apps on offer today are nothing more than custom informational Web sites displayed in their own dedicated app screens (custom Web browser screens).”
    Quote in the paperSubstantial claim
52 of 52

  • The paper relies on this without support.

    As of 2013, little research had been done on the validity and efficacy of mobile medical and health apps in health and healthcare contexts.

    “It is perhaps surprising that relatively very little research has been undertaken so far (as of 2013) to investigate the validity and efficacy of these devices and apps in the contexts of health and healthcare.”

  • The paper relies on this without support.

    The largest number of apps are in the exercise and weight loss category.

    “The most number of apps belong to the exercise and weight loss category.”

  • The paper relies on this without support.

    Apps aimed at healthcare workers are generally more sophisticated, use medical terminology and functions, and are not easily navigable by non-health professionals.

    “These apps are generally more sophisticated, with medical terminology and functions, and not easily navigable by non-health professionals.”

  • The paper relies on this without support.

    Medical professional involvement in app design improves content quality and accountability.

    “With such a little medical professional involvement in the design of the majority of these apps, increased regulation of some kind is definitely required if we were to improve accountability for app content [12].”

  • The paper relies on this without support.

    Regulatory oversight improves app safety and accountability.

    “increased regulation of some kind is definitely required if we were to improve accountability for app content [12].”

  • The paper relies on this without support.

    In-app disclaimers about diagnostic limitations and false negatives can make patient-targeted diagnostic apps useful.

    “Such apps might still be useful, as long as there is an appropriate in-app disclaimer warning users in a clear and simple language about the app’s diagnostic limitations, the possibility and implications of ‘false negatives’, and that the app should not be taken as a substitute for proper professional clinician’s evaluation and advice.”

  • The paper relies on this without support.

    Educating consumers is the best first line of defence against risks from mobile apps.

    “The best first line of defence was, is, and will always be to educate consumers.”

  • The paper relies on this without support.

    App review criteria covering content quality, usability, connectivity, security and privacy allow end users to make informed comparisons and decisions.

    “Medical and health app reviewers, such as the curators of the above mentioned NHS Health Apps Library, should cover (and perhaps rate in some way) these aspects in any app review or assessment they undertake (where applicable), to allow end users to make informed comparisons and decisions about which apps to download and use.”

  • The paper relies on this without support.

    Voluntary certification schemes such as HACP can act as standard setters and improve app quality.

    “Voluntary app certification schemes such as HACP, while very welcome (as standard setters in the domain), are reminiscent of the early days of the Web, when many “similar” quality benchmarking tools and codes of conduct for information publishers were proposed and developed to appraise and rate online medical and health information [141].”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Little empirical research is undertaken to analyze the usability of mHealth apps before they are launched.

    “Despite this, little empirical research is undertaken to analyze the usability of mHealth apps before they are launched [15].”

    From Usability of Mobile Health Apps for Postoperative Care: Systematic Review (2020) · cites this paper as [15]

  • The lack of research evidence on the effectiveness of mobile health apps is likely the most important issue to consider.

    “The lack of research evidence on the effectiveness of mobile health apps is likely the most important issue to consider [207,208].”

    From Smartphone Apps for the Treatment of Mental Disorders: Systematic Review (2020) · cites this paper as [208]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Sickle-cell diary study cites an unrelated reference

    Minor

    The cited reference [34] concerns self-reporting of psychotic symptoms, not an e-diary for children and adolescents with sickle cell disease. The sickle-cell e-diary study appears as reference [35].

1 of 1

Cites

Flagged first, then by citations

1 of the 87 references couldn’t be matched to a known paper.

Research works printed but not linked: 8

  1. [3]More Smartphones Were Shipped in Q1 2013 Than Feature Phones, An Industry First According to IDC (Press Release, 25 Apr 2013) - http://www.idc.com/getdoc.jsp?containerId=prUS24085413
  2. [23]Lange Microbiology & Infectious Diseases Flash Cards, 2nd Edition, by Modality Inc. (iTunes store) - https://itunes.apple.com/gb/app/lange-microbiology-infectious/id496299826?mt=8
  3. [55]Kamel Boulos MN. Quo vadis smartphones and apps in emergency medicine? International Hospital and Equipment (IHE) 2012, 38:14-15. http://www.ihe-online.com/index.php?id=3315
  4. [67]Kamel Boulos MN. Smartphones in diabetes care and management. CyberTherapy & Rehabilitation (C&R) Magazine 2011, 4(3):20-21. http://www.cybertherapyandrehabilitation.com/past-issues/2011/issue-3-2011/smartphones-in-diabetes-care-and-management/
  5. [104]Strickland E. The FDA Takes On Mobile Health Apps--Regulation is coming soon to health apps. Here’s what to expect. IEEE Spectrum 2012, Sep 12: online - http://spectrum.ieee.org/biomedical/devices/the-fda-takes-on-mobile-health-apps
  6. [121]Caprani N, O’Connor NE, Gurrin C. Chapter 5: Touch Screens for the Older User. In Assistive Technologies Edited by Auat Cheein FA. Intech; 2012:95-118 (ISBN 978-953-51-0348-6). Available at http://www.intechopen.com/books/assistive-technologies/touch-screens-for-the-older-user
  7. [129]Almost half of Europeans possess limited health literacy (Maastricht University - press release, 22 Nov 2011) - http://www.maastrichtuniversity.nl/web/Main/Sitewide/PressRelease/AlmostHalfOfEuropeansPossessLimitedHealthLiteracy.htm
  8. [142]PatientView. European Directory of Health Apps 2012-2013 - A review by patient groups and empowered consumers London, UK: PatientView; 2012. Available at http://www.patient-view.com/uploads/6/5/7/9/6579846/pv_appdirectory_final_web_300812.pdf

Cited by

Sorted by citations

Related

Authors
Kamel Boulos, M. N., Brewer, A. C., Karimkhani, C., Buller, D. B., & Dellavalle, R. P.
Affiliations
  • Mayo Clinic in Arizona (US)
  • Klein Buendel (United States) (US)
  • Denver VA Medical Center (US)
ORCID
Venue
Online Journal of Public Health Informatics
Publisher
University of Illinois Libraries
Published
2014-02-05
Volume
5
Issue
3
Pages
229-229
ISSN
1947-2579 (ISSN-L)
DOI
10.5210/ojphi.v5i3.4814
PMC
PMC3959919
OpenAlex
W2102560179
Semantic Scholar
b14ff21a349e8395178e1f009ad5e1cf41072d2b
Cited by
659
References
87
Citations by year
2026: 6 · 2025: 27 · 2024: 40 · 2023: 45 · 2022: 38 · 2021: 55 · 2020: 85 · 2019: 88 · 2018: 93 · 2017: 80 · 2016: 55 · 2015: 36 · 2014: 11
FWCI
108.74
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Online Journal of Public Health Informatics (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, other-oa)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Health Sciences › Health Professions › Health Information Management › Health, Technology, Consumer Behavior
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
Keywords
mobile health apps, health apps, regulatory control, app quality assessment, usability, health literacy, device connectivity, mobile application security, user privacy, consumer education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review