CoChat
Sign Up
Cited by 126Mobile Health and mHealth ApplicationsDigital Mental Health InterventionsPhysical Activity and Health

Evaluating the Impact of Physical Activity Apps and Wearables: Interdisciplinary Review

Open access
McCallum, Claire, Rooksby, John, Gray, Cindy M·2018·JMIR mhealth and uhealth
DOI 10.2196/mhealth.9054PMC PMC5889496

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Although many smartphone apps and wearables have been designed to improve physical activity, their rapidly evolving nature and complexity present challenges for evaluating their impact. Traditional methodologies, such as randomized controlled trials (RCTs), can be slow. To keep pace with rapid technological development, evaluations of mobile health technologies must be efficient. Rapid alternative research designs have been proposed, and efficient in-app data collection methods, including in-device sensors and device-generated logs, are available. Along with effectiveness, it is important to measure engagement (ie, users' interaction and usage behavior) and acceptability (ie, users' subjective perceptions and experiences) to help explain how and why apps and wearables work. OBJECTIVES: This study aimed to (1) explore the extent to which evaluations of physical activity apps and wearables: employ rapid research designs; assess engagement, acceptability, as well as effectiveness; use efficient data collection methods; and (2) describe which dimensions of engagement and acceptability are assessed. METHOD: An interdisciplinary scoping review using 8 databases from health and computing sciences. Included studies measured physical activity, and evaluated physical activity apps or wearables that provided sensor-based feedback. Results were analyzed using descriptive numerical summaries, chi-square testing, and qualitative thematic analysis. RESULTS: A total of 1829 abstracts were screened, and 858 articles read in full. Of 111 included studies, 61 (55.0%) were published between 2015 and 2017. Most (55.0%, 61/111) were RCTs, and only 2 studies (1.8%) used rapid research designs: 1 single-case design and 1 multiphase optimization strategy. Other research designs included 23 (22.5%) repeated measures designs, 11 (9.9%) nonrandomized group designs, 10 (9.0%) case studies, and 4 (3.6%) observational studies. Less than one-third of the studies (32.0%, 35/111) investigated effectiveness, engagement, and acceptability together. To measure physical activity, most studies (90.1%, 101/111) employed sensors (either in-device [67.6%, 75/111] or external [23.4%, 26/111]). RCTs were more likely to employ external sensors (accelerometers: P=.005). Studies that assessed engagement (52.3%, 58/111) mostly used device-generated logs (91%, 53/58) to measure the frequency, depth, and length of engagement. Studies that assessed acceptability (57.7%, 64/111) most often used questionnaires (64%, 42/64) and/or qualitative methods (53%, 34/64) to explore appreciation, perceived effectiveness and usefulness, satisfaction, intention to continue use, and social acceptability. Some studies (14.4%, 16/111) assessed dimensions more closely related to usability (ie, burden of sensor wear and use, interface complexity, and perceived technical performance). CONCLUSIONS: The rapid increase of research into the impact of physical activity apps and wearables means that evaluation guidelines are urgently needed to promote efficiency through the use of rapid research designs, in-device sensors and user-logs to assess effectiveness, engagement, and acceptability. Screening articles was time-consuming because reporting across health and computing sciences lacked standardization. Reporting guidelines are therefore needed to facilitate the synthesis of evidence across disciplines.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/mhealth.9054

Cites

206 of 206 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Knowledge on these apps' potential is lacking, particularly in regard to testing apps in practical settings and using objective data.

    “However, knowledge on these apps’ potential is lacking, particularly in regard to testing apps in practical settings and using objective data, and health apps generally are not evidence-based [63].”

    From Advantages and disadvantages of mobile applications for workplace health promotion: A scoping review (2024) · cites this paper as [63]

    The Discussion (Principal Findings; page not provided) identifies a real-world evaluation concern: some sensor procedures require lab visits and may reduce generalizability. But the claim’s suggestion that objective data are lacking is contrary to the review’s finding that 90.9% of included studies used sensors to measure physical activity (Results, Effectiveness; page not provided). (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Health apps generally are not evidence-based.

    “However, knowledge on these apps’ potential is lacking, particularly in regard to testing apps in practical settings and using objective data, and health apps generally are not evidence-based [63].”

    From Advantages and disadvantages of mobile applications for workplace health promotion: A scoping review (2024) · cites this paper as [63]

    The Discussion > Limitations says commercially developed apps often lack behavior-change techniques that improve effectiveness. But the review covers physical-activity apps and wearables, and explicitly says its findings cannot be generalized to other health behaviors; it does not establish that health apps generally are not evidence-based. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. [1]Blair SN Physical inactivity: the biggest public health problem of the 21st century Br J Sports Med 2009 01 43 1 1 2 19136507 43/1/1 19136507
  2. [42]Kharrazi H Vincz L Increasing physical activity by implementing a behavioral change intervention using pervasive personal health record system: an exploratory study 2011 6th International Conference on Universal Access in Human-Computer Interaction (UAHCI). HCI International 2011 July 9-14 Orlando, FL 10.1007/978-3-642-27-2_39 · DOI
  3. [176]Connelly K On developing a technology acceptance model for pervasive computing Workshop of Ubiquitous System Evaluation (USE) 2007 9th International Conference on Ubiquitous Computing (UBICOMP) 2007 September 16-19 Innsbruck, Austria

Cited by

Sorted by citations

Related

Authors
McCallum, C., Rooksby, J., & Gray, C. M.
Affiliations
University of Glasgow (GB)
Corresponding author
McCallum, Claire
ORCID
Venue
JMIR mhealth and uhealth
Publisher
JMIR Publications Inc.
Published
2018-03-23
Volume
6
Issue
3
Pages
e58
ISSN
2291-5222 (ISSN-L)
DOI
10.2196/mhealth.9054
PMC
PMC5889496
OpenAlex
W2789449717
Cited by
126
References
206
Citations by year
2026: 4 · 2025: 10 · 2024: 9 · 2023: 18 · 2022: 22 · 2021: 22 · 2020: 18 · 2019: 14 · 2018: 9
FWCI
2.87
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR mhealth and uhealth (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion, cc-by-sa)
  • Northumbria Research Link (Northumbria University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Medicine › Physiology › Physical Activity and Health
Keywords
physical activity apps, wearable devices, mobile health, single-case experimental design, multiphase optimization strategy, user engagement, acceptability, effectiveness, accelerometer, research design evaluation, physical activity monitoring

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review