Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.

Best practices for analyzing large-scale health data from wearables and smartphone apps

Open access

Abstract

Smartphone apps and wearable devices for tracking physical activity and other health behaviors have become popular in recent years and provide a largely untapped source of data about health behaviors in the free-living environment. The data are large in scale, collected at low cost in the “wild”, and often recorded in an automatic fashion, providing a powerful complement to traditional surveillance studies and controlled trials. These data are helping to reveal, for example, new insights about environmental and social influences on physical activity. The observational nature of the datasets and collection via commercial devices and apps pose challenges, however, including the potential for measurement, population, and/or selection bias, as well as missing data. In this article, we review insights gleaned from these datasets and propose best practices for addressing the limitations of large-scale data from apps and wearables. Our goal is to enable researchers to effectively harness the data from smartphone apps and wearable devices to better understand what drives physical activity and other health behaviors.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41746-019-0121-1

Cites

62 of 62 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Datasets from apps and wearables are helping researchers identify novel worldwide trends in activity and health. Our team has analyzed data from 717,527 users of the Argus app for tracking physical activity and other health metrics.11 This analysis revealed worldwide inequality in levels of physical activity that varied from country to country. In the map, country area is scaled by the country’s obesity rate, as calculated from the app-reported BMI of users. The countries are shaded according to activity inequality, where warm colors (reds and oranges) indicate high levels of activity inequality (some people are very active and some people are minimally active) and cool colors (blues) indicate low levels of activity inequality (individuals within the country get similar levels of activity). Countries with larger than normal areas (indicative of high obesity) also tend to be shaded with warm colors (indicative of high activity inequality). The map was generated using the Scape Toad software63 and the world borders dataset from the Thematic Mapping API64
Fig. 1 Datasets from apps and wearables are helping researchers identify novel worldwide trends in activity and health. Our team has analyzed data from 717,527 users of the Argus app for tracking physical activity and other health metrics.11 This analysis revealed worldwide inequality in levels of physical activity that varied from country to country. In the map, country area is scaled by the country’s obesity rate, as calculated from the app-reported BMI of users. The countries are shaded according to activity inequality, where warm colors (reds and oranges) indicate high levels of activity inequality (some people are very active and some people are minimally active) and cool colors (blues) indicate low levels of activity inequality (individuals within the country get similar levels of activity). Countries with larger than normal areas (indicative of high obesity) also tend to be shaded with warm colors (indicative of high activity inequality). The map was generated using the Scape Toad software63 and the world borders dataset from the Thematic Mapping API64CC BY · PubMed Central
Fig. 2 Overview of best practices for analyzing large-scale physical activity and health behavior datasets from commercial smartphone apps and wearable devices. The process is highly iterative, as indicated by the arrows flowing in both directions. By sharing results, along with data and software tools, your work can help inspire new research, completing the circle
Fig. 2 Overview of best practices for analyzing large-scale physical activity and health behavior datasets from commercial smartphone apps and wearable devices. The process is highly iterative, as indicated by the arrows flowing in both directions. By sharing results, along with data and software tools, your work can help inspire new research, completing the circleCC BY · PubMed Central
Fig. 3 Difference in life expectancy as a function of the difference in activity volume between females and males. When the gap in steps between females and males gets smaller (i.e., less negative), females outlive males by more years (using World Bank data65; grey line is LOESS fit; R2 = 0.20). These results indicate that the reduced activity levels recorded by women in countries with high activity inequality may have significant implications for health. (Plot shows countries with more than 1000 subjects from the Argus dataset described in our previous work.11)
Fig. 3 Difference in life expectancy as a function of the difference in activity volume between females and males. When the gap in steps between females and males gets smaller (i.e., less negative), females outlive males by more years (using World Bank data65; grey line is LOESS fit; R2 = 0.20). These results indicate that the reduced activity levels recorded by women in countries with high activity inequality may have significant implications for health. (Plot shows countries with more than 1000 subjects from the Argus dataset described in our previous work.11)CC BY · PubMed Central
Fig. 4 Comparison of demographics of U.S. users of a smartphone and data from traditional surveillance studies. a Body Mass Index (BMI) distribution of users of the Argus app (blue) vs. the U.S. population as measured in the National Health and Nutrition Examination Survey66 (NHANES; red). b Age distribution of users of the Argus app vs. NHANES. The counts for the NHANES sample are weighted according to the NHANES-provided sample weights, thus the distributions approximate the general U.S. population and the total of the weighted counts in the histogram matches the number of individuals in the 2011–2012 NHANES study year. While there are differences between the distributions, the app dataset, due to its massive size, has large coverage of users between the ages of 15 and 70 and BMIs from 20 to 40. For example, the dataset includes 32,000 individuals in the U.S. over age 60 and 113,000 individuals in the U.S. whose BMI classifies them as obese
Fig. 4 Comparison of demographics of U.S. users of a smartphone and data from traditional surveillance studies. a Body Mass Index (BMI) distribution of users of the Argus app (blue) vs. the U.S. population as measured in the National Health and Nutrition Examination Survey66 (NHANES; red). b Age distribution of users of the Argus app vs. NHANES. The counts for the NHANES sample are weighted according to the NHANES-provided sample weights, thus the distributions approximate the general U.S. population and the total of the weighted counts in the histogram matches the number of individuals in the 2011–2012 NHANES study year. While there are differences between the distributions, the app dataset, due to its massive size, has large coverage of users between the ages of 15 and 70 and BMIs from 20 to 40. For example, the dataset includes 32,000 individuals in the U.S. over age 60 and 113,000 individuals in the U.S. whose BMI classifies them as obeseCC BY · PubMed Central
Fig. 5 Verifying that a smartphone app dataset reproduces previously reported relationships between physical activity, geographic location, age, and gender. In our study of activity inequality,11 we conducted extensive analyses comparing the app dataset to previously published datasets. a WHO physical activity measure67 versus smartphone activity measure (LOESS fit). The WHO measure corresponds to the percentage of the population meeting the WHO guidelines for moderate to vigorous physical activity based on self-report. The smartphone activity measure is based on accelerometer-defined average daily steps. We found a correlation of r = 0.3194 between the two measures (P < 0.05). Note that this comparison is limited because there is no direct correspondence between the two measures—values of self-reported and accelerometer-defined activity can differ,40 and the WHO confidence intervals are very large for many countries. b WHO obesity estimates68 based on self-reports to survey conductors, versus obesity estimates in our dataset, based on height and weight reported to the activity-tracking app (LOESS fit). We found a significant correlation of r = 0.691 between the two estimates (P < 10−6). c Gender gap in activity estimated from smartphones is strongly correlated with previously reported estimates based on self-report (LOESS fit). We found that the difference in average steps per day between females and males is strongly correlated to the difference in the fraction of each gender who report being sufficiently active according to the WHO (r = 0.52, P < 10−3). d Daily step counts are shown across age for all users. Error bars correspond to bootstrapped 95% confidence intervals. Observed trends in the dataset are consistent with previous findings; that is, activity decreases with increasing BMI69–71 and is lower in females than in males.70,72–74 This figure is adapted from our previous work11 and reproduced with permission
Fig. 5 Verifying that a smartphone app dataset reproduces previously reported relationships between physical activity, geographic location, age, and gender. In our study of activity inequality,11 we conducted extensive analyses comparing the app dataset to previously published datasets. a WHO physical activity measure67 versus smartphone activity measure (LOESS fit). The WHO measure corresponds to the percentage of the population meeting the WHO guidelines for moderate to vigorous physical activity based on self-report. The smartphone activity measure is based on accelerometer-defined average daily steps. We found a correlation of r = 0.3194 between the two measures (P < 0.05). Note that this comparison is limited because there is no direct correspondence between the two measures—values of self-reported and accelerometer-defined activity can differ,40 and the WHO confidence intervals are very large for many countries. b WHO obesity estimates68 based on self-reports to survey conductors, versus obesity estimates in our dataset, based on height and weight reported to the activity-tracking app (LOESS fit). We found a significant correlation of r = 0.691 between the two estimates (P < 10−6). c Gender gap in activity estimated from smartphones is strongly correlated with previously reported estimates based on self-report (LOESS fit). We found that the difference in average steps per day between females and males is strongly correlated to the difference in the fraction of each gender who report being sufficiently active according to the WHO (r = 0.52, P < 10−3). d Daily step counts are shown across age for all users. Error bars correspond to bootstrapped 95% confidence intervals. Observed trends in the dataset are consistent with previous findings; that is, activity decreases with increasing BMI69–71 and is lower in females than in males.70,72–74 This figure is adapted from our previous work11 and reproduced with permissionCC BY · PubMed Central
Fig. 6 Example of a natural experiment using observational data from a smartphone app for tracking activity. The Argus smartphone app (Azumio, Inc.), includes a social network that users can opt to join. Althoff and colleagues19 sought to uncover if and how forming social connections affects social activity. Since users who join and are active in the social network may be more intrinsically motivated to increase their activity, they used a natural experiment to isolate the effects of social influence from other factors that could influence activity. In particular, they compared the change in activity between a individuals who sent out a friend request (question mark) that was immediately accepted (check mark) and b individuals whose friend request was not accepted for more than 7 days. Note the curves in a and b are for illustrative purposes and do not represent actual subjects. Once a friendship is accepted, the user receives notifications of their connections’ activities (e.g., going for a run), and can comment on their connections’ activity posts (denoted by the heart, text box, and notification bell in a and b). Since the two groups were similar in all aspects except whether their friend request was accepted within 7 days, the additional increase in activity of the direct acceptance group can be attributed to social influence. c This social influence resulted in users taking 400 more steps per day on average. Error bars indicate bootstrapped 95% confidence intervals
Fig. 6 Example of a natural experiment using observational data from a smartphone app for tracking activity. The Argus smartphone app (Azumio, Inc.), includes a social network that users can opt to join. Althoff and colleagues19 sought to uncover if and how forming social connections affects social activity. Since users who join and are active in the social network may be more intrinsically motivated to increase their activity, they used a natural experiment to isolate the effects of social influence from other factors that could influence activity. In particular, they compared the change in activity between a individuals who sent out a friend request (question mark) that was immediately accepted (check mark) and b individuals whose friend request was not accepted for more than 7 days. Note the curves in a and b are for illustrative purposes and do not represent actual subjects. Once a friendship is accepted, the user receives notifications of their connections’ activities (e.g., going for a run), and can comment on their connections’ activity posts (denoted by the heart, text box, and notification bell in a and b). Since the two groups were similar in all aspects except whether their friend request was accepted within 7 days, the additional increase in activity of the direct acceptance group can be attributed to social influence. c This social influence resulted in users taking 400 more steps per day on average. Error bars indicate bootstrapped 95% confidence intervalsCC BY · PubMed Central
Fig. 7 Example of propensity scoring to isolate the effects of a treatment that comes in different doses (physical activity) from other confounding factors. The Argus smartphone app (Azumio, Inc.) collects minute by minute step counts. a For each user, we can construct a plot of activity bout length (X) vs. the average number of minutes per day spent in activity of at least X minutes. We call the area under this curve an individual’s activity persistence. In the figure we include users with at least 10 days of step tracking data. b We next want to understand how activity persistence influences quantities like BMI. Since individuals with higher or lower activity persistence may be different in other ways that influence BMI (such as age and gender), we used inverse probability of treatment weighting (IPTW) to isolate the effects of activity. The grey curve shows the BMI of individuals in each decile of activity persistence (where higher deciles indicate more bouts of longer duration), without any weighting. The green curve shows the relationship after we have used IPTW to minimize the influence of other factors like age and gender on the estimated BMI for each decile of activity persistence. Error bars correspond to bootstrapped 95% confidence intervals
Fig. 7 Example of propensity scoring to isolate the effects of a treatment that comes in different doses (physical activity) from other confounding factors. The Argus smartphone app (Azumio, Inc.) collects minute by minute step counts. a For each user, we can construct a plot of activity bout length (X) vs. the average number of minutes per day spent in activity of at least X minutes. We call the area under this curve an individual’s activity persistence. In the figure we include users with at least 10 days of step tracking data. b We next want to understand how activity persistence influences quantities like BMI. Since individuals with higher or lower activity persistence may be different in other ways that influence BMI (such as age and gender), we used inverse probability of treatment weighting (IPTW) to isolate the effects of activity. The grey curve shows the BMI of individuals in each decile of activity persistence (where higher deciles indicate more bouts of longer duration), without any weighting. The green curve shows the relationship after we have used IPTW to minimize the influence of other factors like age and gender on the estimated BMI for each decile of activity persistence. Error bars correspond to bootstrapped 95% confidence intervalsCC BY · PubMed Central
Fig. 8 Relationship between walkability and activity inequality holds within cities in the USA of similar income. We found that walkability was associated with lower levels of activity inequality. To help account for potential confounding due to socioeconomic factors, we grouped the 69 cities in our analysis into quartiles based on median household income (data from the 2015 American Community Survey75). We found that walkable environments were associated with lower levels of activity inequality for all four groups (LOESS fit). The effect appears to be attenuated for cities in the lowest median household income quartile. These results suggest that our main result—activity inequality predicts obesity and is mediated by factors of the physical environment—is independent of potential socioeconomic bias in our sample. This figure is adapted from our previous work11 and reproduced with permission
Fig. 8 Relationship between walkability and activity inequality holds within cities in the USA of similar income. We found that walkability was associated with lower levels of activity inequality. To help account for potential confounding due to socioeconomic factors, we grouped the 69 cities in our analysis into quartiles based on median household income (data from the 2015 American Community Survey75). We found that walkable environments were associated with lower levels of activity inequality for all four groups (LOESS fit). The effect appears to be attenuated for cities in the lowest median household income quartile. These results suggest that our main result—activity inequality predicts obesity and is mediated by factors of the physical environment—is independent of potential socioeconomic bias in our sample. This figure is adapted from our previous work11 and reproduced with permissionCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 62 references

Cited by

See all 183

Metadata

Authors
Hicks, J. L., Althoff, T., Sosic, R., Kuhar, P., Bostjancic, B., King, A. C., Leskovec, J., & Delp, S. L.
Affiliations
  • Stanford University (US)
  • University of Washington (US)
  • Stanford Medicine (US)
  • Chan Zuckerberg Biohub San Francisco (US)
Corresponding author
Hicks, Jennifer L.
Venue
npj Digital Medicine
Published
2019-06-03
Volume
2
Issue
1
Pages
45-45
ISSN
2398-6352 (ISSN-L)
OpenAlex
W2948756736
Cited by
183
References
62
Citations by year
2026: 22 · 2025: 19 · 2024: 27 · 2023: 17 · 2022: 36 · 2021: 35 · 2020: 22 · 2019: 5
FWCI
4.58
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
  • Robert Wood Johnson Foundation (ID#7334, #7334)
  • Silicon Valley Community Foundation (101518)
  • National Institutes of Health
  • National Cancer Institute (P20CA217199, R01CA211048, 5R01CA211048 and P20CA217199)
  • National Institute of Diabetes and Digestive and Kidney Diseases (R01DK102016)
  • National Institute of Biomedical Imaging and Bioengineering (U54EB020405)
  • National Institute on Minority Health and Health Disparities (U54MD010724)
  • Eunice Kennedy Shriver National Institute of Child Health and Human Development (P2CHD065690)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Physical Activity and Health
  • Social Sciences › Transportation › Human Mobility and Location-Based Analysis
Keywords
wearable devices, mobile application, physical activity, health behavior, measurement bias, selection bias, missing data, data quality

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Best practices for analyzing large-scale health data from wearables and smartphone apps. https://app.cochat.ai/papers/10.1038/s41746-019-0121-1
The paper
Hicks, J. L., Althoff, T., Sosic, R., Kuhar, P., Bostjancic, B., King, A. C., Leskovec, J., & Delp, S. L. (2019). Best practices for analyzing large-scale health data from wearables and smartphone apps. npj Digital Medicine. https://doi.org/10.1038/s41746-019-0121-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.