A Library of Analytic Indicators to Evaluate Effective Engagement with Consumer mHealth Apps for Chronic Conditions: Scoping Review

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The first authors of reviewed studies were affiliated with institutions in the United States (46%, 19/41), Canada (20%, 8/41), the United Kingdom (10%, 4/41), Australia (5%, 2/41), Germany (5%, 2/41), the Netherlands (5%, 2/41), France (2%, 1/41), India (2%, 1/41), Singapore (2%, 1/41), Spain (2%, 1/41), Sweden (2%, 1/41), and Switzerland (2%, 1/41).

“The first authors of reviewed studies were affiliated with institutions in the United States (46%, 19/41), Canada (20%, 8/41), the United Kingdom (10%, 4/41), Australia (5%, 2/41), Germany (5%, 2/41), the Netherlands (5%, 2/41), France (2%, 1/41), India (2%, 1/41), Singapore (2%, 1/41), Spain (2%, 1/41), Sweden (2%, 1/41), and Switzerland (2%, 1/41).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Over half of the apps had a hybrid structure (59%, 24/41).

“Over half of the apps had a hybrid structure (59%, 24/41)”

May combine two claims.

A wide range of chronic conditions were targeted through the apps under study, with mental health (29%, 12/41), chronic pain (12%, 5/41), asthma (10%, 4/41), cardiovascular disease (7%, 3/41), and diabetes (type 1 and 2; 15%, 6/41) leading the clinical charge.

“A wide range of chronic conditions were targeted through the apps under study, with mental health (29%, 12/41), chronic pain (12%, 5/41), asthma (10%, 4/41), cardiovascular disease (7%, 3/41), and diabetes (type 1 and 2; 15%, 6/41) leading the clinical charge.”

May combine two claims.

Researchers also evaluated apps for cancer (5%, 2/41), hypertension (5%, 2/41), obesity (5%, 2/41), chronic kidney disease (2%, 1/41), chronic obstructive pulmonary disease (2%, 1/41), cystic fibrosis and inflammatory bowel disease (2%, 1/41), Parkinson disease (2%, 1/41), and sleep apnea (2%, 1/41).

“Researchers also evaluated apps for cancer (5%, 2/41), hypertension (5%, 2/41), obesity (5%, 2/41), chronic kidney disease (2%, 1/41), chronic obstructive pulmonary disease (2%, 1/41), cystic fibrosis and inflammatory bowel disease (2%, 1/41), Parkinson disease (2%, 1/41), and sleep apnea (2%, 1/41).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The remaining 13 studies included for review were observational in design (32%, 13/41).

“The remaining 13 studies included for review were observational in design (32%, 13/41).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Studies were on average 5 months long (median 152 days, IQR 106).

“Studies were on average 5 months long (median 152 days, interquartile range, IQR 106)”

May combine two claims.

There was significant variation in how constructs were defined across studies, which limited the authors' ability to extract reliable definitions, map analytic indicators to specific constructs, and conduct cross-construct comparisons.

“There was significant variation in how constructs were defined across studies, which limited our ability to (1) extract reliable definitions for each construct, (2) map analytic indicators to specific constructs, and (3) conduct cross-construct comparisons of analytic indicators.”

May combine two claims.

Owen et al measured the number of sessions logged by users on their PTSD self-management app and the number of users who logged at least one session on the first day, week, and month post download.

“Owen et al made both these associations by measuring the number of sessions logged by users on their PTSD self-management app, as well as the number of users who logged at least one session on the first day, week, and month post download [42].”

• cites [42] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The longest reviewed observational study conducted by Serrano et al was 7 years long with over 1 million participants.

“The longest reviewed observational study conducted by Serrano et al was 7 years long with over 1 million participants [34].”

• cites [34] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Nearly half of all structured apps were aimed at improving mental health (40%, 4/10).

“Nearly half of all structured apps were aimed at improving mental health (40%, 4/10).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Studies had a sample size of over 100 participants (median 103, IQR 252).

“with a sample size of over 100 participants (median 103, IQR 252)”

Probably not a substantial claim: may not pass S3 (load-bearing).

A total of 1873 articles were identified through the database search.

“A total of 1873 articles were identified through the database search.”

Probably not a substantial claim: may not pass S3 (load-bearing).

10 apps were unstructured (24%).

“10 apps were unstructured (24%)”

Probably not a substantial claim: may not pass S3 (load-bearing).

In total, 14 of the reviewed studies were published in the Journal of Medical Internet Research or its sister journals (34%, 14/41).

“In total, 14 of the reviewed studies were published in the Journal of Medical Internet Research or its sister journals (34%, 14/41).”

May combine two claims.

Quasi-experimental design selection (17%, 7/41) was more diverse and included cohort (29%, 2/7), interrupted time-series (14%, 1/7), and single case (14%, 1/7) studies.

“Quasi-experimental design selection (17%, 7/41) was more diverse and included cohort (29%, 2/7), interrupted time-series (14%, 1/7), and single case (14%, 1/7) studies.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Of the 60 full texts screened, 19 were excluded, 8 of which did not include objective, quantifiable measurements using log data analytics.

“Of the 60 full texts screened, 19 were excluded, 8 of which did not include objective, quantifiable measurements using log data analytics.”

Probably not a substantial claim: may not pass S3 (load-bearing).

7 apps were structured (17%).

“and 7 apps were structured (17%)”

Probably not a substantial claim: may not pass S3 (load-bearing).

The one-group pretest-posttest design was the second most prevalent experimental study design (43%, 9/21).

“followed by the one-group pretest-posttest design (43%, 9/21)”

May combine two claims.

Reviewed studies assessed the number of videos watched from a playlist, educational modules completed, or self-care advice accessed.

“Others assessed the number of videos watched from a playlist [36,55], educational modules completed [52], or self-care advice accessed [69].”

• cites [36] sourceNot checked yet

  Not checked in the free run.

• cites [55] sourceNot checked yet

  Not checked in the free run.

• cites [52] sourceNot checked yet

  Not checked in the free run.

• cites [69] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Davies et al segmented sessions by those in the 30- to 60-second range.

“Davies et al also segmented sessions by those that were in the 30- to 60-second range [38].”

• cites [38] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Reviewed researchers measured time spent on unique sections, unique pages, length of a unique session, length between unique sessions, and total time spent on an app.

“Researchers measured the time spent on unique sections of an app [66], the time spent on unique pages [56], the length of a unique session [38,42,43,71], the length between unique sessions [72], and the total time spent on an app [62,68,73].”

• cites [66] sourceNot checked yet

  Not checked in the free run.

• cites [56] sourceNot checked yet

  Not checked in the free run.

• cites [38] sourceNot checked yet

  Not checked in the free run.

• cites [42] sourceNot checked yet

  Not checked in the free run.

• cites [43] sourceNot checked yet

  Not checked in the free run.

• cites [71] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Taki et al combined session analytics with page analytics and measured the number of pages viewed per session in their obesity self-management app.

“Taki et al combined session analytics with page analytics and measured the number of pages viewed per session in their app for obesity self-management [72].”

• cites [72] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Reviewed studies that compared attrition curves used Kaplan-Meier survival curves, Cox regression models, and latent class growth models.

“Those that conducted comparisons across attrition curves used Kaplan-Meier survival curves (10%, 1/10), Cox regression models (10%, 2/10), and latent class growth models (10%, 1/10).”

May combine two claims.

Researchers used the number of log-ins or sessions to reflect the shift from adoption to habituation, with greater numbers denoting greater engagement.

“Researchers used this analytic indicator to reflect the shift from adoption to habituation, with a greater number of log-ins or sessions denoting greater engagement.”

Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

The number of pages accessed by users was logged by reviewed researchers to reflect overall patterns of app engagement and discoverability of specific content.

“The number of pages accessed by users was logged by researchers to reflect overall patterns of app engagement and discoverability of specific content.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Owen et al recorded click stream data documenting users' navigation through page content.

“Owen et al recorded click stream data documenting their users’ navigation through page content [42].”

• cites [42] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Goh et al identified 3 distinct 8-week app engagement trajectories based on frequency of interactions and associated patient characteristics with these trajectories.

“They were able to identify 3 distinct app trajectories based on the frequency of interactions and also associate patient characteristics with these trajectories.”

• cites [63] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Ben-Zeev et al found that participants who engaged with their schizophrenia self-management app for 5 to 6 months had a higher frequency of interactions and engaged 4.3 days per week on average.

“Ben-Zeev et al found that participants who engaged with their schizophrenia self-management app for a period of 5 to 6 months also had a higher frequency of interactions and engaged 4.3 days per week on average [37].”

• cites [37] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Kuhn et al measured the number and content of pages visited by users in their PTSD self-management app, as did other researchers.

“Kuhn et al measured the number and content of pages visited by users in their app for PTSD self-management, as did other researchers [38,41,71].”

• cites [38] sourceNot checked yet

  Not checked in the free run.

• cites [41] sourceNot checked yet

  Not checked in the free run.

• cites [71] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Researchers studying apps with modular content considered module completion indicative of engagement and tracked module progression and completion rates.

“Overall, researchers studying apps with modular content considered module completion to be indicative of engagement and consequently, tracked module progression and completion rates.”

May combine two claims.

Torous et al found that future use of a schizophrenia self-management app is directly related to prior app use, suggesting those who engage more often have a higher probability of future engagement.

“They found that future use with the app is directly related to prior app use, suggesting that those who engage with the app more often will have a higher probability of app engagement in the future.”

• cites [44] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Studies of apps for diabetes self-management often measured the number of blood glucose readings but never measured the number of modules or lessons because these features were not offered to users.

“For example, studies of apps for diabetes self-management often measured the number of blood glucose readings due to the popularity of this feature but never measured the number of modules or lessons because these features were not offered to users.”

May combine two claims.

Although researchers measured, on average, 3 indicators in a single study, the majority reported findings descriptively and did not further investigate how engagement with an app contributed to its impact on health and well-being.

“Although researchers measured, on average, 3 indicators in a single study, the majority reported findings descriptively and did not further investigate how engagement with an app contributed to its impact on health and well-being.”

May combine two claims.

Some studies had users self-report their engagement, whereas others omitted reporting engagement altogether and solely related findings on app efficacy.

“Some studies had users self-report their engagement, whereas others omitted reporting engagement altogether and solely related findings on app efficacy.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Evaluating engagement is a priority and calls for the integration of nonintrusive measures of this construct in mobile health evaluation methodology.

“Evaluating engagement is therefore a priority and calls for the integration of nonintrusive measures of this construct in mHealth evaluation methodology [14].”

• cites [14] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The construct of engagement has been quantitatively conceptualized as the amount, duration, breadth, and depth of intervention usage.

“The construct of engagement has been quantitatively conceptualized as the amount, duration, breadth, and depth of intervention usage [11,12].”

• cites [11] sourceNot checked yet

  Not checked in the free run.

• cites [12] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

These digitally mediated mechanisms of action require more granular evaluations capable of analyzing multilevel, temporally dense engagement data.

“These digitally mediated mechanisms of action require more granular evaluations capable of analyzing multilevel, temporally dense engagement data [13].”

• cites [13] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Many researchers have included analytic indicators as a study measure when evaluating apps, but they are not consistent or systematic in their selection.

“While many researchers have included analytic indicators as a study measure when evaluating apps, they are not consistent or systematic in their selection [26].”

• cites [26] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

Analytic insights from microengagement can be coupled with measures of macroengagement to identify the mediating mechanisms that motivate effective engagement.

“In turn, these analytic insights can be coupled with measures of macroengagement to identify the mediating mechanisms that motivate effective engagement.”

Probably not a substantial claim: may not pass S2 (checkable).

There is benefit to understanding how engagement with mobile health apps for chronic conditions has been defined, measured, and analyzed across evaluations.

“We propose that there is benefit to understanding how engagement with mHealth apps for chronic conditions has been defined, measured, and analyzed across evaluations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Layering the frequency of interactions over other indicators added temporal context to better understand how users were engaging over time.

“Layering this analytic indicator over other indicators added temporal context to better understand how users were engaging over time.”

Probably not a substantial claim: may not pass S2 (checkable).

Measuring and reporting on microengagement is fundamental to understanding how people actually use an app to improve their health and well-being.

“However, we posit that measuring and reporting on microengagement is fundamental to understanding how people actually use an app to improve their health and well-being.”

May combine two claims.

Analytic indicators of engagement indicate that users may be engaging effectively with a digital health intervention but do not definitively confirm a relationship between engagement and intended outcomes.

“We propose that analytic indicators of engagement do exactly this: they indicate that users may be engaging effectively with a digital health intervention but do not definitively confirm a relationship between engagement and intended outcomes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A data extraction form was piloted on a sample of included articles to validate proposed codes and add emergent codes.

“The form was piloted on a sample of included articles to validate proposed codes and add emergent codes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Full papers of included abstracts were assessed for final selection by all study authors.

“Following title and abstract review, full papers of included abstracts were assessed for final selection by all study authors.”

Classified as probably not a substantial claim.

The application of analytics is an emerging approach to study and interpret engagement with mobile health interventions.

“The application of analytics, defined as the use of data to generate new insights [18], is an emerging approach to study and interpret engagement with mHealth interventions [19].”

• cites [18] sourceNot checked yet

  Not checked in the free run.

• cites [19] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

This scoping review was guided by the methodological framework developed by Arksey and O'Malley and advanced by Levac et al.

“This scoping review was guided by the methodological framework developed by Arksey and O’Malley [27] and advanced by Levac et al [28].”

• cites [27] sourceNot checked yet

  Not checked in the free run.

• cites [28] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Establishing the relationship between engagement and intended outcomes requires adopting a mixed-methods multidimensional approach to measure effective engagement using multiple assessment strategies.

“Establishing this relationship requires adopting a mixed-methods multidimensional approach to measure effective engagement using multiple assessment strategies [14,25].”

• cites [14] sourceNot checked yet

  Not checked in the free run.

• cites [25] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

A subset of researchers are either not measuring the breadth and duration of engagement in their mHealth evaluations or underreporting the findings.

“TThese findings suggest that a subset of researchers are either not measuring the breadth and duration of engagement in their mHealth evaluations or underreporting the findings.”

May combine two claims.

Measuring time spent engaging with an app helped researchers distinguish exploratory from purposeful engagement, with rapid short page views indicating scanning and prolonged viewing suggesting greater intention and interest.

“Measuring time spent engaging with an app helped researchers to distinguish between exploratory and purposeful engagement; a rapid succession of short page views was indicative of scanning through content, whereas prolonged viewing suggested greater intention and interest in content.”

May combine two claims.

The limited sample of reviewed studies suggests that the relationship between engagement and utilization outcomes is underdeveloped and warrants further study.

“Altogether, the limited sample of reviewed studies suggests that the relationship between engagement and utilization outcomes is underdeveloped and warrants further study.”

May combine two claims.

Insights from the number of pages accessed provided researchers with a broader understanding of the user journey through an app and drew attention to specific content that might drive engagement.

“Insights gleaned from this analytic indicator provided researchers with a broader understanding of the user journey through an app and drew attention to specific content that might drive engagement.”

May combine two claims.

Users perceive apps to be a short-term commitment and access app-based content sporadically for shorter periods of time compared with Web-based interventions.

“Users perceive apps to be a short-term commitment [81] and access app-based content sporadically for shorter periods of time compared with Web-based interventions [82].”

• cites [81] sourceNot checked yet

  Not checked in the free run.

• cites [82] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

We did not map analytic indicators to the 14 identified engagement-related constructs for analysis.

“Finally, we did not map analytic indicators to the 14 identified engagement-related constructs for analysis.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Only 1 reviewer conducted the electronic searches and screened all titles and abstracts against eligibility criteria.

“Due to limited resources, only 1 reviewer conducted the electronic searches and screened all titles and abstracts against eligibility criteria”

Classified as probably not a substantial claim.

Sieverink et al concluded that a limited but deliberate set of only one of 2 different indicators in accordance with the goal of the technology is sufficient to operationalize adherence.

“The authors conclude that a limited but deliberate set of only one of 2 different indicators in accordance with the goal of the technology is sufficient to operationalize adherence.”

• cites [17] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Simblett et al found that studies were reporting idiosyncratic measures of engagement and adherence that were not comparable across studies.

“In a recent review on the barriers and facilitators of engagement with remote measurement technology for managing health, Simblett et al found that studies were reporting idiosyncratic measures of engagement and adherence that were not comparable across studies [26].”

• cites [26] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

It is possible to use multiple constructs in the same study.

“and it is possible to use multiple constructs in the same study.”

Probably not a substantial claim: may not pass S2 (checkable).

Raising the standard of mHealth app efficacy through measuring analytic indicators of engagement may enable greater confidence in the causal impact of apps on improved chronic health and well-being.

“Raising the standard of mHealth app efficacy through measuring analytic indicators of engagement may enable greater confidence in the causal impact of apps on improved chronic health and well-being.”

Classified as probably not a substantial claim.

We reviewed each construct and its analytic operationalizations separately during our data extraction process.

“However, we reviewed each construct and its analytic operationalizations separately during our data extraction process”

May combine two claims.

Perski et al reviewed 113 studies up to November 2015, of which 13 were on mobile phone-based interventions, and only 4 applied log data analyses to study engagement.

“Perski et al conducted a systematic review on engagement with digital behavior change interventions that comprised all studies up to November 2015 [11]. They reviewed 113 studies, of which 13 were on mobile phone–based interventions. Only 4 of those studies applied log data analyses to study engagement with the intervention.”

• cites [11] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Without objective knowledge of how users engage with an app to care for themselves, the mechanisms of action that underlie complex models of digitally mediated behavior change cannot be identified.

“Without objective knowledge of how users engage with an app to care for themselves, the mechanisms of action that underlie complex models of digitally mediated behavior change cannot be identified.”

Classified as probably not a substantial claim.

Eysenbach observed that a substantial portion of participants in eHealth trials stop using the intervention before study end.

“In 2005, Eysenbach published landmark work on the law of attrition [10], which was his observation that a substantial portion of participants in eHealth trials stop using the intervention before study end.”

• cites [10] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

We have provided accurate insights into how researchers have used analytic indicators to measure engagement.

“and provided accurate insights into how researchers have used analytic indicators to measure engagement.”

Classified as probably not a substantial claim. May combine two claims.

Yardley et al assert that effective engagement is defined in relation to the purpose of a specific intervention and can only be established empirically in the context of that intervention.

“Their findings align with our own, and support Yardley et al’s assertion that effective engagement is defined in relation to the purpose of a specific intervention and can only be established empirically in the context of that intervention [14].”

• cites [14] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

This finding suggests that researchers are gaining nuanced insights into how users are engaging with their apps but are not conducting inferential analyses to characterize effective engagement for improved outcomes.

“This finding suggests that researchers are gaining nuanced insights into how users are engaging with their apps but are not conducting inferential analyses to characterize effective engagement for improved outcomes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

We did not assess the quality of included articles.

“We did not assess the quality of included articles”

May combine two claims.

Sieverink et al reviewed 62 papers, of which 7 were on smartphone-based interventions; 5 of those 7 were published after 2016 and 2 in 2013.

“The systematic review on usage-based adherence to eHealth interventions conducted by Sieverink et al reviewed 62 papers, of which 7 were on smartphone-based interventions [17]. Of those 7 papers, 5 were published after 2016, and the other 2 were both published in 2013.”

• cites [17] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Our sampling frame of November 2015 to November 2017 directly follows Perski et al's review and includes 41 studies to address our specific research questions.

“Our sampling frame of November 2015 to November 2017 directly follows Perski et al’s review and includes 41 studies to address our specific research questions.”

• cites Perski et al’s reviewNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Eysenbach posits that attrition is a fundamental characteristic and methodological challenge in the evaluation of eHealth interventions and recommends that usage metrics and determinants of attrition should be highlighted, measured, analyzed, and discussed.

“He posits that attrition is a fundamental characteristic and methodological challenge in the evaluation of eHealth interventions and recommends that “usage metrics and determinants of attrition should be highlighted, measured, analyzed, and discussed” [10].”

• cites [10] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Not assessing the quality of included articles is in line with our review framework, which does not mandate this methodological practice.

“however, this is in line with our review framework, which does not mandate this methodological practice.”

Probably not a substantial claim: may not pass S2 (checkable).

Our review supports researchers to harness these natural attributes for conducting data-driven evaluations of digitally mediated behavior change.

“We propose that our review supports researchers to harness these natural attributes for conducting data-driven evaluations of digitally mediated behavior change.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

This variance suggests that researchers are starting to recognize a conceptual and methodological distinction between the constructs of engagement and adherence.

“This variance suggests that researchers are starting to recognize a conceptual and methodological distinction between the constructs of engagement and adherence.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Conceptual differences exist between some of the 14 engagement-related constructs (eg, usage, feasibility, and adherence).

“We acknowledge that conceptual differences exist between some of these constructs (eg, usage, feasibility, and adherence)”

Classified as probably not a substantial claim.

Morrison et al found that app users logged in twice as often but spent half as much time engaging compared with website users.

“They subsequently found that app users logged in twice as often but spent half as much time engaging compared with website users.”

• cites [83] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Sieverink et al found that over half of all reviewed studies measured adherence to eHealth interventions using a single analytic indicator, and a quarter used 2 indicators.

“In their systematic review, Sieverink et al found that over half of all reviewed studies measured adherence to eHealth interventions using a single analytic indicator, and a quarter used 2 indicators [17].”

• cites [17] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Benchmarking engagement by time range provides more context on a user's intentions and needs than just the total amount of engagement.

“Benchmarking engagement by time range provides more context on a user’s intentions and needs than just the total amount of engagement.”