Chronic Health Conditions and Internet Behavioral Interventions: A Review of Factors to Enhance User Engagement
Open accessAbstract
The objective of this study was to review the evidence about what factors influence user engagement in Internet-based behavioral interventions for chronic illness. We conducted a systematic review of the recent published literature. Searches of MEDLINE (using Ovid and PubMed), The Cochrane Library, and PsycINFO, from January 2000 to December 2008, were completed. Additional articles were identified from searching the bibliographies of retrieved articles. We identified studies of interactive health communication interventions delivered via the Internet that, apart from delivering health information, had another component such as interactive tools to manage illness, decision support for treatment, or social support. We restricted the age range to adulthood. The search identified 186 abstracts; 46 articles were reviewed. We used a qualitative approach called "positive deviance" to study those interventions that have succeeded in engaging users where most have failed. Some ways to improve user engagement in Internet interventions suggested by our review include addressing health concerns that are important and relevant to an individual patient or consumer and an individualized approach, such as personally tailored advice and feedback. Interventions that are part of larger health management programs that include clinicians appear to be especially promising.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 22 cited sources checked · 0 supported · 0 flagged
- Findingreasoning
When judging adherence, the appropriate dosage or usage was often unclear.
“In assessing adherence to the intervention, it was often not clear what the dosage or usage should be.”
Quote in the paperNot a substantial claim - Findingown data
Usage reporting varied across the included studies.
“We found that the reporting of usage was not uniform across studies.”
Quote in the paperNot a substantial claim - Findingown data
The studies with low attrition reported rates from 0% to 8%.
“The studies described below reported low attrition rates, ranging from 0% to 8%.”
Quote in the paperNot a substantial claim - Premisecitation
Trials of online self-help programs generally support developing such interventions for various chronic conditions.
“The results of randomized controlled trials (RCTs) testing Internet-delivered self-help programs generally provide evidence to support development for a range of chronic health conditions.5”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [5] source
Not checked in the free run.
- Study design and conductreasoning
As online interventions grow more complex, isolating the features that help or hinder engagement becomes harder.
“Another study limitation relates to the fact that Internet interventions are increasingly complex, and it is a challenge to tease out the specific features that contribute or detract from the intervention’s ability to keep users engaged.”
Quote in the paperNot a substantial claim
- Interpretationcitation
Adding a clinician coach to an intervention seems particularly promising.
“Interventions that include a clinician coach appeared especially promising.”
Quote in the paperSubstantial claim - Study design and conductreasoning
Restricting to peer-reviewed English articles may have excluded interventions relevant to the research question.
“Because this was a review of peer-reviewed, published journal articles in English, we may have eliminated interventions that otherwise would have served our research question.”
Quote in the paperNot a substantial claim - Premisecitation
Cousineau et al. found that 36% of participants reached an estimated appropriate dose of at least 90 minutes.
“Cousineau et al32,73 reported that 36% of participants in their intervention for infertile women spent an appropriate estimated dose of 90 minutes or more.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Cousineau et al32,73
Not checked in the free run.
- Premisenone given
In online interventions, log-in frequency and average session length are used as proxies for dose.
“For Internet interventions, log-in rates and average session times serve as proxy measures.”
Quote in the paperNot a substantial claim - Interpretationcitation
Interventions embedded in broader clinician-inclusive health management programs seem particularly promising.
“Interventions that are part of larger health management programs that include clinicians appear to be especially promising.”
- Also stated in: § CONCLUSION · hedged
“Interventions that are part of a larger health management program that includes clinicians appear to be especially promising.61”
Quote in the paperSubstantial claim - Also stated in: § CONCLUSION · hedged
- Interpretationreasoning
Because Internet interventions are multifaceted and complex, it is hard to know for certain which specific features work best.
“In conclusion, the complexity of multifaceted Internet interventions makes it difficult to determine with certainty what individual features of these interventions are most effective”
Quote in the paperSubstantial claim- Rests on an assumption: The features that distinguish low-attrition from high-attrition interventions are causally responsible for differences in user engagement, rather than being confounded by other study characteristics.
- Rests on an assumption: The included studies' descriptions of intervention features are accurate enough to attribute engagement differences to them.
- Premisecitation
When judging health outcomes, few formal evaluations account for user engagement or adherence.
“However, little is actually known about the mechanisms or components of the interventions that have the greatest impact,11 and few formal evaluations consider user engagement or adherence to the Internet interventions when assessing the overall impact on health outcomes.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [11] source
Not checked in the free run.
- Premisecitation
The mechanisms and components that make Internet interventions most effective remain largely unknown.
“However, little is actually known about the mechanisms or components of the interventions that have the greatest impact,11 and few formal evaluations consider user engagement or adherence to the Internet interventions when assessing the overall impact on health outcomes.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [11] source
Not checked in the free run.
- Study design and conductreasoning
Since many studies lacked detailed descriptions of intervention features, the review could not be certain which features helped or hurt engagement and attrition.
“Also, many of the studies did not describe the features of the Web-based intervention in detail. Thus, it was not possible to determine with certainty what features may have enhanced or hindered user engagement and impacted attrition.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Internet interventions that adapt to individual needs engage users.
“This research identified two characteristics contributing to Internet interventions that engage users: (1) the intervention targets participants with pressing health concerns, and (2) the intervention adapts to individual needs.”
- Also stated in: § DISCUSSION · hedged
“Methods that adapt the intervention to an individual’s specific needs are more likely to engage users.”
Quote in the paperSubstantial claim- Rests on an assumption: The features that distinguish low-attrition from high-attrition interventions are causally responsible for differences in user engagement, rather than being confounded by other study characteristics.
- Rests on an assumption: The 10 studies at the extremes of attrition are representative enough of Internet interventions for chronic illness to identify general features that influence engagement.
- Also stated in: § DISCUSSION · hedged
- Premisecitation
Computer tailoring is generally more effective than nontailored equivalent applications.
“Computer tailoring has become increasingly popular in the last decade and generally has been found to be more effective than nontailored equivalent applications.70,71”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [70] source
Not checked in the free run.
- cites [71]
Not checked in the free run.
- Premisecitation
Van Straten et al. found that among 97 people who finished the first assignment, 39% completed the full 4-week course.
“The study reported that 39% of the 97 people who completed the first assignment completed the entire 4-week course.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Van Straten et al24
Not checked in the free run.
- Premisecitation
Steele et al. showed that tailoring interventions to individual needs improves success.
“Steele et al58 demonstrated that interventions tailored to individual needs are more successful.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Steele et al58
Not checked in the free run.
- Interpretationreasoning
Internet interventions that target participants with pressing health concerns engage users.
“This research identified two characteristics contributing to Internet interventions that engage users: (1) the intervention targets participants with pressing health concerns, and (2) the intervention adapts to individual needs.”
- Also stated in: § DISCUSSION · hedged
“For example, interventions that target a pressing health concern for a specific population, such as secondary prevention of cardiac failure61 or distress in women undergoing fertility treatment, appear to be more effective.32”
Quote in the paperSubstantial claim- Rests on an assumption: The features that distinguish low-attrition from high-attrition interventions are causally responsible for differences in user engagement, rather than being confounded by other study characteristics.
- Rests on an assumption: The 10 studies at the extremes of attrition are representative enough of Internet interventions for chronic illness to identify general features that influence engagement.
- Also stated in: § DISCUSSION · hedged
- Premisecitation
Steele et al. found that those exposed to at least 75% of the material improved more than those who completed less.
“Steele et al58 determined that participants who were exposed to at least 75% of the material had greater improvement than those who completed less”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Steele et al58
Not checked in the free run.
- Premisecitation
Barrera et al. found that diabetic patients in arms with coach and peer support reported significantly greater perceived social support.
“In a study of diabetic patients by Barrera et al,52 participants in the arms with coach and peer support had significantly increased perceived availability of social support.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Barrera et al,52
Not checked in the free run.
- Interpretationreasoning
Interventions that include social networking and support can meet individual needs.
“Interventions that include functionality for social networking and support, either between the patient and clinician or peer-to-peer, can meet the individual’s specific needs.”
Quote in the paperSubstantial claim - Premisecitation
Andersson et al. found no difference between a headache self-help program with e-mail and weekly phone support and the same program without phone support.
“Andersson et al48 compared a self-help program for headache that included e-mail support and weekly telephone support to the same program without telephone support and found no differences.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Andersson et al48
Not checked in the free run.
- May combine two claims. 19
- Classified as probably not a substantial claim. 10
May combine two claims.
Findingown dataAmong the 39 studies that reported attrition, the range was 0% to 85%, with a median of 24%.
“Attrition ranged from zero to 85% (median = 24%) in the 39 studies reporting attrition.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe review identified 46 articles for inclusion.
“The search identified 186 abstracts; 46 articles were reviewed.”
- Also stated in: § Analysis · plain
“Additional articles were identified from the bibliographies, and 46 studies were subsequently identified for inclusion and are listed in Table 1.”
Quote in the paperNot a substantial claim- Also stated in: § Analysis · plain
May combine two claims.
Findingnone givenOver time, log-in rates generally fell, and the steepest declines occurred early.
“Not surprisingly, log-in rates tended to decline over time (even for studies reporting lower attrition), with the largest drops experienced early in the intervention.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationSouthard et al. found that the Internet intervention group had fewer cardiac events; weight changes were significant, but blood pressure and lipid differences were not.
“Fewer cardiac events occurred in the Internet intervention group. Weight changes were significant, whereas other differences such as blood pressure and lipid levels did not reach statistical significance.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Southard et al61
Not checked in the free run.
May combine two claims.
PremisecitationHarvey-Berino et al. found that the Internet arm was less effective than minimal or frequent in-person therapist support, yet attrition remained low.
“Although the Internet intervention was not as effective as minimal or frequent in-person therapist support, low attrition was attained.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Harvey-Berino et al37,41
Not checked in the free run.
May combine two claims.
InterpretationreasoningIn the Schulz et al. study, professional feedback, social support, and dynamic Web site content seemed to be the main factors engaging users and producing zero attrition.
“The availability of professional feedback and social support and the dynamic nature of the Web site appear to be the key features that engaged users, contributing to a zero attrition rate.”
Quote in the paperSubstantial claim- Rests on an assumption: The features that distinguish low-attrition from high-attrition interventions are causally responsible for differences in user engagement, rather than being confounded by other study characteristics.
Probably not a substantial claim: may not pass S4 (based).
Premisenone givenProgram dose is a critical part of any behavior change intervention.
“An essential component of a behavior change intervention is program dose.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationStrom et al. reported that 56% of 102 eligible participants dropped out; only 20 of 45 remained in the treatment arm over the 6-week study.
“Fifty-six percent of the 102 participants who met inclusion criteria dropped out of the study, leaving only 20 of 45 in the treatment group for the 6-week study duration.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Strom et al50
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningWhen dropout is high, the statistical handling of dropouts matters for judging effectiveness.
“Also, in studies with high dropout rates, how dropouts are treated statistically is important in determining the intervention’s effectiveness.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationHouston and Ford found that adding a motivational introductory page improved outcomes, but concurrent changes left the results inconclusive.
“For example, Houston and Ford29 found that the addition of a motivational introductory page improved the health outcomes studies, but because other changes were made, the results were far from conclusive.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Houston and Ford29
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThe short duration of the Cousineau et al. study and its $100 cash incentive may have boosted participation.
“Participation may have been enhanced by the short duration of the study and $100 cash incentive.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe MeSH database helped avoid incorrect search terms.
“However, using the MeSH database reduced the possibility of searching incorrect keywords.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationBickel et al. retained 58% in the standard arm, 53% in the therapist-delivered arm, and 62% in the computer-assisted arm.
“The study retained 58%, 53%, and 62% of participants in the standard, therapist-delivered, and computer-assisted treatment study arms, respectively.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Bickel et al54
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningExcluding losses to follow-up can overstate the likely effects.
“For example, if losses to follow-up are excluded, the results of the likely effects might be overstated.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationIn Schulz et al., Internet participants reported decreased pain and pain medication use, increased activity, and fewer medical consultations compared with control.
“Compared with the control group that had no exposure to the Internet intervention, Internet participants reported a decrease in pain and use of pain medication, increase in activity, and fewer medical consultations.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Schulz et al47
Not checked in the free run.
May combine two claims.
PremisecitationAndersson et al. had 27 nonresponders (51%) in the treatment arm; only seven showed clinically significant improvement, though most found the treatment helpful.
“There were 27 nonresponders (51%) in the treatment group. Only seven participants reported clinically significant improvement; however, the majority found the treatment to be beneficial.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Andersson et al27
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationnone givenThe use of Internet-supported behavioral health interventions will probably grow.
“Internet-supported interventions in delivering behavioral health interventions are likely to increase.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningFeatures of randomized trials may create barriers that lower enrollment rates.
“The features of randomized trials (a chance of being assigned to the control group, repeated reminders to complete assessments, burdensome questionnaires) may create barriers that lower enrollment rates.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationMcKay et al. reported 13% overall attrition; mean log-ins fell from 2.7 in the first two weeks to 0.5 in the final two weeks.
“McKay et al57 reported that overall attrition was 13%, yet usage dropped from a mean of 2.7 log-ins during the first 2 weeks to 0.5 log-ins during the last 2 weeks.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites McKay et al57
Not checked in the free run.
May combine two claims.
InterpretationreasoningUser characteristics, environment, program content, attractiveness, and interaction influence use of Internet interventions.
“In addition to the technical specifications of the intervention, there are other factors to consider that influence the use of the Internet or Web-based program/intervention.”
- Also stated in: § Study Limitations · plain
“These include characteristics of the user and the user’s environment (eg, demographic variables, disease pathology and severity, cognitive factors, attitudes and beliefs, motivations, influences of family/friends, etc), in addition to the program content”
- Also stated in: § Study Limitations · plain
“the overall attractiveness or quality of the Web site, and the interaction between peers or with the therapist group leader (eg, e-mail, phone, face-to-face support).”
Quote in the paperNot a substantial claim- Also stated in: § Study Limitations · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningHow easy it is to enroll in an online study probably affects adherence.
“The ease of enrolling in Internet studies is likely to impact adherence.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationcitationIn a study setting, incentives and telephone reminders may improve adherence.
“On the other hand, the ability to offer incentives may increase adherence in the study setting, as may telephone reminders.23”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [23] source
Not checked in the free run.
May combine two claims.
InterpretationreasoningThe review concluded that several features-social support, fresh engaging content, cognitive skill-building theory, case manager interaction, tailored feedback, and phone support-helped reduce attrition.
“As a result of these studies, social support, new and engaging material, a theoretical framework of cognitive skill building, interaction with a case manager, tailored feedback, and phone support were features that contributed to decreased attrition.”
Quote in the paperSubstantial claim- Rests on an assumption: The features that distinguish low-attrition from high-attrition interventions are causally responsible for differences in user engagement, rather than being confounded by other study characteristics.
- Rests on an assumption: The 10 studies at the extremes of attrition are representative enough of Internet interventions for chronic illness to identify general features that influence engagement.
May combine two claims.
PremisecitationBuhrman et al. added telephone contact to boost support, motivation, and compliance, and reported 8% attrition.
“In addition to providing close monitoring and tailored feedback via the Internet, the intervention used telephone contact to increase support, motivation, and compliance. This study reported attrition of 8%.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Buhrman et al46
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningMixing RCTs with natural-setting studies may have influenced attrition in either direction.
“We reviewed studies that included RCTs as well as studies conducted in natural settings; this difference may have impacted user attrition in either a positive direction or a negative direction.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductreasoningBecause finding every relevant article was hard, the review might have missed some.
“Some limitations of our review were that it was difficult to find all relevant articles in this area, although the scope of chronic illness and health conditions was intentionally broad. As a result, we may have overlooked relevant studies.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationDevineni and Blanchard reported a 38% dropout rate, yet their intervention group showed a significantly larger reduction in headache activity than symptom monitoring alone.
“Devineni and Blanchard49 studied an Internet intervention for chronic headache and reported 38% dropout rate; however, compared with symptom monitoring alone, the intervention group reported significantly greater decrease in headache activity.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Devineni and Blanchard49
Not checked in the free run.
May combine two claims.
Premisenone givenA growing research base indicates that effective online interventions change behavior, reduce symptoms, and improve health.
“There is a growing body of research to support that effective Internet interventions produce changes in participants’ behaviors that reduce symptoms and improve health.”
Quote in the paperSubstantial claim
The paper relies on this without support.
The features that distinguish low-attrition from high-attrition interventions are causally responsible for differences in user engagement, rather than being confounded by other study characteristics.
“Because these studies differed substantially in possible reasons for attrition (ie, studies with demanding expectations early in the study), it is difficult to determine exact causes for attrition.”
“As a result of these studies, social support, new and engaging material, a theoretical framework of cognitive skill building, interaction with a case manager, tailored feedback, and phone support were features that contributed to decreased attrition.”
The paper relies on this without support.
The 10 studies at the extremes of attrition are representative enough of Internet interventions for chronic illness to identify general features that influence engagement.
“We used the positive deviance approach, described previously, to select the five studies with the lowest attrition studies (those who “succeeded”) and the five studies with highest attrition (those who “failed”).”
“Thus, our qualitative review of the manuscripts focused on 10 studies, representing the two extremes of lowest and highest attrition.”
The paper relies on this without support.
The included studies' descriptions of intervention features are accurate enough to attribute engagement differences to them.
“Also, many of the studies did not describe the features of the Web-based intervention in detail. Thus, it was not possible to determine with certainty what features may have enhanced or hindered user engagement and impacted attrition.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some authors have used literature review to identify retrospectively which factors are associated with success of DBCIs, but the strength of conclusions is limited by correlational nature of evidence and incomplete descriptions of interventions.
“Some authors have used literature review to identify retrospectively which factors are associated with success of DBCIs, 6,[12][13][14] but the strength of the conclusions that can be drawn is limited by the correlational nature of the evidence and incomplete descriptions of the interventions.”
From Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions (2016) · cites this paper as [14]
Increased personal relevance, an individualized approach such as tailored advice and feedback, and including clinicians are promising avenues for improving adherence.
“Two qualitative systematic reviews investigated the influence of specific characteristics on adherence and indicated that increased personal relevance, an individualized approach (eg, involving tailored advice and feedback), and including clinicians are promising avenues [28].”
From How to increase reach and adherence of web-based interventions: a design research viewpoint (2015) · cites this paper as [28]
Cites
Eysenbach, G. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e11 · Cited by 2,524 · see citing works
Eysenbach, G. (2001) · Journal of Medical Internet Research · doi:10.2196/jmir.3.2.e20 · Cited by 2,449 · see citing works
Wantland, D. J., Portillo, C. J., Holzemer, W. L., Slaughter, R., & McGhee, E. M. (2004) · Journal of Medical Internet Research · doi:10.2196/jmir.6.4.e40 · Cited by 966 · see citing works
Using Internet technology to deliver a behavioral weight loss program
Tate, D. F. (2001) · JAMA · doi:10.1001/jama.285.9.1172 · Cited by 925 · cited 1× · see citing works
“Successful programs for weight loss have been demonstrated 39,41,69 and often include record keeping, personalized feedback, and accountability.”
Strategies for Reducing the Door-to-Balloon Time in Acute Myocardial Infarction
Bradley, E. H., Herrin, J. S., Wang, Y., Barton, B. A., Webster, T. R., Mattera, J. A., Roumanis, S. A., Curtis, J. P., Nallamothu, B. K., Magid, D. J., et al. (2006) · New England Journal of Medicine · doi:10.1056/nejmsa063117 · Cited by 838 · see citing works
The power of positive deviance.
Marsh, D. R., Schroeder, D. G., Dearden, K. A., Sternin, J., & Sternin, M. (2004) · BMJ · doi:10.1136/bmj.329.7475.1177 · Cited by 794 · see citing works
Defining Internet-Supported Therapeutic Interventions
Barak, A., Klein, B., & Proudfoot, J. G. (2009) · Annals of Behavioral Medicine · doi:10.1007/s12160-009-9130-7 · Cited by 731 · cited 1× · see citing works
“6 10 This section provides the definitions that we have adopted for our study.”
Tate, D. F., Jackvony, E. H., & Wing, R. R. (2003) · JAMA · doi:10.1001/jama.289.14.1833 · Cited by 666 · see citing works
A Behavior Change Model for Internet Interventions
Ritterband, L. M., Thorndike, F. P., Cox, D. J., Kovatchev, B. P., & Gonder-Frederick, L. A. (2009) · Annals of Behavioral Medicine · doi:10.1007/s12160-009-9133-4 · Cited by 591 · cited 2× · see citing works
“11 Attrition refers to dropping out of the study before completion or stopping use of the Internet program.”
Research in action: using positive deviance to improve quality of health care
Bradley, E. H., Curry, L. A., Ramanadhan, S., Rowe, L. A., Nembhard, I. M., & Krumholz, H. M. (2009) · Implementation Science · doi:10.1186/1748-5908-4-25 · Cited by 524 · see citing works
Lange, A., Rietdijk, D., Hudcovicova, M., van de Ven, J.-P., Schrieken, B., & Emmelkamp, P. M. G. (2003) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.71.5.901 · Cited by 495 · see citing works
Internet-Based Chronic Disease Self-Management: A Randomized Trial
Lorig, K. R., Ritter, P. L., Laurent, D. D., & Plant, K. (2006) · Medical Care · doi:10.1097/01.mlr.0000233678.80203.c1 · Cited by 450 · see citing works
Effect of computer support on younger women with breast cancer
Gustafson, D. H., Hawkins, R., Pingree, S., McTavish, F., Arora, N. K., Mendenhall, J., Cella, D. F., Serlin, R. C., Apantaku, F. M., Stewart, J., et al. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016007435.x · Cited by 387 · see citing works
Internet-based physical activity interventions: a systematic review of the literature.
van den Berg, M. H., Schoones, J. W., & Vliet Vlieland, T. P. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.3.e26 · Cited by 377 · see citing works
The Diabetes Network Internet-Based Physical Activity Intervention
McKay, H. G., King, D., Eakin, E. G., Seeley, J. R., & Glasgow, R. E. (2001) · Diabetes Care · doi:10.2337/diacare.24.8.1328 · Cited by 363 · cited 1× · see citing works
“McKay et al 57 reported that overall attrition was 13%, yet usage dropped from a mean of 2.7 log-ins during the first 2 weeks to 0.5 log-ins during the last 2 weeks.”
InfluentialBarrera, M., Glasgow, R. E., McKay, H. G., Boles, S. M., & Feil, E. G. (2002) · American Journal of Community Psychology · doi:10.1023/a:1016369114780 · Cited by 333 · see citing works
Web-based tailored nutrition education: results of a randomized controlled trial
Oenema, A. (2001) · Health Education Research · doi:10.1093/her/16.6.647 · Cited by 316 · see citing works
Andersson, G. (2002) · Psychosomatic Medicine · doi:10.1097/01.psy.0000031577.42041.f8 · Cited by 312 · cited 1× · see citing works
“Andersson et al 27 evaluated an Internet intervention using cognitive behavioral self-help treatment for distress associated with tinnitus.”
Internet-Based Treatment for Insomnia: A Controlled Evaluation.
Ström, L., Pettersson, R., & Andersson, G. (2004) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.72.1.113 · Cited by 311 · see citing works
van Straten, A., Cuijpers, P., & Smits, N. (2008) · Journal of Medical Internet Research · doi:10.2196/jmir.954 · Cited by 297 · see citing works
Christensen, H., Griffiths, K. M., & Korten, A. (2002) · Journal of Medical Internet Research · doi:10.2196/jmir.4.1.e3 · Cited by 292 · cited 1× · see citing works
“Southard et al (2003)(61) (USA) Prevention of coronary heart disease RCT, 6 mo 4 1–3, 5, 6 + 1: Internet case management (53) 2: Usual care (51) Harvey-Berino et al (2002)(37,41) (USA) Weight loss maintenance RCT, 22 wk 4 1–7 j 1: Internet (15) 2: In-person (14) 3: Control (15) Buhrman et al (2004)(46) (Sweden) Chroni…”
Effectiveness of an Internet-based program for reducing risk factors for eating disorders.
Winzelberg, A. J., Eppstein, D., Eldredge, K. L., Wilfley, D., Dasmahapatra, R., Dev, P., & Taylor, C. B. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.68.2.346 · Cited by 291 · see citing works
A controlled trial of self-help treatment of recurrent headache conducted via the Internet.
Ström, L., Pettersson, R., & Andersson, G. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.68.4.722 · Cited by 267 · cited 2× · see citing works
“Strom et al 50 recruited participants for an Internet relaxation training intervention for recurrent headache using advertisements in print media.”
Treatment of panic disorder via the internet: A randomized trial of a self-help program
Carlbring, P., Westling, B. E., Ljungstrand, P., Ekselius, L., & Andersson, G. (2001) · Behavior Therapy · doi:10.1016/s0005-7894(01)80019-8 · Cited by 262 · cited 2× · see citing works
“Strom et al (2000)(50) (Sweden) Recurrent headache RCT, 6 wk 56 1–3 + 1: Internet (20) 2: Waitlist (25) Andersson et al (2002)(27) (Sweden) Tinnitus distress RCT, 6 wk, 1 y F/U 51 1, 6 + 1: Internet cognitive behavior therapy (53), 2: wait list (64) van Straten et al (2008)(24) (the Netherlands) Depression, anxiety, w…”
Clarke, G., Reid, E., Eubanks, D., O'Connor, E., DeBar, L. L., Kelleher, C., Lynch, F., & Nunley, S. (2002) · Journal of Medical Internet Research · doi:10.2196/jmir.4.3.e14 · Cited by 256 · cited 2× · see citing works
“For example, Clarke et al 23 had very low enrollment in their study of depression among HMO members but point out that the low rates are not an indication of the acceptability of this intervention offered outside a research study.”
Celio, A. A., Winzelberg, A. J., Wilfley, D. E., Eppstein-Herald, D., Springer, E. A., Dev, P., & Taylor, C. B. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.68.4.650 · Cited by 251 · see citing works
A Randomized Controlled Trial of a Commercial Internet Weight Loss Program
Womble, L. G., Wadden, T. A., McGuckin, B. G., Sargent, S. L., Rothman, R. A., & Krauthamer‐Ewing, E. S. (2004) · Obesity Research · doi:10.1038/oby.2004.124 · Cited by 247 · see citing works
Controlled trial of Internet-based treatment with telephone support for chronic back pain
Buhrman, M., Fältenhag, S., Ström, L., & Andersson, G. (2004) · Pain · doi:10.1016/j.pain.2004.07.021 · Cited by 232 · see citing works
Clinical Trial of an Internet-based Case Management System for Secondary Prevention of Heart Disease
Southard, B. H., Southard, D. R., & Nuckolls, J. (2003) · Journal of Cardiopulmonary Rehabilitation · doi:10.1097/00008483-200309000-00003 · Cited by 216 · see citing works
Print versus website physical activity programs
Marshall, A. L., Leslie, E. R., Bauman, A. E., Marcus, B. H., & Owen, N. (2003) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(03)00111-9 · Cited by 208 · see citing works
A randomized controlled trial of an internet-based treatment for chronic headache
Devineni, T., & Blanchard, E. B. (2005) · Behaviour Research and Therapy · doi:10.1016/j.brat.2004.01.008 · Cited by 203 · cited 1× · see citing works
“Devineni and Blanchard 49 studied an Internet intervention for chronic headache and reported 38% dropout rate; however, compared with symptom monitoring alone, the intervention group reported significantly greater decrease in headache activity.”
Computerized behavior therapy for opioid-dependent outpatients: a randomized controlled trial.
Bickel, W., Marsch, L., Buchhalter, A., & Badger, G. (2008) · Experimental and clinical psychopharmacology · doi:10.1037/1064-1297.16.2.132 · Cited by 202 · see citing works
An online family intervention to reduce parental distress following pediatric brain injury.
Wade, S. L., Carey, J., & Wolfe, C. R. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.3.445 · Cited by 199 · see citing works
Does using the Internet facilitate the maintenance of weight loss?
Harvey-Berino, J., Pintauro, S., Buzzell, P., DiGiulio, M., Casey Gold, B., Moldovan, C., & Ramirez, E. (2002) · International Journal of Obesity · doi:10.1038/sj.ijo.0802051 · Cited by 197 · see citing works
Randomized pilot of a self-guided internet coping group for women with early-stage breast cancer
Owen, J. E., Klapow, J. C., Roth, D. L., Shuster, J. L., Bellis, J., Meredith, R., & Tucker, D. C. (2005) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm3001_7 · Cited by 192 · see citing works
Cook, R. F., Billings, D. W., Hersch, R. K., Back, A. S., & Hendrickson, A. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.2.e17 · Cited by 170 · see citing works
Do Internet interventions for consumers cause more harm than good? A systematic review
Bessell, T. L., McDonald, S., Silagy, C. A., Anderson, J. N., Hiller, J. E., & Sansom, L. N. (2002) · Health Expectations · doi:10.1046/j.1369-6513.2002.00156.x · Cited by 162 · see citing works
Internet and psychological treatment. How well can they be combined?
Carlbring, P., & Andersson, G. (2006) · Computers in Human Behavior · doi:10.1016/j.chb.2004.10.009 · Cited by 151 · see citing works
Nguyen, H. Q., Carrieri-Kohlman, V., Rankin, S. H., Slaughter, R., & Stulbarg, M. S. (2004) · Computers in Biology and Medicine · doi:10.1016/s0010-4825(03)00046-5 · Cited by 147 · see citing works
Design and Pilot Evaluation of an Internet Smoking Cessation Program
Lenert, L. A., Muñoz, R. F., Stoddard, J., Delucchi, K., Bansod, A., Skoczen, S., & Pérez‐Stable, E. J. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1128 · Cited by 140 · see citing works
Internet‐Based Treatment of Headache: Does Telephone Contact Add Anything?
Andersson, G., Lundström, P., & Ström, L. (2003) · Headache: The Journal of Head and Face Pain · doi:10.1046/j.1526-4610.2003.03070.x · Cited by 136 · see citing works
The Feasibility of Using Internet Support for the Maintenance of Weight Loss
Harvey-Berino, J., Pintauro, S. J., & Gold, E. C. (2002) · Behavior Modification · doi:10.1177/0145445502026001006 · Cited by 135 · see citing works
Directions for the International Society for Research on Internet Interventions (ISRII)
Ritterband, L. M., Andersson, G., Christensen, H. M., Carlbring, P., & Cuijpers, P. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.3.e23 · Cited by 132 · see citing works
Online psychoeducational support for infertile women: a randomized controlled trial
Cousineau, T. M., Green, T. C., Corsini, E., Seibring, A., Showstack, M. T., Applegarth, L., Davidson, M., & Perloe, M. (2008) · Human Reproduction · doi:10.1093/humrep/dem306 · Cited by 127 · see citing works
Linke, S., Murray, E., Butler, C., & Wallace, P. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.2.e10 · Cited by 127 · see citing works
Design and Implementation of an Interactive Website to Support Long-Term Maintenance of Weight Loss
Stevens, V. J., Funk, K., Brantley, P. J., Erlinger, T. P., Myers, V. H., Champagne, C. M., Bauck, A. E., Samuel‐Hodge, C. D., & Hollis, J. F. (2008) · Journal of Medical Internet Research · doi:10.2196/jmir.931 · Cited by 124 · see citing works
Maintenance of weight loss in overweight middle-aged women through the Internet
Cussler, E. C., Teixeira, P. J., Going, S. B., Houtkooper, L. B., Metcalfe, L. L., Blew, R. M., Ricketts, J. R., Lohman, J., Stanford, V. A., & Lohman, T. G. (2008) · Obesity · doi:10.1038/oby.2008.19 · Cited by 115 · see citing works
The Science of Internet Interventions
Ritterband, L. M., & Tate, D. F. (2009) · Annals of Behavioral Medicine · doi:10.1007/s12160-009-9132-5 · Cited by 111 · see citing works
Ruwaard, J., Lange, A., Bouwman, M., Broeksteeg, J., & Schrieken, B. (2007) · Cognitive Behaviour Therapy · doi:10.1080/16506070701381863 · Cited by 99 · see citing works
Allen, M., Iezzoni, L. I., Huang, A., Huang, L., & Leveille, S. G. (2008) · Nursing Research · doi:10.1097/01.nnr.0000313478.47379.98 · Cited by 95 · see citing works
Lau, J. T. F., Lau, M., Cheung, A., & Tsui, H. Y. (2008) · AIDS Care · doi:10.1080/09540120701694048 · Cited by 79 · see citing works
Evaluation of an interactive, Internet-based weight loss program: a pilot study
Booth, A., Nowson, C. A., & Matters, H. (2008) · Health Education Research · doi:10.1093/her/cyn007 · Cited by 71 · see citing works
Internet-based rehabilitation for individuals with chronic pain and burnout: a randomized trial
Brattberg, G. (2006) · International Journal of Rehabilitation Research · doi:10.1097/01.mrr.0000210055.17291.f5 · Cited by 69 · cited 1× · see citing works
“Strom et al (2000)(50) (Sweden) Recurrent headache RCT, 6 wk 56 1–3 + 1: Internet (20) 2: Waitlist (25) Andersson et al (2002)(27) (Sweden) Tinnitus distress RCT, 6 wk, 1 y F/U 51 1, 6 + 1: Internet cognitive behavior therapy (53), 2: wait list (64) van Straten et al (2008)(24) (the Netherlands) Depression, anxiety, w…”
Web 2.0 and chronic illness: new horizons, new opportunities.
Seeman, N. (2008) · PubMed · Cited by 67 · see citing works
Brug, J., & van Assema, P. (2000) · Appetite · doi:10.1006/appe.2000.0322 · Cited by 67 · see citing works
Friedman, S. R., Mateu‐Gelabert, P., Sandoval, M., Hagan, H., & Jarlais, D. C. D. (2008) · BMC Public Health · doi:10.1186/1471-2458-8-94 · Cited by 63 · see citing works
Web-based education and support for patients on the cardiac surgery waiting list
Scherrer-Bannerman, A., Fofonoff, D., Minshall, D. K., Downie, S., Brown, M. C., Leslie, F., & McGowan, P. (2000) · Journal of Telemedicine and Telecare · doi:10.1258/1357633001935662 · Cited by 50 · see citing works
Examination of program exposure across intervention delivery modes: face-to-face versus internet
Steele, R. M., Mummery, W. K., & Dwyer, T. (2007) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-4-7 · Cited by 47 · see citing works
Wise, M. E., Gustafson, D. H., Sorkness, C. A., Molfenter, T. D., Staresinic, A. G., Meis, T. M., Hawkins, R. P., Shanovich, K., & Walker, N. P. (2007) · Health Promotion Practice · doi:10.1177/1524839906289983 · Cited by 46 · see citing works
Schulz, P. J., Rubinell, S., & Hartung, U. (2007) · International Journal of Public Health · doi:10.1007/s00038-007-5127-9 · Cited by 43 · see citing works
Houston, T., & Ford, D. E. (2008) · Informatics for Health and Social Care · doi:10.1080/14639230701842240 · Cited by 43 · see citing works
Goessens, B. M. B., Visseren, F. L. J., de Nooijer, J., van den Borne, H. W., Algra, A., Wierdsma, J., & van der Graaf, Y. (2008) · Patient Education and Counseling · doi:10.1016/j.pec.2008.06.004 · Cited by 40 · see citing works
A positive deviance-based antenatal nutrition project improves birth-weight in Upper Egypt.
Ahrari, M., Houser, R. F., Yassin, S., Mogheez, M., Hussaini, Y., Crump, P., Darmstadt, G. L., Marsh, D., & Levinson, F. J. (2006) · PubMed Central · Cited by 39 · see citing works
Cousineau, T. M., Houle, B. C., Bromberg, J., Fernandez, K. C., & Kling, W. C. (2008) · Journal of Nutrition Education and Behavior · doi:10.1016/j.jneb.2007.04.376 · Cited by 37 · see citing works
Kerr, J., Patrick, K., Norman, G. J., Stein, M. B., Calfas, K. J., Zabinski, M. F., & Robinson, A. H. (2007) · Depression and Anxiety · doi:10.1002/da.20320 · Cited by 35 · see citing works
Cullen, K. W. (2008) · American Journal of Health Behavior · doi:10.5993/ajhb.32.1.4 · Cited by 32 · see citing works
Awofeso, N., Irwin, T., & Forrest, G. (2008) · PubMed · Cited by 24 · see citing works
Rural women with chronic illness
Weinert, C., & Hill, W. G. (2005) · Women s Health Issues · doi:10.1016/j.whi.2005.06.002 · Cited by 21 · see citing works
Retention in a computer-based outreach intervention for chronically ill rural women
Weinert, C., Cudney, S. A., & Hill, W. G. (2008) · Applied Nursing Research · doi:10.1016/j.apnr.2006.08.002 · Cited by 12 · see citing works
Outcome Measures of Efficacy Associated with a Web-Enabled Asthma Self-Management Programme
Atherton, M. (2000) · Disease Management & Health Outcomes · doi:10.2165/00115677-200008040-00006 · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Perski, O., Blandford, A., West, R., & Michie, S. (2017) · Translational Behavioral Medicine · doi:10.1007/s13142-016-0453-1 · Cited by 1,193 · see citing works
Can Mobile Phone Apps Influence People’s Health Behavior Change? An Evidence Review
Zhao, J., Freeman, B., & Li, M. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5692 · Cited by 618 · see citing works
Carolan, S., Harris, P. R., & Cavanagh, K. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7583 · Cited by 385 · see citing works
“In this study, the first two steps were adapted and applied to study interventions showing the highest levels of engagement (cf. [ 38]).”
The Effectiveness of Prompts to Promote Engagement With Digital Interventions: A Systematic Review
Alkhaldi, G., Hamilton, F. L., Lau, R., Webster, R., Michie, S., & Murray, E. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.4790 · Cited by 273 · see citing works
The mobile revolution—using smartphone apps to prevent cardiovascular disease
Neubeck, L., Lowres, N., Benjamin, E. J., Freedman, S. B., Coorey, G., & Redfern, J. (2015) · Nature Reviews Cardiology · doi:10.1038/nrcardio.2015.34 · Cited by 232 · see citing works
How to increase reach and adherence of web-based interventions: a design research viewpoint
Ludden, G. D., van Rompay, T. J., Kelders, S. M., & van Gemert-Pijnen, J. E. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4201 · Cited by 185 · see citing works
Depp, C. A., Ceglowski, J., Wang, V. C., Yaghouti, F., Mausbach, B. T., Thompson, W. K., & Granholm, E. L. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.10.053 · Cited by 215 · see citing works
van Beugen, S., Ferwerda, M., Hoeve, D., Rovers, M. M., Spillekom-van Koulil, S., van Middendorp, H., & Evers, A. W. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2777 · Cited by 189 · see citing works
Gilbey, D., Morgan, H., Lin, A., & Perry, Y. (2020) · Journal of Medical Internet Research · doi:10.2196/20158 · Cited by 112 · see citing works
“However, digital health interventions are also commonly reported to have very high rates of attrition and disengagement [37,38], with up to 60% to 80% of users discontinuing their use [39-41], and the quality of evidence supporting the effectiveness of digital health interventions is also frequently lacking [42-45].”
Scholten, M. R., Kelders, S. M., & Van Gemert-Pijnen, J. E. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7351 · Cited by 97 · see citing works
“Nonadherence is an important issue in Web-based interventions for mental health [7] and becomes an even bigger problem when evidence-based therapies are deployed as free-to-access self-guided Web-based therapeutic interventions [8].”
BackgroundInfluentialFerwerda, M., van Beugen, S., van Middendorp, H., Spillekom-van Koulil, S., Donders, A. R. T., Visser, H., Taal, E., Creemers, M. C. W., van Riel, P. C. L. M., & Evers, A. W. M. (2017) · Pain · doi:10.1097/j.pain.0000000000000845 · Cited by 85 · see citing works
Theoretical Perspectives of Adherence to Web-Based Interventions: a Scoping Review
Ryan, C., Bergin, M., & Wells, J. S. (2018) · International Journal of Behavioral Medicine · doi:10.1007/s12529-017-9678-8 · Cited by 86 · see citing works
Vugts, M. A. P., Joosen, M. C. W., van der Geer, J. E., Zedlitz, A. M. E. E., & Vrijhoef, H. J. M. (2018) · PLOS ONE · doi:10.1371/journal.pone.0196467 · Cited by 85 · see citing works
eHealth for Breast Cancer Survivors: Use, Feasibility and Impact of an Interactive Portal
Kuijpers, W., Groen, W. G., Oldenburg, H. S., Wouters, M. W., Aaronson, N. K., & van Harten, W. H. (2016) · JMIR Cancer · doi:10.2196/cancer.5456 · Cited by 77 · see citing works
Chen, A. T., Wu, S., Tomasino, K. N., Lattie, E. G., & Mohr, D. C. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103187 · Cited by 58 · see citing works
“The Engagement (usage) category was used to represent the conceptualization of engagement as participants’ efforts to complete the intervention as prescribed, which is often called adherence in the behavioral science literature on digital interventions (e.g. [52,53]).”
Al-Durra, M., Nolan, R. P., Seto, E., Cafazzo, J. A., & Eysenbach, G. (2018) · Journal of Medical Internet Research · doi:10.2196/11924 · Cited by 32 · see citing works
Lappalainen, P., Keinonen, K., Lappalainen, R., Selinheimo, S., Vuokko, A., Sainio, M., Liesto, S., Tolvanen, A., & Paunio, T. (2024) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2024.111830 · Cited by 17 · see citing works
Dworschak, C., Heim, E., Kuhn, N., Schwager, J., Tröster, A., & Maercker, A. (2023) · doi:10.31219/osf.io/238xz · Cited by 4 · see citing works
- Authors
- SCHUBART, J. R., STUCKEY, H. L., GANESHAMOORTHY, A., & SCIAMANNA, C. N.
- Affiliations
- Pennsylvania State University (US)
- Venue
- CIN: Computers, Informatics, Nursing
- Published
- 2011-02
- Volume
- 29
- Issue
- 2
- Pages
- 81-92
- ISSN
- 1538-2931 (ISSN-L)
- PMC
- PMC6424131
- OpenAlex
- W4319308297
- Semantic Scholar
- 8b91781550274531d88da45010610e45c66256ad
- Cited by
- 135
- References
- 70
- Citations by year
- 2026: 2 · 2025: 10 · 2024: 8 · 2023: 10 · 2022: 10 · 2021: 14 · 2020: 10 · 2019: 10 · 2018: 9 · 2017: 14 · 2016: 12 · 2015: 8 · 2014: 7 · 2013: 7 · 2012: 3
- FWCI
- 0.65
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- CIN Computers Informatics Nursing (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Heart, Lung, and Blood Institute (R01HL088590)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Keywords
- chronic disease, user engagement, social support, decision support, personalized feedback, tailored advice, clinician engagement, positive deviance
- MeSH terms
- Behavior, Chronic Disease, Humans, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read under definition version 3 (extract-v3) on 2026-09-28.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
