Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study
Open accessAbstract
Objective: The objective of this study was to assess whether a version of the Smoke Free app with a supportive chatbot powered by artificial intelligence (versus a version without the chatbot) led to increased engagement and short-term quit success. Methods: Daily or non-daily smokers aged ≥18 years who purchased the ‘pro’ version of the app and set a quit date were randomly assigned (unequal allocation) to receive the app with or without the chatbot. The outcomes were engagement (i.e. total number of logins over the study period) and self-reported abstinence at a one-month follow-up. Unadjusted and adjusted negative binomial and logistic regression models were fitted to estimate incidence rate ratios (IRRs) and odds ratios (ORs) for the associations of interest. Results: A total of 57,214 smokers were included (intervention: 9.3% (5339); control: 90.7% (51,875). The app with the chatbot compared with the standard version led to a 101% increase in engagement (IRR adj = 2.01, 95% confidence interval (CI) = 1.92–2.11, p < .001). The one-month follow-up rate was 10.6% (intervention: 19.9% (1,061/5,339); control: 9.7% (5,050/51,875). Smokers allocated to the intervention had greater odds of quit success (missing equals smoking: 844/5,339 vs. 3,704/51,875, OR adj = 2.38, 95% CI = 2.19–2.58, p < .001; follow-up only: 844/1,061 vs. 3,704/5,050, OR adj = 1.36, 95% CI = 1.16–1.61, p < .001). Conclusion: The addition of a supportive chatbot to a popular smoking cessation app more than doubled user engagement. In view of very low follow-up rates, there is low quality evidence that the addition also increased self-reported smoking cessation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A recent experimental study found that a supportive AI-powered chatbot doubled the engagement with a smoking cessation app and increased its effectiveness.
“A recent experimental study found that a supportive AI-powered chatbot doubled the engagement with a smoking cessation app and increased its effectiveness [80].”
From Influences on the Uptake of and Engagement With Health and Well-Being Smartphone Apps: Systematic Review (2020) · cites this paper as [80]
The Conclusion says the AI-powered chatbot more than doubled engagement with the smoking cessation app. It describes only low-quality evidence of increased self-reported smoking cessation at one month, owing to very low follow-up rates; the unqualified claim that it increased effectiveness overstates that finding. (LLM, conf=0.90)
Experimental — accuracy not yet measuredOf the 15 studies, 4 (27%) assessed efficacy of AI chatbots in smoking cessation.
“Out of the 15 studies, 4 (27%) studies [23-25,27] assessed the efficacy of AI chatbots in smoking cessation.”
From Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review. (2023) · cites this paper as [24]
Perski et al reported that the intervention group had 2.44 times greater odds of abstinence at the 1-month follow-up than the control group (P<.001).
“Perski et al [24] reported that the intervention group had 2.44 times greater odds of abstinence at the 1-month follow-up than the control group (P<.001).”
From Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review. (2023) · cites this paper as [24]
Chatbots may improve access to health care services, support health-promoting behavior change, and supplement educational programs.
“Chatbots may improve access to health care services (e.g. [105]) and support health-promoting behavior change (e.g. [85]), and supplement educational programs [53].”
From Future directions for chatbot research: An interdisciplinary research agenda (2021) · cites this paper as [85]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Azevedo, R. S. S., de Sousa, J. R., Araujo, M. T. F., Martins Filho, A. J., de Alcantara, B. N., Araujo, F. M. C., Queiroz, M. G. L., Cruz, A. C. R., Vasconcelos, B. H. B., Chiang, J. O., et al. (2018) · Scientific Reports · doi:10.1038/s41598-017-17765-5 · Cited by 3,478 · see citing works
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 1× · see citing works
“In a recent RCT of ‘Woebot’, a conversational agent designed to support young adults with symptoms of depression and anxiety, users allocated to the intervention arm engaged with the chatbot an average of 12 times over the two-week study period.(27) 2 DIGITAL HEALTH”
MethodNicotine replacement therapy for smoking cessation
Stead, L. F., Perera, R., Bullen, C., Mant, D., Hartmann‐Boyce, J., Cahill, K., & Lancaster, T. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000146.pub4 · Cited by 2,514 · see citing works
Biochemical verification of tobacco use and cessation
Benowitz, N. L., III, P. J., Ahijevych, K., Jarvis, M. J., Hall, S., LeHouezec, J., Hansson, A., Lichtenstein, E., Henningfield, J., Tsoh, J., et al. (2002) · Nicotine & Tobacco Research · doi:10.1080/14622200210123581 · Cited by 1,800 · see citing works
WHO Report on the Global Tobacco Epidemic, 2019
Alebshehy, R. (2019) · Cited by 1,354 · see citing works
Persuasive System Design Does Matter: a Systematic Review of Adherence to Web-based Interventions
Kelders, S. M., Kok, R. N., Ossebaard, H. C., & Van Gemert-Pijnen, J. E. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2104 · Cited by 1,308 · see citing works
Outcome criteria in smoking cessation trials: proposal for a common standard
West, R., Hajek, P., Stead, L., & Stapleton, J. (2005) · Addiction · doi:10.1111/j.1360-0443.2004.00995.x · Cited by 1,285 · cited 1× · see citing works
“‘missing equals smoking’ (MES)).(32) The predictor variable was group allocation (i.”
BackgroundPerski, O., Blandford, A., West, R., & Michie, S. (2017) · Translational Behavioral Medicine · doi:10.1007/s13142-016-0453-1 · Cited by 1,193 · see citing works
Self-citationMohr, D. C., Cuijpers, P., & Lehman, K. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1602 · Cited by 1,097 · see citing works
Unravelling the Role of O-glycans in Influenza A Virus Infection
Mayr, J., Lau, K., Lai, J. C. C., Gagarinov, I. A., Shi, Y. Y., McAtamney, S., Chan, R. W. Y., Nicholls, J. M., von Itzstein, M., & Haselhorst, T. (2018) · Scientific Reports · doi:10.1038/s41598-018-34175-3 · Cited by 1,068 · see citing works
Individual behavioural counselling for smoking cessation
Lancaster, T., & Stead, L. F. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001292.pub2 · Cited by 1,015 · see citing works
Evaluating Digital Health Interventions
Murray, E., Hekler, E. B., Andersson, G., Collins, L. M., Doherty, A., Hollis, C., Rivera, D. E., West, R., & Wyatt, J. C. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.008 · Cited by 847 · see citing works
Combined pharmacotherapy and behavioural interventions for smoking cessation
Stead, L. F., Koilpillai, P., Fanshawe, T. R., & Lancaster, T. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008286.pub3 · Cited by 829 · see citing works
Nicotine replacement therapy for smoking cessation
Stead, L. F., Perera, R., Bullen, C., Mant, D., & Lancaster, T. (2008) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000146.pub3 · Cited by 721 · see citing works
Internet-based interventions for smoking cessation
Taylor, G. M. J., Dalili, M. N., Semwal, M., Čivljak, M., Sheikh, A., & Car, J. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007078.pub5 · Cited by 651 · see citing works
Michie, S., Hyder, N., Walia, A., & West, R. (2011) · Addictive Behaviors · doi:10.1016/j.addbeh.2010.11.016 · Cited by 459 · see citing works
Embodied conversational agents in clinical psychology: a scoping review
Provoost, S., Lau, H. M., Ruwaard, J., & Riper, H. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6553 · Cited by 433 · see citing works
Smoking cessation with and without assistance
Zhu, S.-H., Melcer, T., Sun, J., Rosbrook, B., & Pierce, J. P. (2000) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(00)00124-0 · Cited by 405 · see citing works
'The smoking toolkit study': a national study of smoking and smoking cessation in England
Fidler, J. A., Shahab, L., West, O., Jarvis, M. J., McEwen, A., Stapleton, J., Vangeli, E., & West, R. (2011) · BMC Public Health · doi:10.1186/1471-2458-11-479 · Cited by 311 · see citing works
Klopfenstein, L. C., Delpriori, S., Malatini, S., & Bogliolo, A. (2017) · Designing Interactive Systems Conference · doi:10.1145/3064663.3064672 · Cited by 257 · see citing works
Borland, R., Partos, T., Yong, H., Cummings, K. M., & Hyland, A. J. (2011) · Addiction · doi:10.1111/j.1360-0443.2011.03685.x · Cited by 227 · see citing works
Crutzen, R., Peters, G.-J. Y., Portugal, S. D., Fisser, E. M., & Grolleman, J. J. (2011) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2010.09.002 · Cited by 205 · see citing works
Combined pharmacotherapy and behavioural interventions for smoking cessation
Stead, L. F., & Lancaster, T. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008286.pub2 · Cited by 203 · see citing works
Buller, D. B., Borland, R., Bettinghaus, E. P., Shane, J. H., & Zimmerman, D. E. (2014) · Telemedicine and e-Health · doi:10.1089/tmj.2013.0169 · Cited by 178 · see citing works
Bricker, J., Wyszynski, C., Comstock, B., & Heffner, J. L. (2013) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntt056 · Cited by 170 · cited 1× · see citing works
“In two separate trials of Weband app-based smoking cessation interventions, users who logged in a median of eight times or more had increased odds of quitting success.(16,33) Prior to implementing the chatbot (i.”
BackgroundAddressing challenges in promoting healthy lifestyles
Fadhil, A., & Gabrielli, S. (2017) · International Conference on Pervasive Computing Technologies for Healthcare · doi:10.1145/3154862.3154914 · Cited by 141 · see citing works
Attrition Revisited: Adherence and Retention in a Web-Based Alcohol Trial
Murray, E., White, I. R., Varagunam, M., Godfrey, C., Khadjesari, Z., & McCambridge, J. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2336 · Cited by 131 · see citing works
Perski, O., Blandford, A., Ubhi, H. K., West, R., & Michie, S. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0422-8 · Cited by 123 · see citing works
Self-citationCrane, D., Garnett, C., Michie, S., West, R., & Brown, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-22420-8 · Cited by 118 · see citing works
Self-citationWatson, N. L., Mull, K. E., Heffner, J. L., McClure, J. B., & Bricker, J. B. (2018) · Journal of Medical Internet Research · doi:10.2196/10351 · Cited by 118 · see citing works
Khadjesari, Z., Murray, E., Kalaitzaki, E., White, I. R., McCambridge, J., Thompson, S. G., Wallace, P., & Godfrey, C. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1523 · Cited by 114 · see citing works
Performance of English stop smoking services in first 10 years: analysis of service monitoring data
West, R., May, S., West, M., Croghan, E., & McEwen, A. (2013) · BMJ · doi:10.1136/bmj.f4921 · Cited by 103 · see citing works
Feature-level analysis of a novel smartphone application for smoking cessation
Heffner, J. L., Vilardaga, R., Mercer, L. D., Kientz, J. A., & Bricker, J. B. (2015) · The American Journal of Drug and Alcohol Abuse · doi:10.3109/00952990.2014.977486 · Cited by 103 · cited 2× · see citing works
“In two separate trials of Weband app-based smoking cessation interventions, users who logged in a median of eight times or more had increased odds of quitting success.(16,33) Prior to implementing the chatbot (i.”
BackgroundOn the medians of gamma distributions and an equation of Ramanujan
Choi, K. P. (1994) · Proceedings of the American Mathematical Society · doi:10.1090/s0002-9939-1994-1195477-8 · Cited by 87 · see citing works
Gordon, L., Graves, N., Hawkes, A., & Eakin, E. (2007) · Chronic Illness · doi:10.1177/1742395307081732 · Cited by 77 · see citing works
Kramer, J.-N., Künzler, F., Mishra, V., Presset, B., Kotz, D., Smith, S., Scholz, U., & Kowatsch, T. (2019) · JMIR Research Protocols · doi:10.2196/11540 · Cited by 74 · see citing works
Edwards, S. A., Bondy, S. J., Callaghan, R. C., & Mann, R. E. (2013) · Addictive Behaviors · doi:10.1016/j.addbeh.2013.10.036 · Cited by 68 · see citing works
Masaki, K., Tateno, H., Kameyama, N., Morino, E., Watanabe, R., Sekine, K., Ono, T., Satake, K., Suzuki, S., Nomura, A., et al. (2019) · JMIR mHealth and uHealth · doi:10.2196/12694 · Cited by 51 · see citing works
Crane, D., Ubhi, H. K., Brown, J., & West, R. (2019) · F1000Research · doi:10.12688/f1000research.16148.2 · Cited by 37 · see citing works
Self-citationCrane, D., Ubhi, H. K., Brown, J., & West, R. (2018) · F1000Research · doi:10.12688/f1000research.16148.1 · Cited by 33 · see citing works
Self-citationHow do I know what my theory predictsPreprint
Dienes, Z. P. (2019) · doi:10.31234/osf.io/kryqt · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review.
Aggarwal, A., Tam, C. C., Wu, D., Li, X., & Qiao, S. (2023) · Journal of medical Internet research · doi:10.2196/40789 · Cited by 683 · see citing works
“%) reinforced motivation through personalized messages based on TTM (DigiQuit [23]), scoreboards and trackers of milestones (SFA [24]), and motivational messages (CASC [25] and SMAG [27]).”
InfluentialArtificially intelligent chatbots in digital mental health interventions: a review
Boucher, E. M., Harake, N. R., Ward, H. E., Stoeckl, S. E., Vargas, J., Minkel, J., Parks, A. C., & Zilca, R. (2021) · Expert Review of Medical Devices · doi:10.1080/17434440.2021.2013200 · Cited by 440 · see citing works
Szinay, D., Jones, A., Chadborn, T., Brown, J., & Naughton, F. (2020) · Journal of Medical Internet Research · doi:10.2196/17572 · Cited by 289 · see citing works
“A recent experimental study found that a supportive AI-powered chatbot doubled the engagement with a smoking cessation app and increased its effectiveness [80].”
BackgroundSelf-citationSchachner, T., Keller, R., & v Wangenheim, F. (2020) · Journal of Medical Internet Research · doi:10.2196/20701 · Cited by 267 · see citing works
Future directions for chatbot research: An interdisciplinary research agenda
Følstad, A., Araujo, T., Law, E. L.-C., Brandtzaeg, P. B., Papadopoulos, S., Reis, L., Baez, M., Laban, G., McAllister, P., Ischen, C., et al. (2021) · Computing · doi:10.1007/s00607-021-01016-7 · Cited by 247 · see citing works
Saleem, M., Kühne, L., De Santis, K. K., Christianson, L., Brand, T., & Busse, H. (2021) · JMIR Mental Health · doi:10.2196/30000 · Cited by 199 · see citing works
Kowatsch, T., Schachner, T., Harperink, S., Barata, F., Dittler, U., Xiao, G., Stanger, C., v Wangenheim, F., Fleisch, E., Oswald, H., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/25060 · Cited by 103 · see citing works
Bricker, J. B., Mull, K. E., Santiago-Torres, M., Miao, Z., Perski, O., & Di, C. (2022) · Journal of Medical Internet Research · doi:10.2196/39208 · Cited by 44 · see citing works
Self-citationLevin, M. E., Krafft, J., Davis, C. H., & Twohig, M. P. (2021) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2020.1846609 · Cited by 21 · see citing works
Schläpfer, S., Schneider, F., Santhanam, P., Eicher, M., Kowatsch, T., Witt, C. M., & Barth, J. (2024) · JMIR Cancer · doi:10.2196/52386 · Cited by 14 · see citing works
- Authors
- Perski, O., Crane, D., Beard, E., & Brown, J.
- Affiliations
- University College London (GB)
- Corresponding author
- Perski, Olga
- Venue
- DIGITAL HEALTH
- Publisher
- SAGE Publications
- Published
- 2019-01
- Volume
- 5
- Pages
- 2055207619880676-2055207619880676
- ISSN
- 2055-2076 (ISSN-L)
- PMC
- PMC6775545
- OpenAlex
- W2977444880
- Cited by
- 135
- References
- 41
- Citations by year
- 2026: 12 · 2025: 23 · 2024: 29 · 2023: 23 · 2022: 19 · 2021: 19 · 2020: 8 · 2019: 2
- FWCI
- 8.23
- Type
- journal article
- Language
- en
- Update notices
- correction (2020-05-22)
- Open access
- gold
- doi.org
- Locations
- Digital Health (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Cancer Research UK (C1417/A22962)
- Topics
- Health Sciences › Medicine › Physiology › Smoking Behavior and Cessation
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- smoking cessation, chatbots, artificial intelligence, mobile health apps, user engagement, abstinence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
