Pilot evaluation of the Sleep Ninja: A smartphone application for adolescent insomnia symptoms
Open accessThe DOI matches its Crossref record; 53 of the 54 references it cites were found in a registry or an index; 19 of the 41 sources its claims cite were checked for support.
Abstract
Objectives The aim of this study was to test the feasibility, acceptability and preliminary effects of a recently developed smartphone application, Sleep Ninja, for adolescent sleep difficulties. Setting The study was conducted online with Australian individuals recruited through the community. Participants Participants were 50 young people aged 12-16 years with sleep difficulties. Design A single-Arm pre-post design was used to evaluate feasibility, acceptability and sleep and mental health variables at baseline and postintervention. Intervention Cognitive-behavioural therapy for insomnia informed the development of the Sleep Ninja. The core strategies covered by the app are psychoeducation, stimulus control, sleep hygiene and sleep-related cognitive therapy. It includes six training sessions (lessons), a sleep tracking function, recommended bedtimes based on sleep guidelines, reminders to start a wind-down routine each night, a series of sleep tips and general information about sleep. Users progress through each training session and conclude the 6-week programme with a black belt in sleep. Outcome measures Feasibility was evaluated based on consent rates, adherence and attrition, acceptability was assessed using questionnaires and a poststudy interview, and sleep, depression and anxiety variables were assessed at baseline and postintervention. Results Data indicated that the Sleep Ninja is a feasible intervention and is acceptable to young people. Findings showed that there were significant improvements on sleep variables including insomnia (within-group effect size d=-0.90), sleep quality (d=-0.46), depression (d=-0.36) and anxiety (d=-0.41). Conclusions The Sleep Ninja is a promising intervention that could assist adolescents who experience sleep difficulties. A follow-up randomised controlled trial is now warranted. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
Overview
CoChat Summary
TL;DR
This pilot study found that the fully automated Sleep Ninja smartphone app was feasible and acceptable for adolescents aged 12–16 years with sleep difficulties. In a single-arm pre–post evaluation, participants showed significant improvements in insomnia, sleep quality, depression, and anxiety, but the authors note that a controlled trial is needed.
Key findings
- Recruitment and retention were reasonably strong: >300 enquiries led to 60 eligible consent forms, 50 participants enrolled, 47/50 (94%) completed baseline, and 34/50 (72%) completed the poststudy battery.
- Uptake/adherence was moderate: 45 participants (96% of those who completed baseline) downloaded the app; 82% completed the first lesson, 51% completed 4 of 6 lessons, and 33% completed all 6.
- Acceptability was high: 97% said the app was “easy” or “very easy” to use, 78% found it useful or very useful, 72% reported behaviour change, and 91% would recommend it to a friend.
- Preliminary effects favored improvement from baseline to poststudy: insomnia severity decreased (ISI, β=−4.29, p<0.001, d=−0.90), sleep quality improved (PSQI, β=−1.88, p<0.001, d=−0.46), and depression and anxiety also decreased (PHQ-A, β=−2.60, p<0.001, d=−0.36; GAD-7, β=−2.56, p<0.001, d=−0.41). Sleep diary outcomes also improved, including going to bed 35 min earlier, falling asleep 21 min faster, sleeping 33 min longer, and reducing night awakenings to 0.87 on average.
Methods
- Single-arm pre–post pilot study conducted online with Australian community-recruited adolescents aged 12–16 years who had at least mild insomnia symptoms.
- Sample: 50 participants; inclusion required a smartphone (iOS or Android), email, internet access, and personal plus parental consent.
- Main analysis used multilevel modelling comparing baseline with poststudy outcomes; feasibility was assessed via consent, adherence, and attrition, and acceptability via questionnaires plus semistructured interviews.
Limitations
- Stated by the paper: the sample had relatively minor insomnia symptoms at entry, there was no control group, and sleep outcomes were exclusively self-reported.
- Evident from the text: the study was small (n=50) and therefore preliminary, so the reported effects cannot establish causality or separate intervention effects from time or other confounders.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
14 Valid · 1 Not valid · 13 Not checked · 81 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Currently, there are no digital CBT-I programmes that are commercially available for youth.21”
§ Pilot evaluation of the Sleep Ninja: a smartphone application for adolescent insomnia symptoms > Introduction
What the sources say
- Disputedcites [21] Werner-Seidler, 2018, Digitally-delivered cognitive-behavioural therapy for youth insomnia: A systematic review, Internet Interv sourceExperimental — accuracy not yet measured
The Results (Studies meeting inclusion criteria; page unavailable) describe a commercially available online CBT-I program, “Insomnia Relief,” tested in university students. Because the review includes that student sample among its studies of young people, it contradicts the claim that no such program is commercially available for youth. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“with adolescents showing a strong preference for digital delivery (97%) when given the choice between face-to-face and digital CBT-I.17”
§ Pilot evaluation of the Sleep Ninja: a smartphone application for adolescent insomnia symptoms > Introduction
What the sources say
- Partly supportedcites [17] sourceExperimental — accuracy not yet measured
The introductory passage labeled Abstract says online CBT-I may be a preferred treatment for adolescents. It does not report a 97% preference or a choice between face-to-face and digital CBT-I; the Participants section instead describes assignment to Internet or group treatment. (LLM, conf=0.86)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“there is accumulating evidence to support the use of digitally delivered CBT-I in both adults and young people.16–19”
§ Pilot evaluation of the Sleep Ninja: a smartphone application for adolescent insomnia symptoms > Introduction
What the sources say
- Supportedcites [16] de Bruin, 2015, Efficacy of Cognitive Behavioral Therapy for Insomnia in Adolescents: A Randomized Controlled Trial with Internet Therapy, Group Therapy and A Waiting List Condition, Sleep source
This adolescent trial supports the young-people component of the multiply cited sentence. The Discussion (page unavailable) reports that internet-delivered CBT-I was effective for adolescent insomnia and recommends promoting it as a treatment option. (LLM, conf=0.91)
- Supportedcites [17] source
The paper's background section (labelled Abstract in our copy) describes five studies reporting positive effects of online CBT-I in adults. The Discussion reports that Internet-delivered CBT-I alleviated insomnia symptoms in adolescents in this pilot study. Together, these passages support the claim of accumulating evidence in both age groups. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites [18] Okajima, 2011, A meta-analysis on the treatment effectiveness of cognitive behavioral therapy for primary insomnia, Sleep Biol Rhythms source
- Not checked — no full text of the sourcecites [19] Zachariae, 2016, Efficacy of internet-delivered cognitive-behavioral therapy for insomnia - A systematic review and meta-analysis of randomized controlled trials, Sleep Med Rev source
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Insomnia is not only a symptom of depression, but is a common precursor, with high-quality longitudinal data having established insomnia as an independent risk factor for depression onset.5–7”
§ Pilot evaluation of the Sleep Ninja: a smartphone application for adolescent insomnia symptoms > Introduction
What the sources say
- Not checked — no full text of the sourcecites [5] source
- Supportedcites [6] Franzen, 2008, Sleep disturbances and depression: risk relationships for subsequent depression and therapeutic implications, Dialogues Clin Neurosci source
The section “Insomnia is a risk factor for developing depression” reports longitudinal studies in which insomnia predicted new-onset depression, including after adjustment for baseline depressive symptoms or other risk factors. The paper cautions that these associations do not prove causation, but the claim describes insomnia as a risk factor, not a cause. Page numbers were not supplied. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [7] source
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“There is emerging evidence that addressing insomnia in individuals with concurrent insomnia and depression improves both sleep and depression outcomes.8–10”
§ Pilot evaluation of the Sleep Ninja: a smartphone application for adolescent insomnia symptoms > Introduction
What the sources say
- Not checked — no full text of the sourcecites [8] source
- Supportedcites [9] source
The Discussion (page not available in our copy) reports that, among patients with major depressive disorder and insomnia, adding cognitive behavioral therapy for insomnia to antidepressant medication produced higher remission rates for both depression and insomnia than the control treatment. This supports the citing sentence’s qualified description of emerging evidence. (LLM, conf=0.92)
- Not checked — no full text of the sourcecites [10] source
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
62 more claims · 3 with their cited sources checked
Create a free account to see all 109 claims and every source verdict.
- Probably not a substantial claim. 17
- It may state two separate claims as one. 33
- The quote only partly states this claim. 1
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910 · cited 2×
“It is comparable to the PHQ-9 in terms of diagnosing depressive disorders.(32) 33 Each item is scored on a 4-point scale and summed together to form a total depression score ranging from 0 to 24.”
Method(1998) · Cited by 36,049
A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.10.1092 · Cited by 31,765 · cited 4×
“The Generalised Anxiety Disorder 7-item scale (GAD-7) evaluates symptoms of GAD.34 All items are scored on a scale from 0 (not at all) to 3 (nearly every day).”
BackgroundMethodInfluentialDeveloping and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995
Validation of the Insomnia Severity Index as an outcome measure for insomnia research
Bastien, C. (2001) · Sleep Medicine · doi:10.1016/s1389-9457(00)00065-4 · Cited by 8,067 · cited 3×
“The ISI is a psychometrically sound, seven-item self-report measure of insomnia symptoms over the previous 2 weeks.(24) 25 Responses are reported on a Likert scale from 0 to 4, producing total scores of 0–28.”
BackgroundInfluentialThe PHQ-8 as a measure of current depression in the general population
Kroenke, K., Strine, T. W., Spitzer, R. L., Williams, J. B. W., Berry, J. T., & Mokdad, A. H. (2009) · Journal of Affective Disorders · doi:10.1016/j.jad.2008.06.026 · Cited by 5,121
Gulliver, A., Griffiths, K. M., & Christensen, H. (2010) · BMC Psychiatry · doi:10.1186/1471-244x-10-113 · Cited by 3,292 · cited 1×
“This preference may in part be explained by the fact that young people are reluctant to seek help for psychological issues, for reasons that include stigma and a preference to manage the problem themselves.(20) Sleep is typically less stigmatised than disorders like depression, suggesting that targetting sleep it may…”
BackgroundSelf-citationBaglioni, C., Battagliese, G., Feige, B., Spiegelhalder, K., Nissen, C., Voderholzer, U., Lombardo, C., & Riemann, D. (2011) · Journal of Affective Disorders · doi:10.1016/j.jad.2011.01.011 · Cited by 2,779 · cited 1×
“For example, a recent meta-analysis found that insomnia was associated with a greater than twofold increase in depression risk.(5) Although depression has multiple causes and maintaining factors that go beyond the presence of sleep problems, the literature suggests that sleep plays an important role, and targeting sle…”
The consensus sleep diary: standardizing prospective sleep self-monitoring
Carney, C. E., Buysse, D. J., Ancoli-Israel, S., Edinger, J. D., Krystal, A. D., Lichstein, K. L., & Morin, C. M. (2012) · Sleep · doi:10.5665/sleep.1642 · Cited by 2,172
Cited by
Systematic review and meta-analysis of the effectiveness of chatbots on lifestyle behaviours
Singh, B., Olds, T., Brinsley, J., Dumuid, D., Virgara, R., Matricciani, L., Watson, A., Szeto, K., Eglitis, E., Miatke, A., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00856-1 · Cited by 224
Bauducco, S., Pillion, M., Bartel, K., Reynolds, C., Kahn, M., & Gradisar, M. (2024) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2024.101933 · Cited by 102
“Additionally, adolescents may use apps specifically designed for emotion regulation, such as those offering guided meditation, mindfulness exercises, or therapy resources [75,76].”
BackgroundFitzpatrick, O. M., Carson, A., & Weisz, J. R. (2020) · Child Psychiatry & Human Development · doi:10.1007/s10578-020-01089-z · Cited by 109
Eaton, C. K., Vallejo, N., McDonald, X., Wu, J., Rodríguez, R., Muthusamy, N., Mathioudakis, N. N., & Riekert, K. A. (2024) · Journal of Medical Internet Research · doi:10.2196/50508 · Cited by 139
Ueafuea, K., Boonnag, C., Sudhawiyangkul, T., Leelaarporn, P., Gulistan, A., Chen, W., Mukhopadhyay, S. C., Wilaiprasitporn, T., & Piyayotai, S. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3046259 · Cited by 82
“, another team developed a smartphone application, Sleep Ninja, to tackle sleep difficulties in adolescents [90].”
Background
Metadata
- Authors
- Werner-Seidler, A., Wong, Q., Johnston, L., O’Dea, B., Torok, M., & Christensen, H.
- Affiliations
- Black Dog Institute, University of New South Wales, Sydney, NSW, Australia
- Black Dog Institute (AU)
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2019-05
- Volume
- 9
- Issue
- 5
- Pages
- e026502
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC6537982
- OpenAlex
- W2946137690
- Cited by
- 82
- References
- 54
- Citations by year
- 2026: 11 · 2025: 14 · 2024: 16 · 2023: 16 · 2022: 10 · 2021: 7 · 2020: 7 · 2019: 1
- FWCI
- 7.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ Open (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- UNSWorks (University of New South Wales, Sydney, Australia) (submittedVersion, cc-by-nc-nd)
- HighWire Press Open Archive (submittedVersion)
- Funders
- Corella Foundation
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and related disorders
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Keywords
- mobile application, adolescent insomnia, sleep disorders, cognitive behavioral therapy for insomnia, feasibility, acceptability, depression, anxiety, sleep hygiene, stimulus control, psychoeducation, sleep monitoring, digital mental health interventions
- MeSH terms
- Smartphone, Adolescent, Anxiety, Australia, Child, Depression, Female, Humans, Sleep Initiation and Maintenance Disorders, Male, Mental Health Services, Patient Acceptance of Health Care, Pilot Projects, Psychiatric Status Rating Scales, Surveys and Questionnaires, Cognitive Behavioral Therapy, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Pilot evaluation of the Sleep Ninja: A smartphone application for adolescent insomnia symptoms. https://app.cochat.ai/papers/10.1136/bmjopen-2018-026502
- The paper
- Werner-Seidler, A., Wong, Q., Johnston, L., O’Dea, B., Torok, M., & Christensen, H. (2019). Pilot evaluation of the Sleep Ninja: A smartphone application for adolescent insomnia symptoms. BMJ Open. https://doi.org/10.1136/bmjopen-2018-026502
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

