Feasibility and Acceptability of Using a Mobile Phone App for Characterizing Auditory Verbal Hallucinations in Adolescents With Early-Onset Psychosis: Exploratory Study
Open accessThe DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Auditory verbal hallucinations (AVH) are the most frequent symptom in early-onset psychosis (EOP) and a risk factor for increased suicide attempts in adolescents. Increased knowledge of AVH characteristics can lead to better prediction of risk and precision of diagnosis and help identify individuals with AVH who need care. As 98% of Norwegian adolescents aged 12 to 16 years own a mobile phone, the use of mobile phone apps in symptom assessment and patient communication is a promising new tool. However, when introducing new technology to patients, their subjective experiences are crucial in identifying risks, further development, and potential integration into clinical care. OBJECTIVE: The objective was to explore the feasibility and acceptability of a newly developed mobile phone app in adolescents with EOP by examining compliance with the app and user experiences. Indication of validity was explored by examining associations between AVH dimensions, which were correlated and analyzed. METHODS: Three adolescents with EOP and active AVH were enrolled. Real-time AVH were logged on an iPod touch using the experience sampling method (ESM), for seven or more consecutive days. The app included five dimensions of AVH characteristics and was programmed with five daily notifications. Feasibility and acceptability were examined using the mean response rate of data sampling and by interviewing the participants. Validity was assessed by examining associations between the AVH dimensions using nonparametric correlation analysis and by visual inspection of temporal fluctuations of the AVH dimensions. RESULTS: One participant was excluded from the statistical analyses but completed the interview and was included in the examination of acceptability. The sampling period of the two participants was mean 12 (SD 6) days with overall completed sampling rate of 74% (SD 30%), indicating adequate to high compliance with the procedure. The user experiences from the interviews clustered into four categories: (1) increased awareness, (2) personal privacy, (3) design and procedure, and (4) usefulness and clinical care. One participant experienced more commenting voices during the sampling period, and all three participants had concerns regarding personal privacy when using electronic devices in symptom assessment. The AVH dimensions of content, control, and influence showed moderate to strong significant correlations with all dimensions (P<.001). Days of data sampling showed weak to moderate correlations with localization (P<.001) and influence (P=.03). Visual inspection indicated that the app was able to capture fluctuations within and across days for all AVH dimensions. CONCLUSIONS: This study demonstrates the value of including patients' experiences in the development and pilot-testing of new technology. Based on the small sample size, the use of mobile phones with ESM seems feasible for patients with EOP, but the acceptability of using apps should be considered. Further investigation with larger samples is warranted before definitive conclusions are made.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 How the app is used by the participants. In the first screen the participant chooses yes or no to whether they hear voices right now. If they answer “yes,” the five following screens with the visual analogue scales for the five auditory verbal hallucinations (AVH) dimensions appear. Each screen represents the following AVH characteristic: content (emotional), localization (perceptual), control (cognitive), intensity (intensity), and influence (influence). Previously presented on a poster at the International Congress on Schizophrenia Research, March 24-28, 2017; San Diego, CA, with the abstract by Bless et al [46]. iPhone frames adopted from a design by Todd Zaki Warfel.](https://figures.cochat.ai/paper-figures/daf48998b7af4ac6a1315444cb6bc877/feasibility-and-acceptability-of-using-a-mobile-phone-app-for-characterizing-figure-1.jpg)

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, the intervention durations ranged from as short as 7 days [58] or 2 sessions [59] to as long as 24 weeks [60] or 24 months [61].”
From Understanding Mobile Health and Youth Mental Health: Scoping Review (2023) · cites this paper as [58]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Positive and Negative Syndrome Scale (PANSS) for schizophrenia.
Kay, S. R., Fiszbein, A., & Opler, L. A. (1987) · Schizophrenia Bulletin · doi:10.1093/schbul/13.2.261 · Cited by 21,378
KAUFMAN, J., BIRMAHER, B., BRENT, D., RAO, U., FLYNN, C., MORECI, P., WILLIAMSON, D., & RYAN, N. (1997) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-199707000-00021 · Cited by 10,148
Ecological momentary assessment
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091415 · Cited by 6,454
A Children's Global Assessment Scale (CGAS)
Shaffer, D. (1983) · Archives of General Psychiatry · doi:10.1001/archpsyc.1983.01790100074010 · Cited by 3,716 · cited 3×
“AVH: auditory verbal hallucinations BAVQ-R: Beliefs About Voices Questionnaire-revised version CGAS: Children’s Global Assessment Scale EOP: early-onset psychosis ESM: experience sampling method IQ: intelligence quotient PANSS: Positive and Negative Syndrome Scale VAS: visual analog scale Edited by G Strudwick; submit…”
MethodInfluentialApplied Statistics for the Behavioral Sciences
Witz, K., Hinkle, D. E., Wiersma, W., & Jurs, S. G. (1990) · Journal of Educational Statistics · doi:10.2307/1164825 · Cited by 2,913
Self-reports: How the questions shape the answers.
Schwarz, N. (1999) · American Psychologist · doi:10.1037/0003-066x.54.2.93 · Cited by 2,846 · cited 1×
“Assessing AVH using traditional retrospective measures, such as the Positive and Negative Syndrome Scale (PANSS) [9] or the Scale for the Assessment of Positive Symptoms (SAPS) [10], may lead to loss of symptom information due to recall errors and state-dependent biases [11-13].”
MethodScale for Assessment of Positive Symptoms (SAPS) 1984
Andreasen, N. C. (1995) · Open Scholarship Institutional Repository (Washington University in St. Louis) · Cited by 2,423
Insel, T. R. (2010) · Nature · doi:10.1038/nature09552 · Cited by 1,783
Self-focused attention and negative affect: A meta-analysis.
Mor, N., & Winquist, J. (2002) · Psychological Bulletin · doi:10.1037/0033-2909.128.4.638 · Cited by 1,069 · cited 1×
“Increased self-awareness is associated with increased negative affect, particularly in individuals with existing high negative affectivity [51,52].”
BackgroundModelling cognition in emotional disorder: the S-REF model
Wells, A., & Matthews, G. (1996) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(96)00050-2 · Cited by 1,067
Research works printed but not linked: 3
- 2018, Barn og medier-undersøkelsen 2018: 9-18-åringer om medievaner og opplevelser
- 2013, Nasjonal faglig retningslinje for utredning, behandling og oppfølging av personer med psykoselidelser
- 1999, Wasi Manual; Wechsler Abbreviated Scale Of Intelligence Manual
Cited by
Security and Privacy of mHealth Applications: A Scoping Review
Nurgalieva, L., O'Callaghan, D., & Doherty, G. (2020) · IEEE Access · doi:10.1109/access.2020.2999934 · Cited by 132
“Studies covered a wide range of mental health conditions , which included stress [3], depression [55], [60], [71], sleep disorders [88], smoking cessation [55], [98], psychosis [4], anxiety [53], and cognitive decline such as dementia [58], [65].”
BackgroundEisele, G. V., Vachon, H., Lafit, G., Tuyaerts, D., Houben, M., Kuppens, P., Myin‐Germeys, I., & Viechtbauer, W. (2022) · Psychological Assessment · doi:10.1037/pas0001177 · Cited by 95
mHealth for schizophrenia spectrum disorders management: A systematic review
Chivilgina, O., Wangmo, T., Elger, B. S., Heinrich, T. W., & Jotterand, F. (2020) · International Journal of Social Psychiatry · doi:10.1177/0020764020933287 · Cited by 60
Camacho, E., Levin, L., & Torous, J. B. (2019) · Journal of Medical Internet Research · doi:10.2196/16393 · Cited by 58
Bell, I., Eisner, E., Allan, S., Cartner, S., Torous, J. B., Bucci, S., & Thomas, N. (2023) · Schizophrenia Bulletin · doi:10.1093/schbul/sbad127 · Cited by 66
Terp, M., Jørgensen, R., Laursen, B. S., Mainz, J., & Bjørnes, C. D. (2018) · JMIR Mental Health · doi:10.2196/10157 · Cited by 41
Understanding Mobile Health and Youth Mental Health: Scoping Review
Ding, X., Wuerth, K., Sakakibara, B., Schmidt, J., Parde, N., Holsti, L., & Barbic, S. (2023) · JMIR mHealth and uHealth · doi:10.2196/44951 · Cited by 30
“For instance, the intervention durations ranged from as short as 7 days [58] or 2 sessions [59] to as long as 24 weeks [60] or 24 months [61].”
Thunnissen, M. R., Rot, M. A. H., van den Hoofdakker, B. J., & Nauta, M. H. (2021) · Child Psychiatry & Human Development · doi:10.1007/s10578-021-01177-8 · Cited by 42
Accessibility and availability of smartphone apps for schizophrenia
Kwon, S., Firth, J., Joshi, D., & Torous, J. B. (2022) · Schizophrenia · doi:10.1038/s41537-022-00313-0 · Cited by 37
Moore, E., Williams, A., Bell, I., & Thomas, N. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.100299 · Cited by 32
Wang, W., Xu, W., Chander, A., Nepal, S., Buck, B., Pakhomov, S., Cohen, T., Ben-Zeev, D., & Campbell, A. (2023) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3610890 · Cited by 19
Wright, A. C., Browne, J., Skiest, H. E., Bhiku, K., Baker, J. T., & Cather, C. (2021) · Schizophrenia Research · doi:10.1016/j.schres.2021.04.010 · Cited by 23
Exploring engagement with the CBT-informed Actissist smartphone application for early psychosis
Eisner, E., Berry, N., Morris, R., Emsley, R., Haddock, G., Machin, M., Hassan, L., & Bucci, S. (2023) · Journal of Mental Health · doi:10.1080/09638237.2023.2182429 · Cited by 13
de Thurah, L., Kiekens, G., Sips, R., Teixeira, A. S., Kasanova, Z., & Myin‐Germeys, I. (2023) · Psychiatry Research · doi:10.1016/j.psychres.2023.115207 · Cited by 16
Michel, C., Lerch, S., Büetiger, J. R., Flückiger, R., Cavelti, M., Koenig, J., Kaess, M., & Kindler, J. (2022) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-022-02003-9 · Cited by 14
Comendador, L., Cebrià, A. I., Sanz, A., Pérez, V., & Palao, D. J. (2023) · Healthcare · doi:10.3390/healthcare11030432 · Cited by 14
Drexl, K., Ralisa, V., Amoussou, J. R., Wen, C. K. F., Urben, S., Plessen, K. J., & Glaus, J. (2025) · Journal of Medical Internet Research · doi:10.2196/65710 · Cited by 14
Privacy and security in digital therapeutics
Nurgalieva, L., & Doherty, G. J. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-90045-4.00002-2 · Cited by 9
Strauss, J., Zhang, J., Jarrett, M. L., Patterson, B. A., & Ameringen, M. V. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-822201-0.00006-x · Cited by 13
Sommer, I. E., & Hugdahl, K. (2019) · Schizophrenia Research · doi:10.1016/j.schres.2019.07.017 · Cited by 5
Self-citationBamigbayan, I. O., Prowse, R. J. L., Twells, L., & Shannahan, K. L. J. (2025) · doi:10.2196/preprints.79306
Comparative analysis of Mobile Crowdsensing and Ecological Momentary Assessment systems
Stach, M., Ramović, A., Reichert, M., & Pryss, R. C. (2025) · Ecological Informatics · doi:10.1016/j.ecoinf.2025.103315 · Cited by 1
Chinarev, V. А., & Малинина, Е. В. (2025) · Medical alphabet · doi:10.33667/2078-5631-2025-21-33-38
Bamigbayan, I. O., Twells, L., Shannahan, K. L. J., & Prowse, R. J. L. (2025) · JMIR Formative Research · doi:10.2196/79306
Fässler, L., Ersöz, B., Hahne, I. M., Koop, S., & Böge, K. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1727156
Metadata
- Authors
- Smelror, R. E., Bless, J. J., Hugdahl, K., & Agartz, I.
- Affiliations
- University of Oslo (NO)
- Haukeland University Hospital (NO)
- Corresponding author
- Smelror, Runar Elle
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-05-14
- Volume
- 3
- Issue
- 2
- Pages
- e13882
- ISSN
- 2561-326X (ISSN-L)
- PMC
- PMC6537505
- OpenAlex
- W2934045611
- Cited by
- 25
- References
- 54
- Citations by year
- 2026: 1 · 2025: 5 · 2023: 6 · 2022: 4 · 2021: 3 · 2020: 2 · 2019: 3
- FWCI
- 2.39
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Formative Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- Funders
- Norges Forskningsråd (213700, 250358)
- Helse Sør-Øst RHF (2016–118)
- FP7 Ideas: European Research Council (249516)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Schizophrenia research and treatment
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Life Sciences › Neuroscience › Cognitive Neuroscience › Hallucinations in medical conditions
- Keywords
- auditory verbal hallucinations, early-onset psychosis, adolescents, mobile phone app, ecological momentary assessment, feasibility, acceptability, user experience, symptom assessment, privacy concerns, localization, suicide risk, psychosis prediction, digital mental health tools
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.