Feasibility and Acceptability of Using a Mobile Phone App for Characterizing Auditory Verbal Hallucinations in Adolescents With Early-Onset Psychosis: Exploratory Study
Open accessThe DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Auditory verbal hallucinations (AVH) are the most frequent symptom in early-onset psychosis (EOP) and a risk factor for increased suicide attempts in adolescents. Increased knowledge of AVH characteristics can lead to better prediction of risk and precision of diagnosis and help identify individuals with AVH who need care. As 98% of Norwegian adolescents aged 12 to 16 years own a mobile phone, the use of mobile phone apps in symptom assessment and patient communication is a promising new tool. However, when introducing new technology to patients, their subjective experiences are crucial in identifying risks, further development, and potential integration into clinical care. OBJECTIVE: The objective was to explore the feasibility and acceptability of a newly developed mobile phone app in adolescents with EOP by examining compliance with the app and user experiences. Indication of validity was explored by examining associations between AVH dimensions, which were correlated and analyzed. METHODS: Three adolescents with EOP and active AVH were enrolled. Real-time AVH were logged on an iPod touch using the experience sampling method (ESM), for seven or more consecutive days. The app included five dimensions of AVH characteristics and was programmed with five daily notifications. Feasibility and acceptability were examined using the mean response rate of data sampling and by interviewing the participants. Validity was assessed by examining associations between the AVH dimensions using nonparametric correlation analysis and by visual inspection of temporal fluctuations of the AVH dimensions. RESULTS: One participant was excluded from the statistical analyses but completed the interview and was included in the examination of acceptability. The sampling period of the two participants was mean 12 (SD 6) days with overall completed sampling rate of 74% (SD 30%), indicating adequate to high compliance with the procedure. The user experiences from the interviews clustered into four categories: (1) increased awareness, (2) personal privacy, (3) design and procedure, and (4) usefulness and clinical care. One participant experienced more commenting voices during the sampling period, and all three participants had concerns regarding personal privacy when using electronic devices in symptom assessment. The AVH dimensions of content, control, and influence showed moderate to strong significant correlations with all dimensions (P<.001). Days of data sampling showed weak to moderate correlations with localization (P<.001) and influence (P=.03). Visual inspection indicated that the app was able to capture fluctuations within and across days for all AVH dimensions. CONCLUSIONS: This study demonstrates the value of including patients' experiences in the development and pilot-testing of new technology. Based on the small sample size, the use of mobile phones with ESM seems feasible for patients with EOP, but the acceptability of using apps should be considered. Further investigation with larger samples is warranted before definitive conclusions are made.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 How the app is used by the participants. In the first screen the participant chooses yes or no to whether they hear voices right now. If they answer “yes,” the five following screens with the visual analogue scales for the five auditory verbal hallucinations (AVH) dimensions appear. Each screen represents the following AVH characteristic: content (emotional), localization (perceptual), control (cognitive), intensity (intensity), and influence (influence). Previously presented on a poster at the International Congress on Schizophrenia Research, March 24-28, 2017; San Diego, CA, with the abstract by Bless et al [46]. iPhone frames adopted from a design by Todd Zaki Warfel.](https://figures.cochat.ai/paper-figures/daf48998b7af4ac6a1315444cb6bc877/feasibility-and-acceptability-of-using-a-mobile-phone-app-for-characterizing-figure-1.jpg)

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, the intervention durations ranged from as short as 7 days [58] or 2 sessions [59] to as long as 24 weeks [60] or 24 months [61].”
From Understanding Mobile Health and Youth Mental Health: Scoping Review (2023) · cites this paper as [58]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Positive and Negative Syndrome Scale (PANSS) for schizophrenia.
Kay, S. R., Fiszbein, A., & Opler, L. A. (1987) · Schizophrenia Bulletin · doi:10.1093/schbul/13.2.261 · Cited by 21,378
KAUFMAN, J., BIRMAHER, B., BRENT, D., RAO, U., FLYNN, C., MORECI, P., WILLIAMSON, D., & RYAN, N. (1997) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-199707000-00021 · Cited by 10,148
Ecological momentary assessment
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091415 · Cited by 6,454
A Children's Global Assessment Scale (CGAS)
Shaffer, D. (1983) · Archives of General Psychiatry · doi:10.1001/archpsyc.1983.01790100074010 · Cited by 3,716 · cited 3×
“AVH: auditory verbal hallucinations BAVQ-R: Beliefs About Voices Questionnaire-revised version CGAS: Children’s Global Assessment Scale EOP: early-onset psychosis ESM: experience sampling method IQ: intelligence quotient PANSS: Positive and Negative Syndrome Scale VAS: visual analog scale Edited by G Strudwick; submit…”
MethodInfluentialApplied Statistics for the Behavioral Sciences
Witz, K., Hinkle, D. E., Wiersma, W., & Jurs, S. G. (1990) · Journal of Educational Statistics · doi:10.2307/1164825 · Cited by 2,913
Self-reports: How the questions shape the answers.
Schwarz, N. (1999) · American Psychologist · doi:10.1037/0003-066x.54.2.93 · Cited by 2,846 · cited 1×
“Assessing AVH using traditional retrospective measures, such as the Positive and Negative Syndrome Scale (PANSS) [9] or the Scale for the Assessment of Positive Symptoms (SAPS) [10], may lead to loss of symptom information due to recall errors and state-dependent biases [11-13].”
MethodScale for Assessment of Positive Symptoms (SAPS) 1984
Andreasen, N. C. (1995) · Open Scholarship Institutional Repository (Washington University in St. Louis) · Cited by 2,423
Insel, T. R. (2010) · Nature · doi:10.1038/nature09552 · Cited by 1,783
Self-focused attention and negative affect: A meta-analysis.
Mor, N., & Winquist, J. (2002) · Psychological Bulletin · doi:10.1037/0033-2909.128.4.638 · Cited by 1,069 · cited 1×
“Increased self-awareness is associated with increased negative affect, particularly in individuals with existing high negative affectivity [51,52].”
BackgroundModelling cognition in emotional disorder: the S-REF model
Wells, A., & Matthews, G. (1996) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(96)00050-2 · Cited by 1,067
Digital Phenotyping: Technology for a New Science of Behavior
Insel, T. R. (2017) · JAMA · doi:10.1001/jama.2017.11295 · Cited by 833
The ecology of adolescent activity and experience
Csikszentmihalyi, M., Larson, R., & Prescott, S. (1977) · Journal of Youth and Adolescence · doi:10.1007/bf02138940 · Cited by 781 · cited 10×
“The experience sampling method (ESM) [16] was originally based on self-reports (such as diary entries) to monitor real-time experiences and symptom frequency daily.”
MethodInfluentialThe Science of Self-report: Implications for Research and Practice
(1999) · Psychology Press eBooks · doi:10.4324/9781410601261 · Cited by 764
Kelleher, I., Connor, D., Clarke, M. C., Devlin, N., Harley, M., & Cannon, M. (2012) · Psychological Medicine · doi:10.1017/s0033291711002960 · Cited by 686
Feasibility, Acceptability, and Preliminary Efficacy of a Smartphone Intervention for Schizophrenia
Ben-Zeev, D., Brenner, C. J., Begale, M., Duffecy, J., Mohr, D. C., & Mueser, K. T. (2014) · Schizophrenia Bulletin · doi:10.1093/schbul/sbu033 · Cited by 493
Can seeking happiness make people unhappy? Paradoxical effects of valuing happiness.
Mauss, I. B., Tamir, M., Anderson, C. L., & Savino, N. S. (2011) · Emotion · doi:10.1037/a0022010 · Cited by 401
The revised Beliefs About Voices Questionnaire (BAVQ–R)
Chadwick, P., Lees, S., & Birchwood, M. (2000) · British Journal of Psychiatry · doi:10.1192/bjp.177.3.229 · Cited by 327
Auditory Verbal Hallucinations in Persons With and Without a Need for Care
Johns, L. C., Kompus, K., Connell, M., Humpston, C., Lincoln, T. M., Longden, E., Preti, A., Alderson-Day, B., Badcock, J. C., Cella, M., et al. (2014) · Schizophrenia Bulletin · doi:10.1093/schbul/sbu005 · Cited by 311
Hollis, C. (2000) · American Journal of Psychiatry · doi:10.1176/appi.ajp.157.10.1652 · Cited by 298
Daalman, K., Boks, M. P. M., Diederen, K. M. J., de Weijer, A. D., Blom, J. D., Kahn, R. S., & Sommer, I. E. C. (2011) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.09m05797yel · Cited by 293
Experience sampling research in individuals with mental illness: reflections and guidance
Palmier-Claus, J. E., Myin-Germeys, I., Barkus, E., Bentley, L., Udachina, A., Delespaul, P. A. E. G., Lewis, S. W., & Dunn, G. (2011) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2010.01596.x · Cited by 287 · cited 4×
“5 hours), in accordance with recommendations from Palmier-Claus and colleagues [48].”
BackgroundResultInfluentialSmartphone Apps for Schizophrenia: A Systematic Review
Firth, J., & Torous, J. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4930 · Cited by 268 · cited 2×
“There are at least 11 mobile phone apps that have been developed for schizophrenia [20,27-33].”
BackgroundPsychotic Symptoms and Population Risk for Suicide Attempt: A Prospective Cohort Study
Kelleher, I., Corcoran, P., Keeley, H., Wigman, J. T. W., Devlin, N., Ramsay, H., Wasserman, C., Carli, V., Sarchiapone, M., Hoven, C., et al. (2013) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2013.140 · Cited by 252
Determinants of occurrence and recovery from hallucinations in daily life
Delespaul, P., deVries, M., & van Os, J. (2002) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s001270200000 · Cited by 247
Cognitive processes in auditory hallucinations: attributional biases and metacognition
BAKER, C. A., & MORRISON, A. P. (1998) · Psychological Medicine · doi:10.1017/s0033291798007314 · Cited by 199
Feasibility and Validity of Computerized Ecological Momentary Assessment in Schizophrenia
Granholm, E., Loh, C., & Swendsen, J. (2007) · Schizophrenia Bulletin · doi:10.1093/schbul/sbm113 · Cited by 191
Palmier-Claus, J. E., Ainsworth, J., Machin, M., Barrowclough, C., Dunn, G., Barkus, E., Rogers, A., Wykes, T., Kapur, S., Buchan, I., et al. (2012) · BMC Psychiatry · doi:10.1186/1471-244x-12-172 · Cited by 189
Ben-Zeev, D., Brian, R., Wang, R., Wang, W., Campbell, A. T., Aung, M. S. H., Merrill, M., Tseng, V. W. S., Choudhury, T., Hauser, M., et al. (2017) · Psychiatric Rehabilitation Journal · doi:10.1037/prj0000243 · Cited by 184
Kimhy, D., Delespaul, P., Corcoran, C. M., Ahn, H., Yale, S. A., & Malaspina, D. (2005) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2005.09.007 · Cited by 180
Auditory hallucinations across the lifespan: a systematic review and meta-analysis
Maijer, K., Begemann, M. J. H., Palmen, S. J. M. C., Leucht, S., & Sommer, I. E. C. (2018) · Psychological Medicine · doi:10.1017/s0033291717002367 · Cited by 180
Experience Sampling: Promises and Pitfalls, Strength and Weaknesses
Napa Scollon, C., Prieto, C.-K., & Diener, E. (2009) · Social Indicators Research Series · doi:10.1007/978-90-481-2354-4_8 · Cited by 175
Palmier-Claus, J. E., Rogers, A., Ainsworth, J., Machin, M., Barrowclough, C., Laverty, L., Barkus, E., Kapur, S., Wykes, T., & Lewis, S. W. (2013) · BMC Psychiatry · doi:10.1186/1471-244x-13-34 · Cited by 159
Stentebjerg-Olesen, M., Pagsberg, A. K., Fink-Jensen, A., Correll, C. U., & Jeppesen, P. (2016) · Journal of Child and Adolescent Psychopharmacology · doi:10.1089/cap.2015.0097 · Cited by 147
Ben‐Zeev, D., McHugo, G. J., Xie, H., Dobbins, K., & Young, M. A. (2012) · Schizophrenia Bulletin · doi:10.1093/schbul/sbr171 · Cited by 123
Why does schizophrenia develop at late adolescence?
Harrop, C. E., & Trower, P. (2001) · Clinical Psychology Review · doi:10.1016/s0272-7358(99)00047-1 · Cited by 114
Lal, S., Dell'Elce, J., Tucci, N., Fuhrer, R., Tamblyn, R., & Malla, A. (2015) · JMIR Mental Health · doi:10.2196/mental.4400 · Cited by 108 · cited 2×
“This is in accordance with a study showing that one-third of a sample of young adults with psychotic disorders experienced discomfort in online settings [37].”
BackgroundResultNiendam, T. A., Tully, L. M., Iosif, A., Kumar, D., Nye, K. E., Denton, J. C., Zakskorn, L. N., Fedechko, T. L., & Pierce, K. M. (2017) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2017.10.017 · Cited by 107
Schlosser, D., Campellone, T., Kim, D., Truong, B., Vergani, S., Ward, C., & Vinogradov, S. (2016) · JMIR Research Protocols · doi:10.2196/resprot.5450 · Cited by 103 · cited 1×
“There are at least 11 mobile phone apps that have been developed for schizophrenia [20,27-33].”
BackgroundMaijer, K., Hayward, M., Fernyhough, C., Calkins, M. E., Debbané, M., Jardri, R., Kelleher, I., Raballo, A., Rammou, A., Scott, J. G., et al. (2018) · Schizophrenia Bulletin · doi:10.1093/schbul/sby119 · Cited by 97
Prevalence of auditory verbal hallucinations in a general population: A group comparison study
Kråkvik, B., Larøi, F., Kalhovde, A. M., Hugdahl, K., Kompus, K., Salvesen, Ø., Stiles, T. C., & Vedul‐Kjelsås, E. (2015) · Scandinavian Journal of Psychology · doi:10.1111/sjop.12236 · Cited by 92
Self-citationSchmidt, M. H., Blanz, B., Dippe, A., Koppe, T., & Lay, B. (1995) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/bf02190735 · Cited by 78
Mor, N., Doane, L. D., Adam, E. K., Mineka, S., Zinbarg, R. E., Griffith, J. W., Craske, M. G., Waters, A., & Nazarian, M. (2010) · Cognition and Emotion · doi:10.1080/02699930802499715 · Cited by 78 · cited 1×
“Increased self-awareness is associated with increased negative affect, particularly in individuals with existing high negative affectivity [51,52].”
BackgroundEffects of Intensive Mobile Happiness Reporting in Daily Life
Conner, T. S., & Reid, K. A. (2012) · Social Psychological and Personality Science · doi:10.1177/1948550611419677 · Cited by 76
Kumar, D., Tully, L. M., Iosif, A., Zakskorn, L. N., Nye, K. E., Zia, A., & Niendam, T. A. (2018) · JMIR Mental Health · doi:10.2196/mental.8551 · Cited by 76
Kelly, J., Gooding, P., Pratt, D., Ainsworth, J., Welford, M., & Tarrier, N. (2012) · Journal of Mental Health · doi:10.3109/09638237.2011.638001 · Cited by 73 · cited 1×
“has been suggested as a means to individualize the number of notifications and variation in questions, and thus increase compliance and engagement with ESM studies [36,57].”
BackgroundBucci, S., Barrowclough, C., Ainsworth, J., Morris, R., Berry, K., Machin, M., Emsley, R., Lewis, S., Edge, D., Buchan, I., et al. (2015) · Trials · doi:10.1186/s13063-015-0943-3 · Cited by 55
Kimhy, D., Wall, M. M., Hansen, M. C., Vakhrusheva, J., Choi, C. J., Delespaul, P., Tarrier, N., Sloan, R. P., & Malaspina, D. (2016) · Schizophrenia Bulletin · doi:10.1093/schbul/sbw219 · Cited by 54
Kim, S., Lee, G., Yu, H., Jung, E., Lee, J., Kim, S., Kim, J., Shin, I., & Yoon, J. (2018) · Early Intervention in Psychiatry · doi:10.1111/eip.12418 · Cited by 44
Characterizing Smartphone Engagement for Schizophrenia: Results of a Naturalist Mobile Health Study
Torous, J., Staples, P., Slaters, L., Adams, J., Sandoval, L., Onnela, J., & Keshavan, M. (2017) · Clinical Schizophrenia & Related Psychoses · doi:10.3371/csrp.jtps.071317 · Cited by 44 · cited 1×
“There are at least 11 mobile phone apps that have been developed for schizophrenia [20,27-33].”
BackgroundTerp, M., Jørgensen, R., Laursen, B. S., Mainz, J., & Bjørnes, C. D. (2018) · JMIR Mental Health · doi:10.2196/10157 · Cited by 41 · cited 1×
“There are at least 11 mobile phone apps that have been developed for schizophrenia [20,27-33].”
The Beliefs about Voices Questionnaire – Revised: A factor structure from 450 participants
Strauss, C., Hugdahl, K., Waters, F., Hayward, M., Bless, J. J., Falkenberg, L. E., Kråkvik, B., Asbjørnsen, A. E., Johnsen, E., Sinkeviciute, I., et al. (2017) · Psychiatry Research · doi:10.1016/j.psychres.2017.09.089 · Cited by 39
Self-citationPalmier‐Claus, J. E., Dunn, G., Taylor, H. E., Morrison, A. P., & Lewis, S. W. (2012) · British Journal of Clinical Psychology · doi:10.1111/j.2044-8260.2012.02043.x · Cited by 37
Thomas, N., Bless, J. J., Alderson‐Day, B., Bell, I., Cella, M., Craig, T., Delespaul, P., Hugdahl, K., Laloyaux, J., Larøi, F., et al. (2018) · Schizophrenia Bulletin · doi:10.1093/schbul/sby103 · Cited by 29
Self-citationBless, J. J., Smelror, R. E., Agartz, I., & Hugdahl, K. (2017) · Schizophrenia Bulletin · doi:10.1093/schbul/sbx023.108 · Cited by 3
Self-citation
Research works printed but not linked: 3
- 2018, Barn og medier-undersøkelsen 2018: 9-18-åringer om medievaner og opplevelser
- 2013, Nasjonal faglig retningslinje for utredning, behandling og oppfølging av personer med psykoselidelser
- 1999, Wasi Manual; Wechsler Abbreviated Scale Of Intelligence Manual
Cited by
Security and Privacy of mHealth Applications: A Scoping Review
Nurgalieva, L., O'Callaghan, D., & Doherty, G. (2020) · IEEE Access · doi:10.1109/access.2020.2999934 · Cited by 132
“Studies covered a wide range of mental health conditions , which included stress [3], depression [55], [60], [71], sleep disorders [88], smoking cessation [55], [98], psychosis [4], anxiety [53], and cognitive decline such as dementia [58], [65].”
BackgroundEisele, G. V., Vachon, H., Lafit, G., Tuyaerts, D., Houben, M., Kuppens, P., Myin‐Germeys, I., & Viechtbauer, W. (2022) · Psychological Assessment · doi:10.1037/pas0001177 · Cited by 95
mHealth for schizophrenia spectrum disorders management: A systematic review
Chivilgina, O., Wangmo, T., Elger, B. S., Heinrich, T. W., & Jotterand, F. (2020) · International Journal of Social Psychiatry · doi:10.1177/0020764020933287 · Cited by 60
Camacho, E., Levin, L., & Torous, J. B. (2019) · Journal of Medical Internet Research · doi:10.2196/16393 · Cited by 58
Bell, I., Eisner, E., Allan, S., Cartner, S., Torous, J. B., Bucci, S., & Thomas, N. (2023) · Schizophrenia Bulletin · doi:10.1093/schbul/sbad127 · Cited by 66
Metadata
- Authors
- Smelror, R. E., Bless, J. J., Hugdahl, K., & Agartz, I.
- Affiliations
- University of Oslo (NO)
- Haukeland University Hospital (NO)
- Corresponding author
- Smelror, Runar Elle
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-05-14
- Volume
- 3
- Issue
- 2
- Pages
- e13882
- ISSN
- 2561-326X (ISSN-L)
- PMC
- PMC6537505
- OpenAlex
- W2934045611
- Cited by
- 25
- References
- 54
- Citations by year
- 2026: 1 · 2025: 5 · 2023: 6 · 2022: 4 · 2021: 3 · 2020: 2 · 2019: 3
- FWCI
- 2.39
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Formative Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- Funders
- Norges Forskningsråd (213700, 250358)
- Helse Sør-Øst RHF (2016–118)
- FP7 Ideas: European Research Council (249516)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Schizophrenia research and treatment
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Life Sciences › Neuroscience › Cognitive Neuroscience › Hallucinations in medical conditions
- Keywords
- auditory verbal hallucinations, early-onset psychosis, adolescents, mobile phone app, ecological momentary assessment, feasibility, acceptability, user experience, symptom assessment, privacy concerns, localization, suicide risk, psychosis prediction, digital mental health tools
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.