Smartphone Application for the Analysis of Prosodic Features in Running Speech with a Focus on Bipolar Disorders: System Performance Evaluation and Case Study
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
Bipolar disorder is one of the most common mood disorders characterized by large and invalidating mood swings. Several projects focus on the development of decision support systems that monitor and advise patients, as well as clinicians. Voice monitoring and speech signal analysis can be exploited to reach this goal. In this study, an Android application was designed for analyzing running speech using a smartphone device. The application can record audio samples and estimate speech fundamental frequency, F0, and its changes. F0-related features are estimated locally on the smartphone, with some advantages with respect to remote processing approaches in terms of privacy protection and reduced upload costs. The raw features can be sent to a central server and further processed. The quality of the audio recordings, algorithm reliability and performance of the overall system were evaluated in terms of voiced segment detection and features estimation. The results demonstrate that mean F0 from each voiced segment can be reliably estimated, thus describing prosodic features across the speech sample. Instead, features related to F0 variability within each voiced segment performed poorly. A case study performed on a bipolar patient is presented.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Recent advances in speech processing (112) also suggest that mobile platforms could be employed to identify and monitor speech characteristics that may be altered by DBS.”
From Could Wearable and Mobile Technology Improve the Management of Essential Tremor? (2018) · cites this paper as [112]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Design and Implementation of FFTW3
Frigo, M., & Johnson, S. G. (2005) · Proceedings of the IEEE · doi:10.1109/jproc.2004.840301 · Cited by 5,086 · cited 1×
“For this purpose, two libraries were linked to the application for fast Fourier transform [30] and managing sound files in wav format [31].”
MethodPrevalence and Correlates of Bipolar Spectrum Disorder in the World Mental Health Survey Initiative
Merikangas, K. R., Jin, R., He, J.-P., Kessler, R. C., Lee, S., Sampson, N. A., Viana, M. C., Andrade, L. H., Hu, C., Karam, E. G., et al. (2011) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.12 · Cited by 2,627 · cited 1×
“Bipolar disorders are defined as the periodical oscillations of pathological moods, eventually alternating between euthymic or normal states [4,5].”
BackgroundA review of depression and suicide risk assessment using speech analysis
Cummins, N., Scherer, S., Krajewski, J., Schnieder, S., Epps, J., & Quatieri, T. F. (2015) · Speech Communication · doi:10.1016/j.specom.2015.03.004 · Cited by 1,003 · cited 1×
“In fact, speech production is a complex phenomenon that is influenced by the autonomic and somatic nervous systems, through the modulation of breathing activity, vocal muscles tension, salivation and mucus secretion [11–13].”
BackgroundNeuropsychology of bipolar disorder: a review
Quraishi, S., & Frangou, S. (2002) · Journal of Affective Disorders · doi:10.1016/s0165-0327(02)00091-5 · Cited by 571 · cited 1×
“Bipolar disorders are defined as the periodical oscillations of pathological moods, eventually alternating between euthymic or normal states [4,5].”
BackgroundExpression of emotion in voice and music
Scherer, K. R. (1995) · Journal of Voice · doi:10.1016/s0892-1997(05)80231-0 · Cited by 470 · cited 1×
“In fact, speech production is a complex phenomenon that is influenced by the autonomic and somatic nervous systems, through the modulation of breathing activity, vocal muscles tension, salivation and mucus secretion [11–13].”
BackgroundA sawtooth waveform inspired pitch estimator for speech and music
Camacho, A., & Harris, J. G. (2008) · The Journal of the Acoustical Society of America · doi:10.1121/1.2951592 · Cited by 426 · cited 2×
“This step is accomplished using a variation of the sawtooth waveform inspired pitch estimator algorithm (SWIPE’) [24], which estimates F0 at given time instants using a spectral matching approach.”
MethodBauer, M. S. (1991) · Archives of General Psychiatry · doi:10.1001/archpsyc.1991.01810330031005 · Cited by 407 · cited 1×
“Our results are in line with clinical models of bipolar disorders that consider depressive and maniac symptoms at the extremities of a continuous scale, while euthymia is in the middle [34].”
ResultDetection of Clinical Depression in Adolescents’ Speech During Family Interactions
Low, L.-S. A., Maddage, M. C., Lech, M., Sheeber, L. B., & Allen, N. B. (2011) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2010.2091640 · Cited by 256 · cited 1×
“Voice-related features were found to have a predictive value for developing depressive symptoms in adolescents [14].”
BackgroundJava Native Interface: Programmer's Guide and Reference
Sheng, L. (1999) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 236
Voice acoustical measurement of the severity of major depression
Cannizzaro, M., Harel, B., Reilly, N., Chappell, P., & Snyder, P. J. (2004) · Brain and Cognition · doi:10.1016/j.bandc.2004.05.003 · Cited by 233 · cited 1×
“A relationship between speech-related features describing prosodic changes and depression severity was observed [15].”
BackgroundMoore, E., Clements, M. A., Peifer, J. W., & Weisser, L. (2008) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2007.900562 · Cited by 220 · cited 1×
“Glottal, prosodic and vocal tract features could discriminate depressed subjects with respect to controls [16].”
BackgroundOzdas, A., Shiavi, R. G., Silverman, S. E., Silverman, M. K., & Wilkes, D. M. (2004) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2004.827544 · Cited by 204 · cited 2×
“This behavior was observed elsewhere [17] and may be related to agitated depression.”
BackgroundMobile Voice Health Monitoring Using a Wearable Accelerometer Sensor and a Smartphone Platform
Mehta, D. D., Zañartu, M., Feng, S. W., Cheyne, H. A., & Hillman, R. E. (2012) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2012.2207896 · Cited by 202 · cited 1×
“A smartphone device and an accelerometer were the key elements of the device shown in [21].”
MethodNeuroprogression and Cognitive Functioning in Bipolar Disorder: A Systematic Review
Cardoso, T., Bauer, I. E., Meyer, T. D., Kapczinski, F. P., & Soares, J. C. (2015) · Current Psychiatry Reports · doi:10.1007/s11920-015-0605-x · Cited by 175 · cited 1×
“Mood disorders, especially bipolar disorders, have a great impact on people’s lives, and currently, large efforts are being made both to determine their causes and to improve therapy [1–3].”
BackgroundLanata, A., Valenza, G., Nardelli, M., Gentili, C., & Scilingo, E. P. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2014.2360711 · Cited by 164 · cited 1×
“The system allows for compiling a mood agenda, answering daily self-administered questionnaires, storing data coming from a physiological signal monitoring system and acquiring and analyzing speech data [25].”
BackgroundSelf-citationEcologically valid long-term mood monitoring of individuals with bipolar disorder using speech
Karam, Z. N., Provost, E. M., Singh, S., Montgomery, J., Archer, C., Harrington, G., & Mcinnis, M. G. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6854525 · Cited by 136 · cited 1×
“In [23], a mobile phone was used to collect speech data both using structured and unstructured tasks, with the aim of classifying the mood state in bipolar disorders.”
MethodAdaptation of a Pocket PC for Use as a Wearable Voice Dosimeter
Popolo, P. S., Švec, J. G., & Titze, I. R. (2005) · Journal of Speech Language and Hearing Research · doi:10.1044/1092-4388(2005/054 · Cited by 107
Tsanas, A., Zañartu, M., Little, M. A., Fox, C., Ramig, L. O., & Clifford, G. D. (2014) · The Journal of the Acoustical Society of America · doi:10.1121/1.4870484 · Cited by 69 · cited 1×
“The SWIPE’ algorithm was recently shown to have very good performance as compared to established algorithms [26].”
MethodSpeech analysis for mood state characterization in bipolar patients
Vanello, N., Guidi, A., Gentili, C., Werner, S., Bertschy, G., Valenza, G., Lanata, A., & Scilingo, E. P. (2012) · 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/embc.2012.6346375 · Cited by 67 · cited 2×
“In the first step, voiced segments are detected using both signal intensity and zero crossing rate as in [18].”
BackgroundMethodSelf-citationPSYCHE: Personalised monitoring systems for care in mental health
Paradiso, R., Bianchi, A. M., Lau, K. T., & Scilingo, E. P. (2010) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2010.5627469 · Cited by 66 · cited 2×
“At the European level, several projects have been funded to tackle this issue, offering different approaches, such as monitoring physiological parameters and using behavioral questionnaires [3,8–10].”
BackgroundMethodSelf-citationDesign Issues for a Portable Vocal Analyzer
Carullo, A., Vallan, A., & Astolfi, A. (2013) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2012.2236724 · Cited by 48 · cited 1×
“In [20], the performance of an accelerometer and a contact microphone were evaluated for the design of a voice analyzer.”
MethodAutomatic analysis of speech F0 contour for the characterization of mood changes in bipolar patients
Guidi, A., Vanello, N., Bertschy, G., Gentili, C., Landini, L., & Scilingo, E. P. (2015) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2014.10.011 · Cited by 43 · cited 3×
“This approach was also tested in [32] and compared to the segmentation that can be obtained using an electroglottographic signal.”
MethodInfluentialSelf-citationSchärer, L., Hartweg, V, Valerius, G., Graf, M., Hoern, M., Biedermann, C., Walser, S., Boensch, A., Dittmann, S., Forsthoff, A., et al. (2002) · Bipolar Disorders · doi:10.1034/j.1399-5618.4.s1.51.x · Cited by 38 · cited 1×
“The possibility of monitoring a subject’s status and predicting depressive symptoms would allow one to modulate interventions, thereby minimizing the risks for the patients [6,7].”
BackgroundToward stratified treatments for bipolar disorders
Hasler, G., & Wolf, A. (2015) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2014.12.006 · Cited by 30 · cited 1×
“Mood disorders, especially bipolar disorders, have a great impact on people’s lives, and currently, large efforts are being made both to determine their causes and to improve therapy [1–3].”
BackgroundStrategies for Monitoring Outcomes in Patients With Bipolar Disorder
Ketter, T. A. (2010) · The Primary Care Companion to The Journal of Clinical Psychiatry · doi:10.4088/pcc.9064su1c.02 · Cited by 30 · cited 1×
“The possibility of monitoring a subject’s status and predicting depressive symptoms would allow one to modulate interventions, thereby minimizing the risks for the patients [6,7].”
BackgroundMayora, O., Bert Arnrich, A., Bardram, J. E., Dräger, C., Finke, A., Frost, M. M., Giordano, S., Gravenhorst, F., Grünerbl, A., Häring, C., et al. (2013) · Proceedings of the ICTs for improving Patients Rehabilitation Research Techniques · doi:10.4108/icst.pervasivehealth.2013.252123 · Cited by 29 · cited 1×
“At the European level, several projects have been funded to tackle this issue, offering different approaches, such as monitoring physiological parameters and using behavioral questionnaires [3,8–10].”
BackgroundAutonomic function, voice, and mood states
Park, C., Lee, S., Park, H., Baik, Y.-S., Park, Y.-B., & Park, Y.-J. (2010) · Clinical Autonomic Research · doi:10.1007/s10286-010-0095-1 · Cited by 24 · cited 1×
“In fact, speech production is a complex phenomenon that is influenced by the autonomic and somatic nervous systems, through the modulation of breathing activity, vocal muscles tension, salivation and mucus secretion [11–13].”
BackgroundOnline Predictive Tools for Intervention in Mental Illness: The OPTIMI Project
Cristina, B., Inés, M., Baños, R., & Azucena, G.-P. (2011) · Studies in Health Technology and Informatics · doi:10.3233/978-1-60750-706-2-86 · Cited by 20 · cited 1×
“At the European level, several projects have been funded to tackle this issue, offering different approaches, such as monitoring physiological parameters and using behavioral questionnaires [3,8–10].”
BackgroundPersonal Health Systems for Mental Health: The European Projects
Giuseppe, R., Rosa, B., Botella, C., Gaggioli, A., & Wiederhold, B. K. (2011) · Studies in health technology and informatics · doi:10.3233/978-1-60750-706-2-496 · Cited by 17 · cited 2×
“At the European level, several projects have been funded to tackle this issue, offering different approaches, such as monitoring physiological parameters and using behavioral questionnaires [3,8–10].”
BackgroundDetection Of Pitched/Unpitched Sound Using Pitch Strength Clustering.
Camacho, A. (2008) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1418168 · Cited by 12
New computer aided device for real time analysis of speech of people with Parkinson’s disease
Vásquez-Correa, J. C., Orozco‐Arroyave, J. R., Arias-Londoño, J. D., Vargas‐Bonilla, J. F., & Nöth, E. (2022) · Revista Facultad de Ingeniería Universidad de Antioquia · doi:10.17533/udea.redin.327870 · Cited by 10
Evaluation of a Pitch Estimation Algorithm for Speech Emotion Recognition
Vanello, N., Martini, N., Milanesi, M., Keiser, H. N., Calisti, M., Bocchi, L., Manfredi, C., & Landini, L. (2009) · CINECA IRIS Institutial research information system (University of Pisa) · Cited by 7
Self-citationMayora, O., Bert Arnrich, A., Bardram, J. E., Dräger, C., Finke, A., Frost, M. M., Giordano, S., Gravenhorst, F., Grünerbl, A., Häring, C., et al. (2013) · Proceedings of the ICTs for improving Patients Rehabilitation Research Techniques · doi:10.4108/pervasivehealth.2013.252123 · Cited by 4
Cited by
Dogan, E., Sander, C., Wagner, X., Hegerl, U., & Kohls, E. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7006 · Cited by 177
“Processing data locally on the smartphone has been suggested as one possible solution [47].”
BackgroundSeppälä, J., De Vita, I., Jämsä, T., Miettunen, J., Isohanni, M., Rubinstein, K., Feldman, Y., Grasa, E., Corripio, I., Berdun, J., et al. (2019) · JMIR Mental Health · doi:10.2196/mental.9819 · Cited by 148
Rusz, J., Hlavnička, J., Tykalová, T., Novotný, M., Dušek, P., Šonka, K., & Růžička, E. (2018) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2018.2851787 · Cited by 131
“phones are useful for detecting vocal changes [29], [41]–[44].”
BackgroundObjective assessment of depressive symptoms with machine learning and wearable sensors data
Ghandeharioun, A., Fedor, S., Sangermano, L., Ionescu, D., Alpert, J., Dale, C., Sontag, D., & Picard, R. (2017) · 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii.2017.8273620 · Cited by 116
“Most previous work has addressed a classification problem, usually binary, within this area [19], [20].”
BackgroundSmartphone-based objective monitoring in bipolar disorder: status and considerations
Faurholt-Jepsen, M., Bauer, M., & Kessing, L. V. (2018) · International Journal of Bipolar Disorders · doi:10.1186/s40345-017-0110-8 · Cited by 98
Metadata
- Authors
- Guidi, A., Salvi, S., Ottaviano, M., Gentili, C., Bertschy, G., De Rossi, D., Scilingo, E., & Vanello, N.
- Affiliations
- University of Pisa (IT)
- Universidad Politécnica de Madrid (ES)
- Inserm (FR)
- Dipartimento di Ingegneria dell’Informazione, University of Pisa, Via G. Caruso 16, Pisa 56122, Italy
- Corresponding author
- Guidi, Andrea
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2015-11-06
- Volume
- 15
- Issue
- 11
- Pages
- 28070-28087
- ISSN
- 1424-8220 (ISSN-L)
- PMC
- PMC4701269
- OpenAlex
- W2153933853
- Cited by
- 46
- References
- 33
- Citations by year
- 2025: 4 · 2024: 2 · 2023: 3 · 2022: 2 · 2021: 11 · 2020: 5 · 2019: 5 · 2018: 5 · 2017: 6 · 2016: 3
- FWCI
- 7.05
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Sensors (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UPM Digital Archive (Technical University of Madrid) (acceptedVersion, cc-by)
- HAL (submittedVersion, cc-by)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Hispana (submittedVersion, cc-by)
- Padua Research Archive (University of Padova) (submittedVersion, cc-by)
- Funders
- European Commission (247777)
- FP7 Information and Communication Technologies
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- bipolar disorder, prosodic features, fundamental frequency estimation, mobile application, voice monitoring, mood tracking, decision support system, Android application
- MeSH terms
- Smartphone, Adult, Bipolar Disorder, Female, Humans, Male, Monitoring, Physiologic, Pilot Projects, Speech, Voice, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.