Virtual human as a new diagnostic tool, a proof of concept study in the field of major depressive disorders
Open accessThe DOI matches its Crossref record; 38 of the 39 references it cites were found in a registry or an index.
Abstract
Embodied Conversational Agents (ECAs) are promising software to communicate with patients but no study has tested them in the diagnostic field of mental disorders. The aim of this study was 1) to test the performance of a diagnostic system for major depressive disorders (MDD), based on the identification by an ECA of specific symptoms (the MDD DSM 5 criteria) in outpatients; 2) to evaluate the acceptability of such an ECA. Patients completed two clinical interviews in a randomized order (ECA versus psychiatrist) and filled in the Acceptability E-scale (AES) to quantify the acceptability of the ECA. 179 outpatients were included in this study (mean age 46.5 ± 12.9 years, 57.5% females). Among the 35 patients diagnosed with MDD by the psychiatrist, 14 (40%) patients exhibited mild, 12 (34.3%) moderate and 9 (25.7%) severe depressive symptoms. Sensitivity increased across the severity level of depressive symptoms and reached 73% for patients with severe depressive symptoms, while specificity remained above 95% for all three severity levels. The acceptability of the ECA evaluated by the AES was very good (25.4). We demonstrate here the validity and acceptability of an ECA to diagnose major depressive disorders. ECAs are promising tools to conduct standardized and well-accepted clinical interviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Two studies comparing diagnostic performance of conversational agents and clinicians found acceptable sensitivity and specificity.”
From Conversational agents in healthcare: a systematic review (2018) · cites this paper as [22]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The investigated uses for these systems include triage, diagnostics, counseling, health promotion, and training of health care professionals [1,4,11-16].”
From Effectiveness of Conversational Agents (Virtual Assistants) in Health Care: Protocol for a Systematic Review (2020) · cites this paper as [1]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A crossover RCT evaluated the diagnostic performance for depression of an embodied conversational agent compared to a psychiatrist, finding a sensitivity of 49% and specificity of 93%.”
From Conversational agents in healthcare: a systematic review (2018) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“The spoken system for a major depressive disorder used embodied CAs, and showed sensitivity (49%) and specificity (93%) [46].”
From A systematic review on healthcare artificial intelligent conversational agents for chronic conditions (2022) · cites this paper as [46]
The cited source says what the claim attributes to it.
In the citing paper
“The largest proportion of them (12/31, 39%) addressed mental health issues [13,32-42]”
From The Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review (2020) · cites this paper as [38]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
(1998) · The Journal of Clinical Psychiatry · Cited by 19,139
(1998) · PubMed · Cited by 10,216
Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine
Zweig, M. H., & Campbell, G. (1993) · Clinical Chemistry · doi:10.1093/clinchem/39.4.561 · Cited by 6,569
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,026
Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients
Beck, A. T., Steer, R. A., Ball, R., & Ranieri, W. F. (1996) · Journal of Personality Assessment · doi:10.1207/s15327752jpa6703_13 · Cited by 5,965
(1997) · Choice Reviews Online · doi:10.5860/choice.34-3702 · Cited by 4,887
Cross-national epidemiology of DSM-IV major depressive episode
Bromet, E., Andrade, L. H., Hwang, I., Sampson, N. A., Alonso, J., de Girolamo, G., de Graaf, R., Demyttenaere, K., Hu, C., Iwata, N., et al. (2011) · BMC Medicine · doi:10.1186/1741-7015-9-90 · Cited by 2,221
Embodied Conversational Agents
(2000) · The MIT Press eBooks · doi:10.7551/mitpress/2697.001.0001 · Cited by 1,799
Selection and interpretation of diagnostic tests and procedures. Principles and applications.
Griner, P. F., Mayewski, R. J., Mushlin, A. I., & Greenland, P. (1981) · PubMed · Cited by 765
Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S., & Stone, M. (1994) · Proceedings of the 21st annual conference on Computer graphics and interactive techniques - SIGGRAPH '94 · doi:10.1145/192161.192272 · Cited by 665
Postpartum mood disorders: diagnosis and treatment guidelines.
Nonacs, R. M., & Cohen, L. S. (1998) · PubMed · Cited by 637
Selection and interpretation of diagnostic tests and procedures
Griner, P. F., Mayewski, R. J., Mushlin, A. I., & Greenland, P. (1981) · Annals of Internal Medicine · Cited by 595
The Precision Medicine Initiative: A New National Effort
Ashley, E. A. (2015) · JAMA · doi:10.1001/jama.2015.3595 · Cited by 560
Waller, R., & Gilbody, S. (2009) · Psychological Medicine · doi:10.1017/s0033291708004224 · Cited by 393
PROUDFOOT, J., GOLDBERG, D., MANN, A., EVERITT, B., MARKS, I., & GRAY, J. A. (2003) · Psychological Medicine · doi:10.1017/s0033291702007225 · Cited by 389
Bourque, P., & Beaudette, D. (1982) · Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement · doi:10.1037/h0081254 · Cited by 334
Distribution of Major Health Risks: Findings from the Global Burden of Disease Study
Rodgers, A., Ezzati, M., Vander Hoorn, S., Lopez, A. D., Lin, R.-B., Murray, C. J. L., & Group Comparative Risk Assessment Collaborating (2004) · PLoS Medicine · doi:10.1371/journal.pmed.0010027 · Cited by 280
Patient Education and Counseling
(2019) · The Physician Assistant Student’s Guide to the Clinical Year: Behavioral Health · doi:10.1891/9780826195388.0004 · Cited by 234
International Journal on Disability and Human Development
(2010) · International Journal on Disability and Human Development · doi:10.1515/ijdhd · Cited by 187
Hands-on Help: computer-aided Psychotherapy
Marks, I. M., Cavanagh, K., & Gega, L. (2007) · doi:10.25405/ncl.34083078 · Cited by 182
The American Psychiatric Association Practice Guidelines for the Psychiatric Evaluation of Adults
Silverman, J. J., Galanter, M., Jackson-Triche, M., Jacobs, D. G., Lomax, J. W., Riba, M. B., Tong, L. D., Watkins, K. E., Fochtmann, L. J., Rhoads, R. S., et al. (2015) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2015.1720501 · Cited by 178
Marks, I. M., Cavanagh, K., & Gega, L. (2007) · Psychology Press eBooks · doi:10.4324/9780203962152 · Cited by 167
The American Psychiatric Association Practice Guidelines for the Psychiatric Evaluation of Adults
(2015) · American Psychiatric Association Publishing eBooks · doi:10.1176/appi.books.9780890426760 · Cited by 166
Virtual reality for psychotherapy: Current reality and future possibilities.
Glanz, K., Rizzo, A., & Graap, K. M. (2003) · Psychotherapy · doi:10.1037/0033-3204.40.1-2.55 · Cited by 160
Tariman, J. D., Berry, D. L., Halpenny, B., Wolpin, S., & Schepp, K. (2011) · Applied Nursing Research · doi:10.1016/j.apnr.2009.04.003 · Cited by 157
What is Connected Health and why will it change your practice?
Caulfield, B. M., & Donnelly, S. C. (2013) · QJM · doi:10.1093/qjmed/hct114 · Cited by 140
Using computer agents to explain medical documents to patients with low health literacy
Bickmore, T. W., Pfeifer, L. M., & Paasche-Orlow, M. K. (2009) · Patient Education and Counseling · doi:10.1016/j.pec.2009.02.007 · Cited by 137
Psychiatric Interviewing: The Art of Understanding
Shea, S. C. (1988) · Medical Entomology and Zoology · Cited by 118
Grime, P. R. (2004) · Occupational Medicine · doi:10.1093/occmed/kqh077 · Cited by 104
Connected health and integrated care: Toward new models for chronic disease management
Chouvarda, I., Goulis, D. G., Lambrinoudaki, I. V., & Maglaveras, N. (2015) · Maturitas · doi:10.1016/j.maturitas.2015.03.015 · Cited by 98
SimCoach: an intelligent virtual human system for providing healthcare information and support
Rizzo, A., Lange, B., Buckwalter, J. G., Forbell, E., Kim, J., Sagae, K., Williams, J., Difede, J., Rothbaum, B. O., Reger, G., et al. (2011) · International Journal on Disability and Human Development · doi:10.1515/ijdhd.2011.046 · Cited by 94
Protocoles et échelles d'évaluation en psychiatrie et psychologie
Bouvard, M. A., & Cottraux, J. A. (2010) · CERN Document Server (European Organization for Nuclear Research) · Cited by 81
An Intelligent Virtual Human System for Providing Healthcare Information and Support
Rizzo Albert, A., Belinda, L., Buckwalter John, G., Eric, F., Julia, K., Kenji, S., Josh, W., Rothbaum Barbara, O., JoAnn, D., Greg, R., et al. (2011) · Studies in Health Technology and Informatics · doi:10.3233/978-1-60750-706-2-503 · Cited by 67
Computer Simulation and Virtual Reality in the Diagnosis and Treatment of Psychiatric Disorders
Gorrindo, T., & Groves, J. E. (2009) · Academic Psychiatry · doi:10.1176/appi.ap.33.5.413 · Cited by 36
Validation of the French version of the Acceptability E-scale (AES) for mental E-health systems
Micoulaud‐Franchi, J., Sauteraud, A., Olive, J., Sagaspe, P., Bioulac, S., & Philip, P. (2016) · Psychiatry Research · doi:10.1016/j.psychres.2016.01.043 · Cited by 29
Self-citationCould a virtual human be used to explore excessive daytime sleepiness in patients?
Philip, P., Bioulac, S., Sauteraud, A., Chaufton, C., & Olive, J. (2014) · Presence: Teleoperators and Virtual Environments · doi:10.1162/pres_a_00197 · Cited by 25
Self-citationAutomatic Behavior Analysis During a Clinical Interview with a Virtual Human.
Rizzo, A., Lucas, G., Gratch, J. S., Stratou, G., Morency, L., Chavez, K., Shilling, R. D., & Scherer, S. J. (2016) · PubMed · Cited by 19
Cited by
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388
“All the studies evaluating satisfaction with the system reported high overall satisfaction 24,28,30 but only one of these studies used a validated questionnaire (Acceptability e-Scale).(22) Some of the most frequent user experience issues were related to spoken language understanding or dialogue Table 1.”
InfluentialChatbots and Conversational Agents in Mental Health: A Review of the Psychiatric Landscape
Vaidyam, A. N., Wisniewski, H., Halamka, J. D., Kashavan, M. S., & Torous, J. B. (2019) · The Canadian Journal of Psychiatry · doi:10.1177/0706743719828977 · Cited by 1,048
The Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review
Milne-Ives, M., de Cock, C., Lim, E., Shehadeh, M. H., de Pennington, N., Mole, G., Normando, E., & Meinert, E. (2020) · Journal of Medical Internet Research · doi:10.2196/20346 · Cited by 626
“9 (82) 1 1 0 1 0 1 1 1 1 1 1 Philip (2017) [38]”
InfluentialLaestadius, L., Bishop, A., Gonzalez, M., Illenčík, D., & Campos-Castillo, C. (2024) · New Media & Society · doi:10.1177/14614448221142007 · Cited by 420
“Furthermore, embodied chatbots like Replika are preferred to text-only chatbots and may be seen as more trustworthy (Lisetti et al., 2015; Philip et al., 2017).”
BackgroundZhang, J., Oh, Y. J., Lange, P., Yu, Z., & Fukuoka, Y. (2020) · Journal of Medical Internet Research · doi:10.2196/22845 · Cited by 363
“Most previous chatbot research relied on either finite-state (ie, dialog consisting of a sequence of predetermined steps or states) or frame-based systems (ie, dialog is not predetermined but dependent on the content of the user’s input and the information that the system has to elicit) [34-36].”
Metadata
- Authors
- Philip, P., Micoulaud-Franchi, J.-A., Sagaspe, P., Sevin, E. D., Olive, J., Bioulac, S., & Sauteraud, A.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Corresponding author
- Philip, Pierre
- Venue
- Scientific Reports
- Published
- 2017-02-16
- Volume
- 7
- Issue
- 1
- Pages
- 42656-42656
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC5311989
- OpenAlex
- W2588443021
- Cited by
- 128
- References
- 39
- Citations by year
- 2026: 9 · 2025: 17 · 2024: 6 · 2023: 20 · 2022: 19 · 2021: 19 · 2020: 27 · 2019: 5 · 2018: 5 · 2017: 1
- FWCI
- 11.87
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Oskar-Bordeaux (Universite de Bordeaux) (submittedVersion, cc-by)
- HAL (submittedVersion, cc-by)
- Funders
- Equipex
- Agence Nationale de la Recherche (ANR‐10‐EQPX‐12‐01, ANR-10-EQPX-0012, ANR-10, ANR-10-EQPX)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- embodied conversational agents, major depressive disorder, automated diagnosis, DSM-5 diagnostic criteria, diagnostic accuracy, acceptability, clinical interview, virtual humans, outpatients, depression severity, psychiatric diagnosis
- MeSH terms
- Proof of Concept Study, Adult, Major Depressive Disorder, Female, Humans, Interview, Psychological, Male, Middle Aged, Patient Acceptance of Health Care, ROC Curve, Severity of Illness Index, Software, User-Computer Interface, Reproducibility of Results, Diagnostic and Statistical Manual of Mental Disorders
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.