The Diagnostic Validity and Reliability of an Internet-Based Clinical Assessment Program for Mental Disorders
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Internet-based assessment has the potential to assist with the diagnosis of mental health disorders and overcome the barriers associated with traditional services (eg, cost, stigma, distance). Further to existing online screening programs available, there is an opportunity to deliver more comprehensive and accurate diagnostic tools to supplement the assessment and treatment of mental health disorders. OBJECTIVE: The aim was to evaluate the diagnostic criterion validity and test-retest reliability of the electronic Psychological Assessment System (e-PASS), an online, self-report, multidisorder, clinical assessment and referral system. METHODS: Participants were 616 adults residing in Australia, recruited online, and representing prospective e-PASS users. Following e-PASS completion, 158 participants underwent a telephone-administered structured clinical interview and 39 participants repeated the e-PASS within 25 days of initial completion. RESULTS: With structured clinical interview results serving as the gold standard, diagnostic agreement with the e-PASS varied considerably from fair (eg, generalized anxiety disorder: κ=.37) to strong (eg, panic disorder: κ=.62). Although the e-PASS' sensitivity also varied (0.43-0.86) the specificity was generally high (0.68-1.00). The e-PASS sensitivity generally improved when reducing the e-PASS threshold to a subclinical result. Test-retest reliability ranged from moderate (eg, specific phobia: κ=.54) to substantial (eg, bulimia nervosa: κ=.87). CONCLUSIONS: The e-PASS produces reliable diagnostic results and performs generally well in excluding mental disorders, although at the expense of sensitivity. For screening purposes, the e-PASS subclinical result generally appears better than a clinical result as a diagnostic indicator. Further development and evaluation is needed to support the use of online diagnostic assessment programs for mental disorders. TRIAL REGISTRATION: Australian and New Zealand Clinical Trials Registry ACTRN121611000704998; http://www.anzctr.org.au/trial_view.aspx?ID=336143 (Archived by WebCite at http://www.webcitation.org/618r3wvOG).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pilkonis, P. A., Choi, S. W., Reise, S. P., Stover, A. M., Riley, W. T., Cella, D., & PROMIS Cooperative Group (2011) · Assessment · doi:10.1177/1073191111411667 · Cited by 2,132
Brown, T. A., Di Nardo, P. A., Lehman, C. L., & Campbell, L. A. (2001) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.110.1.49 · Cited by 887 · cited 1×
“Participants who endorsed MINI-Plus questions indicating some level of anxiety symptoms were also presented the anxiety disorder modules of the Anxiety Disorders Interview Schedule for DSM-IV-TR (ADIS-IV), a “gold standard” semistructured interview with demonstrated reliability [21,22].”
Defining Internet-Supported Therapeutic Interventions
Barak, A., Klein, B., & Proudfoot, J. G. (2009) · Annals of Behavioral Medicine · doi:10.1007/s12160-009-9130-7 · Cited by 731 · cited 1×
“The Internet offers various benefits to assist the assessment of mental disorders [6,7].”
Self-citationWittchen, H.-U., Lachner, G., Wunderlich, U., & Pfister, H. (1998) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s001270050095 · Cited by 668 · cited 1×
“In general, the e-PASS’ test-retest reliability measures are comparable to those of a computer-assisted administration of the CIDI [31] and a clinician-administered MINI [32].”
Comparability of telephone and face-to-face interviews in assessing axis I and II disorders
Rohde, P., Lewinsohn, P. M., & Seeley, J. R. (1997) · American Journal of Psychiatry · doi:10.1176/ajp.154.11.1593 · Cited by 522 · cited 1×
“The use of telephone interviewing for assessing mental health disorders has support in the literature [17-19].”
BRUGHA, T. S., BEBBINGTON, P. E., & JENKINS, R. (1999) · Psychological Medicine · doi:10.1017/s0033291799008880 · Cited by 191 · cited 1×
“For example, clinicians typically favor unstructured interviewing despite being prone to bias and error [4], whereas the more reliable structured interviewing format is often overlooked for being cumbersome and costly to administer in everyday practice [5].”
A Brief Web-Based Screening Questionnaire for Common Mental Disorders: Development and Validation
Donker, T., Straten, A. V., Marks, I., & Cuijpers, P. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1134 · Cited by 166
What About Telepsychiatry?: A Systematic Review
García-Lizana, F., & Muñoz-Mayorga, I. (2010) · The Primary Care Companion to The Journal of Clinical Psychiatry · doi:10.4088/pcc.09m00831whi · Cited by 125 · cited 1×
“In light of its mixed diagnostic performance, Internet-based screening could also be followed up with clinician interviewing via videoconferencing (eg, [35,36]) or Web chat (eg, [37,38]).”
Klein, B., Meyer, D., Austin, D. W., & Kyrios, M. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1918 · Cited by 118
Self-citationKobak, K. A., Williams, J. B. W., Jeglic, E., Salvucci, D., & Sharp, I. R. (2008) · Depression and Anxiety · doi:10.1002/da.20392 · Cited by 50 · cited 1×
“” The use of telephone interviewing for assessing mental health disorders has support in the literature [17-19].”
Cited by
Indicators of mental disorders in UK Biobank—A comparison of approaches
Davis, K. A. S., Cullen, B., Adams, M., Brailean, A., Breen, G., Coleman, J. R. I., Dregan, A., Gaspar, H. A., Hübel, C., Lee, W., et al. (2019) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1796 · Cited by 111
Bot, M., Middeldorp, C. M., de Geus, E. J. C., Lau, H. M., Sinke, M., van Nieuwenhuizen, B., Smit, J. H., Boomsma, D. I., & Penninx, B. W. J. H. (2017) · Psychological Medicine · doi:10.1017/s0033291716002312 · Cited by 56
“79, reference Mini International Neuropsychiatric Interview; MINI-plus) (Nguyen et al. 2015), and the Web-Based Depression and Anxiety Test (sensitivity 0.”
BackgroundMethodInfluentialFerreri, F., Bourla, A., Peretti, C.-S., Segawa, T., Jaafari, N., & Mouchabac, S. (2019) · JMIR Mental Health · doi:10.2196/11643 · Cited by 53
“…Psychological Assessment System, a Web-based, self-report, multidisorder, clinical assessment and referral system showed a fair agreement with the clinical interview for OCD (kappa=0.39) [19] XSL • FO RenderX With an EMA, patients respond to a series of questions based on their reported symptoms.”
InfluentialBortolon, C., & Raffard, S. (2019) · Cognitive Neuropsychiatry · doi:10.1080/13546805.2019.1670152 · Cited by 19
Metadata
- Authors
- Nguyen, D. P., Klein, B., Meyer, D., Austin, D. W., & Abbott, J.-A. M.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2015-09-21
- Volume
- 17
- Issue
- 9
- Pages
- e218
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W2123245872
- Cited by
- 35
- References
- 10
- FWCI
- 4.58
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Diagnostic Validity and Reliability of an Internet-Based Clinical Assessment Program for Mental Disorders. https://app.cochat.ai/papers/10.2196/jmir.4195
- The paper
- Nguyen, D. P., Klein, B., Meyer, D., Austin, D. W., & Abbott, J.-A. M. (2015). The Diagnostic Validity and Reliability of an Internet-Based Clinical Assessment Program for Mental Disorders. Journal of Medical Internet Research. https://doi.org/10.2196/jmir.4195
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.