Considering patient safety in autonomous e-mental health systems – detecting risk situations and referring patients back to human care
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Digital health interventions can fill gaps in mental healthcare provision. However, autonomous e-mental health (AEMH) systems also present challenges for effective risk management. To balance autonomy and safety, AEMH systems need to detect risk situations and act on these appropriately. One option is sending automatic alerts to carers, but such 'auto-referral' could lead to missed cases or false alerts. Requiring users to actively self-refer offers an alternative, but this can also be risky as it relies on their motivation to do so. This study set out with two objectives. Firstly, to develop guidelines for risk detection and auto-referral systems. Secondly, to understand how persuasive techniques, mediated by a virtual agent, can facilitate self-referral. METHODS: In a formative phase, interviews with experts, alongside a literature review, were used to develop a risk detection protocol. Two referral protocols were developed - one involving auto-referral, the other motivating users to self-refer. This latter was tested via crowd-sourcing (n = 160). Participants were asked to imagine they had sleeping problems with differing severity and user stance on seeking help. They then chatted with a virtual agent, who either directly facilitated referral, tried to persuade the user, or accepted that they did not want help. After the conversation, participants rated their intention to self-refer, to chat with the agent again, and their feeling of being heard by the agent. RESULTS: Whether the virtual agent facilitated, persuaded or accepted, influenced all of these measures. Users who were initially negative or doubtful about self-referral could be persuaded. For users who were initially positive about seeking human care, this persuasion did not affect their intentions, indicating that a simply facilitating referral without persuasion was sufficient. CONCLUSION: This paper presents a protocol that elucidates the steps and decisions involved in risk detection, something that is relevant for all types of AEMH systems. In the case of self-referral, our study shows that a virtual agent can increase users' intention to self-refer. Moreover, the strategy of the agent influenced the intentions of the user afterwards. This highlights the importance of a personalised approach to promote the user's access to appropriate care.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Michie, S., van Stralen, M. M., & West, R. (2011) · Implementation Science · doi:10.1186/1748-5908-6-42 · Cited by 13,697
Epidemiology of insomnia: what we know and what we still need to learn
Ohayon, M. M. (2002) · Sleep Medicine Reviews · doi:10.1053/smrv.2002.0186 · Cited by 3,743
Grant, B. F., Stinson, F. S., Dawson, D. A., Chou, S. P., Dufour, M. C., Compton, W., Pickering, R. P., & Kaplan, K. (2004) · Archives of General Psychiatry · doi:10.1001/archpsyc.61.8.807 · Cited by 2,889
Suicide and suicidal behaviour
Turecki, G., & Brent, D. A. (2016) · The Lancet · doi:10.1016/s0140-6736(15)00234-2 · Cited by 2,338
A behavior model for persuasive design
Fogg, B. (2009) · Proceedings of the 4th International Conference on Persuasive Technology · doi:10.1145/1541948.1541999 · Cited by 2,328 · cited 1×
“Fogg (2009) describes a theory where the combination of motivation and ability need to be high enough for a trigger or que to do the behaviour to have effect [34].”
BackgroundBower, P., & Gilbody, S. (2005) · British Journal of Psychiatry · doi:10.1192/bjp.186.1.11 · Cited by 1,084
Measuring Patients’ Trust in their Primary Care Providers
Hall, M. A., Zheng, B., Dugan, E., Camacho, F., Kidd, K. E., Mishra, A., & Balkrishnan, R. (2002) · Medical Care Research and Review · doi:10.1177/1077558702059003004 · Cited by 661
Simon, G. E., Rutter, C. M., Peterson, D., Oliver, M., Whiteside, U., Operskalski, B., & Ludman, E. J. (2013) · Psychiatric Services · doi:10.1176/appi.ps.201200587 · Cited by 575
(2014) · doi:10.5555/2615731.2617415 · Cited by 553
Self-citationEconomic evaluations of Internet interventions for mental health: a systematic review
Donker, T., Blankers, M., Hedman, E., Ljótsson, B., Petrie, K., & Christensen, H. (2015) · Psychological Medicine · doi:10.1017/s0033291715001427 · Cited by 316 · cited 2×
“Furthermore, AEMH’s offer these advantages for comparatively low cost [4].”
Background
Cited by
Van Daele, T., Karekla, M., Kassianos, A. P., Compare, A., Haddouk, L., Salgado, J., Ebert, D. D., Trebbi, G., Bernaerts, S., Van Assche, E., et al. (2020) · Journal of Psychotherapy Integration · doi:10.1037/int0000218 · Cited by 122
Connected Mental Health: Systematic Mapping Study
(2020) · doi:10.2196/19950 · Cited by 33
Bendig, E., Erb, B., Meißner, E., Bauereiß, N., & Baumeister, H. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100377 · Cited by 24
“People at risk could then be referred back to human contact persons (Tielman et al., 2019).”
Background
Metadata
- Authors
- Tielman, M. L., Neerincx, M. A., Pagliari, C., Rizzo, A., & Brinkman, W.-P.
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2019-03-18
- Volume
- 19
- Issue
- 1
- Pages
- 47-47
- ISSN
- 1472-6947 (ISSN-L)
- OpenAlex
- W2940378240
- Cited by
- 54
- References
- 16
- FWCI
- 12.06
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Considering patient safety in autonomous e-mental health systems – detecting risk situations and referring patients back to human care. https://app.cochat.ai/papers/10.1186/s12911-019-0796-x
- The paper
- Tielman, M. L., Neerincx, M. A., Pagliari, C., Rizzo, A., & Brinkman, W.-P. (2019). Considering patient safety in autonomous e-mental health systems – detecting risk situations and referring patients back to human care. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-019-0796-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.