The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index; 1 of the 31 sources its claims cite were checked for support.
Closing the accessibility gap to mental health treatment with a personalized self-referral chatbot
Open accessPublished version of a preprint
Abstract
Inequality in treatment access is a pressing issue in most healthcare systems across many medical disciplines. In mental healthcare, reduced treatment access for minorities is ubiquitous but remedies are sparse. Here we demonstrate that digital tools can reduce the accessibility gap by addressing several key barriers. In a multisite observational study of 129,400 patients within England's NHS services, we evaluated the impact of a personalized artificial intelligence-enabled self-referral chatbot on patient referral volume and diversity in ethnicity, gender and sexual orientation. We found that services that used this digital solution identified substantially increased referrals (15% increase versus 6% increase in control services). Critically, this increase was particularly pronounced in minorities, such as nonbinary (179% increase) and ethnic minority individuals (29% increase). Using natural language processing to analyze qualitative feedback from 42,332 individuals, we found that the chatbot's human-free nature and the patients' self-realization of their need for treatment were potential drivers for the observed improvement in the diversity of access. This provides strong evidence that digital tools may help overcome the pervasive inequality in mental healthcare.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
In a multi-site observational study of ∼129,400 patients across 28 NHS Talking Therapies services, an AI-enabled self-referral chatbot was associated with a 15% increase in total referrals versus a 6% increase in matched control services. The tool disproportionately increased referrals among minority groups, and NLP analysis of 42,332 feedback entries suggested human-free interaction and self-realisation as key mechanisms.
Key findings
- Services using the AI-enabled self-referral tool had a 15% increase in total referrals, compared with 6% in matched services using traditional self-referral methods; the difference was significant (χ²(1)=86.3, p<0.0001).
- Referral increases were larger for minority groups: non-binary 235%, bisexual 30%, and ethnic minority 31%, versus female 18%, male 16%, heterosexual 14%, and White 15%. Odds ratios were non-binary vs female 2.83 (CI 2.264–3.545), non-binary vs male 2.90 (CI 2.314–3.629), bisexual vs heterosexual 1.14 (CI 1.074–1.219), ethnic minorities vs White 1.14 (CI 1.081–1.202), Asian/Asian British 1.21 (CI 1.116–1.318), and Black/Black British 1.24 (CI 1.108–1.378).
- The tool did not negatively affect access to assessments: a two-way ANOVA found no significant interaction between time period and tool usage (F(1,13)=0.09, p=0.764).
- NLP classification of 42,332 feedback entries found 89% positive, 7% neutral, and 4% negative feedback. Gender minority and bisexual individuals mentioned the tool’s human-free nature more than majority groups (χ²(1)=22.7, p<0.0001; χ²(1)=62.4, p<0.0001), while Asian and Black ethnic groups mentioned self-realisation more than White individuals (χ²(1)=46.6, p<0.0001) and convenience less (χ²(1)=24.2, p<0.0001).
Methods
- Multi-site real-world retrospective observational study using routine NHS Talking Therapies data from 28 services: 14 implemented the AI-enabled self-referral tool (Limbic Access) and 14 matched controls did not; ∼129,400 patients were included, with three-month pre- and post-implementation periods and staggered service launch times.
- Services were matched using Euclidean distance on two normalized features from the pre-implementation period; total referrals were compared with Chi-squared tests, demographic changes with percentage change and logistic regression odds ratios, and a self-referral-only sensitivity analysis was conducted.
- Qualitative feedback from September 2021 to March 2023: 157,416 patients used the tool, 29% (N=46,166) gave free-text feedback, and after excluding responses under 10 characters, 42,332 entries remained. Reflexive thematic analysis identified nine themes; SBERT, PCA, and multi-class logistic regression with 100-fold cross-validation achieved a micro-average F1 score of 0.64; Chi-squared tests with Bonferroni correction compared theme proportions.
Limitations
- The paper states possible cultural differences between NHS Talking Therapies services and that services adopting the tool may be inherently more open to digital health innovation, which could influence referral numbers; it notes implementation was not part of wider digital innovation programs and the increase was tightly linked to launch.
- Evident from the text: the observational design cannot establish causality, and matching was based only on two normalized pre-period referral features, leaving potential residual confounding.
- Evident from the text: the NLP analysis covers only the 29% of users who provided free-text feedback and excludes entries under 10 characters, and the classification model had moderate performance (micro-average F1=0.64), so feedback findings may not represent all users.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
44 Own result · 31 Own interpretation · 20 Not checked yet · 13 Own method · 2 No basis given · 1 Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“There is growing evidence that technology can improve help-seeking behaviour [Johnson et al., 2022] and that patients prefer interacting with chatbots for self-reported measures [Te Pas et al., 2020] and discussing sensitive topics [Bhakta et al., 2014].”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 1 Introduction
What the sources say
- Supportedcites Te Pas et al., 2020 source
The Conclusions state that patients preferred the chatbot questionnaire over a regular computer questionnaire. The Principal Findings identify these as patients completing questionnaires in an anesthesiology outpatient setting, so the claim is supported in that context. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We identified a 31% increase in referrals for ethnic minorities, whereas the increase for the White group was 15% (Figure 3C), suggesting that different ethnic groups were more represented after the implementation of the AI-enabled self-referral tool.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.1 Increase in the number and diversity of referrals
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The data used are recorded in routine practice by all NHS Talking Therapies services and reported by NHS Digital.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We particularly saw an increase for Asian and Asian British groups (odds-ratio =1.21, CI = [1.116,1.318],p < 0.0001) and Black and Black British groups (odds-ratio = 1.24,CI = [1.108,1.378],p < 0.0001) compared to White individuals (Figure 3D).”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.1 Increase in the number and diversity of referrals
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The data used are recorded in routine practice by all NHS Talking Therapies services and reported by NHS Digital.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A two-way ANOVA revealed no significant interaction between the time period and tool usage (F(1, 13) = 0.09, p = 0.764).”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.1 Increase in the number and diversity of referrals
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We did not see any differences in the number of referrals for individuals from other ethnic groups (odds-ratio =1.08, CI = [0.913,1.272],p = 0.377) and Mixed ethnic group (odds-ratio =1.01, CI = [0.917,1.107],p = 0.877) compared to White individuals.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.1 Increase in the number and diversity of referrals
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The data used are recorded in routine practice by all NHS Talking Therapies services and reported by NHS Digital.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that referrals by people whose sexual orientation was bi-sexual increased by 30%, referrals by people whose sexual orientation is gay/lesbian increased by 19%, whereas referrals from heterosexuals increased by 14% (Figure 3B).”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.1 Increase in the number and diversity of referrals
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The data used are recorded in routine practice by all NHS Talking Therapies services and reported by NHS Digital.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the increase in individuals identifying as homosexual did not significantly differ from the increase in referrals for heterosexual individuals (odds-ratio = 1.05,CI= [0.922,1.189],p =< 0.216).”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.1 Increase in the number and diversity of referrals
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The data used are recorded in routine practice by all NHS Talking Therapies services and reported by NHS Digital.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We particularly saw an increase for Asian and Asian British groups (odds-ratio =1.21, CI = [1.116,1.318],p < 0.0001) and Black and Black British groups (odds-ratio = 1.24,CI = [1.108,1.378],p < 0.0001) compared to White individuals (Figure 3D).”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.1 Increase in the number and diversity of referrals
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The data used are recorded in routine practice by all NHS Talking Therapies services and reported by NHS Digital.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The increase in ethnic minorities was higher than for White individuals (odds-ratio = 1.14, CI= [1.081,1.202],p < 0.0001), suggesting that the tool facilitates more access for people from ethnic minorities.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.1 Increase in the number and diversity of referrals
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The data used are recorded in routine practice by all NHS Talking Therapies services and reported by NHS Digital.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We identified a 31% increase in referrals for ethnic minorities, whereas the increase for the White group was 15% (Figure 3C), suggesting that different ethnic groups were more represented after the implementation of the AI-enabled self-referral tool.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.1 Increase in the number and diversity of referrals
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The data used are recorded in routine practice by all NHS Talking Therapies services and reported by NHS Digital.
- It may state two separate claims as one. 29
- Probably not a substantial claim. 15
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, 89% of the free text feedback was classified as positive, 7% neutral and 4% negative.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.2 Mechanism for the increase in referrals
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This was followed by the hope of having taken the first step to access support, the absence of human involvement reducing stigma and judgment and increased self-realization that they might suffer from a mental health condition and need support caused by the self-referral process.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.2 Mechanism for the increase in referrals
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Specifically, services that used the AI-enabled self-referral tool showed a 15% increase in referrals, whereas the matched services that did not use the tool showed a 6% increase.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“At the time of writing, this self-referral tool has facilitated the referral of ∼157,000 patients, which means that ∼14,000 additional patients have found their way into treatment owing to this technical solution.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
plaindirectn = 157,000- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A positive percentage change in referrals was observed for people with all sexual orientations, with the highest percentage increase in referrals for individuals who are bi-sexual.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A positive percentage change in referrals was observed for all genders, with the highest percentage increase in referrals for individuals who identify as non-binary.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A positive percentage change in referrals was observed for all ethnic groups, with the highest percentage increase in referrals for ethnic minorities.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Furthermore, the AI-enabled self-referral tool automatically opens on the service's website, eliminating the need for patients to search for contact information or referral form.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Firstly, the supply of mental healthcare is not matching the demand as mental health services are widely underfunded and understaffed [World Health Organization, 2021b, Mahase, 2020].”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 1 Introduction
What the sources say
- Not checked yetcites World Health Organization, 2021b
- Not checked yetcites Mahase, 2020 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Digital technologies and Artificial Intelligence (AI) have been proposed as potential solutions to these challenges [Rudd and Beidas, 2020, Taylor et al., 2020, Kola et al., 2021].”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 1 Introduction
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“However, existing solutions may not be optimised or digitally streamlined, leading to lower completion rates and limited access to services [NHS Digital, 2019].”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 1 Introduction
What the sources say
- Not checked yetcites NHS Digital, 2019
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This suggests that digital technologies can remove some of the barriers to accessing mental health services, and thus increase accessibility.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 1 Introduction
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Secondly, not every individual experiencing mental health problems seeks help, or they delay seeking it for years [Wang et al., 2007, Department of Health andSocial Care, 2023], resulting in unmet needs and inadequate support at the right time.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 1 Introduction
What the sources say
- Not checked yetcites Wang et al., 2007
- Not checked yetcites Department of Health and Social Care, 2023
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Excitingly, digital solutions might be especially well-positioned to help individuals who are seeking support to overcome barriers, for example by offering more flexibility and reducing stigma [Rodríguez-Rivas et al., 2022].”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 1 Introduction
What the sources say
- Not checked yetcites Rodríguez-Rivas et al., 2022 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
In the paper
“Notably, a significant proportion of mental health problems remain undiagnosed and untreated, with estimates suggesting that as many as two-thirds of individuals with depression do not receive the necessary care [Moitra et al., 2022].”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 1 Introduction
What the sources say
- Not checked yetcites Moitra et al., 2022 source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In order to match services, we calculated the Euclidean distance based on these two (normalized) features and took the service with the lowest distance to the NHS Talking Therapies service using the AI-enabled self-referral tool, where a low distance implies a high degree of similarity.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 2.2 Study design
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To account for multiple comparisons, we used Bonferroni correction.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > Other negative feedback Individuals mentioned any other negative feedback
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, for minority groups, we identified two significant drivers: the tool's human-free nature, which allowed patients to seek help without fear of being judged or discriminated against, and its ability to increase patients' awareness of their treatment needs.”
- Also stated in: § Abstract · plain
“We found that the tool's humanfree nature and its ability to improve the perceived need for treatment were the main drivers for improved diversity.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 1 Introduction
plaindirectn = 42,332- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The feedback entries provided by patients are representative of the experiences of all patients who used the tool.
- Also stated in: § Abstract · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“indicating that different groups might experience different advantages which might contribute to the increased accessibility for these groups.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.2 Mechanism for the increase in referrals
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In conclusion, these results suggest that the AI-enabled self-referral tool facilitates a general increase in referrals, as well as improves the diversity of referrals.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.1 Increase in the number and diversity of referrals
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Initial codes were discussed with the larger group of researchers, and a consensus was reached on the resulting themes (listed in the table below) after two meetings.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 2.3.3 NLP topic classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To ensure that the total number of referrals did not increase due to other forms of referrals other than self-referrals (e.g.GP referrals) increasing in the time period, we repeated the analysis for only self-referrals in the same services at the same time period (Supplementary Figure 1).”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 2.3.1 Total number of referrals
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We used a multi-class logistic regression model and evaluated its accuracy through a 100-fold cross-validation approach.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > Other negative feedback Individuals mentioned any other negative feedback
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“To ensure our analysis only included meaningful insights, we excluded any responses containing fewer than 10 characters, resulting in a final sample of 42,332 feedback entries.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 2.3.3 NLP topic classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These findings suggest that AI-enabled chatbots have the potential to increase accessibility to mental health services for all, and to alleviate barriers faced by disadvantaged populations.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > Abstract
hedgedconditionaln = 42,332- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Our results further support the notion that digital technologies can support reaching the SDGs [Lancet, 2021] and support more people in accessing the services they need.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
What the sources say
- Not checked yetcites Lancet, 2021 source
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Nevertheless, implementation of the AI-enabled self-referral tool was not part of wider digital innovation programs for any of the services and the increase in referrals was tightly linked to the launch of the AI-enabled self-referral tool.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Addressing these barriers with an AI-enabled self-referral tool has likely lowered the threshold for seeking help, resulting in higher referral rates as indicated by our data.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
hedgedconditionaln = 42,332- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Utilizing an AI-enabled self-referral tool could reduce barriers to access, resulting in more referrals and improved treatment efficacy [Rollwage et al., 2022] and a decrease in the burden on individuals and society.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
What the sources say
- Not checked yetcites Rollwage et al., 2022
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As the data was collected as part of a standard national data reporting process that undergoes quality checks, there is no concern about missing data or that the data is reported in different ways.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, the staggered design of the self-referral tool implementation at different time points accounted for any natural changes in referral numbers.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some of these barriers might be especially well addressed by an AI-enabled self-referral tool's human-free aspect and the reduced stigma.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.2 Mechanism for the increase in referrals
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Ethnic minorities mainly mentioned how the referral process helped them to realise that they might be suffering from mental health problems or the need for treatment which may reflect the negative attitudes and lack of perceived need for treatment these individuals might face.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The findings of this research demonstrate the potential of AI-enabled conversational tools to facilitate the self-referral process for mental health services and make it more accessible for different groups of people.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
boostedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such solutions may remove these barriers and may be valued more by these minority groups, leading to a higher number of referrals.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.2 Mechanism for the increase in referrals
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition, the data was collected under real-world conditions from multiple services, which provides a larger sample size and thereby improves the generalisability of the results.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In conclusion, our findings suggest that minority groups may face different barriers and value different aspects when seeking mental health treatment than majority groups.”
- Also stated in: § 4 Discussion · hedged
“Interestingly, we found that minority groups mention different themes in their feedback, suggesting that the barriers they face and their needs are different to majority groups.”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 3.2 Mechanism for the increase in referrals
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § 4 Discussion · hedged
None of the sources cited for this claim have been checked yet.
In the paper
“Findings from previous research suggest that digital technologies may facilitate help-seeking for mental health [Johnson et al., 2022] and make it easier for people to disclose personal information [Te Pas et al., 2020, Bhakta et al., 2014].”
§ Closing the accessibility gap to mental health treatment with a conversational AI-enabled self-referral tool > 4 Discussion
What the sources say
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
NHS Digital publicly available data undergoes quality checks and is comprehensive enough for high-quality analysis.
“The publicly available data undergoes quality checks and is a comprehensive dataset that facilitates high-quality analysis.”
The paper relies on this without support.
The routine questionnaires and screening questions have not been typically collected at the point of referral in NHS Talking Therapies, but rather during the first assessment on a phone call.
“These routine questionnaires and screening questions have not been typically collected at the point of referral in NHS Talking Therapies, but rather during the first assessment on a phone call.”
The paper relies on this without support.
The feedback entries provided by patients are representative of the experiences of all patients who used the tool.
“From September 2021 to March 2023, a total of 157,416 patients used this novel self-referral approach, with 29% of these patients offering free-text feedback on their experience (N = 46,166).”
The paper relies on this without support.
Mental health support is effective for many mental disorders.
“Mental health support can reduce the burden and has been found to be highly effective for many mental disorders.”
The paper relies on this without support.
Less is known about the marginal impact of AI technologies in supporting individuals of differing demographics seeking help.
“less is known about the marginal impact of AI technologies in supporting individuals of differing demographics seeking help.”
The paper relies on this without support.
The data used are recorded in routine practice by all NHS Talking Therapies services and reported by NHS Digital.
“The data used in this study are recorded in routine practice by all NHS Talking Therapies services and are reported to the public monthly, quarterly, and annually by NHS Digital (NHS Digital).”
The paper relies on this without support.
An increase in referrals can potentially lead to longer wait times for clinical assessment and may reduce the number of people who access these assessments.
“An increase in referrals can potentially lead to longer wait times for clinical assessment and may reduce the number of people who access these assessments.”
The paper relies on this without support.
The baseline increase in referrals in matched services is likely due to the national initiative to boost access across all NHS Talking Therapies services.
“This baseline increase is likely to reflect the national initiative to boost access across all NHS Talking Therapies services.”
The paper relies on this without support.
Non-binary, bisexual, and ethnic minority groups have been previously underrepresented in mental healthcare treatment.
“These groups have been previously underrepresented in mental healthcare treatment”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A personalized self-referral chatbot in the UK, geared towards mental health and available to the general population, was able to increase referrals from 6% to 15%, with a particular impact on ethnic minorities (29% increase) [131].”
From Polycystic Ovary Syndrome and the Internet of Things: A Scoping Review (2024) · cites this paper as [131]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Studies have shown that chatbots are able to generate quality and empathetic responses to patients [130] and substantially increase referrals [131].”
From Polycystic Ovary Syndrome and the Internet of Things: A Scoping Review (2024) · cites this paper as [131]
The cited source says what the claim attributes to it.
In the citing paper
“The duration of chatbot intervention ranged from 20 minutes [59] to 3 months [56] (mean 3.63, SD 1.74 wk).”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [56]
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Not confirmedχ²(1) = 14.6reported p = .0036 · recomputed p < .001
“χ 2 (1) = 14.6, p = .0036”
- ConsistentF(1, 13) = 0.09reported p = .764 · recomputed p = .769
“F(1, 13) = 0.09, p = .764”
- Consistentχ²(1) = 30.8reported p < .0001 · recomputed p < .001
“χ 2 (1) = 30.8,p < .0001”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.10.1092 · Cited by 31,765
Clarke, V., & Braun, V. (2017) · The Journal of Positive Psychology · doi:10.1080/17439760.2016.1262613 · Cited by 6,543
Santomauro, D. F., Mantilla Herrera, A. M., Shadid, J., Zheng, P., Ashbaugh, C., Pigott, D. M., Abbafati, C., Adolph, C., Amlag, J. O., Aravkin, A. Y., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(21)02143-7 · Cited by 5,444
The National Institute for Clinical Excellence
(2001) · Nursing Management · doi:10.7748/nm.8.2.39.s15 · Cited by 3,868
Clement, S., Schauman, O., Graham, T., Maggioni, F., Evans-Lacko, S., Bezborodovs, N., Morgan, C., Rüsch, N., Brown, J. S. L., & Thornicroft, G. (2015) · Psychological Medicine · doi:10.1017/s0033291714000129 · Cited by 3,152
The global prevalence of common mental disorders: a systematic review and meta-analysis 1980–2013
Steel, Z., Marnane, C., Iranpour, C., Chey, T., Jackson, J. W., Patel, V., & Silove, D. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyu038 · Cited by 2,846
The Work and Social Adjustment Scale: a simple measure of impairment in functioning
Mundt, J. C., Marks, I. M., Shear, M. K., & Greist, J. M. (2002) · British Journal of Psychiatry · doi:10.1192/bjp.180.5.461 · Cited by 2,515
Construction and Evaluation of a User Experience Questionnaire
Laugwitz, B., Held, T., & Schrepp, M. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-89350-9_6 · Cited by 2,114
The Impact of Mental Illness Stigma on Seeking and Participating in Mental Health Care
Corrigan, P. W., Druss, B. G., & Perlick, D. A. (2014) · Psychological Science in the Public Interest · doi:10.1177/1529100614531398 · Cited by 1,642
Research works printed but not linked: 1
- Bhakta, R., Savin-Baden, M. & Tombs, G. Sharing secrets with robots? In Proc. EdMedia+ Innovate Learning (eds Viteli, J. & Leikomaa, M.) 2295–2301 (Association for the Advancement of Computing in Education, 2014).
Cited by
Sorted by citations
Torous, J., Linardon, J., Goldberg, S. B., Sun, S., Bell, I., Nicholas, J., Hassan, L., Hua, Y., Milton, A., & Firth, J. (2025) · World Psychiatry · doi:10.1002/wps.21299 · Cited by 323
Zhong, W., Luo, J., & Zhang, H. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.04.057 · Cited by 224
The generative era of medical AI
Fahrner, J., Chen, E., Topol, E. J., & Rajpurkar, P. (2025) · Cell · doi:10.1016/j.cell.2025.05.018 · Cited by 120
Practical AI application in psychiatry: historical review and future directions
Sun, J., Lu, T., Shao, X., Han, Y., Xia, Y., Zheng, Y., Wang, Y., Li, X., Ravindran, A. V., Fan, L., et al. (2025) · Molecular Psychiatry · doi:10.1038/s41380-025-03072-3 · Cited by 88
“Personalized self-referral chatbots currently in use in mental health services, such as NHS Talking Therapies in the UK [81], utilize AI-powered NLP algorithms to enable patient self-referral and streamline access to mental health support [76].”
Rollwage, M., Habicht, J., Juechems, K., Carrington, B., Viswanathan, S., Stylianou, M., Hauser, T. U., & Harper, R. J. (2023) · JMIR AI · doi:10.2196/44358 · Cited by 66
Self-citation
Metadata
- Authors
- Habicht, J., Viswanathan, S., Carrington, B., Hauser, T. U., Harper, R., & Rollwage, M.
- Affiliations
- University College London (GB)
- Corresponding author
- Habicht, Johanna
- Venue
- Nature Medicine
- Published
- 2024-02
- Volume
- 30
- Issue
- 2
- Pages
- 595-602
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- OpenAlex
- W4391542646
- Semantic Scholar
- 8f3fb98b9e03c9b4f2ad9d48ddb32f1873d61653
- Cited by
- 132
- References
- 49
- Citations by year
- 2026: 63 · 2025: 46 · 2024: 22
- FWCI
- 51.95
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Nature Medicine (publishedVersion)
- medRxiv (acceptedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- access to mental health care, artificial intelligence, health disparities, ethnic minorities, nonbinary patients, gender diversity, digital health tools, NHS mental health services, patient feedback
- MeSH terms
- Artificial Intelligence, Ethnicity, Female, Health Services Accessibility, Humans, Male, Mental Health, Minority Groups, Referral and Consultation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
