CoChat
Sign Up
Cited by 233Mental Health via WritingSentiment Analysis and Opinion MiningTopic Modeling

The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.

MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare

Open access
Ji, Shaoxiong, Zhang, Tianlin, Ansari, Luna, Fu, Jie, Tiwari, Prayag, Cambria, Erik·2022·Proceedings of the Language Resources and Evaluation Conference
DOI 10.63317/2d7umxifj8wz
Open this paper in CoChat — free accountOpen PDF

Abstract

Mental health is a critical issue in modern society, and mental disorders could sometimes turn to suicidal ideation without adequate treatment.Early detection of mental disorders and suicidal ideation from social content provides a potential way for effective social intervention.Recent advances in pretrained contextualized language representations have promoted the development of several domain-specific pretrained models and facilitated several downstream applications.However, there are no existing pretrained language models for mental healthcare.This paper trains and releases two pretrained masked language models, i.e., MentalBERT and MentalRoBERTa, to benefit machine learning for the mental healthcare research community.Besides, we evaluate our trained domain-specific models and several variants of pretrained language models on several mental disorder detection benchmarks and demonstrate that language representations pretrained in the target domain improve the performance of mental health detection tasks.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.63317/2d7umxifj8wz

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 38 references

Research works printed but not linked: 11

  1. Ansari, L., Ji, S., Chen, Q., and Cambria, E. (2022). Ensemble hybrid learning methods for automated depression detection. IEEE Transactions on Com- putational Social Systems.
  2. Cambria, E., Hussain, A., Durrani, T., Havasi, C., Eckl, C., and Munro, J. (2010). Sentic computing for patient centered applications. In IEEE ICSP, pages 1279-1282.
  3. Cambria, E., Benson, T., Eckl, C., and Hussain, A. (2012). Sentic PROMs: Application of sentic com- puting to the development of a novel unified frame- work for measuring health-care quality. Expert Sys- tems with Applications, 39(12):10533-10543.
  4. Cambria, E., Liu, Q., Decherchi, S., Xing, F., , and Kwok, K. (2022). SenticNet 7: A commonsense- based neurosymbolic ai framework for explainable sentiment analysis. In LREC.
  5. Grassi, M., Cambria, E., Hussain, A., and Piazza, F. (2011). Sentic web: A new paradigm for managing social media affective information. Cognitive Com- putation, 3(3):480-489.
  6. Lee, A., Kummerfeld, J. K., An, L. C., and Mihalcea, R. (2021). Micromodels for efficient, explainable, and reusable systems: A case study on mental health. arXiv preprint arXiv:2109.13770. · arXiv
  7. Lin, C., Miller, T., Dligach, D., Bethard, S., and Savova, G. (2019). A BERT-based universal model for both within-and cross-sentence clinical temporal relation extraction. In Proceedings of the 2nd Clin- ical Natural Language Processing Workshop, pages 65-71.
  8. Martínez-Castaño, R., Htait, A., Azzopardi, L., and Moshfeghi, Y. (2021). BERT-Based Transform- ers for Early Detection of Mental Health Illnesses. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 189-200. Springer.
  9. Mehta, Y., Majumder, N., Gelbukh, A., and Cambria, E. (2020). Recent trends in deep learning based personality detection. Artificial Intelligence Review, 53:2313-2339.
  10. Shickel, B., Siegel, S., Heesacker, M., Benton, S., and Rashidi, P. (2020). Automatic detection and clas- sification of cognitive distortions in mental health text. In 2020 IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE), pages 275-280. IEEE.
  11. Wolf, T., Chaumond, J., Debut, L., Sanh, V., Delangue, C., Moi, A., Cistac, P., Funtowicz, M., Davison, J., Shleifer, S., et al. (2020). Transformers: State-of- the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstra- tions, pages 38-45.

Cited by

Sorted by citations

Items 201–233 of 233

Related

See all

Metadata

Authors
Ji, S., Zhang, T., Ansari, L., Fu, J., Tiwari, P., & Cambria, E.
Affiliations
  • Aalto University (FI)
  • University of Manchester (GB)
  • Mila - Quebec Artificial Intelligence Institute (CA)
  • Nanyang Technological University (SG)
ORCID
Venue
Proceedings of the Language Resources and Evaluation Conference
Publisher
European Language Resources Association (ELRA)
Published
2022-06
Pages
7184-7190
ISSN
2522-2686
DOI
10.63317/2d7umxifj8wz
OpenAlex
W3209409148
Cited by
233
References
38
Citations by year
2026: 68 · 2025: 30 · 2024: 62 · 2023: 55 · 2022: 18
FWCI
79.56
Type
conference paper
Language
en
Open access
Licence
cc-by-nc
Locations
  • Language Resources and Evaluation Conference (LREC) (publishedVersion, cc-by-nc)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
China Scholarship Council
Topics
  • Social Sciences › Psychology › Social Psychology › Mental Health via Writing
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
pre-trained language models, mental disorder detection, suicidal ideation detection, masked language models, social media text analysis, mental health services

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review