Empath: Understanding Topic Signals in Large-Scale Text
Open accessAbstract
Human language is colored by a broad range of topics, but existing text analysis tools only focus on a small number of them. We present Empath, a tool that can generate and validate new lexical categories on demand from a small set of seed terms (like "bleed" and "punch" to generate the category violence). Empath draws connotations between words and phrases by deep learning a neural embedding across more than 1.8 billion words of modern fiction. Given a small set of seed words that characterize a category, Empath uses its neural embedding to discover new related terms, then validates the category with a crowd-powered filter. Empath also analyzes text across 200 built-in, pre-validated categories we have generated from common topics in our web dataset, like neglect, government, and social media. We show that Empath's data-driven, human validated categories are highly correlated (r=0.906) with similar categories in LIWC.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Term sets constructed using Empath have a strong correlation with corresponding LIWC categories.
“Term sets constructed in this way have a strong correlation with the corresponding LIWC categories, which were constructed in a completely manual process.”
From Behavioral Activation and Depression Symptomatology: Longitudinal Assessment of Linguistic Indicators in Text-Based Therapy Sessions (2021) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · see citing works
Journal of personality and social psychology
Armstrong, J. (1988) · International Journal of Forecasting · doi:10.1016/0169-2070(88)90126-4 · Cited by 10,544 · see citing works
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024 · see citing works
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878 · see citing works
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087 · see citing works
The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods
Tausczik, Y. R., & Pennebaker, J. W. (2010) · Journal of Language and Social Psychology · doi:10.1177/0261927x09351676 · Cited by 5,774 · see citing works
Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
Dietterich, T. G. (1998) · Neural Computation · doi:10.1162/089976698300017197 · Cited by 3,840 · see citing works
Experimental evidence of massive-scale emotional contagion through social networks
Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1320040111 · Cited by 3,687 · see citing works
Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
Peter, P., & Turney (2002) · Cited by 3,646 · see citing works
Emotion knowledge: further exploration of a prototype approach
Shaver, P., Schwartz, J., Kirson, D., & O'Connor, C. (1987) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.52.6.1061 · Cited by 3,072 · see citing works
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · Cited by 2,913 · see citing works
CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON
Mohammad, S. M., & Turney, P. D. (2012) · Computational Intelligence · doi:10.1111/j.1467-8640.2012.00460.x · Cited by 2,757 · see citing works
SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining
Esuli, A., & Sebastiani, F. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/54vo5wjya4cw · Cited by 2,494 · see citing works
Turney, P. D. (2001) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/1073083.1073153 · Cited by 2,354 · see citing works
Thumbs up? Sentiment Classification using Machine Learning TechniquesPreprint
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0205070 · Cited by 2,211 · see citing works
Linguistic Inquiry and Word Count (LIWC)
Chung, C. K., & Pennebaker, J. W. (2012) · IGI Global eBooks · doi:10.4018/978-1-60960-741-8.ch012 · Cited by 2,152 · see citing works
5.7 The WORDNET lexical database of English
Ooi, V. B. Y. (2019) · Edinburgh University Press eBooks · doi:10.1515/9781474471459-050 · Cited by 2,034 · see citing works
ConceptNet — A Practical Commonsense Reasoning Tool-Kit
Liu, H., & Singh, P. K. (2004) · BT Technology Journal · doi:10.1023/b:bttj.0000047600.45421.6d · Cited by 1,592 · see citing works
Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of ReviewsPreprint
Turney, P. D. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0212032 · Cited by 1,584 · see citing works
Predicting Depression via Social Media
Choudhury, M. D., Gamon, M., Counts, S., & Horvitz, E. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14432 · Cited by 1,583 · see citing works
Predicting the semantic orientation of adjectives
Hatzivassiloglou, V., & McKeown, K. R. (1997) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/976909.979640 · Cited by 1,445 · see citing works
(1999) · Cited by 1,435 · see citing works
Get another label? improving data quality and data mining using multiple, noisy labelers
Sheng, V. S., Provost, F. J., & Ipeirotis, P. G. (2008) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1401890.1401965 · Cited by 1,180 · see citing works
The general inquirer: A computer approach to content analysis.
Smith, M. S., Stone, P. J., & Dunphy, D. C. (1966) · Medical Entomology and Zoology · Cited by 1,165 · see citing works
Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures
Golder, S. A., & Macy, M. W. (2011) · Science · doi:10.1126/science.1202775 · Cited by 1,155 · cited 2× · see citing works
“We show how the open-ended nature of Empath’s model can replicate and extend classic work in classifying deceptive language [31], identifying the patterns of language in movie reviews [32], and analyzing mood on twitter [13].”
BackgroundMethodFinding Deceptive Opinion Spam by Any Stretch of the Imagination
Ott, M., Choi, Y., Cardie, C., & Hancock, J. T. (2011) · arXiv · doi:10.48550/arxiv.1107.4557 · Cited by 1,008 · see citing works
The General Inquirer: A Computer Approach to Content Analysis.
Hartman, J. J., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvia, D. M. (1967) · American Sociological Review · doi:10.2307/2092070 · Cited by 723 · see citing works
Toma, C. L., Hancock, J. T., & Ellison, N. B. (2008) · Personality and Social Psychology Bulletin · doi:10.1177/0146167208318067 · Cited by 655 · see citing works
Playing the other: gender and society in classical Greek literature
(1996) · Choice Reviews Online · doi:10.5860/choice.33-6124 · Cited by 577 · see citing works
Small, T. A. (2011) · Information Communication & Society · doi:10.1080/1369118x.2011.554572 · Cited by 433 · see citing works
Unsupervised learning of narrative schemas and their participants
Chambers, N., & Jurafsky, D. (2009) · doi:10.3115/1690219.1690231 · Cited by 408 · see citing works
Modeling public mood and emotion: Twitter sentiment and socio-economic phenomenaPreprint
Bollen, J., Pepe, A., & Mao, H. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0911.1583 · Cited by 319 · see citing works
Salehi, N., Irani, L. C., Bernstein, M. S., Alkhatib, A., Ogbe, E., Milland, K., & Clickhappier (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702508 · Cited by 274 · see citing works
Self-citationSentiment, emotion, purpose, and style in electoral tweets
Mohammad, S. M., Zhu, X., Kiritchenko, S., & Martin, J. (2014) · Information Processing & Management · doi:10.1016/j.ipm.2014.09.003 · Cited by 274 · see citing works
Gao, T. Y., Dontcheva, M., Adar, E., Liu, Z., & Karahalios, K. G. (2015) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2807442.2807478 · Cited by 244 · see citing works
Kumar, R., Satyanarayan, A., Torres, C., Lim, M., Ahmad, S., Klemmer, S., & Talton, J. O. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466420 · Cited by 141 · see citing works
We feel fine and searching the emotional web
Kamvar, S. D., & Harris, J. (2011) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/1935826.1935854 · Cited by 118 · see citing works
Mitra, T., Hutto, C. J., & Gilbert, E. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702553 · Cited by 102 · see citing works
Do You Feel What I Feel? Social Aspects of Emotions in Twitter Conversations
Kim, S., Bak, J., & Oh, A. H. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v6i1.14310 · Cited by 88 · see citing works
The general inquirer: A computer approach to content analysis
Grieb, W. E. (1968) · Information Processing & Management · doi:10.1016/0020-0271(68)90031-4 · Cited by 67 · see citing works
Linguistic Harbingers of Betrayal: A Case Study on an Online Strategy Game
Niculae, V., Kumar, S., Boyd-Graber, J. L., & Danescu-Niculescu-Mizil, C. (2015) · doi:10.3115/v1/p15-1159 · Cited by 49 · see citing works
Emergent, crowd-scale programming practice in the IDE
Fast, E., Steffee, D., Wang, L. L., Brandt, J., & Bernstein, M. S. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2556998 · Cited by 36 · see citing works
Self-citationGenerating Music from Literature
Davis, H., & Mohammad, S. M. (2014) · Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL) · doi:10.3115/v1/w14-0901 · Cited by 33 · see citing works
Generating emotionally relevant musical scores for audio stories
Rubin, S., & Agrawala, M. (2014) · ACM Symposium on User Interface Software and Technology · doi:10.1145/2642918.2647406 · Cited by 33 · see citing works
Fast, E., McGrath, W., Rajpurkar, P., et al. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858528 · Cited by 29 · see citing works
Self-citationNeviarouskaya, A., Prendinger, H., & Ishizuka, M. (2007) · Cited by 19 · see citing works
Kim, S., Bak, J., & Oh, A. H. (2012) · ACM SIGWEB Newsletter · doi:10.1145/2362631.2362634 · Cited by 10 · see citing works
You're happy, I'm happy: diffusion of mood expression on twitter
Choudhury, M. D. (2014) · International Conference on Human-Computer Interaction · doi:10.5555/2729485.2729511 · Cited by 6 · see citing works
Learning Word Vectors Efficiently Using Shared Representations and Document Representations
Luo, Q., & Xu, W. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9711 · Cited by 4 · see citing works
Zeitlin, F. I. (2020) · Princeton University Press eBooks · doi:10.2307/j.ctv131bvsk.8 · Cited by 4 · see citing works
Generating Music from LiteraturePreprint
Davis, H., & Mohammad, S. M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.2124 · Cited by 4 · see citing works
Towards Learning a Knowledge Base of Actions from Experiential Microblogs.
Kıcıman, E. (2015) · National Conference on Artificial Intelligence · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [5]5. Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. A computational approach to politeness with application to social factors. In Proc of ACL 2013 .
- [10]10. Ethan Fast, Will McGrath, Pranav Rajpurkar, and Michael Bernstein. Mining Human Behaviors from Fiction to Power Interactive Systems. In Proc. CHI 2016 .
Cited by
Sorted by citations
Psychological inoculation improves resilience against misinformation on social media
Roozenbeek, J., van der Linden, S., Goldberg, B., Rathje, S., & Lewandowsky, S. (2022) · Science Advances · doi:10.1126/sciadv.abo6254 · Cited by 472 · see citing works
Long-Term Trends in the Public Perception of Artificial Intelligence
Fast, E., & Horvitz, E. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10635 · Cited by 467 · see citing works
Self-citationGender and Representation Bias in GPT-3 Generated Stories
Lucy, L., & Bamman, D. (2021) · Proceedings of the Third Workshop on Narrative Understanding · doi:10.18653/v1/2021.nuse-1.5 · Cited by 287 · see citing works
Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence
Singh, N., Bernal, G., Savchenko, D., & Glassman, E. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3511599 · Cited by 119 · see citing works
Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover
Fröhling, L., & Zubiaga, A. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.443 · Cited by 106 · see citing works
De Silva, D., Ranasinghe, W., Bandaragoda, T., Adikari, A., Mills, N., Iddamalgoda, L., Alahakoon, D., Lawrentschuk, N., Persad, R., Osipov, E., et al. (2018) · PLOS ONE · doi:10.1371/journal.pone.0205855 · Cited by 86 · see citing works
Teng, T. H., & Varathan, K. D. (2023) · IEEE Access · doi:10.1109/access.2023.3275130 · Cited by 77 · see citing works
Burkhardt, H. A., Alexopoulos, G. S., Pullmann, M. D., Hull, T. D., Areán, P. A., & Cohen, T. (2021) · Journal of Medical Internet Research · doi:10.2196/28244 · Cited by 68 · see citing works
PANDORA Talks: Personality and Demographics on Reddit
Gjurković, M., Karan, M., Vukojević, I., Bošnjak, M., & Snajder, J. (2021) · Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/2021.socialnlp-1.12 · Cited by 58 · see citing works
“I Can’t Reply with That”: Characterizing Problematic Email Reply Suggestions
Robertson, R. E., Olteanu, A., Diaz, F., Shokouhi, M., & Bailey, P. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445557 · Cited by 43 · see citing works
Quillivic, R., Gayraud, F., Auxéméry, Y., Vanni, L., Peschanski, D., Eustache, F., Dayan, J., & Mesmoudi, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-61557-7 · Cited by 24 · see citing works
Identifying Sexual Predators in Chats Using SVM and Feature Ensemble
Fauzi, M. A., Wolthusen, S., Yang, B., Bours, P., & Yeng, P. (2023) · 2023 International Conference on Emerging Trends in Networks and Computer Communications (ETNCC) · doi:10.1109/etncc59188.2023.10284950 · Cited by 4 · see citing works
- Authors
- Fast, E., Chen, B., & Bernstein, M. S.
- Affiliations
- Stanford University (US)
- Venue
- Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2016-05-07
- Pages
- 4647-4657
- OpenAlex
- W2273847690
- Cited by
- 413
- References
- 53
- Citations by year
- 2026: 25 · 2025: 31 · 2024: 26 · 2023: 48 · 2022: 59 · 2021: 77 · 2020: 59 · 2019: 33 · 2018: 31 · 2017: 21 · 2016: 3
- FWCI
- 15.81
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- parts of speech, topic modeling, word embeddings, connotation, text analysis, LIWC
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
