Natural language analysis and the psychology of verbal behavior: the past, present, and future states of the field
Open accessAbstract
Throughout history, scholars and laypeople alike have believed that our words contain subtle clues about what we are like as people, psychologically speaking. However, the ways in which language has been used to infer psychological processes has seen dramatic shifts over time and, with modern computational technologies and digital data sources, we are on the verge of a massive revolution in language analysis research. In this article, we discuss the past and current states of research at the intersection of language analysis and psychology, summarizing the central successes and shortcomings of psychological text analysis to date. We additionally outline and discuss a critical need for language analysis practitioners in the social sciences to expand their view of verbal behavior. Lastly, we discuss the trajectory of interdisciplinary research on language and the challenges of integrating analysis methods across paradigms, recommending promising future directions for the field along the way.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Dictionary-based text analysis of keywords in qualitative user commands recorded on SVA oversimplifies older people's interactions with SVA to a one-way initiative by the user and fails to capture the communicative nature of such interactions.
“However, such a dictionary-based approach oversimplifies older people's interactions with SVA to a one-way initiative by the user and fails to capture the communicative nature of such interactions—the transmission and coordination of complex information with others (7).”
From Understanding older people's voice interactions with smart voice assistants: a new modified rule-based natural language processing model with human input (2024) · cites this paper as [7]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
Indexing by latent semantic analysis
Deerwester, S. C., Dumais, S., Furnas, G. W., Landauer, T. K., & Harshman, R. A. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6 · Cited by 12,758 · see citing works
Ambrose, A., Wittgenstein, L., & Anscombe, G. E. M. (1954) · Philosophy and Phenomenological Research · doi:10.2307/2103745 · Cited by 12,289 · see citing works
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works
The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods
Tausczik, Y. R., & Pennebaker, J. W. (2010) · Journal of Language and Social Psychology · doi:10.1177/0261927x09351676 · Cited by 5,774 · see citing works
Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts
Grimmer, J., & Stewart, B. M. (2013) · Political Analysis · doi:10.1093/pan/mps028 · Cited by 3,297 · cited 2× · see citing works
“For example, political science papers describing computational text analysis (Grimmer & Stewart, 2013); psychology papers describing machine learning (Stachl et al., 2020), and machine learning papers describing social sciences (O’Connor et al., 2011).”
BackgroundQuantitative analysis of culture using millions of digitized books
Michel, J.-B., Shen, Y., Aiden, A. P., Veres, A., Gray, M. K., Pickett, J. P., Hoiberg, D., Clancy, D., Norvig, P., Orwant, J., et al. (2010) · Science · doi:10.1126/science.1199644 · Cited by 2,972 · see citing works
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · see citing works
Beyond valence: Toward a model of emotion-specific influences on judgement and choice
Lerner, J. S., & Keltner, D. (2000) · Cognition & Emotion · doi:10.1080/026999300402763 · Cited by 2,386 · see citing works
Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Shah, A., Kosinski, M., Stillwell, D., Seligman, M. E. P., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073791 · Cited by 1,700 · see citing works
Self-citationWord of mouth and interpersonal communication: A review and directions for future research
Berger, J. (2014) · Journal of Consumer Psychology · doi:10.1016/j.jcps.2014.05.002 · Cited by 1,469 · see citing works
Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures
Golder, S. A., & Macy, M. W. (2011) · Science · doi:10.1126/science.1202775 · Cited by 1,155 · cited 1× · see citing works
“Even in the absence of other psychological measures, the existing nomological network for language analysis facilitates meaningful theory-testing, ranging from the study of daylight cycles on mood (Golder & Macy, 2011) to personality-driven business outcomes (Block et al., 2019).”
BackgroundUsing linguistic cues for the automatic recognition of personality in conversation and text
Mairesse, F., Walker, M. A., Mehl, M. R., & Moore, R. K. (2007) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2349 · Cited by 1,046 · see citing works
The General Inquirer: A Computer Approach to Content Analysis
Green, B. F., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvie, D. M. (1967) · American Educational Research Journal · doi:10.2307/1161774 · Cited by 1,013 · see citing works
WHY SUMMARIES OF RESEARCH ON PSYCHOLOGICAL THEORIES ARE OFTEN UNINTERPRETABLE
Meehl, P. E. (1990) · Psychological Reports · doi:10.2466/pr0.1990.66.1.195 · Cited by 901 · see citing works
Collaborative Research Across Disciplinary and Organizational Boundaries
Cummings, J. N., & Kiesler, S. (2005) · Social Studies of Science · doi:10.1177/0306312705055535 · Cited by 759 · see citing works
Proceedings of the 28th International Conference on Neural Information Processing Systems
Cortes, C., Lee, D., Sugiyama, M., & Garnett, R. (2015) · Cited by 747 · see citing works
The Measurement of Psychological States Through the Content Analysis of Verbal Behavior
Gottschalk, L. A., & Gleser, G. C. (1969) · University of California Press eBooks · doi:10.1525/9780520376762 · Cited by 657 · see citing works
When Small Words Foretell Academic Success: The Case of College Admissions Essays
Pennebaker, J. W., Chung, C. K., Frazee, J., Lavergne, G. M., & Beaver, D. I. (2014) · PLoS ONE · doi:10.1371/journal.pone.0115844 · Cited by 568 · see citing works
Social Computing: From Social Informatics to Social Intelligence
Wang, F., Carley, K. M., Zeng, D. D., & Mao, W. (2007) · IEEE Intelligent Systems · doi:10.1109/mis.2007.41 · Cited by 568 · see citing works
Pronoun Use Reflects Standings in Social Hierarchies
Kacewicz, E., Pennebaker, J., Davis, M., Jeon, M., & Graesser, A. (2014) · Journal of Language and Social Psychology · doi:10.1177/0261927x13502654 · Cited by 512 · see citing works
Psychological Language on Twitter Predicts County-Level Heart Disease Mortality
Eichstaedt, J. C., Schwartz, H. A., Kern, M. L., Park, G., Labarthe, D. R., Merchant, R. M., Jha, S., Agrawal, M., Dziurzynski, L., Sap, M., et al. (2015) · Psychological Science · doi:10.1177/0956797614557867 · Cited by 487 · see citing works
Self-citationWhat is the central feature of extraversion? Social attention versus reward sensitivity
Ashton, M. C., Lee, K., & Paunonen, S. V. (2002) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.83.1.245 · Cited by 481 · see citing works
Trait explanations in personality psychology
McCrae, R. R., & Costa, P. T. (1995) · European Journal of Personality · doi:10.1002/per.2410090402 · Cited by 426 · see citing works
How Interpersonal Motives Clarify the Meaning of Interpersonal Behavior: A Revised Circumplex Model
Horowitz, L. M., Wilson, K. R., Turan, B., Zolotsev, P., Constantino, M. J., & Henderson, L. (2006) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr1001_4 · Cited by 405 · see citing works
Language style matching in writing: synchrony in essays, correspondence, and poetry.
Ireland, M. E., & Pennebaker, J. W. (2010) · Journal of Personality and Social Psychology · doi:10.1037/a0020386 · Cited by 380 · see citing works
Danescu-Niculescu-Mizil, C., West, R., Jurafsky, D., Leskovec, J., & Potts, C. (2013) · International World Wide Web Conference (WWW) · doi:10.1145/2488388.2488416 · Cited by 342 · see citing works
Language-based personality: a new approach to personality in a digital world
Boyd, R. L., & Pennebaker, J. W. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.07.017 · Cited by 296 · see citing works
Self-citationThe Use of Personal Documents in Psychological Science.
Park, R. E., & Āllport, G. W. (1942) · American Sociological Review · doi:10.2307/2085375 · Cited by 281 · see citing works
Continuum companion to discourse analysis
Paltridge, B., & Hyland, K. (2011) · Cited by 261 · see citing works
Stone, P. J., Bales, R. F., Namenwirth, J. Z., & Ogilvie, D. M. (2007) · Systems Research and Behavioral Science · doi:10.1002/bs.3830070412 · Cited by 220 · see citing works
Collective Emotions and Social Resilience in the Digital Traces After a Terrorist Attack
García, D., & Rimé, B. (2019) · Psychological Science · doi:10.1177/0956797619831964 · Cited by 211 · see citing works
Garten, J., Hoover, J., JOHNSON, K., Boghrati, R., Iskiwitch, C., & Dehghani, M. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0875-9 · Cited by 181 · see citing works
Sounds like a narcissist: Behavioral manifestations of narcissism in everyday life
Holtzman, N. S., Vazire, S., & Mehl, M. R. (2010) · Journal of Research in Personality · doi:10.1016/j.jrp.2010.06.001 · Cited by 180 · see citing works
Personality Research and Assessment in the Era of Machine Learning
Stachl, C., Pargent, F., Hilbert, S., Harari, G. M., Schoedel, R., Vaid, S. S., Gosling, S. D., & Bühner, M. (2020) · European Journal of Personality · doi:10.1002/per.2257 · Cited by 158 · see citing works
Verbal behavior in everyday life
Weintraub, W. (1989) · Cited by 143 · see citing works
Predicting Final Course Performance From Students’ Written Self-Introductions
Robinson, R. L., Navea, R., & Ickes, W. J. (2013) · Journal of Language and Social Psychology · doi:10.1177/0261927x13476869 · Cited by 136 · see citing works
Natural emotion vocabularies as windows on distress and well-being
Vine, V., Boyd, R. L., & Pennebaker, J. W. (2020) · Nature Communications · doi:10.1038/s41467-020-18349-0 · Cited by 136 · see citing works
Self-citationBardi, A., Calogero, R. M., & Mullen, B. R. (2008) · Journal of Applied Psychology · doi:10.1037/0021-9010.93.3.483 · Cited by 123 · see citing works
Linguistic Bases of Social Perception
Berry, D. S., Pennebaker, J. W., Mueller, J., & Hiller, W. S. (1997) · Personality and Social Psychology Bulletin · doi:10.1177/0146167297235008 · Cited by 119 · see citing works
Suicide Risk Assessment with Multi-level Dual-Context Language and
Matero, M., Idnani, A., Son, Y., Giorgi, S., Vu, H., Zamani, M., Limbachiya, P., Guntuku, S. C., & Schwartz, H. A. (2019) · Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology · doi:10.18653/v1/w19-3005 · Cited by 102 · see citing works
Self-citationThe Secret Life of Pronouns. What Our Words Say About Us
Nerbonne, J. (2014) · Literary and Linguistic Computing · doi:10.1093/llc/fqt006 · Cited by 93 · see citing works
No Place Like Home: Home Tours Correlate With Daily Patterns of Mood and Cortisol
Saxbe, D., & Repetti, R. L. (2009) · Personality and Social Psychology Bulletin · doi:10.1177/0146167209352864 · Cited by 73 · see citing works
Use of Social Words in Autobiographies and Longevity
Pressman, S. D., & Cohen, S. (2007) · Psychosomatic Medicine · doi:10.1097/psy.0b013e31803cb919 · Cited by 72 · see citing works
The Nature of Interpersonal Communication: A Message-Centered Approach
Burleson, B. R. (2010) · Sage eBooks · doi:10.4135/9781412982818.n9 · Cited by 72 · see citing works
A personality perspective on business angel syndication✰
Block, J., Fisch, C. O., Obschonka, M., & Sandner, P. (2018) · Journal of Banking & Finance · doi:10.1016/j.jbankfin.2018.10.006 · Cited by 67 · see citing works
Linguistic predictors of post-traumatic stress disorder symptoms following 11 September 2001
D'Andrea, W., Chiu, P. H., Casas, B. R., & Deldin, P. (2012) · Applied Cognitive Psychology · doi:10.1002/acp.1830 · Cited by 62 · see citing works
Linguistic positivity in historical texts reflects dynamic environmental and psychological factors
Iliev, R. I., Hoover, J., Dehghani, M., & Axelrod, R. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1612058113 · Cited by 60 · see citing works
Tong, S. T., Corriero, E. F., Wibowo, K. A., Makki, T. W., & Slatcher, R. B. (2019) · New Media & Society · doi:10.1177/1461444819872678 · Cited by 55 · see citing works
What Words Are Worth: National Science Foundation Grant Abstracts Indicate Award Funding
Markowitz, D. M. (2019) · Journal of Language and Social Psychology · doi:10.1177/0261927x18824859 · Cited by 54 · see citing works
Chung, C. K., & Pennebaker, J. W. (2018) · SAGE Publications Ltd eBooks · doi:10.4135/9781526451163.n16 · Cited by 53 · see citing works
Natural language indicators of differential gene regulation in the human immune system
Mehl, M. R., Raison, C. L., Pace, T. W. W., Arevalo, J. M. G., & Cole, S. W. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1707373114 · Cited by 52 · see citing works
A Weighted Referential Activity Dictionary
Bucci, W., & Maskit, B. (2006) · Springer eBooks · doi:10.1007/1-4020-4102-0_6 · Cited by 49 · see citing works
Linguistic Harbingers of Betrayal: A Case Study on an Online Strategy Game
Niculae, V., Kumar, S., Boyd-Graber, J. L., & Danescu-Niculescu-Mizil, C. (2015) · doi:10.3115/v1/p15-1159 · Cited by 49 · see citing works
The Words that Predict the Outbreak of Wars
Hogenraad, R. L. (2003) · Empirical Studies of the Arts · doi:10.2190/hjwq-qrbx-0c2e-vjya · Cited by 45 · see citing works
The 2016 U.S. Presidential Candidates and How People Tweeted About Them
Jordan, K., Pennebaker, J. W., & Ehrig, C. (2018) · SAGE Open · doi:10.1177/2158244018791218 · Cited by 45 · see citing works
Validating automated integrative complexity: Natural language processing and the Donald Trump Test
Conway, L. G., Conway, K., & Houck, S. C. (2020) · Journal of Social and Political Psychology · doi:10.5964/jspp.v8i2.1307 · Cited by 43 · see citing works
The Personality Panorama: Conceptualizing Personality through Big Behavioural Data
Boyd, R. L., Pasca, P., & Lanning, K. (2020) · European Journal of Personality · doi:10.1002/per.2254 · Cited by 42 · see citing works
Self-citationPersonality development through natural language
Lanning, K., Pauletti, R. E., King, L. A., & McAdams, D. P. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0329-0 · Cited by 37 · see citing works
Bayram, A. B., & Ta, V. P. (2018) · Negotiation and Conflict Management Research · doi:10.1111/ncmr.12142 · Cited by 37 · see citing works
Hablo Inglés y Español: Cultural Self-Schemas as a Function of Language
Rodríguez-Arauz, G., Ramírez‐Esparza, N., Perez‐Brena, N. J., & Boyd, R. L. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00885 · Cited by 32 · see citing works
Self-citationPersonality Research and Assessment in the Era of Machine LearningPreprint
Stachl, C., Pargent, F., Hilbert, S., et al. (2019) · doi:10.31234/osf.io/efnj8 · Cited by 31 · see citing works
Are Words of the Rorschach Predictors of Disease and Death? The Case of “Whirling”
Thomas, C. B., & Duszynski, K. R. (1985) · Psychosomatic Medicine · doi:10.1097/00006842-198503000-00012 · Cited by 26 · see citing works
Deception and Language: The Contextual Organization of Language and Deception (COLD) Framework
Markowitz, D. M., & Hancock, J. T. (2019) · Springer eBooks · doi:10.1007/978-3-319-96334-1_10 · Cited by 22 · see citing works
Interactional Adjustment: Three Approaches in Language and Social Psychology
Gasiorek, J., Weatherall, A., & Watson, B. M. (2020) · Journal of Language and Social Psychology · doi:10.1177/0261927x20965652 · Cited by 22 · see citing works
Smith, M. S. (1968) · Journal of School Psychology · doi:10.1016/0022-4405(68)90017-4 · Cited by 21 · see citing works
Carpenter, J. M., Preoțiuc-Pietro, D., Flek, L., Giorgi, S., Hagan, C. A., Kern, M. L., Buffone, A. E. K., Ungar, L., & Seligman, M. E. P. (2016) · Social Psychological and Personality Science · doi:10.1177/1948550616671998 · Cited by 21 · see citing works
Language Matters: The Double-Edged Role of Linguistic Style Matching in Work Groups
Heuer, K., Müller-Frommeyer, L. C., & Kauffeld, S. (2019) · Small Group Research · doi:10.1177/1046496419874498 · Cited by 21 · see citing works
Comparison of Themes in Folktales by the General Inquirer System
Colby, B. N., Collier, G. A., & Postal, S. K. (1963) · Journal of American Folklore · doi:10.2307/537928 · Cited by 20 · see citing works
Reciprocal language style matching in psychotherapy research
Doorn, K. A., & Müller-Frommeyer, L. C. (2020) · Counselling and Psychotherapy Research · doi:10.1002/capr.12298 · Cited by 19 · see citing works
Development and Application of a Content Analysis Dictionary for Body Boundary Research
Wilson, A. (2005) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqi014 · Cited by 18 · see citing works
Robinson, M. D., Boyd, R. L., Fetterman, A. K., & Persich, M. R. (2016) · Journal of Language and Social Psychology · doi:10.1177/0261927x16668376 · Cited by 17 · see citing works
Self-citationLumma, A., Valk, S. L., Böckler, A., Vrtička, P., & Singer, T. (2018) · Brain and Behavior · doi:10.1002/brb3.940 · Cited by 17 · see citing works
Kranz, P. (1970) · Systems Research and Behavioral Science · doi:10.1002/bs.3830150311 · Cited by 16 · see citing works
National character stereotypes mirror language use: A study of Canadian and American tweets
Snefjella, B., Schmidtke, D., & Kuperman, V. (2018) · PLoS ONE · doi:10.1371/journal.pone.0206188 · Cited by 14 · see citing works
Incorporating Demographic Embeddings Into Language Understanding
Garten, J., Kennedy, B., Hoover, J., Sagae, K., & Dehghani, M. (2019) · Cognitive Science · doi:10.1111/cogs.12701 · Cited by 14 · see citing works
The general inquirer: Useful or not?
Psathas, G. (1969) · Computers and the Humanities · doi:10.1007/bf02401609 · Cited by 13 · see citing works
Martindale, A. E., & Martindale, C. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.55.5.836 · Cited by 11 · see citing works
Laffal, J. (1964) · The Psychoanalytic Quarterly · doi:10.1080/21674086.1964.11926307 · Cited by 10 · see citing works
Developing Latent Semantic Similarity in Initial, Unstructured Interactions
Ta, V. P., Babcock, M. J., & Ickes, W. J. (2016) · Journal of Language and Social Psychology · doi:10.1177/0261927x16638386 · Cited by 9 · see citing works
Using Syntactic and Semantic Context to Explore Psychodemographic Differences in Self-reference
Rouhizadeh, M., Ungar, L., Buffone, A. E. K., & Schwartz, H. A. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1219 · Cited by 8 · see citing works
Self-citationEast-West Conflict and Sino-Soviet Relations
Hoisti, O. R. (1965) · The Journal of Applied Behavioral Science · doi:10.1177/002188636500100202 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Althoff T, Danescu-Niculescu-Mizil C, & Jurafsky D (2014). How to ask for a favor: A case study on the success of altruistic requests. In Adar E, Resnick P, Choudhury MD, Hogan B, & Oh AH(Eds.), Proceedings of the Eighth International Conference on Weblogs and Social Media (pp. 12– 21). The AAAI Press.
- Freud S (1915). The unconscious. Penguin Classics.
- O’Connor B, Bamman D, & Smith NA (2011). Computational text analysis for social science: Model assumptions and complexity. In Mason W, Vaughan JW, & Wallach H(Eds.), Second Workshop on Computational Social Science and Wisdom of the Crowds (pp. 1– 7). Curran Associates.
- Prasad R, Dinesh N, Lee A, Miltsakaki E, Robaldo L, Joshi A, & Webber B (2008, 5). The Penn Discourse TreeBank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08). LREC 2008, Marrakech, Morocco. http://www.lrec-conf.org/proceedings/lrec2008/pdf/754_paper.pdf
Cited by
Sorted by citations
Using large language models in psychology
Demszky, D., Yang, D., Yeager, D. S., Bryan, C. J., Clapper, M., Chandhok, S., Eichstaedt, J. C., Hecht, C., Jamieson, J., Johnson, M., et al. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00241-5 · Cited by 369 · see citing works
From Text to Thought: How Analyzing Language Can Advance Psychological Science
Jackson, J. C., Watts, J., List, J., Puryear, C., Drabble, R., & Lindquist, K. A. (2021) · Perspectives on Psychological Science · doi:10.1177/17456916211004899 · Cited by 183 · see citing works
Large Language Model–Based Responses to Patients’ In-Basket Messages
Small, W. R., Wiesenfeld, B. M., Brandfield-Harvey, B., Jonassen, Z., Mandal, S., Stevens, E. R., Major, V. J., Lostraglio, E., Szerencsy, A. C., Jones, S., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.22399 · Cited by 108 · see citing works
Macanovic, A. (2022) · Social Science Research · doi:10.1016/j.ssresearch.2022.102784 · Cited by 99 · see citing works
Koutsoumpis, A., Oostrom, J. K., Holtrop, D., van Breda, W., Ghassemi, S., & de Vries, R. E. (2022) · Psychological Bulletin · doi:10.1037/bul0000381 · Cited by 95 · see citing works
Artificial Intelligence-Enabled Chatbots in Mental Health: A Systematic Review
Omarov, B. S., Narynov, S., & Zhumanov, Z. (2022) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2023.034655 · Cited by 84 · see citing works
Gillespie, A., Glăveanu, V. P., & Laurent, C. D. S. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009031066 · Cited by 88 · see citing works
Kjell, O., Kjell, K., & Schwartz, H. A. (2023) · Psychiatry Research · doi:10.1016/j.psychres.2023.115667 · Cited by 96 · see citing works
Self-citationAuthentic First Impressions Relate to Interpersonal, Social, and Entrepreneurial Success
Markowitz, D. M., Kouchaki, M., Francesca Gino, B., Hancock, J. T., & Boyd, R. L. (2022) · Social Psychological and Personality Science · doi:10.1177/19485506221086138 · Cited by 65 · see citing works
Self-citationHansen, K., & Świderska, A. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02218-x · Cited by 79 · see citing works
Text Analysis for Psychology: Methods, Principles, and PracticesPreprint
Kennedy, B., Ashokkumar, A., Boyd, R. L., & Dehghani, M. (2021) · doi:10.31234/osf.io/h2b8t · Cited by 57 · see citing works
Self-citationMarkowitz, D. M., Hancock, J. T., & Bailenson, J. N. (2023) · Journal of Language and Social Psychology · doi:10.1177/0261927x231200201 · Cited by 71 · see citing works
The Emergence and Evolution of Consumer Language Research
Packard, G. M., & Berger, J. A. (2022) · Journal of Consumer Research · doi:10.1093/jcr/ucad013 · Cited by 59 · see citing works
Social Media Discussions Predict Mental Health Consultations on College Campuses
Saha, K., Yousuf, A., Boyd, R. L., Pennebaker, J., & Choudhury, M. D. (2022) · Scientific Reports · doi:10.1038/s41598-021-03423-4 · Cited by 68 · see citing works
Self-citationArtificial Intelligence and the Psychology of Human Connection
Boyd, R. L., & Markowitz, D. M. (2026) · Perspectives on Psychological Science · doi:10.1177/17456916251404394 · Cited by 61 · see citing works
Self-citationRaffaelli, Q., Mills, C., de Stefano, N.-A., Mehl, M. R., Chambers, K., Fitzgerald, S. A., Wilcox, R. R., Kalina Christoff, A., Andrews, E. S., Grilli, M. D., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-98138-x · Cited by 45 · see citing works
‘Against the cult of veganism’: Unpacking the social psychology and ideology of anti-vegans
Gregson, R., Piazza, J. R., & Boyd, R. L. (2022) · Appetite · doi:10.1016/j.appet.2022.106143 · Cited by 45 · see citing works
Self-citationGender and ethnicity bias in medicine: a text analysis of 1.8 million critical care records
Markowitz, D. M. (2022) · PNAS Nexus · doi:10.1093/pnasnexus/pgac157 · Cited by 43 · see citing works
Reading dies in complexity: Online news consumers prefer simple writing
Shulman, H. C., Markowitz, D. M., & Rogers, T. (2024) · Science Advances · doi:10.1126/sciadv.adn2555 · Cited by 42 · see citing works
Moreno, J. D., Martínez‐Huertas, J. Á., Olmos, R., Jorge‐Botana, G., & Botella, J. (2021) · Personality and Individual Differences · doi:10.1016/j.paid.2021.110818 · Cited by 45 · see citing works
The Grievance Dictionary: Understanding threatening language use
van der Vegt, I. W. J., Mozes, M., Kleinberg, B., & Gill, P. (2021) · UCL Discovery (University College London) · Cited by 40 · see citing works
Subtle Linguistic Cues Affecting Gender In(Equality)
Formanowicz, M., & Hansen, K. (2021) · Journal of Language and Social Psychology · doi:10.1177/0261927x211035170 · Cited by 40 · see citing works
Tracking group identity through natural language within groups
Ashokkumar, A., & Pennebaker, J. W. (2022) · PNAS Nexus · doi:10.1093/pnasnexus/pgac022 · Cited by 36 · see citing works
Dudău, D. P., & Sava, F. A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.570568 · Cited by 36 · see citing works
Fifty years of research on leader communication: What we know and where we are going
Liu, E. H.-C., Chambers, C. R., & Moore, C. (2023) · The Leadership Quarterly · doi:10.1016/j.leaqua.2023.101734 · Cited by 42 · see citing works
Computational personality assessment
Stachl, C., Boyd, R. L., Horstmann, K. T., Khambatta, P., Matz, S., & Harari, G. M. (2021) · Personality Science · doi:10.5964/ps.6115 · Cited by 42 · see citing works
Self-citationEmotional contagion in online groups as a function of valence and status
Kane, A. A., Swol, L. M. V., & Sarmiento-Lawrence, I. G. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107543 · Cited by 34 · see citing works
A Practical Guide to Conversation Research: How to Study What People Say to Each Other
Yeomans, M., Boland, F. K., Collins, H. K., Abi-Esber, N., & Brooks, A. W. (2023) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459231183919 · Cited by 38 · see citing works
Fyffe, S., Lee, P., & Kaplan, S. A. (2023) · Organizational Research Methods · doi:10.1177/10944281231155771 · Cited by 35 · see citing works
What Gets Shared, and Why? Interpersonal Communication and Word of Mouth
Berger, J. A. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-013024-031524 · Cited by 38 · see citing works
How developments in natural language processing help us in understanding human behaviour
Mihalcea, R. F., Biester, L., Boyd, R. L., Jin, Z., Pérez‐Rosas, V., Wilson, S. R., & Pennebaker, J. W. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01938-0 · Cited by 34 · see citing works
Self-citationKerz, E., Qiao, Y., Zanwar, S., & Wiechmann, D. (2022) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/2022.wassa-1.17 · Cited by 29 · see citing works
Analytic thinking as revealed by function words: What does language really measure?
Markowitz, D. M. (2023) · Applied Cognitive Psychology · doi:10.1002/acp.4057 · Cited by 29 · see citing works
The Social Construction of Narratives and Arguments in Animal Welfare Discourse and Debate
Muhammad, M., Stokes, J. E., Morgans, L. C., & Manning, L. (2022) · Animals · doi:10.3390/ani12192582 · Cited by 28 · see citing works
Psychological trauma and emotional upheaval as revealed in academic writing: The case of COVID-19
Markowitz, D. M. (2021) · Cognition & Emotion · doi:10.1080/02699931.2021.2022602 · Cited by 27 · see citing works
Social Sharing of Emotion During the COVID-19 Pandemic
Kim, D., Park, C., Kim, E., Han, J., & Song, H. (2022) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2021.0270 · Cited by 30 · see citing works
Gronholm, P. C., Kline, S., Lamba, M., Lempp, H. K., Mahkmud, A., Cano, G. M., Vashisht, K., Juan, N. V. S., & Sunkel, C. (2024) · EClinicalMedicine · doi:10.1016/j.eclinm.2024.102509 · Cited by 26 · see citing works
An inclusive, real-world investigation of persuasion in language and verbal behavior
Ta, V. P., Boyd, R. L., Seraj, S., Keller, A., Griffith, C., Loggarakis, A., & Medema, L. (2021) · Journal of Computational Social Science · doi:10.1007/s42001-021-00153-5 · Cited by 26 · see citing works
Self-citationInstrumental goal activation increases online petition support across languages.
Markowitz, D. M. (2022) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000333 · Cited by 20 · see citing works
Word embeddings reveal growing moral concern for people, animals and the environment
Leach, S., Kitchin, A. P., & Sutton, R. M. (2023) · British Journal of Social Psychology · doi:10.1111/bjso.12663 · Cited by 28 · see citing works
de Roos, M. S., Veldhuizen-Ochodničanová, L., & Hanna, A. (2024) · Journal of Interpersonal Violence · doi:10.1177/08862605241239451 · Cited by 24 · see citing works
AI-based analysis of social media language predicts addiction treatment dropout at 90 days
Curtis, B. L., Giorgi, S., Ungar, L., Vu, H., Yaden, D. B., Liu, T., Yadeta, K., & Schwartz, H. A. (2023) · Neuropsychopharmacology · doi:10.1038/s41386-023-01585-5 · Cited by 27 · see citing works
Self-citationComputational social science in smart power systems: Reliability, resilience, and restoration
Valinejad, J., Mili, L., Yu, X., van der Wal, C. N., & Xu, Y. (2023) · Energy Conversion and Economics · doi:10.1049/enc2.12087 · Cited by 26 · see citing works
Tracing the Adoption and Effects of Open Science in Communication Research*
Markowitz, D. M., Song, H., & Taylor, S. H. (2021) · Journal of Communication · doi:10.1093/joc/jqab030 · Cited by 24 · see citing works
Schnepper, R., Roemmel, N., Schaefert, R., Lambrecht-Walzinger, L., & Meinlschmidt, G. (2024) · JMIR Mental Health · doi:10.2196/57986 · Cited by 23 · see citing works
Markowitz, D. M. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107684 · Cited by 22 · see citing works
Moral universals: A machine-reading analysis of 256 societies
Alfano, M., Cheong, M., & Curry, O. S. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e25940 · Cited by 22 · see citing works
Goddard, A., & Gillespie, A. (2023) · Social Science Computer Review · doi:10.1177/08944393231156629 · Cited by 21 · see citing works
Atari, M., Omrani, A., & Dehghani, M. (2023) · doi:10.31234/osf.io/m93pd · Cited by 21 · see citing works
Markowitz, D. M. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02344-0 · Cited by 19 · see citing works
Dirty laundry: The nature and substance of seeking relationship help from strangers online
Entwistle, C., Horn, A. B., Meier, T., & Boyd, R. L. (2021) · Journal of Social and Personal Relationships · doi:10.1177/02654075211046635 · Cited by 19 · see citing works
Self-citationDerner, E., Kučera, D., Oliver, N., & Zahálka, J. (2024) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2024.100088 · Cited by 18 · see citing works
Markowitz, D. M. (2022) · Communication Research · doi:10.1177/00936502221097041 · Cited by 18 · see citing works
Validating the use of large language models for psychological text classification
Bunt, H. L., Goddard, A., Reader, T. W., & Gillespie, A. (2025) · Frontiers in Social Psychology · doi:10.3389/frsps.2025.1460277 · Cited by 19 · see citing works
Speciesism in everyday language
Leach, S., Kitchin, A. P., Sutton, R. M., & Dhont, K. (2022) · British Journal of Social Psychology · doi:10.1111/bjso.12561 · Cited by 16 · see citing works
Radisavljević, D., Rzepka, R., & Araki, K. (2023) · Applied Sciences · doi:10.3390/app13074506 · Cited by 17 · see citing works
Assessing verbal interaction of adult learners in computer‐supported collaborative problem solving
Buseyne, S., Rajagopal, K., Danquigny, T., Depaepe, F., Heutte, J., & Raes, A. (2023) · British Journal of Educational Technology · doi:10.1111/bjet.13391 · Cited by 17 · see citing works
Semantic speech analysis using machine learning and deep learning techniques: a comprehensive review
Tyagi, S., & Szénási, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17769-6 · Cited by 22 · see citing works
The Topics of Nostalgic Recall: The Benefits of Nostalgia Depend on the Topics That One Recalls
Fetterman, A. K., Evans, N. D., Ravey, E. P., Henderson, P. R., Tran, B. H. L., & Boyd, R. L. (2024) · Social Psychological and Personality Science · doi:10.1177/19485506241229305 · Cited by 17 · see citing works
Self-citationIs Sustainability a Liability? Green Marketing and Consumer Beliefs About Eco-Friendly Products
Chernev, A., Blair, S., Böckenholt, U., & Mishra, H. (2021) · Journal of Public Policy & Marketing · doi:10.1177/07439156241264286 · Cited by 17 · see citing works
The construction of emotional meaning in language
Hoemann, K., Lee, Y., Dussault, È., Devylder, S., Ungar, L., Geeraerts, D., & Mesquita, B. (2025) · Communications Psychology · doi:10.1038/s44271-025-00255-0 · Cited by 27 · see citing works
Gillespie, A., & Reader, T. W. (2022) · Risk Analysis · doi:10.1111/risa.14002 · Cited by 16 · see citing works
Diaries as Technologies for Sense-making and Self-transformation in Times of Vulnerability
Marcos, M. J. B., Zittoun, T., & Gillespie, A. (2023) · Integrative Psychological and Behavioral Science · doi:10.1007/s12124-023-09765-0 · Cited by 17 · see citing works
Yahya, N. H., & Rahim, H. A. (2023) · Language and Health · doi:10.1016/j.laheal.2023.10.001 · Cited by 15 · see citing works
Macanovic, A., & Przepiorka, W. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02381-9 · Cited by 17 · see citing works
Pfeifer, V. A., Chilton, T., Grilli, M. D., & Mehl, M. R. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02440-1 · Cited by 15 · see citing works
Lewis, S. C., Markowitz, D. M., & Bunquin, J. B. (2025) · Social Media + Society · doi:10.1177/20563051251325597 · Cited by 20 · see citing works
Kjell, O., Kjell, K., & Schwartz, H. A. (2023) · doi:10.31234/osf.io/yfd8g · Cited by 15 · see citing works
Self-citationYan, Z., Dube, V., Heselton, J., Johnson, K., Yan, C., Jones, V., Blaskewicz Boron, J., & Shade, M. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1329910 · Cited by 14 · see citing works
Style, content, and the success of ideas
Boghrati, R., Berger, J. A., & Packard, G. M. (2023) · Journal of Consumer Psychology · doi:10.1002/jcpy.1346 · Cited by 14 · see citing works
Markowitz, D. M., Kittelman, A., Girvan, E. J., Santiago‐Rosario, M. R., & McIntosh, K. (2023) · Educational Researcher · doi:10.3102/0013189x231189444 · Cited by 14 · see citing works
The chronospatial revolution in psychology
Atari, M., Henrich, J., & Schulz, J. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02229-y · Cited by 15 · see citing works
Tracing the Adoption and Effects of Open Science in Communication ResearchPreprint
Markowitz, D. M., Song, H., & Taylor, S. H. (2021) · doi:10.31234/osf.io/dsf67 · Cited by 14 · see citing works
Li, M. (2020) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2021.104278 · Cited by 13 · see citing works
Classification of Poverty Condition Using Natural Language Processing
Santa, G. M., Escobar-Grisales, D., López-Pabón, F. O., Pérez-Toro, P. A., & Orozco‐Arroyave, J. R. (2022) · Social Indicators Research · doi:10.1007/s11205-022-02883-z · Cited by 13 · see citing works
Abedin, E., Ferreira, M., Reimann, R., Cheong, M., Grossmann, I., & Alfano, M. (2023) · Acta Psychologica · doi:10.1016/j.actpsy.2023.103979 · Cited by 13 · see citing works
Distant Handshakes: Conveying Social Intentions Through Multi-Modal Soft Haptic Gloves
Tong, Q., Wei, W., Guo, Y. C., Jin, T., Wang, Z., Zhang, H., Zhang, Y., & Wang, D. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3438761 · Cited by 14 · see citing works
Gillespie, A., Glăveanu, V. P., Laurent, C. D. S., Zittoun, T., & Marcos, M. J. B. (2024) · Journal of Mixed Methods Research · doi:10.1177/15586898241284696 · Cited by 14 · see citing works
Sikka, P., Valli, K., Revonsuo, A., & Tuominen, J. (2021) · Consciousness and Cognition · doi:10.1016/j.concog.2021.103189 · Cited by 13 · see citing works
Kyröläinen, A.-J., Luke, J., Libben, G., & Kuperman, V. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01740-0 · Cited by 13 · see citing works
Zhao, Y., Zhang, P., & Shi, J. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103113 · Cited by 14 · see citing works
The Fill-Mask Association Test (FMAT): Measuring propositions in natural language.
Bao, H. (2024) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000396 · Cited by 12 · see citing works
Changing channels? A comparison of Fox and MSNBC in 2012, 2016, and 2020
Lanning, K., Wetherell, G. A., Warfel, E. A., & Boyd, R. L. (2021) · Analyses of Social Issues and Public Policy · doi:10.1111/asap.12265 · Cited by 12 · see citing works
Self-citationNatural emotion vocabularies and borderline personality disorder
Entwistle, C., Horn, A. B., Meier, T., Hoemann, K., Miano, A., & Boyd, R. L. (2023) · Journal of Affective Disorders Reports · doi:10.1016/j.jadr.2023.100647 · Cited by 10 · see citing works
Self-citationOrvell, A., Gelman, S. A., & Kross, E. F. (2022) · Social and Personality Psychology Compass · doi:10.1111/spc3.12665 · Cited by 14 · see citing works
PassivePy : A tool to automatically identify passive voice in big text data
Sepehri, A. A., Mirshafiee, M. S., & Markowitz, D. M. (2023) · Journal of Consumer Psychology · doi:10.1002/jcpy.1377 · Cited by 17 · see citing works
Presence and Pronouns: An Exploratory Investigation into the Language of Social VR
DeVeaux, C., Markowitz, D. M., Han, E., Miller, M. R., Hancock, J. T., & Bailenson, J. N. (2024) · Journal of Language and Social Psychology · doi:10.1177/0261927x241248646 · Cited by 10 · see citing works
Kyprianou, C., Vedula, S., Pu, W., & Fitza, M. (2024) · Strategic Entrepreneurship Journal · doi:10.1002/sej.1519 · Cited by 12 · see citing works
Mesquiti, S., Seraj, S., Weyland, A., Ashokkumar, A., Boyd, R. L., Mihalcea, R. F., & Pennebaker, J. W. (2025) · Scientific Reports · doi:10.1038/s41598-025-89165-z · Cited by 10 · see citing works
Self-citationA transparency statement improves trust in community-police interactions
Dobson, K. S. H., Dittmann, A. G., & Yeager, D. S. (2025) · Nature Communications · doi:10.1038/s41467-024-55709-6 · Cited by 21 · see citing works
Simchon, A., Turkin, C., Svoray, T., Kloog, I., Dorman, M., & Gilead, M. (2021) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2021.104221 · Cited by 10 · see citing works
PassivePy: A Tool to Automatically Identify Passive Voice in Big Text DataPreprint
Sepehri, A. A., Mir, M., & Markowitz, D. M. (2022) · doi:10.31234/osf.io/bwp3t · Cited by 10 · see citing works
Beibei, S. (2023) · Journal of Psycholinguistic Research · doi:10.1007/s10936-023-09977-w · Cited by 9 · see citing works
RetractedMarkowitz, D. M. (2024) · Applied Cognitive Psychology · doi:10.1002/acp.4215 · Cited by 9 · see citing works
Content Analysis of Textbooks via Natural Language Processing
Esfahani, M. N. (2024) · American Journal of Education and Practice · doi:10.47672/ajep.2252 · Cited by 9 · see citing works
Why do some academic articles receive more citations from policy communities?
Ma, J., & Cheng, Y. (2024) · Public Administration Review · doi:10.1111/puar.13857 · Cited by 9 · see citing works
Qian, X., & Xu, K. (2024) · Habitat International · doi:10.1016/j.habitatint.2024.103257 · Cited by 11 · see citing works
Rethinking PowerPoint Slide Design for Multimedia Learning
LeFebvre, L., Parsons, M. M., Entwistle, C., Boyd, R. L., & Allen, M. R. (2022) · Journal of Educational Technology Systems · doi:10.1177/00472395221077628 · Cited by 9 · see citing works
Self-citationPolitical ideology shapes risk and benefit judgments of COVID‐19 vaccines
Rubaltelli, E., Dickert, S., Markowitz, D. M., & Slovic, P. (2023) · Risk Analysis · doi:10.1111/risa.14150 · Cited by 9 · see citing works
Text classification models for personality disorders identification
Jain, D., Arora, S., Jha, C. K., & Malik, G. (2024) · Social Network Analysis and Mining · doi:10.1007/s13278-024-01219-8 · Cited by 9 · see citing works
- Authors
- Boyd, R. L., & Schwartz, H. A.
- Affiliations
- Lancaster University (GB)
- Stony Brook University (US)
- Corresponding author
- Boyd, Ryan L.
- Venue
- Journal of Language and Social Psychology
- Publisher
- SAGE Publications
- Published
- 2020-10-28
- Volume
- 40
- Issue
- 1
- Pages
- 21-41
- ISSN
- 0261-927X (ISSN-L)
- 1552-6526
- PMC
- PMC8373026
- OpenAlex
- W3094938719
- Cited by
- 317
- References
- 83
- Citations by year
- 2026: 47 · 2025: 52 · 2024: 82 · 2023: 60 · 2022: 43 · 2021: 29 · 2020: 4
- FWCI
- 37.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Language and Social Psychology (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Lancaster EPrints (Lancaster University) (submittedVersion)
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (196255)
- National Institute on Alcohol Abuse and Alcoholism (R01AA028032)
- Topics
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- natural language analysis, verbal behavior, language and personality, computational text analysis, digital data sources, interdisciplinary research, methodological integration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
