DLATK: Differential Language Analysis ToolKit
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
H. Andrew Schwartz, Salvatore Giorgi, Maarten Sap, Patrick Crutchley, Lyle Ungar, Johannes Eichstaedt. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2017.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 1×
“When used through the interface script, DLATK by default corrects for multiple comparisons using the Benjamini-Hochberg method of FDR correction (Benjamini and Hochberg, 1995).”
MethodScikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 1×
“Where controls are not needed, there are many other options, often less computationally complex, such as TF-IDF , Informative Dirichlet Prior , 3 or classification accuracy metrics like Area Under the ROC Curve (Fawcett, 2006).”
MethodConstruct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195 · cited 1×
“DLATK can provide researchers with enough information to generate hypotheses and clarify the “nomological net” of a construct (Cronbach and Meehl, 1955); That is, help identify the psychological and social processes and constructs that relate to (are sufficiently correlated with) the outcome under…”
BackgroundLinear Statistical Inference and its Applications: Second Editon
Rao, C. R. (1973) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316436 · Cited by 10,489
Statsmodels: Econometric and Statistical Modeling with Python
Seabold, S., & Perktold, J. (2010) · Proceedings of the Python in Science Conference · doi:10.25080/majora-92bf1922-011 · Cited by 6,465 · cited 1×
“…sits on top of many popular open source packages used for data analysis and machine learning (scikit-learn (Pedregosa et al., 2011) and statsmodels (Seabold and Perktold, 2010)) as well as NLP spe-cific packages (Stanford parser (Chen and Man-ning, 2014), TweetNLP (Gimpel et al., 2011) and NLTK…”
BackgroundMultiple comparisons among means
Dunn, O. J. (1961) · Journal of the American Statistical Association · doi:10.1080/01621459.1961.10482090 · Cited by 5,006 · cited 1×
“Other options, such as the more conservative Bon-ferroni correction (Dunn, 1961) are also available.”
MethodApplied Logistic Regression Analysis
(1996) · Technometrics · doi:10.1080/00401706.1996.10484485 · Cited by 3,598
NLTK: the Natural Language Toolkit
Loper, E., & Bird, S. (2002) · Proceedings of the ACL-02 Workshop on Effective tools and methodologies for teaching natural language processing and computational linguistics - · doi:10.3115/1118108.1118117 · Cited by 3,218 · cited 1×
“NLTK (Loper and Bird, 2002)).”
Applied Logistic Regression Analysis
Menard, S. (2002) · Sage eBooks · doi:10.4135/9781412983433 · Cited by 2,989
Cited by
Items 101–130 of 130
Modeling Empathy and Distress in Reaction to News StoriesPreprint
Buechel, S., Buffone, A. E. K., Slaff, B., Ungar, L., & Sedoc, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.10399 · Cited by 37
“The Differential Language Analysis Toolkit (Schwartz et al., 2017) was utilized for this analysis.”
MethodSelf-citationOnline Reviews of Health Care Facilities
Sehgal, N. K. R., Guntuku, S. C., Southwick, L., Merchant, R. M., & Agarwal, A. K. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.24505 · Cited by 8
Situation-Based Interpretable Learning for Personality Prediction in Social Media
Zhang, L., Zhao, L., Zhang, X., Kong, W., Sheng, Z., & Lu, C. (2018) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2018.8622016 · Cited by 7
“They improved their method in another paper [19] and published the toolkit based on open vocabulary [20].”
MethodBig Data, artificial intelligence and the geography of entrepreneurship in the United StatesPreprint
Obschonka, M., Lee, N., Rodríguez‐Pose, A., Eichstaedt, J. C., & Ebert, T. (2018) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/c62tn · Cited by 7
Self-citationQuantifying generalized trust in individuals and counties using language
Giorgi, S., Jones, J., Buffone, A. E. K., Eichstaedt, J., Crutchley, P., Yaden, D., Elstein, J., Zamani, M., Kregor, J., Smith, L. K., et al. (2024) · Frontiers in social psychology · doi:10.3389/frsps.2024.1384262 · Cited by 2
“Since all questions in the Gallup-Sharecare Wellbeing index are not available for each year, we adjusted N r based on the total number of years the question was available ( N r = 200 for < 4 years or N r = 300 for 5-plus years).”
Self-citationAnalyzing Linguistic Differences between Owner and Staff Attributed Tweets
Preoțiuc-Pietro, D., & Marier, R. D. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1274 · Cited by 3
“We use DLATK, which handles social media content and markup such as emoticons or hashtags (Schwartz et al., 2017).”
MethodA Cross-Modal Study of Pain Across Communities in the United States
Aggarwal, A., Rai, S., Giorgi, S., Havaldar, S., Sherman, G. T., Mittal, J., & Guntuku, S. C. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543873.3587642 · Cited by 4
Self-citationQuillivic, R., Auxéméry, Y., Gayraud, F., Dayan, J., & Mesmoudi, S. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf075 · Cited by 3
A Multimodal Analysis of Influencer Content on Twitter
Villegas, D. S., Goanță, C., & Αλέτρας, Ν. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.15 · Cited by 17
Point-of-Interest Type Inference from Social Media TextPreprint
Villegas, D. S., Preoțiuc-Pietro, D., & Αλέτρας, Ν. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.14734 · Cited by 2
“We tokenize text using a Twitter-aware tokenizer (Schwartz et al., 2017).”
MethodKey language markers of depression on social media depend on racePreprint
Rai, S., Stade, E. C., Giorgi, S., Francisco, A., Ungar, L., Curtis, B. L., & Guntuku, S. C. (2023) · doi:10.31234/osf.io/uvamz · Cited by 2
Self-citationThe online language of work-personal conflict
Liou, G., Mittal, J., Sehgal, N. K. R., Tay, L., Ungar, L., & Guntuku, S. C. (2023) · Scientific Reports · doi:10.1038/s41598-023-48193-3 · Cited by 3
Self-citationAnalyzing and Predicting Persistence of News Tweets
Liu, M., Wang, J., & Preoțiuc-Pietro, D. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.46 · Cited by 1
“We tok-enize tweet text using the DLATK Twitter-aware tokenizer (Schwartz et al., 2017).”
MethodNilsson, A., Eijsbroek, V. C., Gu, Z., Kjell, K., Giorgi, S., Kotov, R., Ganesan, A. V., Schwartz, H. A., & Kjell, O. (2026) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459261419036 · Cited by 4
“There exist other packages for advanced language analysis, such as DLATK (Schwartz et al., 2017), Keras (Chollet et al., 2015), and PyTorch (Paszke et al., 2019) in Python, but in this tutorial, we focus on the text package in R, which streamlines these analyses in a user-friendly way tailored for social and behaviora…”
Self-citationLane, J. M., Zhang, X., Alcala, C. S., Midya, V., Nagdeo, K. P., Li, R., & Wright, R. O. (2024) · Preventive Medicine · doi:10.1016/j.ypmed.2024.108081 · Cited by 3
A Computational Linguistic Study of Personal Recovery in Bipolar Disorder
Jagfeld, G. (2019) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/p19-2003 · Cited by 2
“The Differential Language Analysis ToolKit (DLATK) (Schwartz et al., 2017) facilitates social-scientific language analyses, including tools for preprocessing, such as emoticon-aware tokenisers, filtering according to meta data, and analysis, e.”
MethodModeling the Severity of Complaints in Social MediaPreprint
Jin, M., & Αλέτρας, Ν. (2021) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.2103.12428 · Cited by 4
Investigating Parody from Social Media Accounts
Talha, M. A., & Zafar, A. (2021) · South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM) · doi:10.1109/seeda-cecnsm53056.2021.9566233 · Cited by 1
Ganesan, A. V., Varadarajan, V., Mittal, J., Subrahmanya, S., Matero, M., Soni, N., Guntuku, S. C., Eichstaedt, J. C., & Schwartz, H. A. (2022) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/2022.clpsych-1.25 · Cited by 14
“(1) Curr uses only the current message as input features, (2) Curr+Prev uses both current and previous message representations concatenated, (3) Diff uses the difference in representation between the current and the previous messages as shown in figure 1, (4 source python library for language analysis at scale, DLATK…”
MethodSelf-citationWard, G., Schwartz, H. A., Giorgi, S., Menges, J. I., & Matz, S. (2023) · doi:10.31234/osf.io/qfxr9 · Cited by 1
Self-citationQuillivic, R., Gayraud, F., Auxéméry, Y., Vanni, L., Peschanski, D., Eustache, F., Dayan, J., & Mesmoudi, S. (2023) · Research Square · doi:10.21203/rs.3.rs-3747882/v1 · Cited by 1
Automatic implicit motives codings are as accurate as humans’ and 99% fasterPreprint
Nilsson, A., Runge, J. M., Kjell, O., Soni, N., Ganesan, A. V., & Nilsson, C. V. (2024) · doi:10.31234/osf.io/7s6jp · Cited by 1
Extracting Temporally-Anchored Spatial Knowledge
Vempala, A. (2019) · doi:10.12794/metadc1505146
Incremental Validity of Language-Based Assessments of Personality in World Trade Center RespondersPreprint
Oltmanns, J. R., Schwartz, H. A., Ruggero, C. J., Son, Y., Vu, H., Gill, G., Miao, J., Waszczuk, M. A., Clouston, S., Bromet, E. J., et al. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/7graf
Self-citationRegional and Temporal Patterns of COVID-19 Vaccine Misinformation in Sub-Saharan AfricaPreprint
Jamalova, R., Jain, D., Rai, S., Mittal, J., Andy, A., Buttenheim, A. M., & Guntuku, S. C. (2025) · medRxiv · doi:10.1101/2025.08.19.25334033
A new Twitter based credit rating model methodology
Goldmann, L. T., Crook, J. N., & Calabrese, R. (2026) · Annals of Operations Research · doi:10.1007/s10479-026-07067-3
“When using linear or logistic regression 8 for performing DLA the n-grams are represented in terms of their β regression coefficients 9 .”
InfluentialKjell, O., Ganesan, A. V., Boyd, R. L., Oltmanns, J. R., Rivero, A., Feltman, S. M., Carr, M. A., Alves, J., Luft, B. J., Kotov, R., et al. (2026) · Clinical Psychological Science · doi:10.1177/21677026261439026 · Cited by 1
Self-citationNatural language processing of youth speech predicts psychopathology across adolescence
Antonacci, C., Uy, J. P., Kwan, K., Giampetruzzi, E., Jones, S., Pennebaker, J. W., & Gotlib, I. H. (2026) · Nature Mental Health · doi:10.1038/s44220-026-00683-9 · Cited by 1
“To identify latent thematic content within the stress interviews, we utilized LDA 56 using the Differential Language Analysis Toolkit 57 with the MALLET backend 58 .”
Multiple Methods for Visualizing Human Language: A Tutorial for Social and Behavioral Scientists
Eijsbroek, V. C., Nilsson, A., Ackermann, L., Gu, Z., Wiebel, C., Ganesan, A. V., Kjell, K., Schwartz, H. A., & Kjell, O. (2026) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459261454622
Self-citation
Metadata
- Authors
- Schwartz, H. A., Giorgi, S., Sap, M., Crutchley, P., Ungar, L., & Eichstaedt, J.
- Affiliations
- University of Washington (US)
- Stony Brook University (US)
- Corresponding author
- Giorgi, Salvatore
- ORCID
- Venue
- Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
- Published
- 2017
- Pages
- 55-60
- OpenAlex
- W2757124792
- Semantic Scholar
- 038ad68cae08893958f6c1285dd924df74565372
- Cited by
- 130
- References
- 40
- Citations by year
- 2026: 6 · 2025: 6 · 2024: 11 · 2023: 25 · 2022: 16 · 2021: 19 · 2020: 15 · 2019: 12 · 2018: 19 · 2017: 1
- FWCI
- 5.86
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) (publishedVersion, cc-by)
- Funders
- University of Pennsylvania
- Templeton Religion Trust (TRT0048)
- Stony Brook University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- language change, language variation, corpus comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.