DLATK: Differential Language Analysis ToolKit
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
H. Andrew Schwartz, Salvatore Giorgi, Maarten Sap, Patrick Crutchley, Lyle Ungar, Johannes Eichstaedt. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2017.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 1×
“When used through the interface script, DLATK by default corrects for multiple comparisons using the Benjamini-Hochberg method of FDR correction (Benjamini and Hochberg, 1995).”
MethodScikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · cited 1×
“Where controls are not needed, there are many other options, often less computationally complex, such as TF-IDF , Informative Dirichlet Prior , 3 or classification accuracy metrics like Area Under the ROC Curve (Fawcett, 2006).”
MethodConstruct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195 · cited 1×
“DLATK can provide researchers with enough information to generate hypotheses and clarify the “nomological net” of a construct (Cronbach and Meehl, 1955); That is, help identify the psychological and social processes and constructs that relate to (are sufficiently correlated with) the outcome under…”
BackgroundLinear Statistical Inference and its Applications: Second Editon
Rao, C. R. (1973) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316436 · Cited by 10,489
Statsmodels: Econometric and Statistical Modeling with Python
Seabold, S., & Perktold, J. (2010) · Proceedings of the Python in Science Conference · doi:10.25080/majora-92bf1922-011 · Cited by 6,465 · cited 1×
“…sits on top of many popular open source packages used for data analysis and machine learning (scikit-learn (Pedregosa et al., 2011) and statsmodels (Seabold and Perktold, 2010)) as well as NLP spe-cific packages (Stanford parser (Chen and Man-ning, 2014), TweetNLP (Gimpel et al., 2011) and NLTK…”
BackgroundMultiple comparisons among means
Dunn, O. J. (1961) · Journal of the American Statistical Association · doi:10.1080/01621459.1961.10482090 · Cited by 5,006 · cited 1×
“Other options, such as the more conservative Bon-ferroni correction (Dunn, 1961) are also available.”
MethodApplied Logistic Regression Analysis
(1996) · Technometrics · doi:10.1080/00401706.1996.10484485 · Cited by 3,598
NLTK: the Natural Language Toolkit
Loper, E., & Bird, S. (2002) · Proceedings of the ACL-02 Workshop on Effective tools and methodologies for teaching natural language processing and computational linguistics - · doi:10.3115/1118108.1118117 · Cited by 3,218 · cited 1×
“NLTK (Loper and Bird, 2002)).”
Applied Logistic Regression Analysis
Menard, S. (2002) · Sage eBooks · doi:10.4135/9781412983433 · Cited by 2,989
Statistical Methods for the Social Sciences
EJD, Agresti, A., & Finlay, B. (1998) · Journal of the American Statistical Association · doi:10.2307/2670147 · Cited by 2,120 · cited 1×
“Primary metrics are based on Pearson Product-Moment Correlation Coefficient (Agresti and Finlay, 2008).”
MethodNLTK: The Natural Language Toolkit
Loper, E., & Bird, S. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0205028 · Cited by 1,844 · cited 1×
“NLTK (Loper and Bird, 2002)).”
A Fast and Accurate Dependency Parser using Neural Networks
Chen, D., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1082 · Cited by 1,795 · cited 1×
“, 2011) and statsmodels (Seabold and Perktold, 2010)) as well as NLP specific packages (Stanford parser (Chen and Manning, 2014), TweetNLP (Gimpel et al.”
BackgroundPersonality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Shah, A., Kosinski, M., Stillwell, D., Seligman, M. E. P., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073791 · Cited by 1,700 · cited 5×
“If these words are ambiguous, interpretations of dictionary-based results can be misleading (Schwartz et al., 2013a).”
BackgroundMethodInfluentialSelf-citationpandas: a Foundational Python Library for Data Analysis and Statistics
McKinney, W. (2011) · Cited by 1,264
The general inquirer: A computer approach to content analysis.
Smith, M. S., Stone, P. J., & Dunphy, D. C. (1966) · Medical Entomology and Zoology · Cited by 1,165
The General Inquirer: A Computer Approach to Content Analysis
Green, B. F., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvie, D. M. (1967) · American Educational Research Journal · doi:10.2307/1161774 · Cited by 1,013 · cited 1×
“According to citations, the most popular tool is Linguistic Inquiry and Word Count (LIWC) (Pen-nebaker et al., 2015), followed by DICTION (Hart, 1984) and the General Inquirer (Stone et al., 1966).”
MethodAutomatic personality assessment through social media language.
Park, G., Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Kosinski, M., Stillwell, D. J., Ungar, L. H., & Seligman, M. E. P. (2015) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000020 · Cited by 802 · cited 1×
“DLATK has been used as a data analysis platform in over 30 peer-reviewed publications, with venues ranging from general-interest (PLoS ONE: Schwartz et al., 2013b) to computer science meth-ods proceedings (EMNLP: Sap et al., 2014) to psychology journals (JPSP: Park et al., 2015).”
MethodSelf-citationPart-of-Speech Tagging for Twitter: Annotation, Features, and Experiments
Gimpel, K., Schneider, N., O'Connor, B., Das, D., Mills, D. P., Eisenstein, J., Heilman, M., Yogatama, D., Flanigan, J., & Smith, N. A. (2018) · Figshare · doi:10.1184/r1/6473609.v1 · Cited by 800
Quantifying Mental Health Signals in Twitter
Coppersmith, G., Dredze, M., & Harman, C. (2014) · Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality · doi:10.3115/v1/w14-3207 · Cited by 722 · cited 1×
“Much of this work has been done using Facebook and Twitter (Coppersmith et al., 2014).”
Multiple Comparisons Among Means
Dunn, O. J. (1961) · Journal of the American Statistical Association · doi:10.2307/2282330 · Cited by 710
Effects of Age and Gender on Blogging
Schler, J., Koppel, M., Argamon, S., & Pennebaker, J. W. (2005) · National Conference on Artificial Intelligence · Cited by 601
Monroe, B. L., Colaresi, M. P., & Quinn, K. M. (2008) · Political Analysis · doi:10.1093/pan/mpn018 · Cited by 542
Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments
Gimpel, K., Schneider, N., O'Connor, B., Das, D., Mills, D., Eisenstein, J., Heilman, M., Yogatama, D., Flanigan, J., & Smith, N. A. (2010) · doi:10.21236/ada547371 · Cited by 491 · cited 2×
“DLATK sits on top of many popular open source packages used for data analysis and machine learning (scikit-learn (Pedregosa et al., 2011) and statsmodels (Seabold and Perktold, 2010)) as well as NLP spe-cific packages (Stanford parser (Chen and Man-ning, 2014), TweetNLP (Gimpel et al., 2011) and NLTK (Loper and Bird, 2…”
BackgroundPsychological Language on Twitter Predicts County-Level Heart Disease Mortality
Eichstaedt, J. C., Schwartz, H. A., Kern, M. L., Park, G., Labarthe, D. R., Merchant, R. M., Jha, S., Agrawal, M., Dziurzynski, L. A., Sap, M., et al. (2015) · Psychological Science · doi:10.1177/0956797614557867 · Cited by 486 · cited 2×
“, 2014), or predicting community-level health outcomes (e.g., heart disease mortality; Eichstaedt et al., 2015).”
BackgroundSelf-citationTowards Assessing Changes in Degree of Depression through Facebook
Schwartz, H. A., Eichstaedt, J., Kern, M. L., Park, G., Sap, M., Stillwell, D., Kosinski, M., & Ungar, L. (2014) · Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality · doi:10.3115/v1/w14-3214 · Cited by 256 · cited 1×
“…predicting aspects of single messages (e.g., tweet-wise temporal orientation; Schwartz et al., 2015), or predicting user-level attributes (e.g., severity of depression; Schwartz et al., 2014), or predicting community-level health outcomes (e.g., heart disease mortality; Eichstaedt et al., 2015).”
BackgroundSelf-citationDeveloping Age and Gender Predictive Lexica over Social Media
Sap, M., Park, G., Eichstaedt, J., Kern, M., Stillwell, D., Kosinski, M., Ungar, L., & Schwartz, H. A. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1121 · Cited by 230 · cited 1×
“DLATK has been used as a data analysis platform in over 30 peer-reviewed publications, with venues ranging from general-interest (PLoS ONE: Schwartz et al., 2013b) to computer science meth-ods proceedings (EMNLP: Sap et al., 2014) to psychology journals (JPSP: Park et al., 2015).”
MethodSelf-citationModelling Valence and Arousal in Facebook posts
Preoţiuc-Pietro, D., Schwartz, H. A., Park, G., Eichstaedt, J., Kern, M., Ungar, L., & Shulman, E. (2016) · Proceedings of the 7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis · doi:10.18653/v1/w16-0404 · Cited by 165
Self-citationWomen are Warmer but No Less Assertive than Men: Gender and Language on Facebook
Park, G., Yaden, D. B., Schwartz, H. A., Kern, M. L., Eichstaedt, J. C., Kosinski, M., Stillwell, D., Ungar, L. H., & Seligman, M. E. P. (2016) · PLOS ONE · doi:10.1371/journal.pone.0155885 · Cited by 158 · cited 1×
“Other works to primarily use DLATK for correlation-type analyses examine outcomes like age (Kern et al., 2014b), gendered language and stereotypes (Park et al., 2016; Carpenter et al., 2016b), and efficacy of app-based well-being interventions (Carpenter et al., 2016a).”
MethodSelf-citationThe Online Social Self: An Open Vocabulary Approach to Personality
Kern, M. L., Eichstaedt, J. C., Schwartz, H. A., Dziurzynski, L., Ungar, L. H., Stillwell, D. J., Kosinski, M., Ramones, S. M., & Seligman, M. E. P. (2014) · Assessment · doi:10.1177/1073191113514104 · Cited by 147
Self-citationJavkin, H. R. (1985) · Language · doi:10.2307/414438 · Cited by 135
Carpenter, J., Crutchley, P., Zilca, R. D., Schwartz, H. A., Smith, L. K., Cobb, A. M., & Parks, A. C. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5725 · Cited by 60 · cited 1×
“Other works to primarily use DLATK for correlation-type analyses examine outcomes like age (Kern et al., 2014b), gendered language and stereotypes (Park et al., 2016; Carpenter et al., 2016b), and efficacy of app-based well-being interventions (Carpenter et al., 2016a).”
MethodSelf-citationVerbal Style and the Presidency: A Computer-Based Analysis.
Stone, P. J., & Hart, R. P. (1986) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2070923 · Cited by 59 · cited 2×
“According to citations, the most popular tool is Linguistic Inquiry and Word Count (LIWC) (Pen-nebaker et al., 2015), followed by DICTION (Hart, 1984) and the General Inquirer (Stone et al., 1966).”
MethodLinear Statistical Inference and its Applications.
Fienberg, S. E., & Rao, C. R. (1975) · Journal of the American Statistical Association · doi:10.2307/2285975 · Cited by 58
From “Sooo excited!!!” to “So proud”: Using language to study development.
Kern, M. L., Eichstaedt, J. C., Schwartz, H. A., Park, G., Ungar, L. H., Stillwell, D. J., Kosinski, M., Dziurzynski, L., & Seligman, M. E. P. (2014) · Developmental Psychology · doi:10.1037/a0035048 · Cited by 56 · cited 2×
“Many of the outcomes (or dependent variables) are continuous (e.g. scores on personality tests), and researchers are often as interested in interpretable insights as they are with predictive accuracy (Kern et al., 2014a).”
BackgroundMethodSelf-citationMental Illness Detection at the World Well-Being Project for the CLPsych 2015 Shared Task
Preoţiuc-Pietro, D., Sap, M., Schwartz, H. A., & Ungar, L. (2015) · Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality · doi:10.3115/v1/w15-1205 · Cited by 54
Self-citationCarpenter, J. M., Preoțiuc-Pietro, D., Flek, L., Giorgi, S., Hagan, C. A., Kern, M. L., Buffone, A. E. K., Ungar, L., & Seligman, M. E. P. (2016) · Social Psychological and Personality Science · doi:10.1177/1948550616671998 · Cited by 31 · cited 1×
“, 2014b), gendered language and stereotypes (Park et al., 2016; Carpenter et al., 2016b), and efficacy of app-based well-being interventions (Carpenter et al.”
BackgroundSelf-citationChoosing the Right Words: Characterizing and Reducing Error of the Word Count Approach
Schwartz, H. A., Eichstaedt, J. C., Blanco, E., DziurzyÅ„ski, L., Kern, M. L., Ramones, S. M., Seligman, M. E. P., & Ungar, L. (2013) · Cited by 21
Self-citationDoes ‘well-being’ translate on Twitter?
Smith, L. K., Giorgi, S., Solanki, R., Eichstaedt, J. C., Schwartz, H. A., Abdul-Mageed, M., Buffone, A. E. K., & Ungar, L. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1217 · Cited by 20 · cited 1×
“Though not specifically designed to be language independent, DLATK has been used in one non-English study (Smith et al., 2016).”
MethodSelf-citationExtracting Human Temporal Orientation from Facebook Language
Schwartz, H. A., Park, G., Sap, M., Weingarten, E., Eichstaedt, J., Kern, M., Stillwell, D., Kosinski, M., Berger, J., Seligman, M., et al. (2015) · Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.3115/v1/n15-1044 · Cited by 13 · cited 1×
“DLATK works to create models at multiple scales, i.e., for predicting aspects of single messages (e.g., tweet-wise temporal orientation; Schwartz et al., 2015), or predicting user-level attributes (e.g., severity of depression; Schwartz et al., 2014), or predicting community-level health outcomes…”
BackgroundSelf-citation
Cited by
Transformer models for text-based emotion detection: a review of BERT-based approaches
Acheampong, F. A., Nunoo-Mensah, H., & Chen, W. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-09958-2 · Cited by 659
Using large language models in psychology
Demszky, D., Yang, D., Yeager, D. S., Bryan, C. J., Clapper, M., Chandhok, S., Eichstaedt, J. C., Hecht, C., Jamieson, J., Johnson, M., et al. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00241-5 · Cited by 369
“This is a Linux-based library that provides a user-friendly interface to many natural language processing Python libraries, including those that provide access to LLMs 113 .”
MethodSelf-citationTrotzek, M., Koitka, S., & Friedrich, C. M. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2885515 · Cited by 226
“With a similar purpose, Differential Language Analysis Toolkit (DLATK) [22], an open-source Python library, was created for text analysis with a psychological, health, or social focus.”
MethodJaidka, K., Giorgi, S., Schwartz, H. A., Kern, M. L., Ungar, L. H., & Eichstaedt, J. C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1906364117 · Cited by 228
“In the primary analysis, we determined the convergent validity between the language-based methods and the Gallup county-level outcomes using an open-source Python codebase (29).”
MethodSelf-citationUnderstanding and Measuring Psychological Stress Using Social Media
Chandra Guntuku, S., Buffone, A., Jaidka, K., Eichstaedt, J. C., & Ungar, L. H. (2019) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v13i01.3223 · Cited by 133
Self-citation
Metadata
- Authors
- Schwartz, H. A., Giorgi, S., Sap, M., Crutchley, P., Ungar, L., & Eichstaedt, J.
- Affiliations
- University of Washington (US)
- Stony Brook University (US)
- Corresponding author
- Giorgi, Salvatore
- ORCID
- Venue
- Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
- Published
- 2017
- Pages
- 55-60
- OpenAlex
- W2757124792
- Semantic Scholar
- 038ad68cae08893958f6c1285dd924df74565372
- Cited by
- 130
- References
- 40
- Citations by year
- 2026: 6 · 2025: 6 · 2024: 11 · 2023: 25 · 2022: 16 · 2021: 19 · 2020: 15 · 2019: 12 · 2018: 19 · 2017: 1
- FWCI
- 5.86
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) (publishedVersion, cc-by)
- Funders
- University of Pennsylvania
- Templeton Religion Trust (TRT0048)
- Stony Brook University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Keywords
- language change, language variation, corpus comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.