Demographic Factors Improve Classification Performance
Open accessAbstract
Extra-linguistic factors influence language use, and are accounted for by speakers and listeners.Most natural language processing (NLP) tasks to date, however, treat language as uniform.This assumption can harm performance.We investigate the effect of including demographic information on performance in a variety of text-classification tasks.We find that by including age or gender information, we consistently and significantly improve performance over demographic-agnostic models.These results hold across three text-classification tasks in five languages.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Previous studies suggest a relationship between demographic factors, smartphone usage behavior, and depression.
“Previous studies suggest a relationship between demographic factors, smartphone usage behavior, and depression [38,85,86].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,882 · see citing works
The Social Stratification of English in New York City.
Crockett, H. J., & Labov, W. (1968) · American Sociological Review · doi:10.2307/2092899 · Cited by 3,091 · see citing works
(2007) · Cited by 2,018 · see citing works
Domain adaptation with structural correspondence learning
Blitzer, J., McDonald, R., & Pereira, F. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610094 · Cited by 1,543 · see citing works
An Introduction to Sociolinguistics
Holmes, J., & Wilson, N. (2017) · Routledge eBooks · doi:10.4324/9781315728438 · Cited by 932 · see citing works
Domain Adaptation for Statistical Classifiers
Daume III, H., & Marcu, D. (2006) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1872 · Cited by 816 · see citing works
Automatically Categorizing Written Texts by Author Gender
Koppel, M. (2002) · Literary and Linguistic Computing · doi:10.1093/llc/17.4.401 · Cited by 623 · see citing works
Social network and social class: Toward an integrated sociolinguistic model
Milroy, L., & Milroy, J. (1992) · Language in Society · doi:10.1017/s0047404500015013 · Cited by 502 · see citing works
Co-Training for Domain Adaptation
Chen, M., Weinberger, K. Q., & Blitzer, J. C. (2011) · Cited by 374 · see citing works
What to do about bad language on the internet
Eisenstein, J. (2013) · Cited by 343 · see citing works
You're like ‘why not?’ The quotative expressions of Glasgow adolescents
Macaulay, R. K. S. (2001) · Journal of Sociolinguistics · doi:10.1111/1467-9481.00135 · Cited by 232 · see citing works
Holmes, J. (1997) · Journal of Sociolinguistics · doi:10.1111/1467-9481.00012 · Cited by 213 · see citing works
Exploring Demographic Language Variations to Improve Multilingual Sentiment Analysis in Social Media
Volkova, S., Wilson, T., & Yarowsky, D. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1187 · Cited by 191 · see citing works
Author Age Prediction from Text using Linear Regression
Nguyen, D., A. Smith, N., & Rosé, C. P. (2018) · Figshare · doi:10.1184/r1/6473069 · Cited by 178 · see citing works
McKeown, K. R., & Rosenthal, S. (2011) · doi:10.7916/d8w384nb · Cited by 173 · see citing works
Frustratingly Easy Semi-Supervised Domain Adaptation
Daumé, H., Kumar, A., & Saha, A. (2010) · Cited by 170 · see citing works
What's in a Name? Using First Names as Features for Gender Inference in Twitter
Liu, W. J., & Ruths, D. A. (2013) · National Conference on Artificial Intelligence · Cited by 165 · see citing works
(2012) · Cited by 163 · see citing works
Inferring Latent User Properties from Texts Published in Social Media
Volkova, S., Bachrach, Y., Armstrong, M., & Sharma, V. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9271 · Cited by 151 · see citing works
Stylometric analysis of bloggers’ age and gender
Goswami, S., Sarkar, S., & Rustagi, M. (2009) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v3i1.13992 · Cited by 140 · see citing works
Embedding Semantic Similarity in Tree Kernels for Domain Adaptation of Relation Extraction
Plank, B., & Moschitti, A. (2013) · Cited by 125 · see citing works
Discovering Sociolinguistic Associations with Structured Sparsity
Eisenstein, J., Smith, N. A., & Xing, E. P. (2018) · Figshare · doi:10.1184/r1/6475556 · Cited by 124 · see citing works
Girlz II women: Age‐grading, language change and stylistic variation
Rickford, J. R., & Price, M. (2013) · Journal of Sociolinguistics · doi:10.1111/josl.12017 · Cited by 122 · see citing works
Gender Attribution: Tracing Stylometric Evidence Beyond Topic and Genre
Sarawgi, R., Gajulapalli, K., & Choi, Y. (2011) · Cited by 107 · see citing works
User Review Sites as a Resource for Large-Scale Sociolinguistic Studies
Hovy, D., Johannsen, A., & Søgaard, A. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2736277.2741141 · Cited by 98 · see citing works
Self-citationSelf-Training for Enhancement and Domain Adaptation of Statistical Parsers Trained on Small Datasets
Reichart, R., & Rappoport, A. (2007) · Cited by 95 · see citing works
Patterns of age‐based linguistic variation in American English1
Barbieri, F. (2008) · Journal of Sociolinguistics · doi:10.1111/j.1467-9841.2008.00353.x · Cited by 94 · see citing works
Distributed Representations of Geographically Situated Language
Bamman, D., Dyer, C., & Smith, N. A. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-2134 · Cited by 93 · see citing works
Bergsma, S., Dredze, M. H., Durme, B. V., et al. (2013) · Cited by 81 · see citing works
Extremely interesting, very interesting, or only quite interesting? Adverbs and social class
Macaulay, R. K. S. (2002) · Journal of Sociolinguistics · doi:10.1111/1467-9481.00194 · Cited by 76 · see citing works
Mapping Dialectal Variation by Querying Social Media
Doyle, G. (2014) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/v1/e14-1011 · Cited by 76 · see citing works
Extracting discriminative concepts for domain adaptation in text mining
Bo, C., Lam, W. P., Tsang, I. W.-H., & Wong, T.-L. (2009) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1557019.1557045 · Cited by 67 · see citing works
Empirical Evaluation of Profile Characteristics for Gender Classification on Twitter
Alowibdi, J. S., Buy, U. A., & Yu, P. S. (2013) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2013.74 · Cited by 62 · see citing works
Phonological Factors in Social Media Writing
Eisenstein, J. (2013) · Cited by 50 · see citing works
Søgaard, A., Johannsen, A., Plank, B., Hovy, D., & Martínez Alonso, H. (2014) · Proceedings of the Eighteenth Conference on Computational Natural Language Learning · doi:10.3115/v1/w14-1601 · Cited by 39 · see citing works
Self-citationAdapting taggers to Twitter with not-so-distant supervision
Plank, B., Hovy, D., McDonald, R., et al. (2014) · Cited by 34 · see citing works
Self-citationRobust Cross-Domain Sentiment Analysis for Low-Resource Languages
Elming, J., Plank, B., & Hovy, D. (2014) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.3115/v1/w14-2602 · Cited by 12 · see citing works
Self-citationWhat is the meaning of 5 *'s? An investigation of the expression and rating of sentiment
Hardt, D., & Wulff, J. (2012) · CBS Research Portal (Copenhagen Business School) · Cited by 11 · see citing works
The Effect of Age on the Style of Discourse among Japanese Women
Barke, A. (2000) · Institutional Repositories DataBase (IRDB) · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 10
- David Bamman, Jacob Eisenstein, and Tyler Schnoe- belen. 2014b. Gender identity and lexical varia- tion in social media. Journal of Sociolinguistics, 18(2):135-160.
- Federica Barbieri. 2008. Patterns of age-based lin- guistic variation in American English. Journal of sociolinguistics, 12(1):58-88.
- Morgane Ciot, Morgan Sonderegger, and Derek Ruths. 2013. Gender inference of twitter users in non- english contexts. In Proceedings of the 2013 Con- ference on Empirical Methods in Natural Language Processing, Seattle, Wash, pages 18-21.
- Jacob Eisenstein. to appear. Systematic patterning in phonologically-motivated orthographic variation. Journal of Sociolinguistics.
- Janet Holmes. 2013. An introduction to sociolinguis- tics. Routledge.
- Dirk Hovy, Barbara Plank, Héctor Martínez Alonso, and Anders Søgaard. 2015b. Mining for unambigu- ous instances to adapt pos taggers to new domains. In Proceedings of NAACL-HLT.
- Lesley Milroy and James Milroy. 1992. Social net- work and social class: Toward an integrated soci- olinguistic model. Language in society, 21(01):1- 26.
- Dong Nguyen, Dolf Trieschnigg, A. Seza Dogruöz, Ri- lana Gravel, Mariet Theune, Theo Meder, and Fran- ciska De Jong. 2014. Predicting Author Gender and Age from Tweets: Sociolinguistic Theories and Crowd Wisdom. In Proceedings of COLING 2014.
- Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014. Learning sentiment- specific word embedding for twitter sentiment clas- sification. In Proceedings of the 52nd Annual Meet- ing of the Association for Computational Linguis- tics, pages 1555-1565.
- Martijn Wieling, John Nerbonne, and R Harald Baayen. 2011. Quantitative social dialectology: Ex- plaining linguistic variation geographically and so- cially. PloS one, 6(9):e23613.
Cited by
Sorted by citations
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
Kiritchenko, S., & Mohammad, S. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2005 · Cited by 384 · see citing works
The social impact of natural language processing
Hovy, D., & Spruit, S. L. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p16-2096 · Cited by 329 · see citing works
Self-citationOpoku Asare, K., Terhorst, Y., Vega, J., Peltonen, E., Lagerspetz, E., & Ferreira, D. (2021) · JMIR mHealth and uHealth · doi:10.2196/26540 · Cited by 145 · see citing works
Cao, Y. T., & Daumé, H. (2021) · Computational Linguistics · doi:10.1162/coli_a_00413 · Cited by 47 · see citing works
Garimella, A., Banea, C., Hovy, D., & Mihalcea, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1339 · Cited by 41 · see citing works
Self-citationSame Author or Just Same Topic? Towards Content-Independent Style Representations
Wegmann, A., Schraagen, M., & Nguyen, D. (2022) · arXiv · doi:10.48550/arxiv.2204.04907 · Cited by 25
- Authors
- Hovy, D.
- Corresponding author
- Hovy, Dirk
- ORCID
- Hovy, Dirk
- Venue
- Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Published
- 2015
- Pages
- 752-762
- OpenAlex
- W2252241921
- Cited by
- 148
- References
- 40
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 3 · 2023: 9 · 2022: 25 · 2021: 20 · 2020: 12 · 2019: 24 · 2018: 16 · 2017: 18 · 2016: 13 · 2015: 4
- FWCI
- 9.58
- Type
- conference paper
- Language
- en
- Open access
- Licence
- other-oa
- Locations
- Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, other-oa)
- Archivio istituzionale della Ricerca - Bocconi (submittedVersion, cc-by-nc-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- demographic data, text classification, age, gender, language variation, extralinguistic factors, language use, cross-linguistic comparison, demographic bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
