The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Obtaining Reliable Human Ratings of Valence, Arousal, and Dominance for 20,000 English Words
Open accessAbstract
Words play a central role in language and thought.Factor analysis studies have shown that the primary dimensions of meaning are valence, arousal, and dominance (VAD).We present the NRC VAD Lexicon, which has human ratings of valence, arousal, and dominance for more than 20,000 English words.We use Best-Worst Scaling to obtain fine-grained scores and address issues of annotation consistency that plague traditional rating scale methods of annotation.We show that the ratings obtained are vastly more reliable than those in existing lexicons.We also show that there exist statistically significant differences in the shared understanding of valence, arousal, and dominance across demographic variables such as age, gender, and personality.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Russell, J. A. (1980) · Journal of Personality and Social Psychology · doi:10.1037/h0077714 · Cited by 16,334 · cited 1×
“The paradigm words were taken from past literature on VAD (Bradley and Lang, 1999; Osgood et al., 1957; Russell, 1980).”
MethodOsgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957) · Cited by 9,433
THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE: A META‐ANALYSIS
BARRICK, M. R., & MOUNT, M. K. (1991) · Personnel Psychology · doi:10.1111/j.1744-6570.1991.tb00688.x · Cited by 8,689
Core affect and the psychological construction of emotion.
Russell, J. A. (2003) · Psychological Review · doi:10.1037/0033-295x.110.1.145 · Cited by 6,113
A law of comparative judgment.
Thurstone, L. L. (1927) · Psychological Review · doi:10.1037/h0070288 · Cited by 5,210
CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON
Mohammad, S. M., & Turney, P. D. (2013) · Computational Intelligence · doi:10.1111/j.1467-8640.2012.00460.x · Cited by 2,634
Self-citationMcLean, C. P., Asnaani, A., Litz, B. T., & Hofmann, S. G. (2011) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2011.03.006 · Cited by 2,519
Norms of valence, arousal, and dominance for 13,915 English lemmas
Warriner, A. B., Kuperman, V., & Brysbaert, M. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0314-x · Cited by 1,959
Questions and answers in attitude surveys: Experiments on question form, wording, and context
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90054-w · Cited by 1,713
(1999) · Cited by 1,435
A GENERAL PSYCHOEVOLUTIONARY THEORY OF EMOTION
PLUTCHIK, R. (1980) · Theories of Emotion · doi:10.1016/b978-0-12-558701-3.50007-7 · Cited by 1,370
The Method of Paired Comparisons.
Curnow, R. N., & David, H. A. (1964) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344014 · Cited by 1,271
Response Styles in Marketing Research: A Cross-National Investigation
Baumgartner, H., & Steenkamp, J.-B. E. M. (2001) · Journal of Marketing Research · doi:10.1509/jmkr.38.2.143.18840 · Cited by 1,255
The General Inquirer: A Computer Approach to Content Analysis
Green, B. F., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvie, D. M. (1967) · American Educational Research Journal · doi:10.2307/1161774 · Cited by 1,013
SemEval-2018 Task 1: Affect in Tweets
Mohammad, S., Bravo-Marquez, F., Salameh, M., & Kiritchenko, S. (2018) · Proceedings of The 12th International Workshop on Semantic Evaluation · doi:10.18653/v1/s18-1001 · Cited by 746
Self-citationGender differences in anxiety disorders and anxiety symptoms in adolescents.
Lewinsohn, P. M., Gotlib, I. H., Lewinsohn, M., Seeley, J. R., & Allen, N. B. (1998) · Journal of Abnormal Psychology · doi:10.1037//0021-843x.107.1.109 · Cited by 741
Johnson, E. O., Roth, T., Schultz, L., & Breslau, N. (2006) · Pediatrics · doi:10.1542/peds.2004-2629 · Cited by 616 · cited 2×
“than men (Bao and Swaab, 2011; Lewinsohn et al., 1998; McLean et al., 2011; Johnson et al., 2006; Chmielewski et al., 1995).”
BackgroundThe Berlin Affective Word List Reloaded (BAWL-R)
Võ, M. L. H., Conrad, M., Kuchinke, L., Urton, K., Hofmann, M. J., & Jacobs, A. M. (2009) · Behavior Research Methods · doi:10.3758/brm.41.2.534 · Cited by 525
Best-Worst Scaling: Theory, Methods and Applications
Louviere, J. J., Flynn, T. N., & Marley, A. A. J. (2015) · Thammasat University Digital Collections · Cited by 435
Pleasure, Arousal, Dominance: Mehrabian and Russell revisited
Bakker, I., van der Voordt, T., Vink, P., & de Boon, J. (2014) · Current Psychology · doi:10.1007/s12144-014-9219-4 · Cited by 415
Using Hashtags to Capture Fine Emotion Categories from Tweets
Mohammad, S. M., & Kiritchenko, S. (2015) · Computational Intelligence · doi:10.1111/coin.12024 · Cited by 408
Self-citationBao, A.-M., & Swaab, D. F. (2011) · Frontiers in Neuroendocrinology · doi:10.1016/j.yfrne.2011.02.007 · Cited by 378
Mohammad, S. M. (2012) · Joint Conference on Lexical and Computational Semantics · Cited by 355
Self-citationThe Spanish adaptation of ANEW (Affective Norms for English Words)
Redondo, J., Fraga, I., Padrón, I., & Comesaña, M. (2007) · Behavior Research Methods · doi:10.3758/bf03193031 · Cited by 345
Norms of valence, arousal, dominance, and age of acquisition for 4,300 Dutch words
Moors, A., De Houwer, J., Hermans, D., Wanmaker, S., van Schie, K., Van Harmelen, A.-L., De Schryver, M., De Winne, J., & Brysbaert, M. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0243-8 · Cited by 297
WASSA-2017 Shared Task on Emotion Intensity
Mohammad, S., & Bravo-Marquez, F. (2017) · Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis · doi:10.18653/v1/w17-5205 · Cited by 270
Self-citationDepeche Mood: a Lexicon for Emotion Analysis from Crowd Annotated News
Staiano, J., & Guerini, M. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.3115/v1/p14-2070 · Cited by 210
SemEval-2012 Task 2 : measuring degrees of relational similarity
Jurgens, D., Turney, P. D., Mohammad, S. M., & Holyoak, K. J. (2012) · NPARC · Cited by 182
Self-citationBest-Worst Scaling More Reliable than Rating Scales: A Case Study on Sentiment Intensity Annotation
Kiritchenko, S., & Mohammad, S. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p17-2074 · Cited by 170
Self-citationBuilding emotion lexicon from weblog corpora
Yang, C., Lin, K. H.-Y., & Chen, H.-H. (2007) · Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions - ACL '07 · doi:10.3115/1557769.1557809 · Cited by 164
Capturing Reliable Fine-Grained Sentiment Associations by Crowdsourcing and Best–Worst Scaling
Kiritchenko, S., & Mohammad, S. M. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1095 · Cited by 126
Self-citationBest-worst scaling: theory and methods
Marley, A. A. J. (2024) · Edward Elgar Publishing eBooks · doi:10.4337/9781800375635.00014 · Cited by 102
Understanding Emotions: A Dataset of Tweets to Study Interactions between Affect Categories
Mohammad, S. M., & Kiritchenko, S. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5or8sy3gqobf · Cited by 99
Self-citationEthnicity and gender in scales of psychosis proneness and mood disorders.
Chmielewski, P. M., Fernandes, L. O. L., Yee, C. M., & Miller, G. A. (1995) · Journal of Abnormal Psychology · doi:10.1037//0021-843x.104.3.464 · Cited by 98
Embracing Ambiguity: A Comparison of Annotation Methodologies for Crowdsourcing Word Sense Labels
Jurgens, D. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 71
Predicting Valence-Arousal Ratings of Words Using a Weighted Graph Method
Yu, L.-C., Wang, J., Lai, K.-H. R., & Zhang, X. (2015) · Annual Meeting of the Association for Computational Linguistics · doi:10.3115/v1/p15-2129 · Cited by 49
Swaab, D. F., & Bao, A. (2012) · Springer eBooks · doi:10.1007/978-1-4614-1997-6_115 · Cited by 37
Cited by
GoEmotions: A Dataset of Fine-Grained Emotions
Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., & Ravi, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.372 · Cited by 873
GoEmotions: A Dataset of Fine-Grained Emotions
Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., & Ravi, S. (2020) · arXiv · doi:10.48550/arxiv.2005.00547 · Cited by 873
SemEval-2018 Task 1: Affect in Tweets
Mohammad, S., Bravo-Marquez, F., Salameh, M., & Kiritchenko, S. (2018) · Proceedings of The 12th International Workshop on Semantic Evaluation · doi:10.18653/v1/s18-1001 · Cited by 746
Self-citationTransformer models for text-based emotion detection: a review of BERT-based approaches
Acheampong, F. A., Nunoo-Mensah, H., & Chen, W. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-09958-2 · Cited by 659
Social IQa: Commonsense Reasoning about Social Interactions
Sap, M., Rashkin, H., Chen, D., Le Bras, R., & Choi, Y. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1454 · Cited by 473
Metadata
- Authors
- Mohammad, S.
- Venue
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2018
- Pages
- 174-184
- OpenAlex
- W2798357113
- Cited by
- 613
- References
- 37
- FWCI
- 56.70
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.