Checking and bootstrapping lexical norms by means of word similarity indexes
Open accessAbstract
In psychology, lexical norms related to the semantic properties of words, such as concreteness and valence, are important research resources. Collecting such norms by asking judges to rate the words is very time consuming, which strongly limits the number of words that compose them. In the present article, we present a technique for estimating lexical norms based on the latent semantic analysis of a corpus. The analyses conducted emphasize the technique's effectiveness for several semantic dimensions. In addition to the extension of norms, this technique can be used to check human ratings to identify words for which the rating is very different from the corpus-based estimate.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Research aiming to expand the breadth of norming studies by predicting responses for unpresented items will be able to provide more accurate predictions, without simply reproducing artefacts of response patterns, if the models predict latent means rather than Likert means.
“Similarly, research which aims to expand the breadth of norming studies by predicting responses for unpresented items, for example via latent semantic analysis (Bestgen & Vincze, 2012), will be able to provide more accurate predictions, without simply reproducing artefacts of response patterns, if the models predict latent means rather than Likert means.”
From Rating norms should be calculated from cumulative link mixed effects models (2022) · cites this paper as Bestgen & Vincze, 2012
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Indexing by latent semantic analysis
Deerwester, S. C., Dumais, S., Furnas, G. W., Landauer, T. K., & Harshman, R. A. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6 · Cited by 12,758 · see citing works
Cross validatory choice and assessment of statistical predictions
Stone, M. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00994.x · Cited by 10,881 · see citing works
Opinion Mining and Sentiment Analysis
Pang, B., & Lee, L. (2008) · now publishers, Inc. eBooks · doi:10.1561/9781601981516 · Cited by 6,957 · see citing works
Opinion mining and sentiment analysis
Pang, B., & Lee, L. (2008) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000011 · Cited by 6,372 · see citing works
An introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880 · see citing works
Introduction to WordNet: An On-line Lexical Database *
Miller, G. A., Beckwith, R., Fellbaum, C., Gross, D., & Miller, K. J. (1990) · International Journal of Lexicography · doi:10.1093/ijl/3.4.235 · Cited by 4,197 · see citing works
Psychological Aspects of Natural Language Use: Our Words, Our Selves
Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003) · Annual Review of Psychology · doi:10.1146/annurev.psych.54.101601.145041 · Cited by 2,624 · see citing works
SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining
Esuli, A., & Sebastiani, F. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/54vo5wjya4cw · Cited by 2,494 · see citing works
Producing high-dimensional semantic spaces from lexical co-occurrence
Lund, K., & Burgess, C. (1996) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03204766 · Cited by 1,808 · see citing works
Handbook of Latent Semantic Analysis
(2007) · Psychology Press eBooks · doi:10.4324/9780203936399 · Cited by 1,571 · see citing works
Measuring praise and criticism
Turney, P. D., & Littman, M. L. (2003) · ACM Transactions on Information Systems · doi:10.1145/944012.944013 · Cited by 1,515 · see citing works
Estimation of Error Rates in Discriminant Analysis
Lachenbruch, P. A., & Mickey, M. R. (1968) · Technometrics · doi:10.1080/00401706.1968.10490530 · Cited by 1,508 · see citing works
Determining the sentiment of opinions
Kim, S.-M., & Hovy, E. H. (2004) · International Conference on Computational Linguistics · doi:10.3115/1220355.1220555 · Cited by 1,499 · see citing works
Predicting the semantic orientation of adjectives
Hatzivassiloglou, V., & McKeown, K. R. (1997) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/976909.979640 · Cited by 1,445 · see citing works
Topics in semantic representation
Griffiths, T. L., Steyvers, M., & Tenenbaum, J. B. (2007) · Psychological Review · doi:10.1037/0033-295x.114.2.211 · Cited by 1,122 · see citing works
The General Inquirer: A Computer Approach to Content Analysis
Green, B. F., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvie, D. M. (1967) · American Educational Research Journal · doi:10.2307/1161774 · Cited by 1,013 · see citing works
Visual Word Recognition of Single-Syllable Words.
Balota, D. A., Cortese, M. J., Sergent-Marshall, S. D., Spieler, D. H., & Yap, M. J.-M. (2004) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.133.2.283 · Cited by 919 · see citing works
The representation of abstract words: Why emotion matters.
Kousta, S. T., Vigliocco, G., Vinson, D., Andrews, M. W., & Campo, E. D. (2010) · Journal of Experimental Psychology General · doi:10.1037/a0021446 · Cited by 897 · see citing works
Age-of-acquisition, imagery, concreteness, familiarity, and ambiguity measures for 1,944 words
Gilhooly, K. J., & Logie, R. H. (1980) · Behavior Research Methods · doi:10.3758/bf03201693 · Cited by 877 · see citing works
Predicting the semantic orientation of adjectives
Hatzivassiloglou, V., & McKeown, K. R. (1997) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/979617.979640 · Cited by 822 · see citing works
The General Inquirer: A Computer Approach to Content Analysis.
Hartman, J. J., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvia, D. M. (1967) · American Sociological Review · doi:10.2307/2092070 · Cited by 723 · see citing works
Cross-Validatory Choice and Assessment of Statistical Predictions (With Discussion)
Stone, M. (1976) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1976.tb01573.x · Cited by 696 · see citing works
Representing word meaning and order information in a composite holographic lexicon.
Jones, M. N., & Mewhort, D. J. K. (2007) · Psychological Review · doi:10.1037/0033-295x.114.1.1 · Cited by 616 · see citing works
Judgment of contingency between responses and outcomes.
Jenkins, H. M., & Ward, W. C. (1965) · The Psychological Monographs · doi:10.1037/h0093874 · Cited by 593 · see citing works
Morrison, C. M., Chappell, T. D., & Ellis, A. W. (1997) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/027249897392017 · Cited by 525 · see citing works
Integrating experiential and distributional data to learn semantic representations.
Andrews, M. W., Vigliocco, G., & Vinson, D. (2009) · Psychological Review · doi:10.1037/a0016261 · Cited by 457 · see citing works
Symbol Interdependency in Symbolic and Embodied Cognition
Louwerse, M. M. (2011) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2010.01106.x · Cited by 399 · see citing works
Unsupervised Learning of Semantic Orientation from a Hundred-Billion-Word CorpusPreprint
Turney, P. D., & Littman, M. L. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0212012 · Cited by 361 · see citing works
Automated writing evaluation: Defining the classroom research agenda
Warschauer, M., & Ware, P. (2006) · Language Teaching Research · doi:10.1191/1362168806lr190oa · Cited by 337 · see citing works
Lexical access for concrete and abstract words.
Kroll, J. F., & Merves, J. S. (1986) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.12.1.92 · Cited by 261 · see citing works
The Concreteness Effect: Evidence for Dual Coding and Context Availability
Jessen, F., Heun, R., Erb, M., et al. (2000) · Brain and Language · doi:10.1006/brln.2000.2340 · Cited by 245 · see citing works
Emotion Vocabulary in Interlanguage
Dewaele, J., & Pavlenko, A. (2002) · Language Learning · doi:10.1111/0023-8333.00185 · Cited by 230 · see citing works
The influence of age of acquisition in word reading and other tasks: A never ending story?
Bonin, P., Barry, C. T., Méot, A., & Chalard, M. (2004) · Journal of Memory and Language · doi:10.1016/j.jml.2004.02.001 · Cited by 226 · see citing works
Developing Affective Lexical Resources
Valitutti, A., Strapparava, C., & Stock, O. (2004) · Cited by 180 · see citing works
Semantic differential profiles for 1,000 most frequent English words.
Heise, D. R. (1965) · The Psychological Monographs · doi:10.1037/h0093884 · Cited by 163 · see citing works
From paragraph to graph: Latent semantic analysis for information visualization
Landauer, T. K., Laham, D., & Derr, M. A. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0400341101 · Cited by 146 · see citing works
Bayer, M., Sommer, W., & Schacht, A. (2010) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2010.09.004 · Cited by 137 · see citing works
Kamps, J., Marx, M., Blockeel, H., et al. (2001) · UvA-DARE (University of Amsterdam) · Cited by 132 · see citing works
Do the effects of subjective frequency and age of acquisition survive better word frequency norms?
Brysbaert, M., & Cortese, M. J. (2010) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2010.503374 · Cited by 131 · see citing works
Chalard, M., Bonin, P., Méot, A., Boyer, B., & Fayol, M. (2003) · The European Journal of Cognitive Psychology · doi:10.1080/09541440244000076 · Cited by 125 · see citing works
Ferrand, L., Bonin, P., Méot, A., et al. (2008) · Behavior Research Methods · doi:10.3758/brm.40.4.1049 · Cited by 110 · see citing works
Subjective frequency and imageability ratings for 3,600 French nouns
Desrochers, A., & Thompson, G. L. (2009) · Behavior Research Methods · doi:10.3758/brm.41.2.546 · Cited by 104 · see citing works
Can emotional valence in stories be determined from words?
Bestgen, Y. (1994) · Cognition & Emotion · doi:10.1080/02699939408408926 · Cited by 91 · see citing works
Self-citationWord familiarity and frequency
Tanaka‐Ishii, K., & Terada, H. (2011) · Studia Linguistica · doi:10.1111/j.1467-9582.2010.01176.x · Cited by 58 · see citing works
Kintsch, W. (2015) · Routledge eBooks · doi:10.4324/9780203936399.ch5 · Cited by 45 · see citing works
A laboratory-based procedure for measuring emotional expression from natural speech
Cohen, A. S., Minor, K. S., Najolia, G. M., & Hong, S. L. (2009) · Behavior Research Methods · doi:10.3758/brm.41.1.204 · Cited by 44 · see citing works
Time flies when we read taboo words
Tipples, J. (2010) · Psychonomic Bulletin & Review · doi:10.3758/pbr.17.4.563 · Cited by 41 · see citing works
A guide to norms, ratings, and lists
Bradshaw, J. L. (1984) · Memory & Cognition · doi:10.3758/bf03198435 · Cited by 37 · see citing works
Building Affective Lexicons from Specific Corpora for Automatic Sentiment Analysis
Bestgen, Y. (2008) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/27nh6ysohqwa · Cited by 30 · see citing works
Self-citationIndex of norms and ratings published in the Psychonomic Society journals
Proctor, R. W., & Vu, K. L. (1999) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03200742 · Cited by 29 · see citing works
Thompson, G. L., & Desrochers, A. (2009) · Behavior Research Methods · doi:10.3758/brm.41.2.452 · Cited by 25 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Desrochers, A., & Saint-Aubin, J. (2008). Sources de matériel en français pour l'élaboration d'épreuves de compétences en lecture et en écriture [Materials for assessing writing and reading skills in French language]. Canadian Journal of Education, 31, 305-326. Retrieved June 22, 2011, from http://www.csse-scee.ca/RCE/ Articles/RCE31-2.html
- Schmid, H. (1994). Probabilistic part-of-speech tagging using decision trees. In: D. Jones (ed) Proceedings of the International Conference on New Methods in Language Processing, 44-49. University of Manchester, Manchester, England.
Cited by
Sorted by citations
Norms of valence, arousal, and dominance for 13,915 English lemmas
Warriner, A. B., Kuperman, V., & Brysbaert, M. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0314-x · Cited by 1,959 · see citing works
Rating norms should be calculated from cumulative link mixed effects models
Taylor, J. E., Rousselet, G. A., Scheepers, C., & Sereno, S. C. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01814-7 · Cited by 126 · see citing works
- Authors
- Bestgen, Y., & Vincze, N.
- Affiliations
- UCLouvain (BE)
- Corresponding author
- Bestgen, Yves
- ORCID
- Bestgen, Yves
- Venue
- Behavior Research Methods
- Published
- 2012-03-07
- Volume
- 44
- Issue
- 4
- Pages
- 998-1006
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- OpenAlex
- W2048795844
- Cited by
- 75
- References
- 51
- Citations by year
- 2026: 4 · 2025: 1 · 2024: 3 · 2023: 5 · 2022: 10 · 2021: 5 · 2020: 7 · 2019: 6 · 2018: 10 · 2017: 6 · 2016: 5 · 2015: 4 · 2014: 5 · 2013: 3 · 2012: 1
- FWCI
- 4.08
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Behavior Research Methods (publishedVersion)
- PubMed (publishedVersion)
- DIAL (Catholic University of Leuven) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- lexical norms, word similarity, latent semantic analysis, concreteness ratings, valence ratings, semantic dimensions, bootstrapping
- MeSH terms
- Emotions, Humans, Language, Semantics, Vocabulary, Imagery, Psychotherapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
