CoChat

Don’t Classify Ratings of Affect; Rank Them!

Open access
Martinez, Hector P., Yannakakis, Georgios N., Hallam, John · 2014 · IEEE Transactions on Affective Computing
Cited by 109

The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.

Abstract

How should affect be appropriately annotated and how should machine learning best be employed to map manifestations of affect to affect annotations? What is the use of ratings of affect for the study of affective computing and how should we treat them? These are the key questions this paper attempts to address by investigating the impact of dissimilar representations of annotated affect on the efficacy of affect modelling. In particular, we compare several different binary-class and pairwise preference representations for automatically learning from ratings of affect. The representations are compared and tested on three datasets: one synthetic dataset (testing “in vitro ”) and two affective datasets (testing “in vivo”). The synthetic dataset couples a number of attributes with generated rating values. The two affective datasets contain physiological and contextual user attributes, and speech attributes, respectively; these attributes are coupled with ratings of various affective and cognitive states. The main results of the paper suggest that ratings (when used) should be naturally transformed to ordinal (ranked) representations for obtaining more reliable and generalisable models of affect. The findings of this paper have a direct impact on affect annotation and modelling research but, most importantly, challenge the traditional state-of-practice in affective computing and psychometrics at large.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/taffc.2014.2352268

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 9 references

Research works printed but not linked: 35

  1. kolmogorov, 1957, Dokl Akad Nauk SSSR
  2. joachims, 0, Text categorization with support vector machines: Learning with many relevant features, Proc 10th Eur Conf Mach Learn, 137
  3. 10.1007/978-3-642-24600-5_55 · doi:10.1007/978-3-642-24600-5_55
  4. 10.1016/j.ijhcs.2008.06.004 · DOI
  5. 10.1007/s11257-010-9078-0 · DOI
  6. 10.2307/2332226 · DOI
  7. 10.1145/1007730.1007733 · DOI · Paper page
  8. crammer, 2001, Pranking with ranking, Adv Neural Inf Process Syst, 14, 641
  9. 10.1145/1102351.1102369 · DOI · Paper page
  10. 10.1016/j.specom.2007.01.010 · DOI · Paper page
  11. 10.1109/ICME.2005.1521579 · DOI · Paper page
  12. 10.1109/ICME.2010.5583101 · DOI
  13. 10.1080/1364557032000081654 · doi:10.1080/1364557032000081654
  14. 10.1109/FG.2013.6553804 · DOI
  15. 10.1109/TASL.2010.2064164 · DOI
  16. 10.1371/journal.pone.0014679 · DOI
  17. 10.1007/s10994-008-5097-z · DOI
  18. morris, 1995, Observations: Sam: The self-assessment manikinan efficient cross-cultural measurement of emotional response, J Advert Res, 35, 63
  19. fürnkranz, 2010, Preference Learning
  20. 10.1007/978-3-642-24600-5_30 · DOI
  21. 10.1007/978-3-642-24600-5_60 · DOI
  22. 10.1007/978-3-642-24600-5_23 · DOI
  23. likert, 1932, A technique for the measurement of attitudes, Arch Psychol, 22, 55
  24. healey, 0, Recording affect in the field: Towards methods and metrics for improving ground truth labels, Proc 4th Int Conf Affective Comput Intell Interaction, 6974, 107, 10.1007/978-3-642-24600-5_14 · DOI
  25. mitchell, 1997, Machine Learning
  26. 10.1109/T-AFFC.2012.15 · DOI
  27. 10.1145/1180995.1181029 · DOI · Paper page
  28. meng, 0, Multi-score learning for affect recognition: The case of body postures, Proc 4th Int Conf Affective Comput Intell Interaction, 6974, 225, 10.1007/978-3-642-24600-5_26 · DOI
  29. agresti, 2010, Analysis of Ordinal Categorical Data, 656, 10.1002/9780470594001 · DOI · Paper page
  30. baker, 0, The dynamics between student affect and behavior occurring outside of educational software, Proc 4th Int Conf Affective Comput Intell Interaction, 6974, 14, 10.1007/978-3-642-24600-5_5 · DOI
  31. georgiou, 0, ‘That’s aggravating, very aggravating’: Is it possible to classify behaviors in couple interactions using automatically derived lexical features?, Proc 4th Int Conf Affective Comput Intell Interaction, 87
  32. 10.1145/1877826.1877832 · DOI
  33. 10.1109/TPAMI.2009.107 · DOI
  34. 10.1007/s12193-013-0122-3 · DOI
  35. 10.1007/978-3-642-24600-5_19 · DOI

Cited by

Metadata

Authors
Martinez, H. P., Yannakakis, G. N., & Hallam, J.
Venue
IEEE Transactions on Affective Computing
Published
2014-07-01
Volume
5
Issue
3
Pages
314-326
ISSN
1949-3045 (ISSN-L)
OpenAlex
W1993008008
Cited by
109
References
9
FWCI
15.08
Type
journal article
Language
en
Open access
green
Funders
  • EU-funded FP7 ICT ILearnRW (318803)
  • Experiments reported in this paper run at the computational facilities procured through the European Regional Development Fund (ERDF-080)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.