Beyond usability: evaluating emotional response as an integral part of the user experience
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
The role of emotion as an integral component of user experience has mostly been overlooked in the HCI literature. Instead, usability has been relied upon as the key indicator of user experience. We developed a methodology that combined verbal and nonverbal emotion scales. A usability study was then conducted, in which we collected both traditional usability metrics and emotional response data. Results indicated insignificant differences in usability metrics but numerous significant differences between emotional responses of users. Exploration of these emotional responses successfully provided additional insight into the user experience.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Measuring emotion: The self-assessment manikin and the semantic differential
Bradley, M. M., & Lang, P. J. (1994) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(94)90063-9 · Cited by 9,538 · cited 2×
“Currently, most publications discuss emotion metrics and their application at a more theoretical level [1, 2] or these metrics are applied in fields other than HCI [5, 6, 7, 9, 15], such as ergonomics and aesthetics.”
BackgroundMethod(1974) · Medical Entomology and Zoology · Cited by 7,295
Hedonic Consumption: Emerging Concepts, Methods and Propositions
Hirschman, E. C., & Holbrook, M. B. (1982) · Journal of Marketing · doi:10.1177/002224298204600314 · Cited by 5,616 · cited 1×
“primary motivation for consumptive behavior [10].”
(2013) · Encyclopedia of Creativity, Invention, Innovation and Entrepreneurship · doi:10.1007/978-1-4614-3858-8_100431 · Cited by 4,799
Frames, Biases, and Rational Decision-Making in the Human Brain
De Martino, B., Kumaran, D., Seymour, B., & Dolan, R. J. (2006) · Science · doi:10.1126/science.1128356 · Cited by 1,590 · cited 1×
“ACM 978-1-60558-247-4/09/04.”
A categorized list of emotion definitions, with suggestions for a consensual definition
Kleinginna, P. R., & Kleinginna, A. M. (1981) · Motivation and Emotion · doi:10.1007/bf00992553 · Cited by 1,228 · cited 1×
“As one might imagine, there are many definitions of "emotion" in the relevant literature [14].”
BackgroundThe Power of Feelings in Understanding Advertising Effects
Edell, J. A., & Burke, M. C. (1987) · Journal of Consumer Research · doi:10.1086/209124 · Cited by 1,119 · cited 9×
“Currently, most publications discuss emotion metrics and their application at a more theoretical level [1, 2] or these metrics are applied in fields other than HCI [5, 6, 7, 9, 15], such as ergonomics and aesthetics.”
BackgroundInfluentialFunology 2: From Usability to Enjoyment
(2018) · Human–Computer Interaction Series · doi:10.1007/978-3-319-68213-6 · Cited by 561
Facial Action Coding System (FACS)
Ekman, P. (2002) · Medical Entomology and Zoology · Cited by 433
EMOTION IN HUMAN-COMPUTER INTERACTION
(2007) · The Human-Computer Interaction Handbook · doi:10.1201/9781410615862-13 · Cited by 344
Research works printed but not linked: 3
- Desmet , P.M.A . Emotion through expression; designing mobile telephones with an emotional fit . Report of Modeling the Evaluation Structure of KANSEI , 3 ( 2000 ), 103 -- 110 . Desmet, P.M.A. Emotion through expression; designing mobile telephones with an emotional fit. Report of Modeling the Evaluation Structure of KANSEI, 3 (2000), 103--110.
- Holbrook , M.B. , & Batra , R. Toward a Standardized Emotional Profile (SEP) Useful in Measuring Responses to the Nonverbal Components of Advertising . In Hecker & ; Stewert (Eds.), Nonverbal Communications in Advertising, D.C. Heath ( 1987 ), 95 -- 109. Holbrook, M.B., & Batra, R. Toward a Standardized Emotional Profile (SEP) Useful in Measuring Responses to the Nonverbal Components of Advertising. In Hecker & Stewert (Eds.), Nonverbal Communications in Advertising, D.C. Heath (1987), 95 -- 109.
- Picard , R.W. Does HAL cry digital tears? Emotions and computers . In D.G. Stork (Ed.), Hal's Legacy: 2001 's Computer as Dream and Reality, M.I.T. Press ( 1997), 279--303. Picard, R.W. Does HAL cry digital tears? Emotions and computers. In D.G. Stork (Ed.), Hal's Legacy: 2001's Computer as Dream and Reality, M.I.T. Press (1997), 279--303.
Cited by
Usability: Lessons Learned … and Yet to Be Learned
Lewis, J. R. (2014) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2014.930311 · Cited by 311
Prather, J., Reeves, B. N., Denny, P., Becker, B. A., Leinonen, J., Luxton-Reilly, A., Powell, G., Finnie-Ansley, J., & Santos, E. A. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3617367 · Cited by 256
“Emotion is a key part of creating usable designs [2, 18, 56].”
BackgroundPartala, T., & Kallinen, A. (2011) · Interacting with Computers · doi:10.1016/j.intcom.2011.10.001 · Cited by 186
“Besides need-related aspects, emotions are also an integral part of user experience and their measurement consequently becomes central in the empirical research conducted in the field of user experience (Agarwal and Meyer, 2009). In the field of human– computer interaction, research on emotions originally focused larg…”
BackgroundInfluentialDon't look at me that way! Understanding User Attitudes Towards Data Glasses Usage
Koelle, M., Kranz, M., & Möller, A. (2015) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/2785830.2785842 · Cited by 143
“Semantic differentials are a well-understood and established method to measure emotional responses in psychology and in HCI [1, 8].”
MethodInfluentialA Systematic Review About User Experience Evaluation
Maia, C. L. B., & Furtado, E. S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-40409-7_42 · Cited by 93
Metadata
- Authors
- Agarwal, A., & Meyer, A.
- Affiliations
- Salesforce (United States) (US)
- Stanford University (US)
- Venue
- CHI '09 Extended Abstracts on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2009-04-04
- Pages
- 2919-2930
- OpenAlex
- W2069106458
- Cited by
- 115
- References
- 15
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 8 · 2023: 9 · 2022: 5 · 2021: 11 · 2020: 7 · 2019: 8 · 2018: 9 · 2017: 11 · 2016: 6 · 2015: 4 · 2014: 13 · 2013: 6 · 2012: 4
- FWCI
- 2.55
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- usability metrics, emotional responses, user experience evaluation, usability evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Beyond usability: evaluating emotional response as an integral part of the user experience. https://app.cochat.ai/papers/10.1145/1520340.1520420
- The paper
- Agarwal, A., & Meyer, A. (2009). Beyond usability: evaluating emotional response as an integral part of the user experience. CHI '09 Extended Abstracts on Human Factors in Computing Systems. https://doi.org/10.1145/1520340.1520420
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.