The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Attitudes towards user experience (UX) measurement
Open accessAbstract
Abstract User experience (UX), as a recently established research area, is still haunted by the challenges of defining the scope of UX in general and operationalising experiential qualities in particular. To explore the basic question whether UX constructs are measurable, we conducted semi-structured interviews with 10 UX researchers from academia and one UX practitioner from the industry where a set of questions in relation to UX measurement were explored (Study 1). The interviewees expressed scepticism as well as ambivalence towards UX measures and shared anecdotes related to such measures in different contexts. Interestingly, the results suggested that design-oriented UX professionals tended to be sceptical about UX measurement. To examine whether such an attitude prevailed in the HCI community, we conducted a survey – UX Measurement Attitudes Survey (UXMAS) – with essentially the same set of 13 questions used in the interviews (Study 2). Specifically, participants were asked to rate a set of five statements to assess their attitude towards UX measurement, to identify (non)measurable experiential qualities with justifications, and to discuss the topic from the theoretical, methodological and practical perspectives. The survey was implemented in a paper-based and an online format. Altogether, 367 responses were received; 170 of them were valid and analysed. The survey provided empirical evidence on this issue as a baseline for progress in UX measurement. Overall, the survey results indicated that the attitude towards UX measurement was more positive than that identified in the interviews, and there were nuanced views on details of UX measurement. Implications for enhancing the acceptance of UX measures and the interplay between UX evaluation and system development are drawn: UX modelling grounded in theories to link experiential qualities with outcomes; the development of UX measurement tools with good measurement properties, and education within the HCI community to disseminate validated models, and measurement tools as well as their successful applications. Mutual recognition of the value of objective measures and subjective accounts of user experience can enhance the maturity of this area.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
(2011) · Cited by 14,317
(1998) · Cited by 13,501
Technology Acceptance Model 3 and a Research Agenda on Interventions
Venkatesh, V., & Bala, H. (2008) · Decision Sciences · doi:10.1111/j.1540-5915.2008.00192.x · Cited by 8,159
On the Theory of Scales of Measurement
Stevens, S. S. (1946) · Science · doi:10.1126/science.103.2684.677 · Cited by 4,761 · cited 2×
“S Stevens (1946), who developed the theory of scale types and imparted strong influences on measurement in social sciences such as intelligence tests.”
BackgroundWhat are emotions? And how can they be measured?
Scherer, K. R. (2005) · Social Science Information · doi:10.1177/0539018405058216 · Cited by 4,264
Emotional design: why we love (or hate) everyday things
(2004) · Choice Reviews Online · doi:10.5860/choice.41-6846 · Cited by 3,085
What is satisfying about satisfying events? Testing 10 candidate psychological needs.
Sheldon, K. M., Elliot, A. J., Kim, Y., & Kasser, T. (2001) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.80.2.325 · Cited by 1,694 · cited 1×
“, users’ loyalty to an online service) or preferably – but not necessarily only – objective outcomes that improve people’s productivity or health (Sheldon et al. 2001).”
Background(2004) · Technology as Experience · doi:10.7551/mitpress/6687.003.0011 · Cited by 1,306
Understanding, scoping and defining user experience: a survey approach
Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A. P. O. S., & Kort, J. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518813 · Cited by 1,127
Self-citationCurrent practice in measuring usability: Challenges to usability studies and research
Hornbæk, K. (2006) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2005.06.002 · Cited by 1,123
Understanding experience in interactive systems
Forlizzi, J., & Battarbee, K. (2004) · Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques · doi:10.1145/1013115.1013152 · Cited by 925
User experience (UX): towards an experiential perspective on product quality
Hassenzahl, M. (2008) · Proceedings of the 20th Conference on l'Interaction Homme-Machine · doi:10.1145/1512714.1512717 · Cited by 791
Bargas-Avila, J. A., & Hornbæk, K. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979336 · Cited by 660
Usability, aesthetics and emotions in human–technology interaction
Thüring, M., & Mahlke, S. (2007) · International Journal of Psychology · doi:10.1080/00207590701396674 · Cited by 622
User experience evaluation methods: current state and development needs
Vermeeren, A. P. O. S., Law, E. L.-C., Roto, V., Obrist, M., Hoonhout, J., & Väänänen-Vainio-Mattila, K. (2010) · Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries · doi:10.1145/1868914.1868973 · Cited by 592
Self-citationMcCarthy, J., & Wright, P. (2004) · The MIT Press eBooks · doi:10.7551/mitpress/6687.001.0001 · Cited by 561
How emotion is made and measured
Boehner, K., DePaula, R., Dourish, P., & Sengers, P. (2007) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2006.11.016 · Cited by 444
Identifying emotional states using keystroke dynamics
Epp, C., Lippold, M., & Mandryk, R. L. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979046 · Cited by 364
The Inference of Perceived Usability From Beauty
Hassenzahl, M., & Monk, A. (2010) · Human-Computer Interaction · doi:10.1080/07370024.2010.500139 · Cited by 330
Measurement issues in emotion research.
Larsen, R. J., & Fredrickson, B. L. (1999) · Cited by 316
The varieties of emotional experience: A meditation on James-Lange theory.
Lang, P. J. (1994) · Psychological Review · doi:10.1037//0033-295x.101.2.211 · Cited by 297
(2008) · doi:10.1016/b978-0-12-373558-4.x0001-5 · Cited by 287
Meta-analysis of correlations among usability measures
Hornbæk, K., & Law, E. L.-C. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240722 · Cited by 276
Self-citationAll You Need is Love: Current Strategies of Mediating Intimate Relationships through Technology
Hassenzahl, M., Heidecker, S., Eckoldt, K., Diefenbach, S., & Hillmann, U. (2012) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2395131.2395137 · Cited by 269
Modelling user experience – An agenda for research and practice
Law, E. L.-C., & van Schaik, P. (2010) · Interacting with Computers · doi:10.1016/j.intcom.2010.04.006 · Cited by 261
Self-citationSmart things: ubiquitous computing user experience design
(2011) · Choice Reviews Online · doi:10.5860/choice.48-5735 · Cited by 230
Correlations among prototypical usability metrics: evidence for the construct of usability
Sauro, J., & Lewis, J. R. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518947 · Cited by 221
Measuring the flow experience of gamers: An evaluation of the DFS-2
Procci, K., Singer, A. R., Levy, K. R., & Bowers, C. A. (2012) · Computers in Human Behavior · doi:10.1016/j.chb.2012.06.039 · Cited by 171
Pixels vs. Paper: Comparing Online and Traditional Survey Methods in Sport Psychology
Lonsdale, C., Hodge, K. P., & Rose, E. A. (2006) · Journal of Sport and Exercise Psychology · doi:10.1123/jsep.28.1.100 · Cited by 159
Measuring the dynamics of remembered experience over time
Karapanos, E., Zimmerman, J., Forlizzi, J., & Martens, J.-B. (2010) · Interacting with Computers · doi:10.1016/j.intcom.2010.04.003 · Cited by 156
Popular Lectures and Addresses
THOMSON, W. J. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511997235 · Cited by 150
The measurability and predictability of user experience
Law, E. L. (2011) · ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS) · doi:10.1145/1996461.1996485 · Cited by 136
Self-citationMumm, J., & Mutlu, B. (2011) · Computers in Human Behavior · doi:10.1016/j.chb.2011.02.002 · Cited by 133
Analyzing the emotional outcomes of the online search behavior with search engines
Flavián, C., Gurrea, R., & Orús, C. (2010) · Computers in Human Behavior · doi:10.1016/j.chb.2010.10.002 · Cited by 113
Theory Construction and Model Building Skills
DR.hosseinigolafshani, Z. (2014) · Cited by 110
User-Experience from an Inference Perspective
van Schaik, P., Hassenzahl, M., & Ling, J. (2012) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2240156.2240159 · Cited by 102
Fehnert, B., & Kosagowsky, A. (2008) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/1409240.1409294 · Cited by 102
Olsson, T., & Salo, M. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208677 · Cited by 88
Hartmann, J., Angeli, A. D., & Sutcliffe, A. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357190 · Cited by 83
Grounding experience: relating theory and method to evaluate the user experience of smartphones
Swallow, D., Blythe, M., & Wright, P. (2005) · doi:10.5555/1124666.1124679 · Cited by 69
Computerized Questionnaires and the Quality of Survey Data
Hanscom, B. S., Lurie, J. D., Homa, K. A., & Weinstein, J. N. (2002) · Spine · doi:10.1097/00007632-200208150-00020 · Cited by 68
Park, D., Lee, J., & Kim, S. (2011) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2011.06.006 · Cited by 43
Why do I like it? investigating the product-specificity of user experience
Groß, A., & Bongartz, S. (2012) · Nordic Conference on Human-Computer Interaction · doi:10.1145/2399016.2399067 · Cited by 39
An Experimental Analysis of Experiential and Cognitive Variables in Web Navigation
van Schaik, P., & Ling, J. (2012) · Human-Computer Interaction · doi:10.1080/07370024.2011.646923 · Cited by 35
Obrist, M., Law, E. L., Väänänen, K., Roto, V., Vermeeren, A. P. O. S., & Kuutti, K. (2011) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1979742.1979526 · Cited by 27
Self-citationA cognitive-experiential approach to modelling web navigation
van Schaik, P., & Ling, J. (2012) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2012.03.003 · Cited by 18
To measure or not to measure UX: an interview study
Law, E. L., & van Schaik, P. (2012) · Cited by 6
Self-citation
Research works printed but not linked: 4
- Bartholomew, D.J., (Ed.). , 2006. Measurement: Sage Benchmarks in Social Research Methods, vols. I–IV, Sage.
- Kaye, J. Buie, E.A., Hoonhout, J., Höök, K., Roto,V., Jenson,S., Wright, P., 2011. Designing for user experience: academia & industry. In: CHI EA 2011, pp. 219–222. · doi:10.1145/1979742.1979486
- Löwgren, 2007, Fluency as an experiential quality in augmented spaces, International Journal of Design, 1, 1
- Wixon, 2011, Measuring fun, trust, confidence, and other ethereal constructs: it isn't that hard, Interactions, 18, 74, 10.1145/2029976.2029995 · doi:10.1145/2029976.2029995
Cited by
Active Involvement of End Users When Developing Web-Based Mental Health Interventions
de Beurs, D., van Bruinessen, I., Noordman, J., Friele, R., & van Dulmen, S. (2017) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00072 · Cited by 97
“UCD is mainly assessed via qualitative (interview) methods (31).”
MethodThe Usability Construct: A Dead End?
Tractinsky, N. (2018) · Human–Computer Interaction · doi:10.1080/07370024.2017.1298038 · Cited by 87
Culturally sensitive user interface design: a case study with German and Vietnamese users
Lachner, F., Nguyen, M.-A., & Butz, A. (2018) · Proceedings of the Second African Conference for Human Computer Interaction: Thriving Communities · doi:10.1145/3283458.3283459 · Cited by 19
Targeting IMPACT: A New Psychological Model of User Experience
Kurta, L., & Freeman, J. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-05637-6_12 · Cited by 8
“Furthermore, in practical terms, without a process for hypothesis development to aid design, insights from measurement tools may not easily be interpreted into design decisions [29].”
Metadata
- Authors
- Law, E. L.-C., van Schaik, P., & Roto, V.
- Venue
- International Journal of Human-Computer Studies
- Publisher
- Elsevier BV
- Published
- 2014-06
- Volume
- 72
- Issue
- 6
- Pages
- 526-541
- ISSN
- 1071-5819 (ISSN-L)
- OpenAlex
- W2041074845
- Cited by
- 270
- References
- 48
- FWCI
- 29.33
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.