The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
User experience evaluation methods: current state and development needs
Abstract
The recent shift of emphasis to user experience (UX) has rendered it a central focus of product design and evaluation. A multitude of methods for UX design and evaluation exist, but a clear overview of the current state of the available UX evaluation methods is missing. This is partly due to a lack of agreement on the essential characteristics of UX. In this paper, we present the results of our multi-year effort of collecting UX evaluation methods from academia and industry with different approaches such as literature review, workshops, Special Interest Groups sessions and an online survey. We have collected 96 methods and analyzed them, among other criteria, based on the product development phase and the studied period of experience. Our analysis reveals development needs for UX evaluation methods, such as early-stage methods, methods for social and collaborative UX evaluation, establishing practicability and scientific quality, and a deeper understanding of UX.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Telling more than we can know: Verbal reports on mental processes.
Nisbett, R. E., & Wilson, T. D. (1977) · Psychological Review · doi:10.1037/0033-295x.84.3.231 · Cited by 11,317
Field Theory in Social Science: Selected Theoretical Papers.
Price, D. O., Lewin, K., & Cartwright, D. (1951) · American Sociological Review · doi:10.2307/2087618 · Cited by 3,501
(2006) · doi:10.1080/01449290500330331 · Cited by 3,026
Gaver, B., Dunne, T., & Pacenti, E. (1999) · Interactions · doi:10.1145/291224.291235 · Cited by 2,303
Field Theory in Social Science
Reed, R., & Lewin, K. (1951) · The American Catholic Sociological Review · doi:10.2307/3707800 · Cited by 2,109
Field Theory in Social Science: Selected Theoretical Papers
Lewin, K., & Cartwright, D. (1975) · Medical Entomology and Zoology · Cited by 1,417
Understanding, scoping and defining user experience: a survey approach
Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A. P. O. S., & Kort, J. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518813 · Cited by 1,127
Self-citationUser experience (UX): towards an experiential perspective on product quality
Hassenzahl, M. (2008) · Proceedings of the 20th Conference on l'Interaction Homme-Machine · doi:10.1145/1512714.1512717 · Cited by 791
User experience over time: an initial framework
Karapanos, E., Zimmerman, J., Forlizzi, J., & Martens, J.-B. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518814 · Cited by 561
User Interface Design and Evaluation
Stone, D., Jarrett, C., Woodroffe, M. R., & Minocha, S. (2005) · Cited by 482
(2008) · doi:10.1016/b978-0-12-373558-4.x0001-5 · Cited by 287
Isomursu, M., Tähti, M., Väinämö, S., & Kuutti, K. (2007) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2006.11.007 · Cited by 156
Value of Information Systems and Products: Understanding the Users’ Perspective and Values
Kujala, S., & Väänänen, K. (2009) · The Journal of Information Technology Theory and Application · Cited by 133
Now let's do it in practice: user experience evaluation methods in product development
Väänänen, K., Roto, V., & Hassenzahl, M. (2008) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1358628.1358967 · Cited by 94
Self-citationObrist, M., Roto, V., & Väänänen, K. (2009) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1520340.1520401 · Cited by 64
Self-citationEvaluating experience-focused HCI
Kaye, J. (2007) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1240866.1240877 · Cited by 36
User Experience Evaluation Methods in Product Development (UXEM’09)
Roto, V., Väänänen, K., Law, E. L., & Vermeeren, A. P. O. S. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-03658-3_141 · Cited by 33
Self-citationMeasuring affect in hci: going beyond the individual
Shami, N. S., Hancock, J. T., Peter, C., Muller, M. J., & Mandryk, R. L. (2008) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1358628.1358952 · Cited by 23
An Investigation into the use of Field Methods in the Design and Evaluation of Interactive Systems
Monahan, K., Lähteenmäki, M., McDonald, S., & Cockton, G. (2008) · Electronic workshops in computing · doi:10.14236/ewic/hci2008.10 · Cited by 23
User experience evaluation with experimental pilots
Isomursu, M. (2008) · Cited by 18
Cited by
Poushneh, A., & Vasquez-Parraga, A. Z. (2017) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2016.10.005 · Cited by 556
“Although prior literature has studied some UX dimensions, no mutual agreement about measuring UX has been reached (Vermeeren et al., 2010).”
BackgroundUX Curve: A method for evaluating long-term user experience
Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., Karapanos, E., & Sinnelä, A. (2011) · Interacting with Computers · doi:10.1016/j.intcom.2011.06.005 · Cited by 485
Self-citationUser experience model for augmented reality applications in urban heritage tourism
Han, D.-I., tom Dieck, M. C., & Jung, T. (2018) · Journal of Heritage Tourism · doi:10.1080/1743873x.2016.1251931 · Cited by 281
Attitudes towards user experience (UX) measurement
Law, E. L.-C., van Schaik, P., & Roto, V. (2014) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2013.09.006 · Cited by 270
Self-citation
Metadata
- Authors
- Vermeeren, A. P. O. S., Law, E. L.-C., Roto, V., Obrist, M., Hoonhout, J., & Väänänen-Vainio-Mattila, K.
- Venue
- Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries
- Publisher
- ACM
- Published
- 2010-10-16
- Pages
- 521-530
- OpenAlex
- W2086956503
- Cited by
- 592
- References
- 20
- FWCI
- 19.60
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.