CoChat
Sign Up
Cited by 66Gaze Tracking and Assistive TechnologyUsability and User Interface DesignElectronic Health Records Systems

Using Eye Trackers for Usability Evaluation of Health Information Technology: A Systematic Literature Review

Open access
Asan, Onur, Yang, Yushi·2015·JMIR Human Factors
DOI 10.2196/humanfactors.4062PMC PMC4797658
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Eye-tracking technology has been used to measure human cognitive processes and has the potential to improve the usability of health information technology (HIT). However, it is still unclear how the eye-tracking method can be integrated with other traditional usability methodologies to achieve its full potential. OBJECTIVE: The objective of this study was to report on HIT evaluation studies that have used eye-tracker technology, and to envision the potential use of eye-tracking technology in future research. METHODS: We used four reference databases to initially identify 5248 related papers, which resulted in only 9 articles that met our inclusion criteria. RESULTS: Eye-tracking technology was useful in finding usability problems in many ways, but is still in its infancy for HIT usability evaluation. Limited types of HITs have been evaluated by eye trackers, and there has been a lack of evaluation research in natural settings. CONCLUSIONS: More research should be done in natural settings to discover the real contextual-based usability problems of clinical and mobile HITs using eye-tracking technology with more standardized methodologies and guidance.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/humanfactors.4062

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 54

  1. [1]Bates DW Gawande AA Improving safety with information technology N Engl J Med 2003 06 19 348 25 2526 2534 10.1056/NEJMsa020847 12815139 348/25/2526 12815139 · DOI · Paper page
  2. [3]Chaudhry B Wang J Wu S Maglione M Mojica W Roth E Morton SC Shekelle PG Systematic review: impact of health information technology on quality, efficiency, and costs of medical care Ann Intern Med 2006 05 16 144 10 742 752 16702590 0000605-200605160-00125 16702590
  3. [4]Buntin MB Burke MF Hoaglin MC Blumenthal D The benefits of health information technology: a review of the recent literature shows predominantly positive results Health Aff (Millwood) 2011 03 30 3 464 471 10.1377/hlthaff.2011.0178 21383365 30/3/464 21383365 · DOI · Paper page
  4. [5]Jha AK Doolan D Grandt D Scott T Bates DW The use of health information technology in seven nations Int J Med Inform 2008 12 77 12 848 854 10.1016/j.ijmedinf.2008.06.007 18657471 S1386-5056(08)00088-9 18657471 · DOI · Paper page
  5. [6]Ford EW Menachemi N Peterson LT Huerta TR Resistance is futile: but it is slowing the pace of EHR adoption nonetheless J Am Med Inform Assoc 2009 05 16 3 274 281 10.1197/jamia.M3042 19261931 M3042 19261931 PMC2732222 · DOI
  6. [8]Rahimi B Vimarlund V Methods to evaluate health information systems in healthcare settings: a literature review J Med Syst 2007 10 31 5 397 432 17918694 17918694
  7. [9]Yen PY Bakken S Review of health information technology usability study methodologies J Am Med Inform Assoc 2012 06 19 3 413 422 10.1136/amiajnl-2010-000020 21828224 amiajnl-2010-000020 21828224 PMC3341772 · DOI · Paper page
  8. [10]Jacob RJK Karn KS Radach R Hyona J Deubel H Eye tracking in human-computer interaction and usability research: ready to deliver the promises The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research 2003 Amsterdam, The Netherlands Elsevier Science BV 573 603
  9. [11]Schall A Bergstrom J Bergstorm J Schall A Introduction to eye tracking Eye Tracking in User Experience Design 2014 Waltham, MA Morgan Kaufmann 3 26
  10. [12]Wickens CD Hollands JG Parasuraman R Banbury S Engineering Psychology and Human Performance. 4th edition 2012 Boston, MA Pearson
  11. [13]Poock GK Information processing vs pupil diameter Percept Mot Skills 1973 12 37 3 1000 1002 10.2466/pms.1973.37.3.1000 4764491 4764491 · DOI
  12. [14]Palinko O Kun AL Shyrokov A Heeman P Estimating cognitive load using remote eye tracking in a driving simulator Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications 2010 ETRA '10 March 22-24, 2010 Austin, TX 141 144 10.1145/1743666.1743701 · DOI
  13. [15]Richardson DC Spivey MJ Wnek GE Bowlin GL Eye tracking: Research areas and applications Encyclopedia of Biomaterials and Biomedical Engineering 2004 London, UK Taylor & Francis 573 582
  14. [16]Bruneau D Sasse MA McCarthy J The eyes never lie: the use of eyetracking data in HCI research Proceedings of the CHI2002 Workshop on Physiological Computing 2002 CHI2002 Workshop on Physiological Computing April 21, 2002 Minneapolis, MN
  15. [17]Djamasbi S Tullis T Hsu J Mazuera E Osberg K Bosch J Gender preferences in Web design: usability testing through eye tracking Proceedings of the Thirteenth Americas Conference on Information Systems (AMCIS) 2007 Thirteenth Americas Conference on Information Systems (AMCIS) August 9-12, 2007 Keystone, CO
  16. [19]Velazquez CE Pasch KE Attention to food and beverage advertisements as measured by eye-tracking technology and the food preferences and choices of youth J Acad Nutr Diet 2014 04 114 4 578 582 10.1016/j.jand.2013.09.030 24332473 S2212-2672(13)01551-7 24332473 · DOI
  17. [20]Goldberg JH Kotval XP Computer interface evaluation using eye movements: methods and constructs Int J Ind Ergon 1999 10 24 6 631 645 10.1016/S0169-8141(98)00068-7 · DOI
  18. [21]McConkie GW Yang SN Radach R Hyona J Deubel H How cognition affects eye movements during reading The Mind's Eye: Cognitive and Applied Aspects of Eye Movement Research 2003 Amsterdam, The Netherlands Elsevier Science BV 413 427
  19. [25]Reingold EM Eye tracking research and technology: towards objective measurement of data quality Vis cogn 2014 03 22 3 635 652 10.1080/13506285.2013.876481 24771998 24771998 PMC3996543 · DOI
  20. [26]Manhartsberger M Zellhofer N Eye tracking in usability research: What users really see Empowering Software Quality: How Can Usability Engineering Reach These Goals? 2005 Usability Symposium November 8, 2005 Vienna, Austria 141 152
  21. [27]Djamasbi S Siegel M Tullis T Generation Y, web design, and eye tracking Int J Hum Comput Stud 2010 5 68 5 307 323 10.1016/j.ijhcs.2009.12.006 · DOI
  22. [28]Dong W Liao H Xu F Liu Z Zhang S Using eye tracking to evaluate the usability of animated maps Sci China Earth Sci 2013 10 10 57 3 512 522 10.1007/s11430-013-4685-3 · DOI
  23. [29]Opach T Nossum A Evaluating the usability of cartographic animations with eye-movement analysis Proceedings of the 25th International Cartographic Conference 2011 25th International Cartographic Conference July 3-8, 2011 Paris, France
  24. [31]Cooper HM Synthesizing Research: A Guide for Literature Reviews. 3rd edition 1998 Thousand Oaks, CA SAGE Publications
  25. [32]Berg BL Lune H Qualitative Research Methods for the Social Sciences. 8th edition 2011 New York, NY Pearson
  26. [33]Bansback N Li LC Lynd L Bryan S Development and preliminary user testing of the DCIDA (Dynamic computer interactive decision application) for 'nudging' patients towards high quality decisions BMC Med Inform Decis Mak 2014 14 62 10.1186/1472-6947-14-62 25084808 1472-6947-14-62 25084808 PMC4130126 · DOI
  27. [34]Erol Barkana D Açık A Improvement of design of a surgical interface using an eye tracking device Theor Biol Med Model 2014 05 7 11 Suppl 1 S4 10.1186/1742-4682-11-S1-S4 25080176 1742-4682-11-S1-S4 25080176 PMC4108870 · DOI
  28. [35]Eghdam A Forsman J Falkenhav M Lind M Koch S Combining usability testing with eye-tracking technology: evaluation of a visualization support for antibiotic use in intensive care Stud Health Technol Inform 2011 169 945 949 21893885 21893885
  29. [36]Forsman J Anani N Eghdam A Falkenhav M Koch S Integrated information visualization to support decision making for use of antibiotics in intensive care: design and usability evaluation Inform Health Soc Care 2013 12 38 4 330 353 10.3109/17538157.2013.812649 23957739 23957739 · DOI
  30. [37]Henneman PL Fisher DL Henneman EA Pham TA Mei YY Talati R Nathanson BH Roche J Providers do not verify patient identity during computer order entry Acad Emerg Med 2008 07 15 7 641 648 19086323 19086323
  31. [38]Kules B Xie B Older adults searching for health information in MedlinePlus – An exploratory study of faceted online search interfaces Proceedings of the 74th ASIS&T Annual Meeting 2011 ASIS&T 2011 October 9-12, 2011 New Orleans, LA 9 13
  32. [39]Liu CJ Kemper S McDowd J The use of illustration to improve older adults' comprehension of health-related information: is it helpful? Patient Educ Couns 2009 08 76 2 283 288 10.1016/j.pec.2009.01.013 19286343 S0738-3991(09)00034-2 19286343 · DOI
  33. [40]Rashid S Soo S Sivaji A Naeni HS Bahri S Preliminary usability testing with eye tracking and FCAT analysis on occupational safety and health websites Procedia Soc Behav Sci 2013 11 97 737 744 10.1016/j.sbspro.2013.10.295 · DOI
  34. [41]Wolpin SE Halpenny B Whitman G McReynolds J Stewart M Lober WB Berry DL Development and usability testing of a web-based cancer symptom and quality-of-life support intervention Health Informatics J 2015 03 21 1 10 23 10.1177/1460458213495744 24406906 1460458213495744 24406906 PMC4747103 · DOI
  35. [42]Cowen L Ball LJ Delin J Faulkner X Finlay J Detienne F An eye movement analysis of web page usability People and Computers XVI-Memorable Yet Invisible 2002 London, UK Springer 317 335
  36. [43]Ehmke C Wilson S Identifying Web usability problems from eye-tracking data People and Computers XXI – HCI… but not as we know it: Proceedings of HCI 2007 2007 The 21st British HCI Group Annual Conference September 3-7, 2007 Lancaster, UK 10.1145/1531294.1531311 · DOI
  37. [44]Ball MJ Weaver C Abbott PA Enabling technologies promise to revitalize the role of nursing in an era of patient safety Int J Med Inform 2003 01 69 1 29 38 12485702 S1386505602000631 12485702
  38. [45]Moody LE Slocumb E Berg B Jackson D Electronic health records documentation in nursing: nurses' perceptions, attitudes, and preferences Comput Inform Nurs 2004 12 22 6 337 344 15602303 15602303
  39. [46]Poissant L Pereira J Tamblyn R Kawasumi Y The impact of electronic health records on time efficiency of physicians and nurses: a systematic review J Am Med Inform Assoc 2005 10 12 5 505 516 10.1197/jamia.M1700 15905487 M1700 15905487 PMC1205599 · DOI · Paper page
  40. [49]Acton T Golden W Gudee S Scott M Usability and acceptance in small-screen information systems Proceedings of 9th European Collaborative Electronic Commerce Technology and Research Conference 2004 The 9th European Collaborative Electronic Commerce Technology and Research Conference January 1, 2004 Surrey, UK
  41. [50]He J Siu C Chaparro B Strohl J Bergstrom J Schall A Mobile Eye Tracking in User Experience Design 2014 Waltham, MA Morgan Kaufmann 255 290
  42. [51]Virzi RA Sokolov J Karis D Usability problem identification using both low-high-fidelity prototypes Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 1996 CHI '96 April 13-18, 1996 Vancouver, BC New York, NY ACM 236 243 10.1145/238386.238516 · DOI
  43. [54]Carayon P Human factors of complex sociotechnical systems Appl Ergon 2006 07 37 4 525 535 10.1016/j.apergo.2006.04.011 16756937 S0003-6870(06)00058-5 16756937 · DOI
  44. [55]Rivera-Rodriguez AJ Karsh BT Interruptions and distractions in healthcare: review and reappraisal Qual Saf Health Care 2010 08 19 4 304 312 10.1136/qshc.2009.033282 20378621 qshc.2009.033282 20378621 PMC3007093 · DOI
  45. [56]Yang Y Rivera AJ Fortier C Abernathy J A macroergonomic usability evaluation of pre-filled medication syringes Proceedings of the International Symposium of Human Factors and Ergonomics in Healthcare 2014 The International Symposium of Human Factors and Ergonomics in Healthcare March 16-19, 2014 Chicago, IL 87 91 10.1177/2327857914031013 · DOI
  46. [57]Meingast M Roosta T Sastry S Security and privacy issues with health care information technology Proceedings of the 28th Annual International Conference of the IEEE: Engineering in Medicine and Biology Society, 2006. EMBS '06 2006 28th Annual International Conference of the IEEE: Engineering in Medicine and Biology Society, 2006 August 30-September 3, 2006 New York, NY IEEE 5453 5458 10.1109/IEMBS.2006.260060 17946702 · DOI
  47. [58]Thomas J Kellogg W Minimizing ecological gaps in interface design IEEE Softw 1989 01 6 1 78 86 10.1109/52.16905 · DOI
  48. [59]Olmsted-Hawala E Holland TH Quach V Bergstrom J Schall A Usability testing Eye Tracking in User Experience Design 2014 Waltham, MA Morgan Kaufmann 49 80
  49. [60]Henderson J Human gaze control during real-world scene perception Trends Cogn Sci 2003 11 7 11 498 504 14585447 S1364661303002481 14585447
  50. [61]Van Someren MW Barnard YF Sandberg JAC The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes. Volume 2 1994 London, UK Academic Press
  51. [62]Hertzum M Holmegaard KD Thinking aloud influences perceived time Hum Factors 2015 02 57 1 101 109 25790572 25790572
  52. [63]Bavelas JB Coates L Johnson T Listener responses as a collaborative process: the role of gaze J Commun 2002 09 52 3 566 580 10.1111/j.1460-2466.2002.tb02562.x · DOI · Paper page
  53. [64]Elling S Lentz L de Jong M Retrospective think-aloud method: using eye movements as an extra cue for participants verbalizations Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 2011 CHI '11 May 7-12, 2011 Vancouver, BC New York, NY ACM 1161 1170 10.1145/1978942.1979116 · DOI
  54. [65]Guan Z Lee S Cuddihy E Ramey J The validity of the stimulated retrospective think-aloud method as measured by eye tracking Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 2006 CHI '06 April 22-27, 2006 Montréal, QC New York, NY ACM 1253 1262 10.1145/1124772.1124961 · DOI

Cited by

Sorted by citations

Authors
Asan, O., & Yang, Y.
Affiliations
  • Medical College of Wisconsin (US)
  • Clemson University (US)
Corresponding author
Asan, Onur
ORCID
Venue
JMIR Human Factors
Publisher
JMIR Publications Inc.
Published
2015-04-14
Volume
2
Issue
1
Pages
e5
ISSN
2292-9495 (ISSN-L)
DOI
10.2196/humanfactors.4062
PMC
PMC4797658
OpenAlex
W2097345068
Cited by
66
References
6
Citations by year
2026: 8 · 2025: 3 · 2024: 7 · 2023: 3 · 2022: 7 · 2021: 7 · 2020: 9 · 2019: 9 · 2018: 7 · 2017: 3 · 2016: 2 · 2015: 1
FWCI
0.54
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Human Factors (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Medical College of Wisconsin
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Keywords
eye tracking, usability evaluation, health information technology, clinical information systems, mobile health, naturalistic assessment, usability problems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review